Department of Computer Science & Engineering

Course Syllabus

• Basics of Linear Algebra, Probability Theory and Optimization: Vectors, Inner product, Outer product, Inverse of a matrix, Eigenanalysis, Singular value decomposition, Probability distributions – Discrete distributions and Continuous distributions; Independence of events, Conditional probability distribution and Joint probability distribution, Bayes theorem, Unconstrained optimization, Constrained optimization – Lagrangian multiplier method. (7 Lectures)
• Methods for Function Approximation: Linear models for regression, Parameter estimation methods - Maximum likelihood method and Maximum a posteriori method; Regularization, Ridge regression, Lasso, Bias-Variance decomposition, Bayesian linear regression. (6 Lectures)
• Probabilistic Models for Classification: Bayesian decision theory, Bayes classifier, Minimum error-rate classification, Normal (Gaussian) density – Discriminant functions, Decision surfaces, Maximum-Likelihood estimation, Maximum a posteriori estimation; Gaussian mixture models -- Expectation-Maximization method for parameter estimation; Naive Bayes classifier, Non-parametric techniques for density estimation -- Parzen-window method, K-nearest neighbors method, Hidden Markov models (HMMs) for sequential pattern classification -- Discrete HMMs and Continuous density HMMs; (15 Lectures)
• Discriminative Learning based Models for Classification: Logistic regression, Perceptron, Multilayer feedforward neural network – Gradient descent method, Error backpropagation method; Support vector machine. (7 Lectures)
• Dimensionality Reduction Techniques: Principal component analysis, Fisher discriminant analysis, Multiple discriminant analysis. (4 Lectures)
• Non-Metric Methods for Classification: Decision trees, CART. ( 3 Lectures)
• Ensemble Methods for Classification: Bagging, Boosting, Gradient boosting (4 Lectures)
• Pattern Clustering: Criterion functions for clustering, Techniques for clustering -- K-means clustering, Hierarchical clustering, Density based clustering and Spectral clustering; Cluster validation. (6 Lectures)

Text Books

• C.M.Bishop, Pattern Recognition and Machine Learning, Springer, 2006
• R.O.Duda, P.E.Hart and D.G.Stork, Pattern Classification, John Wiley, 2001

Reference Books

• S. Theodoridis and K. Koutroumbas, Pattern Recognition, Academic Press, 2009
• E. Alpaydin, Introduction to Machine Learning, Prentice-Hall of India, 2010
• G. James, D. Witten, T. Hastie and R. Tibshirani, Introduction to Statistical Learning, Springer, 2013.