Basics of Probability, Random Processes and Linear Algebra (recap):
Probability: independence of events, conditional and joint probability, Bayes theorem
Random Processes: Stationary and non-stationary processes, Expectation, Autocorrelation, Cross-Correlation, spectra.
Linear Algebra: Inner product, outer product, inverses, eigen values, eigen vectors, singular values, singular vectors.
Bayes Decision Theory : Minimum-error-rate classification. Classifiers, Discriminant functions, Decision surfaces. Normal density and discriminant functions.
Discrete features.
Parameter Estimation Methods : Maximum-Likelihood estimation :Gaussian case. Maximum a Posteriori estimation. Bayesian estimation: Gaussian case.
Unsupervised learning and clustering - Criterion functions for clustering. Algorithms for clustering: K-Means, Hierarchical and other methods. Cluster validation.
Gaussian mixture models, Expectation-Maximization method for parameter estimation.
Maximum entropy estimation. Sequential Pattern Recognition. Hidden Markov Models (HMMs). Discrete HMMs. Continuous HMMs.
Nonparametric techniques for density estimation. Parzen-window method. K-Nearest Neighbour method.
Dimensionality reduction: Principal component analysis - it relationship to eigen analysis. Fisher discriminant analysis - Generalised eigen analysis. Eigen vectors/Singular vectors as dictionaries. Factor Analysis, Total variability space - a dictionary learning methods. Non negative matrix factorisation - a dictionary learning method.
Linear discriminant functions : Gradient descent procedures, Perceptron,
Support vector machines - a brief introduction.
Artificial neural networks: Multilayer perceptron - feedforwark neural network. A brief introduction to deep neural networks, convolutional neural networks, recurrent neural networks.
Non-metric methods for pattern classification : Non-numeric data or nominal data. Decision trees: Classification and Regression Trees (CART).
Text Books:
R.O.Duda, P.E.Hart and D.G.Stork, Pattern Classification, John Wiley, 2001
S.Theodoridis and K.Koutroumbas, Pattern Recognition, 4th Ed., Academic Press, 2009
C.M.Bishop, Pattern Recognition and Machine Learning, Springer, 2006
