Description:
Neural Networks have made tremendous impact on various AI fields in the recent past. In this course, we study the basics of Neural Networks and their various variants such as the Convolutional Neural Networks and Recurrent Neural Networks. We also study the different ways in which they can be used to solve problems in various domains such as Computer Vision, Speech and NLP. We also look at the latest results and trends in the field.
Course Content:
Overview: of the Classification task and motivation for NNs to solve these tasks. Network Organization: Biological Neurons, Idea of computational units, Activation functions, Multi-layer Perceptrons, Convolutional Neural Networks, Convolution and pooling, Higher-level representations, Fea- ture visualization.
Training Algorithms: Loss Functions, Optimization, Stochastic Gradient Descent, Back-propagation, Initialization, Regularization, Update rules, Ensembles, data augmentation, Transfer learning, Dropout, Batch Normalization. Advanced Architectures: Recurrent Neural Networks: RNN, LSTM, GRU, CTC, Residual networks etc. Generative models: Restrictive Boltzmann Machines (RBMs), MCMC and Gibbs Sampling, Variational Auto-encoders, Generative Adversarial Networks. Applications: Application to various problems in different AI fields such as Computer vision, NLP and Speech. A subset of the following topics will be covered: Image Classification, Object Detection, Image Segmentation, Semantic segmentation, Instance segmen- tation, stereo matching, optical flow, style transfer, PixelRNN, Human Pose Estimation, Contour Detec- tion, shape classification, 3D Object Detection and Classification, Video analysis, summarization, label- ing, Language modeling, Image captioning, visual question answering, Attention, Neural Machine Trans- lation, Document Question Answering, Encoder Decoder Models, Text Summarization, and other recent applications from NLP, Speech and Computer Vision. Other latest ideas and trends: such as adversarial examples, network compaction, unsupervised learn- ing, transfer learning etc.
TextBooks
None
ReferenceBooks:
- Deep Learning, An MIT Press book, Ian Goodfellow and Yoshua Bengio and Aaron Courvill.
- Information Theory, Inference, and Learning Algorithms (Ch.5), DavidMacKay.
- Latest research papers from various Computer Vision, Natural Language Processing, Speech and Information Retrieval conferences.
