| Title | : | Neural Collapse in Deep Network Classifiers |
| Speaker | : | Dr. Akshay Rangamani (Post-doctoral fellow @ MIT) |
| Details | : | Thu, 16 Nov, 2023 2:00 PM @ CSE 25 |
| Abstract: | : | Neural Collapse is a phenomenon in which the top-layer feature embeddings of samples from the same class tend to concentrate around their means, and the top layer’s weights align with those features. We first show how it emerges when training deep networks with weight decay and normalization. We then investigate if these properties extend to intermediate layers. We empirically study the evolution of the covariance and mean of representations as we move deeper into a trained neural network. We use the basic tool of studying the within-class and between-class covariances to describe how deep classifiers perform their task, and present analogies between deep networks and associative memories. |
