| Title | : | Memory Networks, Neural Turing Machines and all that |
| Speaker | : | Sarath Chandar A.P. (University of Montreal, Canada) |
| Details | : | Tue, 7 Jun, 2016 3:30 PM @ BSB 361 |
| Abstract: | : | Designing general-purpose learning algorithms is one of the long-standing goals
of artificial intelligence. Despite the success of deep learning in this area, there are still a set of complex tasks that are not well addressed by conventional neural networks. Those tasks often require a neural network to be equipped with an explicit, external memory in which a larger, potentially unbounded, set of facts need to be stored. They include, but are not limited to, episodic question-answering and compact algorithms. Recently two promising approaches based on neural networks to this type of tasks have been proposed: Memory Networks and Neural Turing Machines.
In this talk, I will give an overview of this paradigm of "neural networks with memory". We will focus on two major challenges: how to address the memory efficiently and how to scale up these architectures to larger memory. To address the memory addressing problem, we propose Dynamic Neural Turing Machines (D-NTM), which can learn a complex addressing scheme which could be a non-linear combination of location based addressing and content based addressing. D-NTM addressing can be either soft (based on softmax) or hard (based on REINFORCE). To address the scalability problem, we propose Hierarchical Memory Networks (HMN) which can learn to organize the memory in a hierarchical fashion using approximate Maximum Inner Product Search (MIPS) based algorithms such that it is easy for the reader to selectively read the content from the memory. |
