| Title | : | The Foundation Model Path to Open-World Robots |
| Speaker | : | Dhruv Shah (Assistant Professor @ Princeton University) |
| Details | : | Wed, 19 Jun, 2024 11:00 AM @ RBCDSAI Seminar Hall |
| Abstract: | : | Robot learning methods typically rely either on learning from large-scale simulation modeling and transferring to real-world settings or by collecting real-world interaction data on the target robot. While this paradigm has been successful for solving simple tasks in structured environments, it may fall short for tasks that are hard to simulate accurately (e.g. in off-road racing) and where data collection may be expensive (e.g. micro UAVs with <5 minute battery life). My research proposes an alternative paradigm of “cross-embodiment†robot learning, building algorithms and systems that can leverage internet-scale data to learn intelligent behaviors in unstructured, open-world environments. In this talk, I will discuss the unique challenges and opportunities that motivate building “robot foundation modelsâ€, and present the first instantiation of such a model for the task of visual navigation. I will then discuss how such a model can serve as a pre-trained backbone for a variety of downstream applications, such as autonomous off-road racing and socially-compliant navigation, as well as bootstrap learning for entirely new robots such as drones, quadrupeds, and manipulators. Finally, I will discuss how these robot foundation models can be empowered with current vision and language foundation models using a novel planning framework to build robust robotic systems capable of “in-the-wild†deployment of intelligent robotic systems. |
