Abstract: 

Thanks to advances in imitation and reinforcement learning techniques, we can now train intelligent agents to accomplish a diverse range of goals. But if we want to create household robots or personal assistants that can take advantage of this diversity, we need to give users some way to tell them what to do! This tutorial will focus on humans' favorite tools for communicating goals and plans: natural language. We'll assume basic familiarity with supervised learning and RL, and begin with a review of core machine learning techniques useful for natural language instruction following problems. The body of the talk will focus on modeling techniques for instruction following problems in different kinds of environments and data conditions. We'll conclude with a survey of other applications for the tools we've built, including instruction generation, interpretability, and machine teaching.

Bio: 

Jacob Andreas is an assistant professor at MIT and a senior researcher at Microsoft Semantic Machines. His research focuses on language learning as a window into reasoning, planning and perception, and on more general machine learning problems involving compositionality and modularity. Jacob earned his Ph.D. from UC Berkeley, his M.Phil. from Cambridge (where he studied as a Churchill scholar) and his B.S. from Columbia. He has been the recipient of an NSF graduate fellowship, a Facebook fellowship, and paper awards at NAACL and ICML.