Grounded Language Learning in Virtual Environments (Clark)

Stephen Clark, U Cambridge and Quantinuum, 19 November, 2020

VIDEO

Abstract: Natural Language Processing is currently dominated by the application of text-based language models such as GPT. One feature of these models is that they rely entirely on the statistics of text, without making any connection to the world, which raises the interesting question of whether such models could ever properly “understand” the language. One these models can be grounded is to connect them to images or videos, for example by conditioning the language models on visual input and using them for captioning. In this talk I extend the grounding idea to a simulated virtual world: an environment which an agent can perceive, explore and interact with. A neural-network-based agent is trained — using distributed deep reinforcement learning — to associate words and phrases with things that it learns to see and do in the virtual world.The world is 3D, built in Unity, and contains recognisable objects, including some from the ShapeNet repository of assets. One of the difficulties in training such networks is that they have a tendency to overfit to their training data.We demonstrate how the interactive, first-person perspective of an agent helps it to generalize to out-of-distribution settings. Training the agents typically requires a huge number of training examples. We show how meta-learning can be used to teach the agents to bind words to objects in a one-shot setting. The agent is able to combine its knowledge of words obtained one-shot with its stable knowledge of word meanings learned over many episodes, providing a form of grounded language learning which is both “fast and slow”. Joint work with Felix Hill.

Clark, S., Lerchner, A., von Glehn, T., Tieleman, O., Tanburn, R., Dashevskiy, M., & Bosnjak, M. (2021). Formalising Concepts as Grounded Abstractions. arXiv preprint arXiv:2101.05125.

Tull, S., Shaikh, R. A., Zemljic, S. S., & Clark, S. (2023). From Conceptual Spaces to Quantum Concepts: Formalising and Learning Structured Conceptual Models. arXiv preprint arXiv:2401.08585.