A theoretical framework for grounding language is introduced that provides a computational
path from sensing and motor action to words and speech acts. The approach combines …

We present a visually-grounded language understanding model based on a study of how
people verbally describe objects in scenes. The emphasis of the model is on the …

To build robots that engage in fluid face-to-face spoken conversations with people, robots
must have ways to connect what they say to what they see. A critical aspect of how language …

Robots that interact with humans face-to-face using natural language need to be responsive
to the way humans use language in those situations. We propose a psychologically-inspired …

A common use of language is to refer to visually present objects. Modelling it in computers
requires modelling the link between language and perception. The" words as classifiers" …

Fuse is a situated spoken language understanding system that uses visual context to steer
the interpretation of speech. Given a visual scene and a spoken description, the system finds …

This thesis presents a perceptual system for a humanoid robot that integrates abilities such
as object localization and recognition with the deeper developmental machinery required to …

Human cognition makes extensive use of visualization and imagination. As a first step
towards giving a robot similar abilities, we have built a robotic system that uses a …

Evidence from recent psycholinguistic experiments suggests that humans resolve reference
incrementally in the presence of constraining visual context. In this paper, we present and …

Recent psycholinguistic experiments show that acoustic and syntactic aspects of online
speech processing are influenced by visual context through cross-modal influences. During …