Abstract Controllable Image Captioning (CIC)--generating image descriptions following
designated control signals--has received unprecedented attention over the last few years …

We address the problem of recognizing situations in images. Given an image, the task is to
predict the most salient verb (action), and fill its semantic roles such as who is performing the …

The NLP community has seen substantial recent interest in grounding to facilitate interaction
between language technologies and the world. However, as a community, we use the term …

Abstract Language communication plays an important role in human learning and
knowledge acquisition. With the emergence of a new generation of cognitive robots …

Grounding textual phrases in visual content with standalone image-sentence pairs is a
challenging task. When we consider grounding in instructional videos, this problem …

Our goal is to develop models that allow a robot to efficiently understand or “ground” natural
language instructions in the context of its world representation. Contemporary approaches …

To enable human-robot communication and collaboration, previous works represent
grounded verb semantics as the potential change of state to the physical world caused by …

To enable situated human-robot dialogue, techniques to support grounded language
communication are essential. One particular challenge is to ground human language to …

Abstract Grounded Situation Recognition (GSR), ie, recognizing the salient activity (or verb)
category in an image (eg, buying) and detecting all corresponding semantic roles (eg, agent …

We propose an unsupervised method for reference resolution in instructional videos, where
the goal is to temporally link an entity (eg," dressing") to the action (eg," mix yogurt") that …