Current methods for learning visually grounded language from videos often rely on text
annotation, such as human generated captions or machine generated automatic speech …

In this paper, we present a method for learning discrete linguistic units by incorporating
vector quantization layers into neural models of visually grounded speech. We show that our …

We propose a trilingual semantic embedding model that associates visual objects in images
with segments of speech signals corresponding to spoken words in an unsupervised …

Following the popularisation of media streaming, a number of video streaming services are
continuously buying new video content to mine the potential profit from them. As such, the …

We propose a data expansion method for learning a multilingual semantic embedding
model using disjoint datasets containing images and their multilingual audio captions. Here …

In this paper, we explore self-supervised audio-visual models that learn from instructional
videos. Prior work has shown that these models can relate spoken words and sounds to …

Human learning is inherently multimodal. We watch, listen, read, and communicate to learn
from and understand our surroundings. There have been several advancements in machine …

Speech recognition and machine translation have made major progress over the past
decades, providing practical systems to map one language sequence to another. Although …

Audio-visual co-segmentation is a task to extract segments and regions corresponding to
specific events on unlabeled audio and video signals. It is particularly important to …

Since the popularisation of media streaming, a number of video streaming services are
continually buying new video content to mine the potential profit. As such, newly added …