Human action recognition systems use data collected from a wide range of sensors to
accurately identify and interpret human actions. One of the most challenging issues for …

In this paper, we develop position embedding transformation (PETR) for multi-view 3D
object detection. PETR encodes the position information of 3D coordinates into image …

Recently integrating video foundation models and large language models to build a video
understanding system can overcome the limitations of specific pre-defined vision tasks. Yet …

We introduce LAVILA, a new approach to learning video-language representations by
leveraging Large Language Models (LLMs). We repurpose pre-trained LLMs to be …

Transformer models have shown great success handling long-range interactions, making
them a promising tool for modeling video. However, they lack inductive biases and scale …

Most existing forecasting systems are memory-based methods, which attempt to mimic
human forecasting ability by employing various memory mechanisms and have progressed …

Procedural activity understanding requires perceiving human actions in terms of a broader
task, where multiple keysteps are performed in sequence across a long video to reach a …

Understanding long real-world videos requires modeling of long-range visual
dependencies. To this end we explore video-first architectures building on the common …

Effective modeling of complex spatiotemporal dependencies in long-form videos remains an
open problem. The recently proposed Structured State-Space Sequence (S4) model with its …

Video-language embeddings are a promising avenue for injecting semantics into visual
representations, but existing methods capture only short-term associations between …