Next-gpt: Any-to-any multimodal llm
While recently Multimodal Large Language Models (MM-LLMs) have made exciting strides,
they mostly fall prey to the limitation of only input-side multimodal understanding, without the …
they mostly fall prey to the limitation of only input-side multimodal understanding, without the …
Deep learning and knowledge graph for image/video captioning: A review of datasets, evaluation metrics, and methods
Generating an image/video caption has always been a fundamental problem of Artificial
Intelligence, which is usually performed using the potential of Deep Learning Methods …
Intelligence, which is usually performed using the potential of Deep Learning Methods …
Accurate and fast compressed video captioning
Existing video captioning approaches typically require to first sample video frames from a
decoded video and then conduct a subsequent process (eg, feature extraction and/or …
decoded video and then conduct a subsequent process (eg, feature extraction and/or …
Alignment and generation adapter for efficient video-text understanding
Pre-trained models have demonstrated considerable performance, especially in enhancing
cross-modal understanding between videos and text. However, fine-tuning them at scale …
cross-modal understanding between videos and text. However, fine-tuning them at scale …
Native: Multi-modal knowledge graph completion in the wild
Multi-modal knowledge graph completion (MMKGC) aims to automatically discover the
unobserved factual knowledge from a given multi-modal knowledge graph by collaboratively …
unobserved factual knowledge from a given multi-modal knowledge graph by collaboratively …
Context-Guided Spatio-Temporal Video Grounding
Spatio-temporal video grounding (or STVG) task aims at locating a spatio-temporal tube for
a specific instance given a text query. Despite advancements current methods easily suffer …
a specific instance given a text query. Despite advancements current methods easily suffer …
RTQ: Rethinking Video-language Understanding Based on Image-text Model
Recent advancements in video-language understanding have been established on the
foundation of image-text models, resulting in promising outcomes due to the shared …
foundation of image-text models, resulting in promising outcomes due to the shared …
vid-TLDR: Training Free Token merging for Light-weight Video Transformer
Video Transformers have become the prevalent solution for various video downstream tasks
with superior expressive power and flexibility. However these video transformers suffer from …
with superior expressive power and flexibility. However these video transformers suffer from …