Image captioning is conventionally formulated as the task of generating captions that match
the conditional distribution of reference image-caption pairs. However, reference captions in …

We propose a method to efficiently equip the Segment Anything Model (SAM) with the ability
to generate regional captions. SAM presents strong generalizability to segment anything …

Given recent advances in generative AI technology, a key question is how large language
models (LLMs) can enhance acoustic modeling tasks using text decoding results from a …

Existing automatic captioning methods for visual content face challenges such as lack of
detail content hallucination and poor instruction following. In this work we propose …

The conventional training approach for image captioning involves pre-training a network
using teacher forcing and subsequent fine-tuning with Self-Critical Sequence Training to …

Despite recent advances in multimodal pre-training for visual description, state-of-the-art
models still produce captions containing errors, such as hallucinating objects not present in …

Vision--Language Models (VLMs) have demonstrated success across diverse applications,
yet their potential to assist in relevance judgments remains uncertain. This paper assesses …

In this report, we describe our submission to Track 6 of the DCASE 2024 challenge for the
task of Automated Audio Captioning (AAC). The submitted models utilize an encoder …

In difficult decision-making scenarios, it is common to have conflicting opinions among
expert human decision-makers as there may not be a single right answer. Such decisions …

Fine-tuning image captioning models with hand-crafted rewards like the CIDEr metric has
been a classical strategy for promoting caption quality at the sequence level. This approach …