CLAIR: Evaluating image captions with large language models

D Chan, S Petryk, JE Gonzalez, T Darrell… - arXiv preprint arXiv …, 2023 - arxiv.org
The evaluation of machine-generated image captions poses an interesting yet persistent
challenge. Effective evaluation measures must consider numerous dimensions of similarity …

Mauve scores for generative models: Theory and practice

K Pillutla, L Liu, J Thickstun, S Welleck… - Journal of Machine …, 2023 - jmlr.org
Generative artificial intelligence has made significant strides, producing text
indistinguishable from human prose and remarkably photorealistic images. Automatically …

Ic3: Image captioning by committee consensus

DM Chan, A Myers, S Vijayanarasimhan… - arXiv preprint arXiv …, 2023 - arxiv.org
If you ask a human to describe an image, they might do so in a thousand different ways.
Traditionally, image captioning models are trained to generate a single" best"(most like a …

Scalable and accurate self-supervised multimodal representation learning without aligned video and text data

V Lialin, S Rawls, D Chan, S Ghosh… - Proceedings of the …, 2023 - openaccess.thecvf.com
Scaling up weakly-supervised datasets has shown to be highly effective in the image-text
domain and has contributed to most of the recent state-of-the-art computer vision and …

Mallows-DPO: Fine-Tune Your LLM with Preference Dispersions

H Chen, H Zhao, H Lam, D Yao, W Tang - arXiv preprint arXiv:2405.14953, 2024 - arxiv.org
Direct Preference Optimization (DPO) has recently emerged as a popular approach to
improve reinforcement learning with human feedback (RLHF), leading to better techniques …

Talking Machines: Philosophical Essays on Language Models

N Osman Attah - 2024 - d-scholarship.pitt.edu
This dissertation is a collection of three independent but thematically connected papers on
language models. The first assesses a currently popular account of the nature of the …

Understanding, Building, and Evaluating Models for Context Aware Conditional Natural Language Generation

DM Chan - 2024 - search.proquest.com
If you ask a human to describe an image, they might do so in a thousand different ways.
Each of these descriptions depends not only on the image but also on a rich tapestry of …