The recent success of zero-and few-shot prompting with models like GPT-3 has led to a
paradigm shift in NLP research. In this paper, we study its impact on text summarization …

Despite thousands of researchers, engineers, and artists actively working on improving text-
to-image generation models, systems often fail to produce images that accurately align with …

We introduce INSTRUCTOR, a new method for computing text embeddings given task
instructions: every text input is embedded together with instructions explaining the use case …

By simply composing prompts, developers can prototype novel generative applications with
Large Language Models (LLMs). To refine prototypes into products, however, developers …

Large language models, particularly GPT-3, are able to produce high quality summaries of
general domain news articles in few-and zero-shot settings. However, it is unclear if such …

We address a fundamental challenge in Natural Language Generation (NLG) model
evaluation--the design and evaluation of evaluation metrics. Recognizing the limitations of …

We introduce CheckEval, a novel evaluation framework using Large Language Models,
addressing the challenges of ambiguity and inconsistency in current evaluation methods …

Language technologies that accurately model the dynamics of events must perform
commonsense reasoning. Existing work evaluating commonsense reasoning focuses on …

Abstract Metrics for Inter-Annotator Agreement (IAA), like Cohen's Kappa, are crucial for
validating annotated datasets. Although high agreement is often used to show the reliability …

There are rich opportunities to reduce the language complexity of professional content
(either human-written or computer-generated) and make it accessible to a broad audience …