An empirical survey on long document summarization: Datasets, models, and metrics

HY Koh, J Ju, M Liu, S Pan - ACM computing surveys, 2022 - dl.acm.org
Long documents such as academic articles and business reports have been the standard
format to detail out important issues and complicated subjects that require extra attention. An …

News summarization and evaluation in the era of gpt-3

T Goyal, JJ Li, G Durrett - arXiv preprint arXiv:2209.12356, 2022 - arxiv.org
The recent success of zero-and few-shot prompting with models like GPT-3 has led to a
paradigm shift in NLP research. In this paper, we study its impact on text summarization …

Recursively summarizing books with human feedback

J Wu, L Ouyang, DM Ziegler, N Stiennon… - arXiv preprint arXiv …, 2021 - arxiv.org
A major challenge for scaling machine learning is training models to perform tasks that are
very difficult or time-consuming for humans to evaluate. We present progress on this …

Fantastic Questions and Where to Find Them: FairytaleQA--An Authentic Dataset for Narrative Comprehension

Y Xu, D Wang, M Yu, D Ritchie, B Yao, T Wu… - arXiv preprint arXiv …, 2022 - arxiv.org
Question answering (QA) is a fundamental means to facilitate assessment and training of
narrative comprehension skills for both machines and young children, yet there is scarcity of …

LongEval: Guidelines for human evaluation of faithfulness in long-form summarization

K Krishna, E Bransom, B Kuehl, M Iyyer… - arXiv preprint arXiv …, 2023 - arxiv.org
While human evaluation remains best practice for accurately judging the faithfulness of
automatically-generated summaries, few solutions exist to address the increased difficulty …

Summ^ n: A multi-stage summarization framework for long input dialogues and documents

Y Zhang, A Ni, Z Mao, CH Wu, C Zhu, B Deb… - arXiv preprint arXiv …, 2021 - arxiv.org
Text summarization helps readers capture salient information from documents, news,
interviews, and meetings. However, most state-of-the-art pretrained language models (LM) …

Booookscore: A systematic exploration of book-length summarization in the era of llms

Y Chang, K Lo, T Goyal, M Iyyer - arXiv preprint arXiv:2310.00785, 2023 - arxiv.org
Summarizing book-length documents (> 100K tokens) that exceed the context window size
of large language models (LLMs) requires first breaking the input document into smaller …

Reading subtext: Evaluating large language models on short story summarization with writers

M Subbiah, S Zhang, LB Chilton… - Transactions of the …, 2024 - direct.mit.edu
Abstract We evaluate recent Large Language Models (LLMs) on the challenging task of
summarizing short stories, which can be lengthy, and include nuanced subtext or scrambled …

Multi-lexsum: Real-world summaries of civil rights lawsuits at multiple granularities

Z Shen, K Lo, L Yu, N Dahlberg… - Advances in …, 2022 - proceedings.neurips.cc
With the advent of large language models, methods for abstractive summarization have
made great strides, creating potential for use in applications to aid knowledge workers …

Conditional generation with a question-answering blueprint

S Narayan, J Maynez, RK Amplayo… - Transactions of the …, 2023 - direct.mit.edu
The ability to convey relevant and faithful information is critical for many tasks in conditional
generation and yet remains elusive for neural seq-to-seq models whose outputs often reveal …