The challenging task of summary evaluation: an overview

E Lloret, L Plaza, A Aker - Language Resources and Evaluation, 2018 - Springer
Abstract Evaluation is crucial in the research and development of automatic summarization
applications, in order to determine the appropriateness of a summary based on different …

Better summarization evaluation with word embeddings for ROUGE

JP Ng, V Abrecht - arXiv preprint arXiv:1508.06034, 2015 - arxiv.org
ROUGE is a widely adopted, automatic evaluation measure for text summarization. While it
has been shown to correlate well with human judgements, it is biased towards surface …

MCRMR: Maximum coverage and relevancy with minimal redundancy based multi-document summarization

P Verma, H Om - Expert Systems with Applications, 2019 - Elsevier
In this paper, we propose a novel extraction based method for multi-document
summarization that covers three important features of a good summary: coverage, non …

Finding a balanced degree of automation for summary evaluation

S Zhang, M Bansal - arXiv preprint arXiv:2109.11503, 2021 - arxiv.org
Human evaluation for summarization tasks is reliable but brings in issues of reproducibility
and high costs. Automatic metrics are cheap and reproducible but sometimes poorly …

Automatic Arabic text summarization: a survey

AB Al-Saleh, MEB Menai - Artificial Intelligence Review, 2016 - Springer
This survey investigates several research studies that have been conducted in the field of
Arabic text summarization. Specifically, it addresses summarization and evaluation methods …

Understanding the extent to which content quality metrics measure the information quality of summaries

D Deutsch, D Roth - Proceedings of the 25th Conference on …, 2021 - aclanthology.org
Reference-based metrics such as ROUGE or BERTScore evaluate the content quality of a
summary by comparing the summary to a reference. Ideally, this comparison should …

UICLIP: a data-driven model for assessing user interface design

J Wu, YH Peng, XYA Li, A Swearngin… - Proceedings of the 37th …, 2024 - dl.acm.org
User interface (UI) design is a difficult yet important task for ensuring the usability,
accessibility, and aesthetic qualities of applications. In our paper, we develop a machine …

Creating language resources for under-resourced languages: methodologies, and experiments with Arabic

M El-Haj, U Kruschwitz, C Fox - Language Resources and Evaluation, 2015 - Springer
Abstract Language resources are important for those working on computational methods to
analyse and study languages. These resources are needed to help advancing the research …

Mwi-sum: A multilingual summarizer based on frequent weighted itemsets

E Baralis, L Cagliero, A Fiori, P Garza - ACM Transactions on …, 2015 - dl.acm.org
Multidocument summarization addresses the selection of a compact subset of highly
informative sentences, ie, the summary, from a collection of textual documents. To perform …

Evaluation of text summaries without human references based on the linear optimization of content metrics using a genetic algorithm

J Rojas-Simón, Y Ledeneva… - Expert systems with …, 2021 - Elsevier
Abstract The Evaluation of Text Summaries (ETS) has been a task of constant challenges to
the development of Automatic Text Summarization (ATS). Within the ATS task, the ETS is …