Re-evaluating evaluation in text summarization

M Bhandari, P Gour, A Ashfaq, P Liu… - arXiv preprint arXiv …, 2020 - arxiv.org
Automated evaluation metrics as a stand-in for manual evaluation are an essential part of
the development of text-generation tasks such as text summarization. However, while the …

Automatic text evaluation through the lens of Wasserstein barycenters

P Colombo, G Staerman, C Clavel… - arXiv preprint arXiv …, 2021 - arxiv.org
A new metric\texttt {BaryScore} to evaluate text generation based on deep contextualized
embeddings eg, BERT, Roberta, ELMo) is introduced. This metric is motivated by a new …

Infolm: A new metric to evaluate summarization & data2text generation

PJA Colombo, C Clavel, P Piantanida - Proceedings of the AAAI …, 2022 - ojs.aaai.org
Assessing the quality of natural language generation (NLG) systems through human
annotation is very expensive. Additionally, human annotation campaigns are time …

Investigating entropy for extractive document summarization

A Khurana, V Bhatnagar - Expert Systems with Applications, 2022 - Elsevier
Automatic text summarization aims to cut down readers' time and cognitive effort by reducing
the content of a text document without compromising on its essence. Ergo, informativeness …

A pseudo-metric between probability distributions based on depth-trimmed regions

G Staerman, P Mozharovskyi, P Colombo… - arXiv preprint arXiv …, 2021 - arxiv.org
The design of a metric between probability distributions is a longstanding problem motivated
by numerous applications in Machine Learning. Focusing on continuous probability …

[HTML][HTML] Summarization of biomedical articles using domain-specific word embeddings and graph ranking

M Moradi, M Dashti, M Samwald - Journal of Biomedical Informatics, 2020 - Elsevier
Text summarization tools can help biomedical researchers and clinicians reduce the time
and effort needed for acquiring important information from numerous documents. It has been …

How well do you know your summarization datasets?

P Tejaswin, D Naik, P Liu - arXiv preprint arXiv:2106.11388, 2021 - arxiv.org
State-of-the-art summarization systems are trained and evaluated on massive datasets
scraped from the web. Despite their prevalence, we know very little about the underlying …

Coldgans: Taming language gans with cautious sampling strategies

T Scialom, PA Dray, S Lamprier… - Advances in …, 2020 - proceedings.neurips.cc
Training regimes based on Maximum Likelihood Estimation (MLE) suffer from known
limitations, often leading to poorly generated text sequences that lack of coherence …

Wikicheck: An end-to-end open source automatic fact-checking api based on wikipedia

M Trokhymovych, D Saez-Trumper - Proceedings of the 30th ACM …, 2021 - dl.acm.org
With the growth of fake news and disinformation, the NLP community has been working to
assist humans in fact-checking. However, most academic research has focused on model …

Automatic text summarization using deep reinforced model coupling contextualized word representation and attention mechanism

H Aliakbarpour, MT Manzuri, AM Rahmani - Multimedia Tools and …, 2024 - Springer
With the rapid and unprecedented growth of textual data in recent years, there is a
remarkable need for automatic text summarization models to retrieve useful information from …