Is chatgpt a good nlg evaluator? a preliminary study
Recently, the emergence of ChatGPT has attracted wide attention from the computational
linguistics community. Many prior studies have shown that ChatGPT achieves remarkable …
linguistics community. Many prior studies have shown that ChatGPT achieves remarkable …
Bartscore: Evaluating generated text as text generation
A wide variety of NLP applications, such as machine translation, summarization, and dialog,
involve text generation. One major challenge for these applications is how to evaluate …
involve text generation. One major challenge for these applications is how to evaluate …
TRUE: Re-evaluating factual consistency evaluation
Grounded text generation systems often generate text that contains factual inconsistencies,
hindering their real-world applicability. Automatic factual consistency evaluation may help …
hindering their real-world applicability. Automatic factual consistency evaluation may help …
Automatic machine translation evaluation in many languages via zero-shot paraphrasing
B Thompson, M Post - arXiv preprint arXiv:2004.14564, 2020 - arxiv.org
We frame the task of machine translation evaluation as one of scoring machine translation
output with a sequence-to-sequence paraphraser, conditioned on a human reference. We …
output with a sequence-to-sequence paraphraser, conditioned on a human reference. We …
Natural language watermarking via paraphraser-based lexical substitution
Although powerful pretrained language models generate high-quality output text, they bring
new concerns about the potential misuse of such models for malicious purposes. Natural …
new concerns about the potential misuse of such models for malicious purposes. Natural …
Quality controlled paraphrase generation
Paraphrase generation has been widely used in various downstream tasks. Most tasks
benefit mainly from high quality paraphrases, namely those that are semantically similar to …
benefit mainly from high quality paraphrases, namely those that are semantically similar to …
Data augmentation with paraphrase generation and entity extraction for multimodal dialogue system
Contextually aware intelligent agents are often required to understand the users and their
surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can …
surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can …
Paraphrase generation as zero-shot multilingual translation: Disentangling semantic similarity from lexical and syntactic diversity
B Thompson, M Post - arXiv preprint arXiv:2008.04935, 2020 - arxiv.org
Recent work has shown that a multilingual neural machine translation (NMT) model can be
used to judge how well a sentence paraphrases another sentence in the same language …
used to judge how well a sentence paraphrases another sentence in the same language …
AutoQA: From databases to QA semantic parsers with only synthetic training data
We propose AutoQA, a methodology and toolkit to generate semantic parsers that answer
questions on databases, with no manual effort. Given a database schema and its data …
questions on databases, with no manual effort. Given a database schema and its data …
An investigation of evaluation methods in automatic medical note generation
Recent studies on automatic note generation have shown that doctors can save significant
amounts of time when using automatic clinical note generation (Knoll et al., 2022) …
amounts of time when using automatic clinical note generation (Knoll et al., 2022) …