Uncertainty in natural language generation: From theory to applications
Recent advances of powerful Language Models have allowed Natural Language
Generation (NLG) to emerge as an important technology that can not only perform traditional …
Generation (NLG) to emerge as an important technology that can not only perform traditional …
It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk
Minimum Bayes Risk (MBR) decoding is a method for choosing the outputs of a machine
learning system based not on the output with the highest probability, but the output with the …
learning system based not on the output with the highest probability, but the output with the …
Mbr and qe finetuning: Training-time distillation of the best and most expensive decoding methods
M Finkelstein, S Naskar, M Mirzazadeh, A Shah… - arXiv preprint arXiv …, 2023 - arxiv.org
Recent research in decoding methods for Natural Language Generation (NLG) tasks has
shown that MAP decoding is not optimal, because model probabilities do not always align …
shown that MAP decoding is not optimal, because model probabilities do not always align …
Narrowing the knowledge evaluation gap: Open-domain question answering with multi-granularity answers
Factual questions typically can be answered correctly at different levels of granularity. For
example, both``August 4, 1961''and``1961''are correct answers to the question``When was …
example, both``August 4, 1961''and``1961''are correct answers to the question``When was …
Non-exchangeable conformal language generation with nearest neighbors
Quantifying uncertainty in automatically generated text is important for letting humans check
potential hallucinations and making systems more reliable. Conformal prediction is an …
potential hallucinations and making systems more reliable. Conformal prediction is an …
Faster minimum Bayes risk decoding with confidence-based pruning
J Cheng, A Vlachos - arXiv preprint arXiv:2311.14919, 2023 - arxiv.org
Minimum Bayes risk (MBR) decoding outputs the hypothesis with the highest expected utility
over the model distribution for some utility function. It has been shown to improve accuracy …
over the model distribution for some utility function. It has been shown to improve accuracy …
Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding
Minimum Bayes-Risk (MBR) decoding is shown to be a powerful alternative to beam search
decoding for a wide range of text generation tasks. However, MBR requires a huge amount …
decoding for a wide range of text generation tasks. However, MBR requires a huge amount …
Measuring Uncertainty in Neural Machine Translation with Similarity-Sensitive Entropy
J Cheng, A Vlachos - Proceedings of the 18th Conference of the …, 2024 - aclanthology.org
Uncertainty estimation is an important diagnostic tool for statistical models, and is often used
to assess the confidence of model predictions. Previous work shows that neural machine …
to assess the confidence of model predictions. Previous work shows that neural machine …
QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation
An important challenge in machine translation (MT) is to generate high-quality and diverse
translations. Prior work has shown that the estimated likelihood from the MT model …
translations. Prior work has shown that the estimated likelihood from the MT model …
Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in text
Data quality is a problem that perpetually resurfaces throughout the field of NLP, regardless
of task, domain, or architecture, and remains especially severe for lower-resource …
of task, domain, or architecture, and remains especially severe for lower-resource …