A survey of language model confidence estimation and calibration

J Geng, F Cai, Y Wang, H Koeppl, P Nakov… - arXiv preprint arXiv …, 2023 - arxiv.org
Language models (LMs) have demonstrated remarkable capabilities across a wide range of
tasks in various domains. Despite their impressive performance, the reliability of their output …

A Survey of Confidence Estimation and Calibration in Large Language Models

J Geng, F Cai, Y Wang, H Koeppl… - Proceedings of the …, 2024 - aclanthology.org
Large language models (LLMs) have demonstrated remarkable capabilities across a wide
range of tasks in various domains. Despite their impressive performance, they can be …

Uncertainty estimation of transformer predictions for misclassification detection

A Vazhentsev, G Kuzmin, A Shelmanov… - Proceedings of the …, 2022 - aclanthology.org
Uncertainty estimation (UE) of model predictions is a crucial step for a variety of tasks such
as active learning, misclassification detection, adversarial attack detection, out-of-distribution …

Hybrid uncertainty quantification for selective text classification in ambiguous tasks

A Vazhentsev, G Kuzmin, A Tsvigun… - Proceedings of the …, 2023 - aclanthology.org
Many text classification tasks are inherently ambiguous, which results in automatic systems
having a high risk of making mistakes, in spite of using advanced machine learning models …

Annotation error detection: Analyzing the past and present for a more coherent future

JC Klie, B Webber, I Gurevych - Computational Linguistics, 2023 - direct.mit.edu
Annotated data is an essential ingredient in natural language processing for training and
evaluating machine learning models. It is therefore very desirable for the annotations to be …

CBCT-guided adaptive radiotherapy using self-supervised sequential domain adaptation with uncertainty estimation

N Ebadi, R Li, A Das, A Roy, P Nikos, P Najafirad - Medical Image Analysis, 2023 - Elsevier
Adaptive radiotherapy (ART) is an advanced technology in modern cancer treatment that
incorporates progressive changes in patient anatomy into active plan/dose adaption during …

A structured narrative prompt for prompting narratives from large language models: Sentiment assessment of chatgpt-generated narratives and real tweets

CJ Lynch, EJ Jensen, V Zamponi, K O'Brien… - Future Internet, 2023 - mdpi.com
Large language models (LLMs) excel in providing natural language responses that sound
authoritative, reflect knowledge of the context area, and can present from a range of varied …

LM-polygraph: Uncertainty estimation for language models

E Fadeeva, R Vashurin, A Tsvigun… - arXiv preprint arXiv …, 2023 - arxiv.org
Recent advancements in the capabilities of large language models (LLMs) have paved the
way for a myriad of groundbreaking applications in various fields. However, a significant …

[HTML][HTML] Automatic uncertainty-based quality controlled T1 mapping and ECV analysis from native and post-contrast cardiac T1 mapping images using Bayesian vision …

TW Arega, S Bricq, F Legrand, A Jacquier… - Medical image …, 2023 - Elsevier
Deep learning-based methods for cardiac MR segmentation have achieved state-of-the-art
results. However, these methods can generate incorrect segmentation results which can …

Transformer uncertainty estimation with hierarchical stochastic attention

J Pei, C Wang, G Szarvas - Proceedings of the AAAI Conference on …, 2022 - ojs.aaai.org
Transformers are state-of-the-art in a wide range of NLP tasks and have also been applied
to many real-world products. Understanding the reliability and certainty of transformer …