Robots that ask for help: Uncertainty alignment for large language model planners

AZ Ren, A Dixit, A Bodrova, S Singh, S Tu… - arXiv preprint arXiv …, 2023 - arxiv.org
Large language models (LLMs) exhibit a wide range of promising capabilities--from step-by-
step planning to commonsense reasoning--that may provide utility for robots, but remain …

Trustworthy LLMs: A survey and guideline for evaluating large language models' alignment

Y Liu, Y Yao, JF Ton, X Zhang, RGH Cheng… - arXiv preprint arXiv …, 2023 - arxiv.org
Ensuring alignment, which refers to making models behave in accordance with human
intentions [1, 2], has become a critical task before deploying large language models (LLMs) …

Uncertainty in natural language generation: From theory to applications

J Baan, N Daheim, E Ilia, D Ulmer, HS Li… - arXiv preprint arXiv …, 2023 - arxiv.org
Recent advances of powerful Language Models have allowed Natural Language
Generation (NLG) to emerge as an important technology that can not only perform traditional …

Conformal prediction for natural language processing: A survey

M Campos, A Farinhas, C Zerva… - Transactions of the …, 2024 - direct.mit.edu
The rapid proliferation of large language models and natural language processing (NLP)
applications creates a crucial need for uncertainty quantification to mitigate risks such as …

Benchmarking llms via uncertainty quantification

F Ye, M Yang, J Pang, L Wang, DF Wong… - arXiv preprint arXiv …, 2024 - arxiv.org
The proliferation of open-source Large Language Models (LLMs) from various institutions
has highlighted the urgent need for comprehensive evaluation methods. However, current …

Forking uncertainties: Reliable prediction and model predictive control with sequence models via conformal risk control

M Zecchin, S Park, O Simeone - IEEE Journal on Selected …, 2024 - ieeexplore.ieee.org
In many real-world problems, predictions are leveraged to monitor and control cyber-
physical systems, demanding guarantees on the satisfaction of reliability and safety …

Knowing when to stop: Delay-adaptive spiking neural network classifiers with reliability guarantees

J Chen, S Park, O Simeone - IEEE Journal of Selected Topics …, 2024 - ieeexplore.ieee.org
Spiking neural networks (SNNs) process time-series data via internal event-driven neural
dynamics. The energy consumption of an SNN depends on the number of spikes exchanged …

Conformal autoregressive generation: Beam search with coverage guarantees

N Deutschmann, M Alberts, MR Martínez - Proceedings of the AAAI …, 2024 - ojs.aaai.org
We introduce two new extensions to the beam search algorithm based on conformal
predictions (CP) to produce sets of sequences with theoretical coverage guarantees. The …

Prompt risk control: A rigorous framework for responsible deployment of large language models

TP Zollo, T Morrill, Z Deng, JC Snell, T Pitassi… - arXiv preprint arXiv …, 2023 - arxiv.org
The recent explosion in the capabilities of large language models has led to a wave of
interest in how best to prompt a model to perform a given task. While it may be tempting to …

C-rag: Certified generation risks for retrieval-augmented language models

M Kang, NM Gürel, N Yu, D Song, B Li - arXiv preprint arXiv:2402.03181, 2024 - arxiv.org
Despite the impressive capabilities of large language models (LLMs) across diverse
applications, they still suffer from trustworthiness issues, such as hallucinations and …