Validation metrics are key for tracking scientific progress and bridging the current chasm
between artificial intelligence research and its translation into practice. However, increasing …

Neural networks have achieved remarkable performance across various problem domains,
but their widespread applicability is hindered by inherent limitations such as overconfidence …

The use of deep neural networks in real-world applications require well-calibrated networks
with confidence scores that accurately reflect the actual probability. However, it has been …

Recent work on algorithmic fairness has largely focused on the fairness of discrete
decisions, or classifications. While such decisions are often based on risk score models, the …

A variety of different performance metrics are commonly used in the machine learning
literature for the evaluation of classification systems. Some of the most common ones for …

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an
underestimated global problem. In biomedical image analysis, chosen performance metrics …

For large language models (LLMs) to be trusted by humans they need to be well-calibrated
in the sense that they can accurately assess and communicate how likely it is that their …

Recalibrating probabilistic classifiers is vital for enhancing the reliability and accuracy of
predictive models. Despite the development of numerous recalibration algorithms, there is …

Fine-tuned Large Language Models (LLMs) often suffer from overconfidence and poor
calibration, particularly when fine-tuned on small datasets. To address these challenges, we …

Monitoring and maintaining machine learning models are among the most critical
challenges in translating recent advances in the field into real-world applications. However …