Taxonomy of machine learning safety: A survey and primer
The open-world deployment of Machine Learning (ML) algorithms in safety-critical
applications such as autonomous vehicles needs to address a variety of ML vulnerabilities …
applications such as autonomous vehicles needs to address a variety of ML vulnerabilities …
Agreement-on-the-line: Predicting the performance of neural networks under distribution shift
Recently, Miller et al. showed that a model's in-distribution (ID) accuracy has a strong linear
correlation with its out-of-distribution (OOD) accuracy, on several OOD benchmarks, a …
correlation with its out-of-distribution (OOD) accuracy, on several OOD benchmarks, a …
Rankfeat: Rank-1 feature removal for out-of-distribution detection
The task of out-of-distribution (OOD) detection is crucial for deploying machine learning
models in real-world settings. In this paper, we observe that the singular value distributions …
models in real-world settings. In this paper, we observe that the singular value distributions …
Domain adaptation under open set label shift
S Garg, S Balakrishnan… - Advances in Neural …, 2022 - proceedings.neurips.cc
We introduce the problem of domain adaptation under Open Set Label Shift (OSLS), where
the label distribution can change arbitrarily and a new class may arrive during deployment …
the label distribution can change arbitrarily and a new class may arrive during deployment …
Probvlm: Probabilistic adapter for frozen vison-language models
Large-scale vision-language models (VLMs) like CLIP successfully find correspondences
between images and text. Through the standard deterministic mapping process, an image or …
between images and text. Through the standard deterministic mapping process, an image or …
Can you rely on your model evaluation? improving model evaluation with synthetic test data
Evaluating the performance of machine learning models on diverse and underrepresented
subgroups is essential for ensuring fairness and reliability in real-world applications …
subgroups is essential for ensuring fairness and reliability in real-world applications …
Complementary benefits of contrastive learning and self-training under distribution shift
Self-training and contrastive learning have emerged as leading techniques for incorporating
unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it …
unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it …
Gnnevaluator: Evaluating gnn performance on unseen graphs without labels
Evaluating the performance of graph neural networks (GNNs) is an essential task for
practical GNN model deployment and serving, as deployed GNNs face significant …
practical GNN model deployment and serving, as deployed GNNs face significant …
A survey on evaluation of out-of-distribution generalization
Machine learning models, while progressively advanced, rely heavily on the IID assumption,
which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …
which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …
Rlsbench: Domain adaptation under relaxed label shift
Despite the emergence of principled methods for domain adaptation under label shift, their
sensitivity to shifts in class conditional distributions is precariously under explored …
sensitivity to shifts in class conditional distributions is precariously under explored …