Minimization and estimation of the variance of prediction errors for cross-validation designs

M Fuchs, N Krautenbacher - Journal of Statistical Theory and Practice, 2016 - Springer
We consider the mean prediction error of a classification or regression procedure as well as
its cross-validation estimates, and investigate the variance of this estimate as a function of …

Oracle inequalities for multi-fold cross validation

AW Vaart, S Dudoit, MJ Laan - Statistics & Decisions, 2006 - degruyter.com
We consider choosing an estimator or model from a given class by cross validation
consisting of holding a nonneglible fraction of the observations out as a test set. We derive …

Cross-validation: what does it estimate and how well does it do it?

S Bates, T Hastie, R Tibshirani - Journal of the American Statistical …, 2024 - Taylor & Francis
Cross-validation is a widely used technique to estimate prediction error, but its behavior is
complex and not fully understood. Ideally, one would like to think that cross-validation …

The leave-worst-k-out criterion for cross validation

L Wang - Optimization Letters, 2023 - Springer
Cross validation is widely used to assess the performance of prediction models for unseen
data. Leave-k-out and m-fold are among the most popular cross validation criteria, which …

Asymptotics of cross-validation

M Austern, W Zhou - arXiv preprint arXiv:2001.11111, 2020 - arxiv.org
Cross validation is a central tool in evaluating the performance of machine learning and
statistical models. However, despite its ubiquitous role, its theoretical properties are still not …

Optimality of training/test size and resampling effectiveness in cross-validation

G Afendras, M Markatou - Journal of Statistical Planning and Inference, 2019 - Elsevier
An important question in cross-validation (CV) is whether rules can be established to allow
optimal sample size selection of the training/test set, for fixed values of the total sample size …

Optimality of training/test size and resampling effectiveness of cross-validation estimators of the generalization error

G Afendras, M Markatou - arXiv preprint arXiv:1511.02980, 2015 - arxiv.org
An important question in constructing Cross Validation (CV) estimators of the generalization
error is whether rules can be established that allow" optimal" selection of the size of the …

Consistency of cross validation for comparing regression procedures

Y Yang - 2007 - projecteuclid.org
Theoretical developments on cross validation (CV) have mainly focused on selecting one
among a list of finite-dimensional models (eg, subset or order selection in linear regression) …

Cross-validation

S Arlot - arXiv preprint arXiv:1703.03167, 2017 - arxiv.org
This text is a survey on cross-validation. We define all classical cross-validation procedures,
and we study their properties for two different goals: estimating the risk of a given estimator …

[PDF][PDF] Hypothesis testing for cross-validation

Y Grandvalet, Y Bengio - Montreal Universite de Montreal …, 2006 - iro.umontreal.ca
K-fold cross-validation produces variable estimates, whose variance cannot be estimated
unbiasedly. However, in practice, one would like to provide a figure related to the variability …