The Matthews correlation coefficient (MCC) is more informative than Cohen's Kappa and Brier score in binary classification assessment

D Chicco, MJ Warrens, G Jurman - Ieee Access, 2021 - ieeexplore.ieee.org
Even if measuring the outcome of binary classifications is a pivotal task in machine learning
and statistics, no consensus has been reached yet about which statistical rate to employ to …

[HTML][HTML] A comparison of reliability coefficients for ordinal rating scales

A de Raadt, MJ Warrens, RJ Bosker, HAL Kiers - Journal of Classification, 2021 - Springer
Kappa coefficients are commonly used for quantifying reliability on a categorical scale,
whereas correlation coefficients are commonly applied to assess reliability on an interval …

Definition and Classification of Intraoperative Complications (CLASSIC): Delphi Study and Pilot Evaluation

R Rosenthal, H Hoffmann, PA Clavien, HC Bucher… - World journal of …, 2015 - Springer
Background Standardized reporting of intraoperative adverse events is important to
enhance transparency. To the best of our knowledge, there is no validated definition and …

Bayesian belief network models to analyse and predict ecological water quality in rivers

MAE Forio, D Landuyt, E Bennetsen, K Lock… - Ecological …, 2015 - Elsevier
Economic growth is often based on the intensification of crop production, energy
consumption and urbanization. In many cases, this leads to the degradation of aquatic …

Cohen's kappa is a weighted average

MJ Warrens - Statistical Methodology, 2011 - Elsevier
The κ coefficient is a popular descriptive statistic for summarizing an agreement table. It is
sometimes desirable to combine some of the categories, for example, when categories are …

Conditional inequalities between Cohen's kappa and weighted kappas

MJ Warrens - Statistical Methodology, 2013 - Elsevier
Cohen's kappa and weighted kappa are two standard tools for describing the degree of
agreement between two observers on a categorical scale. For agreement tables with three …

Experienced versus Inexperienced Interexaminer Reliability on Location and Classification of Myofascial Trigger Point Palpation to Diagnose Lateral Epicondylalgia …

R Mora-Relucio, S Núñez-Nagy… - Evidence‐Based …, 2016 - Wiley Online Library
The purpose was to evaluate the interexaminer reliability of experienced and inexperienced
examiners on location and classification of myofascial trigger points (MTrPs) in two …

Some paradoxical results for the quadratically weighted kappa

MJ Warrens - Psychometrika, 2012 - Springer
The quadratically weighted kappa is the most commonly used weighted kappa statistic for
summarizing interrater agreement on an ordinal scale. The paper presents several …

Evaluating quadratic weighted kappa as the standard performance metric for automated essay scoring

A Doewes, N Kurdhi, A Saxena - 16th International Conference on …, 2023 - research.tue.nl
Abstract Automated Essay Scoring (AES) tools aim to improve the efficiency and consistency
of essay scoring by using machine learning algorithms. In the existing research work on this …

Cohen's linearly weighted kappa is a weighted average

MJ Warrens - Advances in Data Analysis and Classification, 2012 - Springer
An n× n agreement table F= f ij with n≥ 3 ordered categories can for fixed m (2≤ m≤ n− 1)
be collapsed into n-1 m-1 distinct m× m tables by combining adjacent categories. It is shown …