In 1960, Cohen introduced the kappa coefficient to measure chance‐corrected nominal
scale agreement between two raters. Since then, numerous extensions and generalizations …

The Cohen (1960) kappa interrater agreement coefficient has been criticized for penalizing
raters (eg, diagnosticians) for their a priori agreement about the base rates of categories (eg …

Pi (π) and kappa (κ) statistics are widely used in the areas of psychiatry and psychological
testing to compute the extent of agreement between raters on nominally scaled data. It is a …

The paper presents inequalities between four descriptive statistics that have been used to
measure the nominal agreement between two or more raters. Each of the four statistics is a …

This paper gives a method for determining a sample size that will achieve a prespecified
bound on confidence interval width for the interrater agreement measure, κ. The same …

Evaluating the extent of agreement between 2 or between several raters is common in
social, behavioral and medical sciences. The objective of this paper is to provide a detailed …

The statistical methods described in the preceding chapter for controlling for error are
applicable only when the rates of misclassification are known from external sources or are …

Interrater reliability studies are used in a diverse set of fields. Often, these investigations
involve three or more raters, and thus, require the use of indices such as Fleiss's kappa …

Previously proposed methods for calculating the kappa measure of nominal rating
agreement among multiple raters are not applicable in many situations. This paper presents …

The kappa statistic is commonly used for quantifying inter-rater agreement on a nominal
scale. In this review article we discuss five interpretations of this popular coefficient. Kappa is …