The aim of this study was to evaluate reader variability in screening mammograms according to the American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) assessment and breast density categories.

A stratified random sample of 100 mammograms was selected from a population-based breast cancer screening programme in Barcelona, Spain: 13 histopathologically confirmed breast cancers and 51 with true-negative and 36 with false-positive results. 21 expert radiologists from radiological units of breast cancer screening programmes in Catalonia, Spain, reviewed the mammography images twice within a 6-month interval. The readers described each mammography using BI-RADS assessment and breast density categories. Inter- and intraradiologist agreement was assessed using percentage of concordance and the kappa (κ) statistic.

Fair interobserver agreement was observed for the BI-RADS assessment [κ=0.37, 95% confidence interval (CI) 0.36–0.38]. When the categories were collapsed in terms of whether additional evaluation was required (Categories III, 0, IV, V) or not (I and II), moderate agreement was found (κ=0.53, 95% CI 0.52–0.54). Intra-observer agreement for BI-RADS assessment was moderate using all categories (κ=0.53, 95% CI 0.50–0.55) and substantial on recall (κ=0.66, 95% CI 0.63–0.70). Regarding breast density, inter- and intraradiologist agreement was substantial (κ=0.73, 95% CI 0.72–0.74 and κ=0.69, 95% CI 0.68–0.70, respectively).

We observed a substantial intra-observer agreement in the BI-RADS assessment but only moderate interobserver agreement. Both inter- and intra-observer agreement in mammographic interpretation of breast density was substantial.

Educational efforts should be made to decrease radiologists' variability in BI-RADS assessment interpretation in population-based breast screening programmes.