BI-RADS Data Should Not Be Used to Estimate ROC Curves

BI-RADS Data Should Not Be Used to Estimate ROC Curves
Yulei Jiang and Charles E. Metz
Radiology 2010;256 29-31

Link to Journal

After applauding the recent trend of employing receiver operating characteristic (ROC) analysis to measure diagnostic performance in large clinical studies, we discuss why Breast Imaging Reporting and Data System data should not be used to estimate ROC curves in screening mammography

However, to truly realize the benefit of this assessment method, one must estimate ROC curves appropriately, addressing additional issues that arise as ROC analysis migrates from the laboratory to the clinic. Hypothetical perturbation of results from a recent study indicates that BI-RADS data do not provide a reliable basis for estimating ROC curves in screening mammography. Although we appreciate the potential benefit of—and therefore, the motivation for—estimating ROC curves from existing clinical cases in which BI-RADS assessments are recorded, we must recommend caution until an approach is found that overcomes the problems we have identified here. Any future methodologic innovations in prospective clinical study design that will allow diagnostic confidence to be reported on a true ordinal scale without introducing additional bias are welcome and should be exploited to help address these problems