Nonparametric bootstrap methods for interval estimation of the area under the ROC curve for correlated diagnostic test data

Abstract

An essential part of contemporary diagnostic research is the development of new diagnostic assays and their performance evaluation. The receiver operating characteristic curve (ROC) and the area under the ROC curve (AUC) are frequently used in evaluating the performance of diagnostic assays. The variation in AUC estimation can be quantified non-parametrically using resampling methods, and then used to construct interval estimation for the AUC. For example, the pROC package is a widely used open-source tool that can calculate interval estimates of the AUC by performing nonparametric bootstrapping. However, when multiple observations are observed from the same subject, which is very common in diagnostic tests evaluation experiments, traditional bootstrap-based methods, such as the one in the pROC package, may fail to provide valid interval estimation of AUC. In particular, the traditional method does not account for the correlation among data observations and would result in interval estimation that does not cover the true AUC at the desired level of confidence. In this paper, we propose two novel methods to calculate the confidence interval of the AUC for correlated diagnostic test data, based on cluster bootstrapping and hierarchical bootstrapping, respectively. Our simulation studies showed that both proposed methods have higher coverage probabilities compared with the existing method when there are intra-subject correlations. We also discuss an application of the proposed methods to the evaluation of a novel whole-virus ELISA diagnostic essay in detection of porcine parainfluenza virus type-1 antibody in swine serum.