Probability of detection (POD) is commonly used to measure a nondestructive evaluation (NDE) inspection procedure’s performance. Due to inherent variability in the inspection procedure caused by variability in factors such as crack morphology and operators, it is important, for some purposes, to model POD as a random function. Traditionally, inspection variabilities are pooled and an estimate of the mean POD (averaged over all sources of variability) is reported. In some applications it is important to know how poor typical inspections might be, and this question is answered by estimating a quantile of the POD distribution. This article shows how to fit and compare different models to repeated-measures hit--miss data with multiple inspections with different operators for each crack and shows how to estimate the mean POD as well as quantiles of the POD distribution for binary (hit--miss) NDE data. We also show how to compute credible intervals (quantifying uncertainty due to limited data) for these quantities using a Bayesian estimation approach. We use NDE for the detection of fatigue cracks as the motivating example, but the concepts apply more generally to other NDE applications areas.