An evaluation of score-based likelihood ratios for glass data

Abstract

<p>The likelihood ratio (LR) provides a numerical statement of the evidential strength in a forensic setting, but requires knowing a complex probability model, particularly for pattern and impression evidence. A proposed alternative relies on using similarity scores to develop score-based likelihood ratios (SLR). Different methods to compute evidential strength have been proposed and evaluated under different metrics. This project considers the model-based LR and SLR already present in the literature and focuses on a less-discussed aspect, dependence on the data used to estimate LRs and SLRs. We discovered that there is no clear winner that outperforms all other methods through our simulation studies. On average, distance-based methods of computing scores resulted in less discriminating power and a higher rate of misleading evidence for KNM. Machine learning methods of computing scores produce highly discriminating evidential values, but requires an additional sample to train. Our results also show that non-parametric estimation of score distributions can lead to non-monotonic behavior of the SLR and even counter-intuitive results. We also present evidence that the methods we studied are susceptible to performance issues when the split into training, testing and validation is modified, and resulting SLRs could even lead examiners to make different conclusions.</p>