Examiner variability in pattern evidence: proficiency, inconclusive tendency, and reporting styles

Thumbnail Image
Luby, Amanda
Major Professor
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Organizational Unit
Center for Statistics and Applications in Forensic Evidence
The Center for Statistics and Applications in Forensic Evidence (CSAFE) carries out research on the scientific foundations of forensic methods, develops novel statistical methods and transfers knowledge and technological innovations to the forensic science community. We collaborate with more than 80 researchers and across six universities to drive solutions to support our forensic community partners with accessible tools, open-source databases and educational opportunities.
Journal Issue
Is Version Of
The current approach to characterizing uncertainty in pattern evidence disciplines has focused on error rate studies, which provide aggregated error rates over many examiners and pieces of evidence. However, decisions are often not unanimous and error frequency is likely to vary depending on the quality of the evidence and proficiency of the examiner. Item Response Theory, a class of statistical methods used prominently in educational testing, is one promising approach that allows for the estimation of both examiner proficiency and item difficulty, and can also be extended to account for differing response styles. We find that even when examiners largely agree on a final source decision, there is considerable variability in their tendency to make inconclusive decisions and how they report the strength of evidence. We also find similar differences across items, and these differences could have downstream impact on the final assessment of guilt by the trier of fact. Using error rate studies in fingerprint and firearm comparisons, I will review some recent advances, outline challenges in applying IRT in practice, and discuss the implications of these findings.
This presentation is from the 2023 Joint Statistical Meetings (JSM), Toronto, Ontario, Canada, August 5-10, 2023. Posted with permission of CSAFE.

Copyright 2023, The Author