Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners

Thumbnail Image
Monson, Keith L.
Smith, Erich D.
Bajic, Stanley J.
Major Professor
Committee Member
Journal Title
Journal ISSN
Volume Title
Iowa State University Digital Repository, Ames IA (United States)
Research Projects
Organizational Units
Organizational Unit
Ames National Laboratory

Ames National Laboratory is a government-owned, contractor-operated national laboratory of the U.S. Department of Energy (DOE), operated by and located on the campus of Iowa State University in Ames, Iowa.

For more than 70 years, the Ames National Laboratory has successfully partnered with Iowa State University, and is unique among the 17 DOE laboratories in that it is physically located on the campus of a major research university. Many of the scientists and administrators at the Laboratory also hold faculty positions at the University and the Laboratory has access to both undergraduate and graduate student talent.

Journal Issue
Is Version Of
This paper describes design and logistical aspects of a decision analysis study to assess the performance of qualified firearms examiners working in accredited laboratories in the United States in terms of accuracy (error rate), repeatability, and reproducibility of decisions involving comparisons of fired bullets and cartridge cases. The purpose of the study was to validate current practice of the forensic discipline of firearms/toolmarks (F/T) examination. It elicited error rate data by counting the number of false positive and false negative conclusions. Preceded by the experimental design, decisions, and logistics described herein, testing was ultimately administered 173 qualified, practicing F/T examiners in public and private crime laboratories. The first round of testing evaluated accuracy, while two subsequent rounds evaluated repeatability and reproducibility of examiner conclusions. This project expands on previous studies by involving many F/T examiners in challenging comparisons and by executing the study in the recommended double-blind format.
This article is published as Monson, Keith L., Erich D. Smith, and Stanley J. Bajic. "Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners." Forensic Science International: Synergy 4 (2022): 100221. DOI: 10.1016/j.fsisyn.2022.100221. Copyright 2022 The Authors. Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0). Posted with permission. DOE Contract Number(s): AC02-07CH11358.