A finely tuned deep transfer learning algorithm to compare outsole images
Date
2023-12
Authors
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Wiley
Abstract
In forensic practice, evaluating shoeprint evidence is challenging because the differences between images of two different outsoles can be subtle. In this paper, we propose a deep transfer learning-based matching algorithm called the Shoe-MS algorithm that quantifies the similarity between two outsole images. The Shoe-MS algorithm consists of a Siamese neural network for two input images followed by a transfer learning component to extract features from outsole impression images. The added layers are finely tuned using images of shoe soles. To test the performance of the method we propose, we use a study dataset that is both realistic and challenging. The pairs of images for which we know ground truth include (1) close non-matches and (2) mock-crime scene pairs. The Shoe-MS algorithm performedwell in terms of prediction accuracy andwas able to determine the source of pairs of outsole images, even when comparisonswere challenging. When using a score-based likelihood ratio, the algorithm made the correct decision with high probability in a test of the hypothesis that images had a common source. An important advantage of the proposed approach is that pairs of images can be compared without alignment. In initial tests, Shoe-MS exhibited better-discriminating power than existing methods.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
Article
Comments
This is a manuscript of an article published as Jang, Moonsoo, Soyoung Park, and Alicia Carriquiry. "A finely tuned deep transfer learning algorithm to compare outsole images." Statistical Analysis and Data Mining: The ASA Data Science Journal 16, no. 6 (2023): 511-527. doi:10.1002/sam.11636. © 2023 Wiley Periodicals LLC. Posted with permission of CSAFE.