A study of examiner accuracy in cartridge case comparisons. Part 2: Examiner use of the AFTE range of conclusions

This report describes an analysis of how examiners used the Association of Firearm and Toolmark Examiners (AFTE) Range of Conclusions in a controlled study undertaken to estimate examiner error rates in comparing cartridge cases. Results of the error rate analysis are reported in [1]; this paper focuses on a broader analysis of how the entire collection of classification categories, especially those in the Inconclusive range, were used by the participating examiners. Volunteer active examiners with AFTE membership or working in laboratories that participate in Association of Crime Laboratory Directors (ASCLD) were provided with 15 sets of three known and one questioned cartridge cases fired from a collection of 25 new Ruger SR9 handguns. Remington 9-mm Luger (manufacturer designation L9MM3) ammunition was used and comparison sets were made up of cartridge cases fired within 100 cartridges of each other for each gun. Examiners were provided with a background survey, an answer sheet allowing for the AFTE Range of Conclusions, and return shipping materials. The participating examiners were provided with both same-source and different-source comparison sets allowing the study to assess both error rates. Responses were received from 218 participating examiners. The overall rate of false-negatives was estimated as 0.367 % from comparisons known to be from the same firearm but reported as eliminations. The overall rate of false-positives was estimated as 1.01 % from comparisons known to be from different firearms but reported as identifications. In the case of true different-source examinations, it is clear that the three Inconclusive categories and the Elimination category are not used consistently by all examiners. We identify five different apparent patterns of use of the AFTE Range of Conclusions scale, and discuss possible reasons for and implications of these differences. •In the reported black-box study, far more different-source sets than same-source sets were graded as Inconclusive.•It is clear that the three Inconclusive categories and the Elimination category are not used consistently by all examiners.•We identify five patterns of use of the AFTE Range of Conclusions, and discuss possible reasons for these differences.