Introduction

            An article entitled “The Field of Firearms Forensics is Flawed” was published by David L. Faigman, Nicholas Scurich, and Thomas D. Albright. The authors start their article by referencing an article entitled “Forensic Science: Oxymoron,” by Donald Kennedy, which agues the point that Forensic Science is an Oxymoron. The authors of this article agree that the statement made in 2003 is still relevant today. They state, “Forensic experts continue to employ unproven techniques, and courts continue to accept their testimony largely unchecked.” They claim that the field of Firearm Examination is built on smoke and mirrors. I would like to provide a response to this article using the knowledge I have as a Firearm Examiner. 

Quantity of Studies

            Their first argument was that there existed only a few studies for the validation of the field, and the ones that did exist, indicated that examiners cannot reliably determine whether bullets or cartridges were fired by a particular gun. This statement is problematic in that they offer no reference to what article(s) they are referring to that shows that an examiner cannot reliably determine the origin of an expended component. During my training as an examiner, I have read hundreds of articles supporting the field, which all produced low error rates. For example, the Hamby and Brundage study examined bullets from ten consecutively manufactured Ruger pistol barrels, the Fadul study examined 10 consecutively manufactured Ruger slides, and the Cazes Study examined 10 consecutively manufactured Hi-Point slides. The Hamby and Brundage test had a 0% error rate that incorporated 502 examiners. The Fadul study established an error rate of 0.000636% and and a durability error rate of 0.0017699%, and both error rates were determined to not be significantly higher than zero. The durability of the Fadul study consisted of giving the participants casings that were fired in a later sequence from the casings they originally received.

The durability portion of the Fadul study was created to see if an examiner’s conclusions would change based on the ware of the markings on the breech face due to the previous test fires. Studies like these focus on consecutively manufactured parts, because it creates the hardest scenario for examiners, but also ensures the examiners are using individual markings rather than class characteristics for their conclusions.

These studies also focus on the manufacturing method rather than the make/model of the firearm used. This is because a manufacturer can use only a few manufacturing methods to produce a firearm. So if the overall method is proven to produce markings that are individual it can be applied to all firearms that are produced with that same method. There are many other foundational studies and their summaries which can be found on the AFTE SWGGUN ARK.

Anti-Experts Experts??

            The authors of the articles suggest the need to create anti-expert experts to combat experts in court. These experts would consist of research scientists, which would not make sense because the people who are researching in the field are publishing in the Journal of Forensic Science and the AFTE Journal that the authors just argued against. These journals are peer-reviewed and are published for anyone to view and allow anyone to retest the conclusions made. Since these articles are peer-reviewed and are in the scientific community I am not sure who would be the research scientist that would become anti-expert experts. These experts would just be the people in the field publishing the work.  

Inconclusive Results

            As with many critiques of the science, they argue that inconclusive results should not be used in research studies, citing them as an “I don’t know” answer. As explained in a previous post the inconclusive conclusion is used to speak for the evidence rather than to get the examiner out of making a conclusion. Depending on the condition of the evidence and the quality of the toolmarks, the examiner may only have the option to report an inconclusive conclusion. The markings present can be enough to prevent the examiner from eliminating the expended evidence, but the poor marking quality and quantity may also prevent the examiner from identifying the expended evidence. If the examiner was forced to conclude an identification or an elimination in this scenario, their basis for the conclusion would be weak because of the quality and quantity of those markings. So the examiners would have to use inconclusive as their conclusion to be able to properly speak for that particular evidence.

Subjective vs Objective

            The authors explain how the examiner’s subjective experience should not be a reliable source, and that a quantitative standard needs to be established. The authors fail to explain the vast amount of articles and scientific background that supports the validity of the field. Some of the studies were discussed above but also the science of toolmarks are heavily documented that tools leave unique markings on surfaces as they perform work. This may be due to the crystalline structure of the material and other factors. These factors can be seen on the molecular level and can be observed on a microscopic scale that shows the observer the chip formation and its effects on toolmarks. Backed by this foundational knowledge the examiner is able to make their conclusion.

An analogy can be used to further push the point that subjectivity does not automatically discount the validity of the science. For example, the house you live in is unique either by the way it was built, the area around it, or the personal touches you have added to the house. Based on these factors you would be able to walk up to the house that belonged to you, because of the features of the house. This selection would be subjective but is supported by the many factors discussed above. A picture of another house of the same design can be shown to the homeowner along with a picture of their house, and by using the previously described factors, they would still be able subjectively select their house from the pictures. 

AMES Study

            The authors also reference the AMES Part II study. In this study participants from the first part of the study were given the same evidence without their knowledge and were told to reach a conclusion. The authors claim that the Part II of this study showed that the same examiners looking at the same bullet reached the same conclusion one-third of the time, and different examiners looking at the same bullets reached the same conclusion less than one-third of the time. This is all the information the author provides with no references. So I tried looking up the article and could not find it published anywhere and later discovered that the FBI removed the article from distribution. I contacted the laboratory that originally produced the article and they stated that they had error rates of only 1% and are frustrated that the FBI took down part II of the study. I am currently in the process of getting access to the second part of the study.

The reason I would like to review the second study before submitting this portion of my response is that the author’s use of the data can be misleading. Their statement could mean that an examiner originally concluded inconclusive and then in part II of the study the examiner could have concluded the actual ground truth. Alternatively, the examiner could have originally reported the ground truth and changed the answer to an inconclusive in part II of the study. Both scenarios would not be an error that would destroy someone’s life as the authors suggest. These changes can be due to many factors. These changes can be based on the quality and quantity of the markings found on the expended evidence as explained above. Originally the examiner that reported inconclusive could now have found more markings due to lighting angles or seeing a small spot on the evidence that was not seen before that provides enough information to meet their threshold for an identification/elimination conclusion. Alternatively, the examiner that originally reported an identification/elimination may report inconclusive now because they can not find the small spots they originally found that supported their conclusion or they can not achieve the angle of light they originally used to properly illuminate the markings.

Conclusion

            Overall, the authors’ view on the science is lacking support and provides little to no references for their claims. I have provided sources and explanations that combat their claims and show the foundation that the Firearm Examiners use for their conclusions. If the authors were to have provided sources for their argument, I would be able to understand their position better and be able to dissect those sources, and provide additional sources if needed. For example, their use of the AMES study lacks reference and explanation of the data, especially with the lack of the source to be viewed and analyzed by the reader. They would also need to better elaborate on who would be an anti-expert expert so that the reader can better understand where these experts would be getting their information from and why they would make a difference. Additionally, the authors need to have a better understanding of the inconclusive conclusion and its use in the field of firearm examination before offering to omit it from research studies.