Literature Review

Response: The Field of Firearms Forensics is Flawed

Introduction

            An article entitled “The Field of Firearms Forensics is Flawed” was published by David L. Faigman, Nicholas Scurich, and Thomas D. Albright. The authors start their article by referencing an article entitled “Forensic Science: Oxymoron,” by Donald Kennedy, which agues the point that Forensic Science is an Oxymoron. The authors of this article agree that the statement made in 2003 is still relevant today. They state, “Forensic experts continue to employ unproven techniques, and courts continue to accept their testimony largely unchecked.” They claim that the field of Firearm Examination is built on smoke and mirrors. I would like to provide a response to this article using the knowledge I have as a Firearm Examiner. 

Quantity of Studies

            Their first argument was that there existed only a few studies for the validation of the field, and the ones that did exist, indicated that examiners cannot reliably determine whether bullets or cartridges were fired by a particular gun. This statement is problematic in that they offer no reference to what article(s) they are referring to that shows that an examiner cannot reliably determine the origin of an expended component. During my training as an examiner, I have read hundreds of articles supporting the field, which all produced low error rates. For example, the Hamby and Brundage study examined bullets from ten consecutively manufactured Ruger pistol barrels, the Fadul study examined 10 consecutively manufactured Ruger slides, and the Cazes Study examined 10 consecutively manufactured Hi-Point slides. The Hamby and Brundage test had a 0% error rate that incorporated 502 examiners. The Fadul study established an error rate of 0.000636% and and a durability error rate of 0.0017699%, and both error rates were determined to not be significantly higher than zero. The durability of the Fadul study consisted of giving the participants casings that were fired in a later sequence from the casings they originally received.

The durability portion of the Fadul study was created to see if an examiner’s conclusions would change based on the ware of the markings on the breech face due to the previous test fires. Studies like these focus on consecutively manufactured parts, because it creates the hardest scenario for examiners, but also ensures the examiners are using individual markings rather than class characteristics for their conclusions.

These studies also focus on the manufacturing method rather than the make/model of the firearm used. This is because a manufacturer can use only a few manufacturing methods to produce a firearm. So if the overall method is proven to produce markings that are individual it can be applied to all firearms that are produced with that same method. There are many other foundational studies and their summaries which can be found on the AFTE SWGGUN ARK.

Anti-Experts Experts??

            The authors of the articles suggest the need to create anti-expert experts to combat experts in court. These experts would consist of research scientists, which would not make sense because the people who are researching in the field are publishing in the Journal of Forensic Science and the AFTE Journal that the authors just argued against. These journals are peer-reviewed and are published for anyone to view and allow anyone to retest the conclusions made. Since these articles are peer-reviewed and are in the scientific community I am not sure who would be the research scientist that would become anti-expert experts. These experts would just be the people in the field publishing the work.  

Inconclusive Results

            As with many critiques of the science, they argue that inconclusive results should not be used in research studies, citing them as an “I don’t know” answer. As explained in a previous post the inconclusive conclusion is used to speak for the evidence rather than to get the examiner out of making a conclusion. Depending on the condition of the evidence and the quality of the toolmarks, the examiner may only have the option to report an inconclusive conclusion. The markings present can be enough to prevent the examiner from eliminating the expended evidence, but the poor marking quality and quantity may also prevent the examiner from identifying the expended evidence. If the examiner was forced to conclude an identification or an elimination in this scenario, their basis for the conclusion would be weak because of the quality and quantity of those markings. So the examiners would have to use inconclusive as their conclusion to be able to properly speak for that particular evidence.

Subjective vs Objective

            The authors explain how the examiner’s subjective experience should not be a reliable source, and that a quantitative standard needs to be established. The authors fail to explain the vast amount of articles and scientific background that supports the validity of the field. Some of the studies were discussed above but also the science of toolmarks are heavily documented that tools leave unique markings on surfaces as they perform work. This may be due to the crystalline structure of the material and other factors. These factors can be seen on the molecular level and can be observed on a microscopic scale that shows the observer the chip formation and its effects on toolmarks. Backed by this foundational knowledge the examiner is able to make their conclusion.

An analogy can be used to further push the point that subjectivity does not automatically discount the validity of the science. For example, the house you live in is unique either by the way it was built, the area around it, or the personal touches you have added to the house. Based on these factors you would be able to walk up to the house that belonged to you, because of the features of the house. This selection would be subjective but is supported by the many factors discussed above. A picture of another house of the same design can be shown to the homeowner along with a picture of their house, and by using the previously described factors, they would still be able subjectively select their house from the pictures. 

AMES Study

            The authors also reference the AMES Part II study. In this study participants from the first part of the study were given the same evidence without their knowledge and were told to reach a conclusion. The authors claim that the Part II of this study showed that the same examiners looking at the same bullet reached the same conclusion one-third of the time, and different examiners looking at the same bullets reached the same conclusion less than one-third of the time. This is all the information the author provides with no references. So I tried looking up the article and could not find it published anywhere and later discovered that the FBI removed the article from distribution. I contacted the laboratory that originally produced the article and they stated that they had error rates of only 1% and are frustrated that the FBI took down part II of the study. I am currently in the process of getting access to the second part of the study.

The reason I would like to review the second study before submitting this portion of my response is that the author’s use of the data can be misleading. Their statement could mean that an examiner originally concluded inconclusive and then in part II of the study the examiner could have concluded the actual ground truth. Alternatively, the examiner could have originally reported the ground truth and changed the answer to an inconclusive in part II of the study. Both scenarios would not be an error that would destroy someone’s life as the authors suggest. These changes can be due to many factors. These changes can be based on the quality and quantity of the markings found on the expended evidence as explained above. Originally the examiner that reported inconclusive could now have found more markings due to lighting angles or seeing a small spot on the evidence that was not seen before that provides enough information to meet their threshold for an identification/elimination conclusion. Alternatively, the examiner that originally reported an identification/elimination may report inconclusive now because they can not find the small spots they originally found that supported their conclusion or they can not achieve the angle of light they originally used to properly illuminate the markings.

Conclusion

            Overall, the authors’ view on the science is lacking support and provides little to no references for their claims. I have provided sources and explanations that combat their claims and show the foundation that the Firearm Examiners use for their conclusions. If the authors were to have provided sources for their argument, I would be able to understand their position better and be able to dissect those sources, and provide additional sources if needed. For example, their use of the AMES study lacks reference and explanation of the data, especially with the lack of the source to be viewed and analyzed by the reader. They would also need to better elaborate on who would be an anti-expert expert so that the reader can better understand where these experts would be getting their information from and why they would make a difference. Additionally, the authors need to have a better understanding of the inconclusive conclusion and its use in the field of firearm examination before offering to omit it from research studies. 

General Laboratory

ANAB Accreditation Explained

Important Terminology

ANAB is a name you will have to know when working for a New York State Crime Laboratory because of the accreditation requirement. This requirement is mandated by The Commission of Forensic Science which is part of the New York Division of Criminal Justice Services (NYDCJS). ANAB stands for The ANSI National Accreditation Board, and its goal is to accredit laboratories that follow the accreditation standards that the board sets.

The ANAB standards are based on the ISO/IEC 17025: International General Requirements for the Competence of Testing and Calibration Laboratories. The calibration portion of this document is not relevant for Forensic Laboratories, therefore it will be ignored for the rest of the article. ANAB takes the ISO standards and adds the AR3125 document, which is a supplement that adds more specific requirements for forensic service providers. Below we will dissect the ISO requirements, which are separated into different numbered sections. These sections are usually mimicked by the Quality Assurance’s (QA) Standard Operating Procedure (SOP) to ensure that the laboratory follows all the standards. The ISO document is generally vague when it comes to requirements, it’s up to the laboratory to get more specific in their QA SOP, and then ensure that all the requirements are followed at all times.

Beginning Sections

The sections that start the document are more introduction based than the actual standards that a laboratory should follow. Section 1 is the “Scope,” which explains the purpose of the document. Section 2 is “Normative References” and Section 3 is “Term and Definitions.” These sections are self-explanatory and they do not go into any specific requirements. 

Section 4 is the first section that creates requirements for the laboratory and it is appropriately titled, “General Requirements.” This section mainly focuses on impartiality and confidentiality. The next section, Section 5: “Structural Requirements,” discusses the management and personnel responsibilities, along with the activities and their requirements. Sections 4 and 5 are small when compared to the next two sections, which are the sections that make up the majority of a Laboratory’s QA manual.

Section 6

Section 6, “Resource Requirements” is one of the biggest sections and is one that the QA section will focus most of their manual on. The sub-sections of Section 6 are split up as follows: General (6.1), Personnel (6.2), Facilities and Environment Conditions (6.3), Equipment (6.4), Metrology Traceability (6.5), and Externally Provided Products and Services (6.6). Below I will explain some of the sub-section for better clarity. I will skip 6.1 because it does not contain anything of value for a general examination of the document.

(6.2): This section goes over the education, qualifications, training, and experience of the personnel. The agency needs to have the requirements for these categories and the records to support them. The personnel should also have competency in their laboratory activities.  

(6.3): Requirement to list and record all the environmental factors that can affect the validity of the results.

(6.4): To keep a record of the equipment that is used in testing and to ensure the equipment conforms to specified requirements: measurement accuracy and measurement uncertainty. There must also be recorded calibration and checks on the instruments.

(6.5): There has to be documentation for metrological traceability. These measurements must be traceable to SI units.

(6.6): Records of what will approve a product and what will make it conform.

Section 7

Next is Section 7 which is the review of requests, tenders, and contracts. This section is split up into 11 sub-sections. This section will affect more of the report writing and the way analyst must conduct their examinations. The sub-sections cover the following:

(7.1): Procedure for the review of requests, tenders, and contracts

(7.2): Selection, verification, and validation of methods that must be appropriate and correct. There must be a manual for the methods, standards if applicable, and a verification process.

(7.3): This section goes into depth about sampling. There must be a plan and must address any factors.

(7.4): This section goes into the handling of test or calibration items. Also requires a procedure for storage, transportation, receipt, etc. of these items.

(7.5): The testing records must be complete so they can be replicated. The original observations, data, and calculations at the time they were made must be recorded.

(7.6): Evaluation on Measurement Uncertainty. There will be another article posted on measurement uncertainty and its importance at a later date.

(7.7): Requiring the monitoring of results so that trends can be detectable. This can be done through functional checks, retesting of retained items, replicate testing, etc.

(7.8): This section dives deeply into the reporting of results. The results must be accurate clear and unambiguous, and objective. The section also provides guidance on what the report should include: title, the method used, data, etc. On top of report writing the section addresses calibration certificates and what they should include. To keep this section brief I would like to conclude that most of the guidance in the final product is found in this sub-section. This sub-section is very important to the QA section and other sections in the laboratory when crafting their own SOPs.

(7.9): This sub-section goes through the complaint process when a customer files a complaint.

(7.10): Anything dealing with non-conformance work is found in this sub-section

(7.11): Lastly this sub-section finishes off with control of data and information management.

Section 8

The last section of this document is the management system requirements, which is section 8. They split this section into an “A” option and a “B” option. The “A” option is usually taken by crime laboratories and the “B” option is taken by laboratories that use the ISO 9001. The “A” option goes over control of documentation of the laboratory, corrective actions, internal audits, identifying areas of improvement, and policies and objectives for the fulfillment of this document.

Conclusion

This article is just to teach you the basic idea and summary of the ISO/IEC 17025:2017 document. Overall, the document is just general requirements that any laboratory can take and mold into its own SOPs. The ANAB Accreditation Requirements (AR 3125) has all the same sections as the ISO 17025 document, but the information is geared more towards forensic laboratories. Although the AR3125 document is still pretty general in nature, it does have more forensic-specific requirements. Once these documents are incorporated into the QA manual and the other section’s SOP is when they become more specific to the specific laboratory they are incorporated into. At this point, the laboratory is responsible for following its own manuals in addition to the ISO and AR documents when an assessment is performed on the laboratory.

This information can be used by the scientist who is preparing to help their lab with accreditation. They can use also use this article as a quick reference or to better understand what the ISO standards are. This information can also prove to be very useful for the student who is looking for a job or internship. This knowledge is sought after when hiring because this is huge for the laboratory, but is usually eclipsed by casework.

General Laboratory

How to Secure Your Forensic Internship

Introduction

Below are some tips that I have learned that will help you to secure an internship in a Forensic Science Laboratory. The first step is scouting your area for laboratories that have the discipline you are looking to work in but be open to other disciplines. Working in another discipline will broaden your horizon and may help you more than what you are searching for.  

Research Project

As a student, you should be preparing for your internship by your second year as an undergraduate.  By this time, you have settled into college and should be aware that an internship requirement must be fulfilled by your senior year. At this time, you should be searching for a professor to work with to conduct your own research or to assist in an ongoing research project. The more time you put into this research before applying to internships the better. Presenting and/or being published with this research will help even more. The research you pick should correlate to the discipline of Forensics that you are most interested in.

Now that you are conducting research and are working in a research lab this can now be applied to your application. The cover letter for your application or the conversation you have with the laboratory should focus mainly on your research. This will show the laboratory that you have a good knowledge of the field and are dedicated to what you do. This will put you above the other students who are applying with no research experience.

Internship Research Project

Hopefully, you should be already conducting your own research you should plan another area of research that may only be achieved with an internship with a laboratory by using their knowledge and resources. Write a proposal for this research and have it on hand. Create a short description of this research and add it to the cover letter that you have the intention of researching this topic at the laboratory if you were selected to intern. This will show your drive and show that you will be utilizing the full potential of the internship. These laboratories want to select students who will want to make the most from it, not just to fulfill a requirement.

What to do after submitting your application

After submitting your application make sure you follow up with the laboratory. If you had an interview, let them know soon after that you are thankful for the opportunity for the interview and that you appreciated meeting the panel and seeing the laboratory. Also, state you look forward to the next steps in the process. This will show your drive and how much this means to you, which should place you higher on their list of who to accept.

If the process only required a submission of an application, try to call the laboratory a week or two after the submission to inquire what the next steps of the process will be, and what you should expect. This will show your drive for the internship and show how important the internship is to you.

My Experience Securing an Internship

In college, I conducted two types of research projects. One project dealt with Firearms and the other dealt with Physics. I used both of these research projects in my cover letter, which was the main focus. I also explained how I wanted to be able to intern at a laboratory that served my local community. I expressed my interest in firearms and talked a little about my research experience, but also stated that I will be willing to intern in any section to broaden my knowledge of the crime laboratory. I also stated that I would like to conduct research at the laboratory for a capstone project I had to complete for my Honors Course. At this point, I didn’t have a specific capstone idea in mind, but now I would recommend that anyone applying should have some idea of what they would like to do to make their cover letter more appealing.

Two weeks later I called the laboratory to ask about the future steps of the process and what I should expect. During this phone call, the person in charge of accepting interns retrieved my file and told me my application looked good, and that they would be selecting interns in the next couple of weeks. I felt like this moment caused my file to reach the top of the list and stand out from the rest. Because of my openness to joining any section, I was picked to intern at the laboratory under the Quality Assurance (QA) Section. I was a little disappointed that I wasn’t selected for Firearms but gladly accepted the offer. But, after speaking with the laboratory I found out that my internship was with the QA section, but the research I mentioned in my cover letter and phone call would be fulfilled. So, I split my time learning and helping the QA section while researching in the Firearm Section. Thankfully I got my internship in the QA section, because little did I know, that this section was really important to the operation of the laboratory. Having the knowledge of the QA section actually helped me land my first Forensic Science position and my second. Also, my research allowed me to continue my internship after the summer and all through my senior year, which allowed me to grow and get recognition at the laboratory, while the other interns left by August.

Concluding Note

Researching at your internship is so important. I cannot stress that enough. It will help you satisfy any college requirement for research and help with your job applications that you will submit during your Senior year. Laboratories that are hiring want to know your internship experience, and nothing sounds better than doing research as you learn the laboratory’s function. Do not do an internship just to satisfy your school’s requirements! Use it to push you forward in the job market. Every Forensic Scientist has taken an internship, but only a few have reaped the full benefits of that internship.

General Laboratory

Why is Forensic Science Important?

The Definition of Forensic Science

            The NIST (The National Institute of Standards and Technology) definition of Forensic Science is the use of the scientific methods or expertise to investigate crimes or examine evidence that might be presented in a court of law.

The Importance of Forensics

            I would like to take Forensic Scientist and look at it under the microscope. Forensic Science is so important because it is unbiased and strictly just follows the scientific method. Forensic Science is the sole voice of the evidence, it isn’t the voice of the defense or the prosecution. This is the most important part to remember.

Speaking for the Evidence

            Physical evidence cannot speak to the jury and some evidence requires expertise to extract the information from the evidence and convert it to layman’s terms. This may be a fingerprint examiner decoding a fingerprint and comparing it to other prints and giving their conclusion/opinion of the comparison based on their experience. This can also be a drug chemist who can take a white powder and determine the makeup and conclude if there is an absence or presence of a controlled substance.  Just remember when analyzing the evidence or testifying to a conclusion that you are the voice of the evidence and you do not care who “side” it benefits. The only benefit is conveying the correct information to the Jury.

Can be Used to Prove Innocence or Guilt

            A Forensic Scientist’s conclusion/opinion may lead the Jury to establish the guilt or innocence of a subject. Many people believe that the conclusion of a Forensic Scientist is mostly used to establish guilt, but just as frequently the conclusion also proves someone’s innocence. Proving innocence can give the subject their freedom back and can allow the investigation to move on to find the correct person if applicable. A note should be made that the establishment of guilt and innocence is up to the Jury to decide and for the lawyers to persuade. The Scientist’s goal is only to give the information that the evidence has provided, no more and no less.