In 1984, the body of 63-year-old Ione Cychosz was found in a vacant lot near her home in Milwaukee, Wisconsin. She had been raped, beaten, stabbed, and bitten. That last fact contributed what became the most important piece of evidence in the case. Lowell Thomas Johnson, a dentist and forensic odontologist brought in by the police as a bite mark expert, examined eight marks found on Cychosz’s body and concluded that the killer was missing an upper front tooth.
This detail was pivotal in the arrest of 21-year-old Robert Lee Stinson, who was brought in for questioning by police because his backyard connected to the lot where Cychosz’s body was found. No physical evidence connected Stinson to the crime, but he was missing an upper front tooth, which led to him being convicted of first-degree murder and sentenced to life in prison. At his trial, Johnson and another forensic odontologist recruited by the prosecution testified that Stinson’s teeth were unequivocally identical to the bite mark pattern. Nobody, from the judge to the defense attorney, scrutinized their techniques.
Of course, they were wrong.
In 2009, a panel of experts commissioned by the Wisconsin Innocence Project determined that Stinson’s teeth didn’t match the bite marks. DNA testing also connected blood and saliva found on the victim’s sweater to another man.
Stinson was released in 2009 after 23 years in prison.
Any person with relevant “knowledge, skill, experience, training, or education” can testify as an expert witness in court, according to the federal rules of evidence. Attorneys can challenge an expert’s credentials, but it’s the judge who decides if a science, method, or expert is acceptable.
It took decades to reach even that loose standard. In 1923, James Alphonso Frye was tried for murder. His defense tried to use the results of a blood pressure-based lie detection test as evidence of his innocence, but the court decided that because the test didn’t have established scientific support, it was inadmissible. The trial resulted in an opinion known as the Frye standard, which states that methods used by scientific experts “must be sufficiently established to have gained general acceptance in the particular field in which it belongs.”
This was the accepted guideline to determining expert witness admissibility until 1993 and Daubert v. Merrell Dow Pharmaceuticals. Jason Daubert, Eric Schuller, and their parents sued a subsidiary of Dow Chemical Company, claiming that one of their drugs caused birth defects. Schuller was born without a hand, Daubert without three fingers and an arm bone.
No published studies had found the drug to cause birth defects, but the plaintiffs’ attorneys presented research from their own expert whose methods weren’t published or peer-reviewed. They were disallowed by the California district court judge who found them to be outside what scientists considered reliable, a decision that led to a shift in how courts handled expert testimony.
“For the court to take up an evidence case and say they’re going to set a new standard for the admission of scientific testimony in court, that was a big deal,” Villanova University law professor David Caudill told The Outline. Prior to Daubert, determination of scientific reliability was largely deferred to the experts themselves and indirectly to the attorneys who selected them. “That shifted to no, we need to be very careful about science. We need to really scrutinize it and we need the expert to explain to the judge what they’re doing,” says Caudill, “In other words, not defer to the expert. Make the expert convince the judge.”
The appeal ruling suggested criteria for judges to consider when assessing experts’ methods including whether the methods have appeared in peer-reviewed journals and been analyzed to determine how often they produce false results, as well as whether they are generally accepted within the scientific community. The goal was to keep out junk science and make sure good science wasn’t turned away. “Daubert brings in this new era, this new system [in which] the judge will be the arbiter, the gatekeeper of good and bad science,” said Caudill.
Initially, these standards were only meant for scientific testimony, but were later expanded to include other types of expertise. Kumho Tire Co. v. Carmichael, initially heard in an Alabama district court in 1998, claimed a tire manufactured by the company caused an accident that resulted in one death and a number of severe injuries.
In one study, 60 percent of overturned convictions featured forensic analysts who either misstated data or drew unsupported conclusions
The plaintiff’s lawyers presented a tire failure expert who stated that the tire failed because of a defect and not over-inflation, as Kumho Tire Co. was claiming. The defense attorneys moved to have the testimony kept out of court based on untested methodology. The judge agreed that the methods weren’t acceptable and used Daubert to bar them from the trial. On appeal, the Eleventh Circuit court reversed the ruling, saying that Daubert was meant for scientific evidence only, not technical evidence like tire failure. Later, the Supreme Court upheld the initial ruling, saying that drawing a line between scientific and technical evidence was both difficult and unnecessary, and Daubert should apply to both. From there on, it would fall on judges to determine which expert testimonies would be allowed.
There’s one obvious problem with this approach. Judges are presented with evidence ranging from forensic accounting to psychology to burn patterns from fires; it’s impossible to have in-depth understandings of them all. As a result, judges must often make an assessment of an expert’s reliability without a preexisting understanding of the underlying discipline. “No judge is familiar with all areas,” said federal district judge Jed Rakoff. “I think what judges have in common is they’re quick studies. The judge may not be an expert in science, but they are an expert in reasoning and logic.”
Rakoff says he asks both sides to tell him what they agree to be the foundational documents in that area, giving him a jumping off point for researching the field and an initial understanding of its basics. He then gets written briefs from each side and holds an evidentiary hearing where the experts testify. At that point, Rakoff poses questions to each expert before determining what and who are admissible during the trial.
This process is better than it was before Daubert, when judges simply assumed experts knew what they were talking about. But bad science can still make its way into court. The advent of DNA forensic evidence in 1987 has not only overturned convictions, it has also highlighted flawed forensic science that leads to wrongful verdicts. Even the most recognizable courtroom sciences — fingerprint analyses, burn patterns in arson cases, microscopic hair comparisons, and bite mark comparisons — have come under scrutiny. Most of these techniques require visual analysis, which opens them up to significant human error, and the development of new, less subjective methods has highlighted past weaknesses. “Other than DNA and toxicology, not much is surviving,” Caudill told The Outline.
Daubert only applies at the federal level. Most states have adopted the standard, but the rules for evaluating experts may be even looser in the nine states that have not. Just this year, the Florida Supreme Court overturned the governor’s 2013 approval of the Daubert adoption.
Another wrinkle is that while prosecutors can typically get an expert to testify, defense attorneys aren’t guaranteed one. “Most criminal defendants are not O.J. and they only get an expert if the court is willing to pay for it,” said Brandon Garrett, a law professor at the University of Virginia School of Law. (This alludes to another issue with courtroom experts: They are typically paid for their work, which gives them an incentive to agree with the side that hired them.)
If the court declines to pay, defendants can’t always afford to pay for one themselves. “That makes it pretty difficult for the defense to even understand the evidence,” said Garrett, “You may need an expert, even if you’re not going to call them to the stand, to explain it to you.” In some states, crime labs represent themselves as independent facilities and are willing to explain evidence to both prosecutors and defense attorneys, but that’s not the case in most places.
The effect of this is evident in a 2009 Virginia Law Review study. Garrett and his coauthors showed that in cases with convictions later overturned by DNA evidence, the shoddy expert testimony often wasn’t effectively cross-examined by defense attorneys who rarely obtained their own experts.
This was the failure in the Stinson case: His lawyer had actually retained an expert but didn’t call them to the stand, saying that the prosecutor’s expert had tainted other potential experts by presenting his analysis at an odontological conference before the trial. But the defense attorney also didn’t make an effort to challenge the experts’ methods during cross examination. According to Rakoff, it has only been in the last few years that defense lawyers have begun submitting meaningful Daubert challenges to expert testimony.
Finally, forensic testimony has a messy history in the courtroom. Garrett’s study found that out of 137 cases where convictions were later overturned based on DNA evidence, 60 percent featured forensic analysts called by the prosecution who either misstated data or drew unsupported conclusions. A 2013 Department of Justice report compared cases with wrongful convictions and cases that were dismissed or acquitted based on clear evidence that the defendant was not guilty — such as DNA evidence or changes in a victim’s statement — to see what factors contribute to one outcome over the other. Misstatements on forensic analysis were a major factor, leading to a 14 percent increase in the likelihood of a wrongful conviction.
Some extreme examples of this are the cases of Annie Dookhan and Joyce Gilchrist, crime lab forensic scientists who falsified evidence. Dookhan would claim she chemically tested samples that she had only visually examined, and in some cases, she deliberately tainted samples with drugs like cocaine. Gilchrist was dismissed from her position as an Oklahoma City forensic chemist after an FBI investigation found a number of her conclusions to be either wrong or overstated. Her department had to review hundreds of her cases. On a larger scale, last year, the Justice Department initiated an expanded review of FBI expert testimony after finding that nearly every FBI expert had overstated hair match evidence.
In recent years, the government began paying attention to where bad science has taken a toll and making efforts to correct it.
In 2013, the Department of Justice and the National Institute of Standards and Technology established the National Commission on Forensic Science with the aim of improving the reliability of forensic science and solidifying the standards by which it’s judged.
The commission has provided a number of recommendations to the Department of Justice about the handling of forensic science, one of which resulted in a review of the methods used to differentiate fingerprints, bullet patterns, and bite marks. Attorney General Jeff Sessions’ predecessor Loretta Lynch enacted accreditation procedures and ethical codes for forensic labs based on its suggestions.
A lot of the progress made in recent years is now hanging in limbo.
However, in April, Sessions said he would not renew the commission and put the expanded FBI review on hold.
Sessions made clear his position on forensic science reform after a 2009 report published by the National Academies of Science highlighted major issues surrounding forensic science. In a Senate Judiciary Committee hearing on the topic, Sessions stated, “I don't accept the idea that they seem to suggest that fingerprints is not a proven technology … I don't think we should suggest that those proven scientific principles that we've been using for decades are somehow uncertain.”
The reforms made so far would likely have kept Robert Lee Stinson out of jail. But Stinson’s prosecutor and bite mark expert both stood by their analyses even after they were shown to be wrong. Faced with the multiple analyses contradicting his findings, Johnson said, “I would have to say that I respectfully disagree with them.” Unfortunately, this attitude, along with Sessions’ unshakable faith in forensic science, shows just how difficult it will be to reform expert testimony.
