DNA testing not so infallible after all?

New Scientist has a very interesting article claiming that DNA testing isn't so clear-cut and definitive as it's made out to be. Here are a couple of excerpts.

... much of the DNA analysis now conducted in crime labs can suffer from worrying subjectivity and bias. We asked forensic analysts to interpret a sample of real DNA evidence and found that they reached opposing conclusions about whether the suspect matched it or not. Our subsequent survey of labs around the world also shows that there are significant inconsistencies in the guidelines on how to interpret a sample. The findings suggest that the difference between prison and freedom could often rest on the opinions of a single individual.

. . .

We took a mixed sample of DNA evidence from an actual crime scene- a gang rape committed in Georgia, US- which helped to convict a man called Kerry Robinson, who is currently in prison. We presented it, and Robinson's DNA profile, to 17 experienced analysts working in the same accredited government lab in the US, without any contextual information that might bias their judgement.

In the original case, two analysts from the Georgia Bureau of Investigation concluded that Robinson "could not be excluded" from the crime scene sample, based on his DNA profile. (A second man convicted of the same crime also testified that Robinson was an assailant, in return for a lesser jail term.) Each of our 17 analysts independently examined the profiles from the DNA mixture, the victim's profile and those of two other suspects and was asked to judge whether the suspects' profiles could be "excluded", "cannot be excluded" or whether the results were "inconclusive".

If DNA analysis were totally objective, then all 17 analysts should reach the same conclusion. However, we found that just one agreed with the original judgement that Robinson "cannot be excluded". Four analysts said the evidence was inconclusive and 12 said he could be excluded.

"Fingerprinting and other forensic disciplines have now accepted that subjectivity and context may affect their judgement and decisions," says Dror. "It is now time that DNA analysts accept that under certain conditions, subjectivity and even bias may affect their work." Dror presented the results at the Green Mountain DNA conference in Burlington, Vermont, last month.

Christine Funk, an attorney in the Office of the Public Defender for the State of Minnesota, says the results of New Scientist's survey have profound implications for criminal justice. "The difference between prison and freedom rests in the hands of the scientist assigned the case," she says.

. . .

So what can be done? This year, the Scientific Working Group on DNA Analysis Methods (SWGDAM), which issues guidance to US labs performing forensic DNA analysis, published new recommendations regarding the interpretation of forensic DNA.

. . .

It seems lab managers would welcome consistent rules. Forensic lab directors at the 19 labs we surveyed also provided their views about how their analysis is currently done: 15 either agreed or strongly agreed that interpretation procedures should be based on national standards, and 11 agreed or strongly agreed that decisions over alleles should not be based on analyst opinion.

Labs must also take steps to avoid bias. Butler says that some labs continue to insist upon seeing suspect profiles before analysing evidence from the crime scene, which could lead to biased decision-making (see "Crime Scene Investigation: Impartiality"). Analysts also often know too much about a suspect and other evidence to be impartial, and public labs often have close ties to police. "Crime labs, including DNA labs, should not be under the control of a law enforcement agency," says one US analyst, who wished to remain anonymous. "We are scientists, not cops or prosecutors."

There's much more at the link. Interesting and thought-provoking reading.

Peter