The Prosecutor's Fallacy

The field of statistics has quite a few types of fallacies up its sleeve. The prosecutor's fallacy is a particularly misleading one. In the courtroom, the prosecutor may not purposely use the fallacy to present evidence. Neither may the defense, who might use it to argue a suspects innocence. Still, sometimes the fallacy is presented by mistake.

Let's say some DNA is found on a murder scene. The police have a DNA databank containing 20,000 people and run the sample through it. There is a match, the suspected murderer is identified and put to trial.
The crime scene analyst testifies that the probability of two DNA profiles matching erroneously is only 1 in 10,000. The jury concludes that this means that there is only a 0,01% chance that the suspect is innocent.

Would you conclude the same?

The odds are actually quite different. First of all, the sample is matched against 20,000 others, so there were 20,000 opportunities to yield an incorrect match.
But here is the kicker: if the suspect is in fact innocent, then there is still a huge chance the DNA databank pops out a match.

100% − (99,99%)²⁰⁰⁰⁰ ≈ 86%

So while the murderer would likely be matched when he is in the databank, when he is not, the chance that an innocent person is matched is more likely than not. This is why these kind of database searches — resulting in “cold hits” — should always only represent a portion of the evidence.

A well known case that may have dealt with this fallacy is the Sally Clark case. The odds seemed very much against her, but working with statistical probabilities is always tricky...