You know that thing that was almost solved

-by a flasher dog in 1980's.

But specifically this story is about

how we increasingly solve crimes using forensic evidence.

It's that thing that is just a staple of TV crime shows.

Pull it from the torso on the left.

Pull it from the boat on the right.

Two hearts beat as one.

Matches up perfectly.

That's a match.

We've got a match.

And it just found us a match.

Visible match.

-We've a match. -Match.

Were you able to determine

which monkey bit him?

The bite marks match those of the monkey found at the scene.

Wow!

That last one was presumably

from one of the crossover episodes where the team from Law & Order,

worked a case with the cast of Monkey Law and Monkey Order.

-But...

on TV and in real life forensic science plays

an important role in criminal convictions.

Prosecutors often complain about a so called C.S.I. effect,

where jurors expect to see forensic evidence in every case.

The problem is, not all forensic science

is as reliable as we've become accustomed to believe.

A report in 2009,by the National Academy of Sciences

found that many forensic scientists

do not meet the fundamental requirements of science.

And a report last year by a Presidential Science Council agreed saying that,

"expert witnesses have often overstated

the value of their evidence, going far beyond

what the relevant science can justify,"

and that's the thing here.

It's not that all forensic science is bad,

'cause it's not, but too often,

it's reliability is dangerously overstated

and one sign of that is that forensic experts in court

are often nudged to use one very convincing phrase.

To a reasonable degree of the scientific certainty...

To a reasonable degree of scientific certainty...

To a reasonable degree of the scientific certainty...

Within reasonable scientific certainty...

To a reasonable degree of scientific certainty...

Are you able to say that within a reasonable degree

-of the scientific certainty? -Yes.

And here's the thing that phrase

does have a persuasive ring to it. Unfortunately,

as that Presidential Council pointed out,

it has no generally accepted meaning in science.

It's one of those terms like, basic or trill

that has no commonly understood definition.

Am I trill?

Is that good or bad?

I mean I do feel trill, so I'm guessing it's awful.

And-- when bad science

is confidently presented, terrible convictions can happen.

In fact, among the hundreds of people

who have been exonerated by DNA testing since 1989,

in nearly half of their cases,

there was some misapplication of forensic science

and there are people behind those numbers.

Take Santae Tribble, who was convicted of murder

and served 26 years. In large part,

thanks to an FBI analyst who testified that his hair

matched hairs found at the scene.

And as he will tell you,

the evidence was presented, as being rock solid.

They said they matched my hair

in all microscopical characteristics.

And that's the way they presented it to the jury

and the jury took it for granted that, that was my hair.

But you know, I can see, why they did.

Because who other than an FBI expert

would possibly know that much about hair?

Except of course,

whoever stalled Amanda Seyfried at the 2009 Oscars.

Breath taking waves,

without loosing any of their body or bounce.

-Stunning.

-Stunning!

-Stunning.

-Stunning.

Stunning! Stunning! Stunning! Stunning! Stunning! Stunning!

The jurors in Tribble's case, were actually told

there was one chance in ten million

that it could be someone else's hair, and guess what?

He was exonerated.

Because once DNA analysis became available, his lawyer tested

the thirteen hairs from the case

and not only were none of them his,

some of what they found was incredible.

Nine of the hairs had come from the same source,

a couple had come from different sources

and one was a dog.

Two different FBI agents who had, eh,

looked at that and analyzed it, didn't recognize that it was dog hair?

It was a K9.

It was a domestic dog, yes.

My personal conclusion was,

-the dog committed the crime.

Okay.

So, first, it is amazing that he is able to laugh at that,

but second, if a dog did commit the crime

there's really no recourse there because there is actually no law

against dogs committing murder

and that's a fact that learned in Air Bud 9,

Fuck the Paw-lice!

And it turns out,

Tribble is not the only case where FBI experts

overstated their confidence in their results.

The Innocence Project

and the National Association of Criminal Defense lawyers

found from the 1970's through 1999,

in 268 cases where FBI hair analysis led to a conviction,

257 or 96 percent of them had errors in analysis.

Oh, it gets worse

because nine of those defendants had already been executed,

which is horrifying.

And you would expect FBI hair analysis

to have a high rate of accuracy than your friend's hair analysis

of you can totally pull off bangs,

because you can't,

you absolutely can't, believe me I couldn't,

just learn--

-learn from our mistakes kids.

Save yourselves!

It's too late for me.

And look,

it's by no means,

just microscopic hair comparison which has had

the reliability of these results overstated.

Those reports that I showed earlier suggests

there is weak scientific support

for some aspects of techniques like

a blood pattern, footwear, firearm and bite mark analysis.

And you must be familiar with that last one from

cool scenes like this:

A little 3D magic for clarity and I give you

the killer's incisors.

(COMPUTER BEEPING)

Oh, Yo!

The computer rated it "Yellow rectangle."

And we all know yellow rectangle is the highest level of match

a computer can give you about teeth.

(AUDIENCE LAUGHING)

Look, in the real world, bite mark analysis

is highly subjective and unreliable.

The President's Council found the entire discipline,

does not meet the scientific standards

for foundational validity.

Which I believe, is science speak for "Bullshit!"

But people have been sent to prison

on the basis of bite mark testimony by experts like,

Dr. Michael West.

The science of bite marks analysis, is very accurate.

NARRATOR 1: When it comes to bite marks,

West consider himself "The maestro."

He's found bite marks on a decomposed body

submerged in a swamp,

on a corpse that had been buried for more than a year.

He's even used a bite mark

taken out of a bologna sandwich to get a conviction.

Now, that sounds impressive matching a killer's teeth

to a bite mark in a bologna sandwich,

although, you should know that the defendant in that case,

got a new trial after an autopsy report

found that the murder victim

had actually eaten a small amount of bologna

consistent with the amounts bitten off the sandwich.

-(AUDIENCE LAUGHING) -So, that sandwich,

was irrelevant to the case.

In fact, you could even argue that it was actually Dr. West,

who was full of, say it with me,

-shit. -(AUDIENCE LAUGHING)

And-- that is not the only issue

that has arisen from his testimony.

There are now five cases

where he testified for the prosecution

and where the charges were dropped

or the conviction was later over turned

and even West himself

has admitted that he no longer believes

in bite mark analysis for identifying perpetrators

and he doesn't think it should be used in court.

And yet, incredibly,

every time a defendant has challenged its validity

the court has ruled it admissible.

And a key reason for that

is that judges often rely on precedent

to decide what to allow in front of a jury. So,

if a particular discipline has been in court before

it is likely that a judge will admit it again.

All of which means that as the co-founder

of the Innocence Project points out,

decisions about the validity of science are being made

by people who don't necessarily know much about it.

Historically, we had a situation where,

two scientifically illiterate lawyers

argued the bonafides of scientific evidence

before a scientifically illiterate judge,

so the 12 scientifically illiterate jurors

could decide the weight of that evidence.

And if you think about it, that's absolutely terrifying.

Trials can often be a situation when no one

really knows what they are doing.

It's like a cooking competition for toddlers,

hosted by a stray cat and judged by goats.

-(AUDIENCE LAUGHING) -Oh.

The tuna was under cooked

and covered in cold spaghetti sauce.

You then for some reason

cover the whole dish in honey nut cheerios.

-I loved it. -(AUDIENCE LAUGHING)

And look, none of this is to say,

that there is not reliable forensic science out there.

Finger prints and DNA

are obvious examples but while we think of them as perfect,

it is important to know

they are by no means infallible.

The FBI has found fingerprint analysis

could have a false positive rate

as high as one error in 306 cases.

And a dramatic example of this

came after the Madrid's train bombings in 2004

when the FBI arrested this Oregon man, Brandon Mayfield.

He had never even been to Spain in his life.

But, three separate examiners, matched his finger prints

to one on a bag of detonators. So, he was at that point,

completely fucked!

Until, investigators happen to determine that,

that fingerprint actually also matched someone else

who was in Spain at the time and that blew the minds

of finger print experts.

MARK ACREE: We always assume

that finger prints are very very unique,

but what the Mayfield case demonstrates, is that

parts of a fingerprint can be

so similar, it's possible for two people to be

identified to one print.

That's true.

It turns out that two people can have finger prints

that are so close that even experts can't tell them apart.

Meaning that we are now this close

to finally proving my theory. There is only one Olsen twin.

She's just moving very fast

-back and forth. -(AUDIENCE APPLAUDING)

She confuses your eye.

Now, I don't know how this new information helps me, yet,

but when it does, the end is-- No! You frauds! You frauds!

(AUDIENCE APPLAUDING)

And then-- there is DNA,

which is the gold standard in forensic science for a reason

because in perfect conditions

it's seen as the most reliable form of evidence, but

not all DNA tests are equal

and crime scenes can produce DNA of widely varying quality.