Failed Forensics: How Forensic Science Lost Its Way and ... · forensic identiﬁcation science is not really science at all. No Basic-Science Origins Although forensic science is

ANRV359-LS04-08 ARI 15 October 2008 14:37

Failed Forensics:How Forensic ScienceLost Its Way and HowIt Might Yet Find ItMichael J. Saks1 and David L. Faigman2

1College of Law and Department of Psychology, Arizona State University, Tempe,Arizona 85287; email: [email protected] College of the Law, University of California, San Francisco, California 94102;email: [email protected]

Annu. Rev. Law Soc. Sci. 2008. 4:149–71

First published online as a Review in Advance onJuly 8, 2008

The Annual Review of Law and Social Science isonline at lawsocsci.annualreviews.org

This article’s doi:10.1146/annurev.lawsocsci.4.110707.172303

1550-3585/08/1201-0149$20.00

Key Words

courts, Daubert, science, scientific evidence

AbstractA group of nonscience forensic sciences has developed over the past cen-tury. These are fields within the broader forensic sciences that have littleor no basis in actual science. They are not applications of establishedbasic sciences, they have not systematically tested their own hypothe-ses, and they make unsupported assumptions and exaggerated claims.This review explains the nature and origins of those nonscience forensicfields, which include the forensic individualization sciences and certainother areas, such as fire and arson investigation. We explore the role ofthe courts in maintaining the underdeveloped state of these fields andconsider suggestions for improving this state of affairs (addressing thepotential role that could be played by these fields themselves, by thecourts, and by normal sciences).

149

Click here for quick links to Annual Reviews content online, including:

• Other articles in this volume• Top cited articles• Top downloaded articles• Our comprehensive search

FurtherANNUALREVIEWS


INTRODUCTION

Surprisingly little of what most people thinkthey know about major aspects of the foren-sic sciences is as they imagine it to be. Thisreview focuses on what a recent report of theNational Institute of Justice (2007) refers toas the soft forensic sciences, or what might bemore accurately described as the nonscienceforensic sciences. The public appears to con-sider the various forensic sciences as one whole,believing them all to be flawless applications ofsound basic science to problems of law. Courtsgenerally share this view. As a Utah Court ofAppeals judge commented in regard to finger-print expert testimony: “In essence, we haveadopted a cultural assumption that a govern-ment representative’s assertion that a defen-dant’s fingerprint was found at a crime sceneis an infallible fact, and not merely the exam-iner’s opinion” (State v. Quintana 2004, Thorne,J., concurring, p. 171). The belief in fingerprinttechnology may be the most extreme example,but it is by no means the only. The trust thatis laid upon the forensic sciences generally fallssomewhere between uncritical faith and manu-factured myth.

The nonscience forensic sciences, as theparadoxical phrase suggests, are those fieldswithin forensic science that have little or nobasis in actual science. They neither borrowfrom established science nor systematically testtheir hypotheses. Their primary claims for va-lidity rest on anecdotal experience and procla-mations of success over time. Hypothesis andsupposition are typically considered sufficient.Whereas in most scientific fields experience andobservation are designated as the first steps ofthe scientific method, for many forensic fieldsthey constitute the final stages of confirma-tion. Indeed, in a way, many practitioners of theforensic arts have turned the scientific methodon its head. So long as their hypotheses andsuppositions have not been tested, they are as-sumed true. Hypotheses that endure over time,rather than actually being tested, are deemedproven. This model was once pervasive inapplied settings, especially in medicine, and

produced such time-tested technologies asbloodletting and phrenology.

The fields that most resemble those ancientnonscience sciences are the forensic identifi-cation sciences, as well as certain other spe-cialties within forensic science. By identifi-cation sciences we mean those subfields thatoften are referred to as criminalistics and thatinvolve pattern matching in an effort to as-sociate a crime scene mark or object with itssource. These subfields include the compar-ison of fingerprints, handwriting, bitemarks,voiceprints, toolmarks, firearms, tire prints,shoe prints, and so on. Their goal is to link alatent fingerprint, a writing, a bitemark, a bul-let, or similar objects to the one and only fin-ger, writer, teeth, gun, or other specific objectin the world that made the markings. The ul-timate objective, and postulated achievement,of forensic identification science is individual-ization, “[t]he process of placing an object ina category which consists of a single, solitaryunit. Individualization implies uniqueness . . .”(Thornton & Peterson 2008, p. 71). “Crim-inalistics is the science of individualization”(Osterburg 1969, p. 97). For example, a firearmsexaminer testifying in a federal court claimed tobe able to identify the unknown weapon “to theexclusion of every other firearm in the world”(U.S. v. Green 2005, p. 107).

Beyond the identification sciences are cer-tain other subfields of forensic science that as-sert knowledge that was neither borrowed fromextant basic science nor rigorously tested byforensic experts before being offered to courts.These fields include the areas of fire, arson andexplosives, gunshot residue, comparative bulletlead analysis, and aspects of forensic pathology.These areas of claimed expertise share with theforensic identification sciences the irrationalreliance on unspecified, unsystematic “experi-ence” coupled with plausible-sounding argu-ments as the nearly exclusive bases for theirhypotheses.

The nonscience forensic sciences have failedon several levels. They are scientific failuresin the sense that science (either in substanceor in methodology) played little more than a

150 Saks · Faigman


rhetorical part in the development of thesefields (Saks 2007). The word and the ac-coutrements of science were exploited to sellthese fields to courts and to the public (Cole2001), but they did not trouble themselves tobuild upon a foundation of substantive scien-tific knowledge or to employ the methods ofscience to self-critically evaluate their wares.They are also technological failures. The spe-cific techniques that have been developed anddeployed in these fields make unknown num-bers of errors—both false negatives (failing todetect what they sought to detect, or to identifywhat they sought to identify) and false positives(seeming to detect something that did not exist,mistakenly identifying an innocent person orobject as having been connected to the crimeunder investigation). Yet despite these corefailures, the nonscience forensic sciences havebeen extraordinarily “successful” as measuredby their admission into trial courts and theirapparent acceptance by the public. The reasonsfor acclaim in the public domain is a subject be-yond the scope of this review, although promi-nence on nightly television programs and infeature-length movies must constitute part ofany explanation. Acceptance among courts andlawyers, however, is somewhat easier to docu-ment, though possibly as difficult to explain.

In this review, we focus on the nonscienceforensic sciences and consider the quandary ofhow such profoundly flawed disciplines couldachieve such worldly success. In examining thispuzzle, we explore three fundamental concerns.The first section considers the basic scientificfailure of these disciplines. This considerationincludes a brief survey of their historical originsand a detailed analysis of their compositions,in an effort to identify how they came to beand to understand how they persist. The secondsection considers the law’s role in perpetuatingthese disciplines. Although courts provide theprimary markets for the experts coming fromthese fields, they have done virtually nothing toevaluate critically the bases for the opinions thatrepresentatives from these fields offer into ev-idence. Finally, the third section looks forwardand asks how these nonscientific forensic sci-

ences might be made scientific and what such aworld might look like.

SCIENTIFIC FAILURES

Although science is not monolithic, the manyforms science assumes share certain character-istics. Perhaps foremost, good science has astrong element of formality associated with it.This formality permits what might be the keyattribute of the scientific method: reproducibil-ity. Although postmodern sensibilities are wontto stress objectivity as a keystone of science, vir-tually all science strives toward what Sir KarlPopper (1949) called intersubjective testability.In a nutshell, this means that the essential char-acteristics of any scientific test can be under-stood well enough to be replicated by anothergroup of researchers. Researchers must specifyall the details employed in testing the hypoth-esis of interest so that outsiders can know justwhat was done. This requires clear statementsof operational definitions, full exposition of re-search methods employed and statistics used,and strict limits on the degree of subjectivenessrelied upon in reaching judgments or conclu-sions. The scientific method, therefore, is notmerely a way of testing hypotheses; it permitsoutsiders to know how the tests were conductedand allows other researchers to repeat and con-firm them.

From the law’s perspective, as a consumer ofscientific opinion, knowing what methods wereused to support various theories or hypothe-ses should be an imperative. Without knowingin detail how (and whether) a field tested itshypotheses, it is impossible for courts to knowhow valuable, if valuable at all, expert opinionsare that come from that field. Although anec-dotal experience sometimes produces valuableknowledge, without verifiable tests it is impos-sible to know when it does so.

Ostensibly scientific expert opinion, there-fore, must be evaluated on the basis of fun-damental principles of good hypothesis test-ing. Unfortunately, as this section describes,much of forensic science fails this basic test.The nonscience forensic sciences have largely

www.annualreviews.org • Failed Forensics 151


developed outside of mainstream science andare not premised on the basic sciences. More-over, their essential hypotheses have been sub-jected to few if any serious tests, and indeed,their primary hypothesis of individualizationmay be inherently impossible to test. Absentreal testing that can be replicated by other re-searchers and independently verified by courts,forensic identification science is not reallyscience at all.

No Basic-Science Origins

Although forensic science is routinely definedas “the application of science to problems oflaw,” this description does not fit the subfieldsof forensic science that are concerned with in-dividualization. Although knowledge from or-ganic chemistry can be brought to bear inidentifying what drugs, poisons, or medica-tions might be discovered in a corpse foundat a crime scene, what knowledge from con-ventional sciences like biology or chemistry orphysics support the notion of individualization?To what university science department does onego, what course does one take, to learn the basicscientific research that undergirds the notionthat no two firearms impart indistinguishablysimilar markings or that reveals how one knowswhen a questioned writing was made by onewriter rather than another?

Even a cursory review of major universitycourse catalogs quickly reveals that nothingcould supply a knowledge base for the foren-sic identification sciences. Only a relative hand-ful of schools nationally have forensic sciencedepartments, and most of those are devotedto teaching the technologies of the past ratherthan testing the limits of these technologies ordeveloping their scientific foundations. Moretelling, and more troubling, forensic issues arerarely taken up by academic scientists of anykind. Statisticians have generally ignored thesubject, although they regularly investigate par-allel problems, such as signal-detection theory(Swets 1973, Phillips et al. 2001).

In the nineteenth century, science was aslikely to be found outside the university struc-

ture as it was to be found inside it. This wasparticularly true in the United States, wherethe university itself was not fully formed untilthe twentieth century (Rudolph 1990 [1962],Thelin 2004). It was not unusual for seriousscience to be explored by wealthy amateurs.Benjamin Franklin and Charles Darwin are par-ticularly well-known exemplars of this model.But, over time, science became increasingly in-stitutionally based, and by the mid-twentiethcentury most science was done by professionalsassociated with universities. These departmentshoused communities of scientists, set standardsof acceptable practice, and educated succeed-ing generations of researchers. Perhaps mostimportantly, these departments embodied andpromoted a culture of scientific curiosity andadventure. In the best science departments, thiscombined a commitment to well-demonstratedmethodological and statistical practices and en-couraged innovation and creativity in hypoth-esis formation. Although science departmentsachieved such cultural ideals imperfectly andnot invariably, the incredible success of theacademic model in the twentieth century isundeniable.

Most of the forensic identification sciences,however, missed the school bus. They neverjoined the university system. Although hardlysurprising, but worse still, they instead becamean integrated component of the police appa-ratus. They became an instrument of law en-forcement, largely controlled by police tech-nicians and their superiors. Innovation, suchas it existed, was done by nonscientificallytrained, albeit mostly well-meaning, bureau-crats. As novel technologies came into being,a bureaucratic mindset controlled their appli-cation. They were not subjected to system-atic empirical testing before being offered astestimony—or, as Cole (2007) puts it, theyfollowed a testify-first-validate-later approach(see also Cooley 2004, Pyrek 2007). Whereasthe scientific ideal promotes constant ques-tioning, the bureaucratic inclination is the re-verse. Forensic identification science embracedthe bureaucratic ideal of perpetuating the sta-tus quo (Wilson 1989). Substantial questioning,

152 Saks · Faigman


testing, rethinking, and innovation in hand-writing or toolmark identification, for instance,would not only mean that those trained in theold ways would be in danger of becoming obso-lete; it also would mean that much of what theyhad testified to in the past, on which courts hadrelied, might come under a cloud of doubt.

This unfortunate state of affairs was com-pounded by the realities of judicial practice andreasoning, in two basic respects. First, the legalculture itself is inconsistent with the scientificperspective. In law, stare decisis operates as thecore working premise, a view in fundamentaltension with the idea of scientific progress. Thehabit of judges to defer to prior decisions disin-clines appellate courts to revisit possible (or ac-tual) errors by trial courts in any given case, andit leads trial judges to admit species of evidencethat appellate courts had approved in the past,regardless of how flawed that type of evidencecan be shown to be with current knowledge.Second, lawyers had, and continue to have, littletraining in the basic methods of science. Theydo not know how to think about validation ofscience claims or how to frame an assessmentof such claims. In particular, most lawyers havelittle or no training in statistics, a key element ofthe applied sciences of which forensics shouldhave been a part. Many lawyers simply couldnot distinguish between real science and pre-tensions to science. Lawyers—on both sides—who did (and do) know the difference oftenwere too overwhelmed by resource limitationsand caseload pressures to be able to mount ameaningful challenge to the evidence. More-over, scientifically naive courts were sometimesromanced by the claim that they were beingpresented with science by scientists—terms ap-propriated by police forensic personnel eventhough much of what they do would be de-scribed in more conventional scientific work-places as the work of technicians. At the sametime, other courts were complicit in what theyrealized was a parody of science but that they,too, saw as useful in achieving convictions.

DNA profiling is the exception that provesthe rule. DNA was discovered in 1953 by scien-tists working in academic science departments

(Morange 2000). The value of this discoveryfor law enforcement was obvious. If everyonehas virtually unique DNA, and a copy of ev-ery person’s DNA is contained in every cell ofhis or her body, then a technology that canprofile DNA found in material left at crimescenes should be a boon to law enforcement.Once DNA technologies permitted individualidentifications, both scientists and lawyers im-mediately appreciated its power. In just over adecade, DNA technology progressed throughseveral iterations and various advances, includ-ing RFLP (restriction fragment length poly-morphism) techniques, PCR (polymerase chainreaction) testing, and mitochondrial DNA anal-ysis. Today, although academic scientists con-tinue to monitor and refine DNA forensics, thetechnology is largely mechanical and operatedby technicians who have little understanding ofthe scientific or statistical bases underlying theprotocols they follow. The history of forensicDNA is almost a paradigm of how basic sciencecan contribute systematically and systemicallyto forensics.

If forensic individualization science hademerged from normal science, its approach andits techniques probably would resemble DNAtyping, with its measurement of attributes, sam-pling of variation in populations, and statisticalbases. Error rates, probability levels, confidenceintervals, and so on would be natural parts ofwhat developed. The elements of subjectivityin forensic examination would themselves betopics of research, to understand both their op-eration and how to tame them.

The nonscience forensic sciences, however,followed a separate path. They were not incor-porated into university science departments andever since have remained orphaned. They be-came instruments of police and prosecutors. Inthis capacity, they have always been evaluatedby their success in court. By this measure, theyhave been a sensation. But their success as sci-ence, measured by such pedestrian concepts asreliability and validity, is considerably less cer-tain. It is uncertain because it is unknown. Theirbasic hypotheses have not been tested in anyrigorous or systematic way. By any reasonable



measure of science, then, these forensic fieldsare failures.

Impossibility of Individualization

Possibly the most fundamental error made bythe identification branches of the nonscienceforensic sciences is the statement of the basichypothesis of their fields. They repeatedly claimthe ability to individualize. In other words, abasic operating premise of their expertise, theyclaim, is the ability to say that some trace ev-idence (e.g., a latent fingerprint, markings ona bullet, or a shoe imprint) was made by aparticular source (e.g., finger, gun, or shoe) tothe exclusion of all other possible sources inthe world. This is a form of Sir Karl Pop-per’s white swan problem (Saks 1998), whichhe used to demonstrate the limits of the in-ductive method (i.e., expertise based on sim-ple experience or observation). Popper (1959)asked his readers to consider the hypothesis that“all swans are white.” This hypothesis mightbe tested by the experience of observing 1000swans, all of which are white. But no matter howmany swans are observed, the possibility alwaysremains of finding a nonwhite swan. Upon find-ing, say, a black swan, a researcher would haveto modify his hypothesis, perhaps positing that“99.9% of swans are white,” a hypothesis sim-ilarly susceptible to rigorous test but immuneto certain proof. Popper concluded that eventhough hypotheses could not be conclusivelyproven true, they could be rigorously tested—that is, scientists could attempt to falsify them.Most working scientists would recognize thisPopperian model as representing the essenceof what they do on a day-to-day basis (as acommunity, though not necessarily as individ-uals). The assertion of the ability to individu-alize not only was not tested; the very framingof the hypothesis is highly vulnerable to falsifi-cation. When falsification inevitably occurs ineach nonscience forensic science subfield—by“different” being mistaken as the “same” or the“same” being mistaken as “different”—ratherthan revise the core hypothesis to a more plau-sible and probabilistic one, practitioners either

completely ignore the evidence of falsificationor seek ways to explain away the falsificationsin order to keep insisting there are none.

In fact, it appears that “[i]ndividualizationis unique to forensic science” (DeForest et al.1983, p. 7). Conventional science has noth-ing to say about individualization. Individual-ization was invented by police investigators tomeet a criminal justice system need, namely,to try to discover who had committed a crime(Thornton 1975) and to secure a conviction.In seeking to establish pinpoint linkages be-tween crime scene evidence and known exem-plars, the forensic identification sciences soughtto do what no other field had ever tried to do andabout which no other field has developed anybasic science knowledge. Although individual-ization is the centerpiece of numerous forensicscience subfields, a recent review argues that notheoretical or empirical basis for individualiza-tion exists, and none is likely to come into be-ing in the foreseeable future (Saks & Koehler2008). Interestingly, numerous forensic scien-tists recognize that a scientific basis for suchidentification does not exist and that practition-ers are making a leap of faith when they offerindividualization testimony in court (Inman &Rudin 2001; Stoney 2001, 1991; Bowers 2008;and others).

Forensic individualization begins with therecognition that the fine physical features ofobjects and the marks they leave (bitemarks ortoolmarks, for example) vary. But rather thanmeasuring that variability and carefully assess-ing the degree to which it reduces the poolof objects that could have been the source ofthe questioned evidence, systematically and rig-orously taking into account the quality andamount of the questioned markings, all but oneor two of the forensic identification sciences(those one or two being DNA typing and micro-scopic hair comparison) quickly adopted a beliefin uniqueness: that no two objects could leaveindistinguishably similar residua or markings.Much rides on this belief. If true, it eliminatesthe need to collect systematic data to assess vari-ability. It leads to the belief that if two marksare indistinguishable they must have originated

154 Saks · Faigman


from the same object “to the exclusion of allothers in the world.” And, by a more obscureroute, it has led many forensic examiners to be-lieve that the possibility of error is (or borderson) zero.

The notion of uniqueness originated in thenineteenth century with Quetelet, who hypoth-esized that “nature never repeats,” offering theproduct rule of probability theory to suggestgreat odds against such repetition (Cole 2001).Bertillon, a police records clerk in Paris, ap-plied the idea to the classification of prison-ers and invented anthropometry. Though neverproven empirically and impossible to prove sta-tistically, several generations of forensic scien-tists have repeated the mantra. The foundersof each forensic identification subfield invokedthe product rule (see Saks & Koehler 2008).Thus, forensic identification science has takena fundamental insight (variability produced byrandom effects) and exaggerated it into an un-founded but strongly held faith in uniquenessand (virtual) freedom from error. For example,Cummins & Midlo (1943 [1961]) undertookan extensive effort to prove that no two fin-gerprints could be indistinguishably alike, butin the end conceded failure: “[I]t is impossi-ble to offer decisive proof that no two finger-prints bear identical patterns.” Speaking of thefutility of relying on probability theory, theywrote, “It is unfortunate that this approachcarries the implication that a complete corre-spondence of two patterns might occur. . . .”More recently, Stoney (1991) constructed anarticle around the question, “What made usever think we could individualize using statis-tics?” In the field of handwriting identification,Harris (1958) showed that many people withthe same name sign their names indistinguish-ably similarly, a vitally important disconfirma-tion of a core assumption that has been al-most completely ignored by forensic documentexaminers.

Let us be clear: Efforts to identify the per-son who is the source of a latent fingerprintdeposited at a crime scene will likely be suc-cessful much of the time, most of the time, ornearly all of the time. That it is impossible to

say how often is symptomatic of the larger prob-lem. No one knows how often errors occur orwhat factors promote or prevent errors. Thatis because systematic empirical research on thefoundations of handwriting identification, onits performance in practice, or on what shouldconstitute best practices had been all but nonex-istent for a century. Research showing founda-tional claims to be untrue or unprovable wassimply ignored by the field (see, e.g., Cole 2005,Cummins & Midlo 1943 [1961]). Research sug-gesting improved practices was disregarded infavor of traditional practices (Risinger et al.2002). When the first Daubert challenge wasposed to asserted fingerprint expertise, propo-nents found themselves with so little researchor any other real evidence to offer in supportof their claims that in an effort to close theknowledge gap they launched two studies inmid-pretrial of that first challenge. This is a re-markable situation for a field to find itself inafter a century of admission to courts.

The absence of knowledge has been filledwith assertions supported by little more thanintuition, anecdote, and ipse dixit. In the placeof systematic research, the field developed a setof faith-based beliefs that are more akin to areligion or an ideology than to a science butthat have largely succeeded in making all possi-ble difficulties disappear, at least in the eyes ofcourts, fact finders, and the public. These un-proved (and sometimes unprovable) assertionsinclude such notions as that the error rate of fin-gerprint identification is zero; that the theoryand the technique are flawless; that errors thatdo come to light always result from misapplica-tion of the technique, not from the technique;that no two people have fingerprints that areindistinguishably alike (or confusable one foranother); and so on.

These beliefs sidestep the real issues that ex-aminers and courts need to be concerned about.The claim of uniqueness—be it true or false,provable or unprovable—is largely irrelevant.By analogy, assuming all faces are unique doesnot mean that one is never mistaken for another.The real problems are that latent prints are typ-ically partial (a fraction of a fingerprint) and/or



of varying quality (smudged, distorted, overlaidwith other marks). They are compared by ex-aminers who work with no objective guidelinesand who must use their judgment to reach aconclusion that is required to be of absolutesubjective certainty (if a conclusion is reached).Like DNA typing, the problem facing finger-print examiners is actually to estimate the pop-ulation frequency of the attributes observed inthe latent print and the known print and to es-timate how many equally good matches existin the population. The likelihood of mistakes(almost certainly) depends on numerous fac-tors that have not been systematically studiedand that are dealt with haphazardly in practice(Budowle 2006). Research to answer the ques-tions implied in this paragraph would bring uscloser to knowing how well examiners performthe task for which they are testifying or, asDaubert and Kumho Tire conceive the inquiry,the task at hand in the case before the court.For now, courts and the public will hear onlythe exaggerated, faith-based answers.

What we have just said about fingerprintscan be multiplied across all the conventionalforensic identification sciences. The good newsis that research has begun, in fits and starts,more in some fields than in others. Some fieldsmight be found to be so weak that they mustbe limited to no more than an investigatoryrole, not courtroom testimony. (This seems tohave been the fate, for example, of voiceprintsand compositional bullet lead analysis.) Othersmight be found to be more solid than they arecurrently regarded by informed commentators(e.g., Thornton & Peterson 2008). Some daybetter answers will be known. But not yet.

Consider the so-called ACE-V technique,claimed primarily by fingerprint examiners,though starting to be picked up by other identi-fication sciences. The acronym stands for Anal-ysis, Comparison, Evaluation, and Verification.The analysis step has the examiner look at thequality and quantity of detail that is present, of-ten aided by magnification of some sort. Thecomparison step usually calls upon the exam-iner to compare the trace evidence to a candi-date source. At the evaluation step, the exam-

iner forms an opinion regarding individualiza-tion, though, as noted above, this is not guidedby any formal standards or statistical criteria.Finally, the last stage involves the reexamina-tion by a second examiner of the trace andreputed source material to verify the match,though that person almost always knows thefirst examiner’s conclusion (Stoney 2008). Justabout everything that is wrong with today’sforensics can be found embedded somewherewithin the ACE-V system. Each step calls fora subjective impression guided by no objectivestandards.

Latent fingerprint identification is a prob-lem that should be readily amenable to testing,considering the amount of time and attentiongiven to it, not to mention the availability ofdigital databases of fingerprints. Latent printstypically are partial fingerprints found at thescene of the crime (or other place of interest)and can vary greatly in general quality as wellas in the quantity of detail available for analy-sis. For the analysis step of the ACE-V method,however, there has yet to be any systematic at-tempt to determine the frequency of the finger-print minutiae (e.g., ridge characteristics) thatare analyzed and subsequently used for com-parison. Suppose an examiner finds five minu-tiae that “match” between the latent print anda known print. The meaning of that degree ofsimilarity depends entirely on the frequency ofoccurrence in the population for each of the fivecharacteristics and, further, whether they arestatistically independent of one another. With-out more, the probability of randomly findingfive characteristics that match is somewhere be-tween 0.0001 and 0.9999. Nothing in the ACE-V method permits fingerprint examiners to saywhat the likelihood of finding a match is. In-deed, fingerprint examiners refuse to estimateprobabilities; they merely insist that their sub-jective judgments amount to certainties.

This discussion is not meant to suggest thatthe kinds of information criminalists bring tothe crime-solving process are useless. They canbe of enormous help in narrowing the pool ofsuspect persons or objects. But everyday exam-iners have ignored their fields’ own cautionary

156 Saks · Faigman


literature, and the claims of the fields havebeen woefully underresearched and oversold tocourts and to the public.

Absence of Empirical Testing

Even though most of the nonscience forensicsciences had no basic science to build upon, asself-proclaimed sciences they might have con-ducted their own empirical research to test thebeliefs on which their work and their con-clusions relied. But they did virtually no suchresearch.

As one example, after reviewing the liter-ature in their own field of forensic odontol-ogy in search of the basis for the field’s be-liefs that it can individualize bitemarks, Pretty& Sweet (2001) concluded, “Despite the con-tinued acceptance of bitemark evidence inEuropean, Oceanic and North AmericanCourts, the fundamental scientific basis forbitemark analysis has never been established.”Risinger et al. (1989) scoured the handwrit-ing identification literature and found essen-tially no research testing whether such assertedexperts could do what they purported to beable to do. In the area of firearms identifica-tion, some firearms and toolmark examinershave been working to place their judgmentson more systematic empirical footing (Biasottiet al. 2008). This program, while admirable, re-mains some distance from completion and hasnot been embraced by most firearms examiners.For a critical review of the field, see Schwartz(2005).

There are no systematic, rigorous, empiri-cal research bases on which the great majorityof beliefs in the nonscience forensic sciences arebuilt. (For detailed discussion of various foren-sic sciences, see relevant chapters of Faigmanet al. 2008.) If called upon to prove their claims,they have little or no data to marshal in theirfield’s defense. Instead, they generally point to aguild of mutually self-reassuring examiners whohave come to believe in the truth of their claims,often sounding more like a faith-based religionthan a data-based science (see, e.g., Moenssens1998).

As Giannelli & Imwinkelried (2000) havenoted, in each area of the nonscience foren-sic sciences, “little rigorous, systematic researchhas been done to validate the discipline’s ba-sic premises and techniques, and in each areathere is no evident reason why such researchwould be infeasible.” For example, until fairlyrecently, the field of fire and arson investigationrelied on intuition and imagination to develop aset of what came to be accepted as indicators ofarson. They came to believe that crazed glass,spalled concrete, certain types of burn patterns,and so on indicated fires that had been deliber-ately set. Untold numbers of cases, both crim-inal and civil, were decided on the strength ofsuch experts applying those beliefs and opiningon whether they thought a particular fire was ar-son or accident. Eventually, those beliefs wereput to empirical tests in which buildings wereset afire in ways that simulated either arson firesor accidental fires. By comparing the effects ofthe arson versus accidental fires on windows,walls, burn patterns, and so on, these concep-tually simple experiments revealed that manyof the accepted indicators of arson did not, infact, distinguish arson from accidental fires. Inlight of the research findings, the field beganto abandon those incorrect beliefs and adoptmore accurate beliefs based on systematic em-pirical testing (Lentini 2008). Of course, someinvestigators learn of or come to accept the newdiscoveries more slowly than others. And an un-addressed concern is all of the past verdicts ofcriminal and civil liability or denial of insuranceclaims that were made in error, relying on theexpert opinions that relied on untested beliefsthat now are recognized to be false.

Little Use of Scientific Methods

Scientists have a way of using the scientificmethod in much of what they do. For exam-ple, in trying to determine what testing proce-dures produce the most accurate results, theymight try out different ones and compare theresults obtained. The ones that consistently fa-cilitate more accurate results would be adoptedas the standard procedure, the state of the art.



Similarly, systematic experiments can be con-ducted on different training procedures to de-termine which work best and which enabletrainees to acquire the most effective skills mostefficiently. Those are then adopted. These sortsof efforts are largely absent from the nonscienceforensic sciences, and in their place are commit-ted beliefs in the validity of the unproven.

Most immediately serious and troubling,whereas conventional scientists design studiesand procedures to maximize the contributionof the phenomenon under scrutiny and mini-mize the contribution of expectations and bi-ases (in particular, using blind or masked testsso as not to be misled into seeing what they wantor hope or expect to see), forensic scientists—alone among those who regard themselves asscientists—do not employ such procedures andinsist that they are not affected by such con-text effects (Risinger et al. 2002). Most scientistswould suspect that knowing other inculpatoryevidence in a case could distort the conclusionsof someone judging and interpreting other ev-idence in a case, but forensic scientists have in-sisted that they can will themselves not to bevulnerable to this phenomenon.

However, recent research by Dror and col-leagues no longer permits such denials. For in-stance, Dror et al. (2006) found that four out offive fingerprint experts who had several yearsearlier identified two prints as a match laterreached different conclusions on their secondencounter with those same prints, when eachexaminer was told that others had declared theprints to be from different persons. Althoughcontext effects are usually found to be small butpersistent (usually leading to error rates of a fewpercent at most), 80% of Dror et al.’s finger-print examiners changed their opinions whenpresented with misleading contextual informa-tion. (See also Dror & Charlton 2006.) A recentmeta-analysis of these data found an effect sizeof r = 0.40, which is quite large for contexteffects (Dror & Rosenthal 2008). A famous fin-gerprint error by the FBI, in which several oftheir most senior examiners one after anothererroneously identified a latent print in a ter-rorist bombing in Madrid as belonging to an

Oregon lawyer who had no connection at allto the bombing, has been attributed partly tocontext effects (Stacey 2004). (For descriptionsof other cases of fingerprint identification er-rors, see Cole 2005.) Such research results andincidents suggest that criminalists should em-ploy the same kind of blind examination pro-cedures that are used widely in other fields. Todate, however, forensic scientists continue withbusiness as usual.

Error Management

Normal scientists do not deny error. Quitethe contrary: They have been obsessed withthe measurement and reporting of error atleast since the Greek astronomers. Today, mostscientists have made statistics—the study ofuncertainty—an integral part of how they thinkand what they do, and they work closely withstatisticians. Research studies report probabil-ity levels, specify confidence intervals, presenterror bars on graphs, and so on. They not onlymeasure and report error, scientists study error,explore its sources, and work to manage it. Instriking contrast, most forensic scientists, espe-cially criminalists, pretend that error does notexist in their work. This is most plain in thework of fingerprint examiners, who insist thattheir error rate is zero and who declare the re-porting of a qualified identification (meaningan identification accompanied by a probabilitystatement of less than 100% certainty) to be anunethical act.

The forensic expert’s crabbed understandingof error is well illustrated in the case of U.S. v.Allen (2002), in which the issue was the admis-sibility of footwear identification techniques.The court observed as follows:

With respect to the rate of error in the ACE-V[i.e., analysis, comparison, evaluation, and ver-ification] process, [the expert] testified that theerror rate of the process itself is zero, meaningthat based upon the science, the shoe eitherdid or did not make the impression. Any errorthat does occur, according to [the expert] iscaused by examiner error in the application of

158 Saks · Faigman


the process or by examiner error in reaching aparticular conclusion (p. 862).

A theoretical error rate of zero should be re-assuring to no one. No technology has a zeroerror rate or failure rate in practice, whetherthe fault lies with the method, the operator, ormost likely both. To say that the suspect shoeeither did or did not make the impression is ametaphysical statement that addresses neitherthe real concerns about the ability to associateany given shoe with such a mark nor the variousrisks of error associated with any inference re-lated to whether it did or did not. Courts, afterall, operate in the real world, not the world ofmetaphysical imaginings. To even suggest thatfootwear identification techniques have a zeroerror rate, based on some theoretical belief thatevery shoeprint is unique, is simply rhetoric, notscience.

Claims of zero error rates are particularlyironic given that all individualization sciences(with the exception of DNA typing) rely ex-clusively on each examiner’s subjective judg-ment to assess the commonness or rarity offeatures (of fingerprints, toolmarks, etc.), be-cause no efforts have been made to determinethe population frequency of those features (ex-cept of DNA). Indeed, DNA evidence, the goldstandard in forensic identification, is itself sus-ceptible to an assortment of potential sourcesof error. These include, among others, errorsin collecting biological material, mislabelingof evidence, accidental contamination of sam-ples, and, sometimes, intentional fraud. All thepotential sources of error that plague DNAidentification also threaten other kinds offorensic evidence. But with the latter, we havelittle or no idea of the random match probabil-ities. As Stoney (2008) has pointed out, “Thecriteria for absolute identification in fingerprintwork are subjective and ill-defined. They arethe product of probabilistic intuitions widelyshared among fingerprint examiners, not of sci-entific research” (p. 360). It may seem para-doxical that a field whose work is so lackingin objective measurement, and whose decisionsturn on subjective judgment, would eschew the

analysis of probability and error. But for sev-eral reasons it should not be surprising. Whenone has no objective measurements, the calcula-tion of error and probabilities is more difficult.Moreover, this scientifically odd behavior likelyis a product of the adversary system. Forensicscientists are trapped in the crossfire of pros-ecutors who want evidence to be as airtight asit can be and defense attorneys who will lookfor any daylight that might be seeping throughthe government’s case. It has been suggestedthat over the decades forensic examiners havebeen pressured to make statements as extremeas they can get—100% certainty, zero errorrates, identification to the exclusion of all oth-ers in the world—not because such statementsgrow out of their science, but because they servethe needs of those who use their work (Saks1998).

Prominent forensic science thinkers havequestioned these implausible conventions.Thornton & Peterson (2008) have observed:“[T]hough individualization is clearly the goaltoward which forensic science strives, it canbe achieved only in a probabilistic sense, ofreducing uncertainty to the smallest possibleamount . . .” (p. 7). “Behind every opinion ren-dered by a forensic scientist there is a statisticalbasis. We may not know what that basis is, andwe may have no feasible means of developingan understanding of that basis, but it is futile todeny that one exists” (p. 22). Champod & Evett(2001) have argued that “there is, at present, amajor contradiction between the scientific sta-tus that is claimed and the operational paradigmto which its practitioners subscribe. This con-tradiction is exemplified by a recent statementby . . . SWGFAST [Scientific Working Groupon Friction Ridge Analysis, Study and Technol-ogy]: ‘Friction ridge identifications are absoluteconclusions. Probable, possible, or likely iden-tification are outside the acceptable limits of thescience of friction ridge identification’” (p. 432).Champod & Evett pose these challenging ques-tions: “Is a statement of an ‘absolute conclusion’compatible with scientific reasoning? Is the de-nial of probabilistic reasoning compatible witha scientific pursuit?”



Moreover, claims of no error and no possi-bility of error fly in the face of studies reveal-ing errors in casework (Cole 2005), in profi-ciency testing (see Thornton & Peterson 2008),and in other studies (e.g., Dror et al. 2006,Dror & Charlton 2006, Kam et al. 1997). (Formore extensive reviews, see Volumes 4 and 5 ofFaigman et al. 2008.) The irony may be thatby denying the possibility of error, forensic sci-entists have denied themselves the opportunityto understand better the sources of error andto manage error better. Lest one think that er-rors hurt only defendants, bear in mind that bysetting a decision threshold so high that one de-clares an identification only when one is “100%sure,” the incidence of false negative errors isincreased (Phillips et al. 2001, Haber & Haber2003).

Surprisingly, forensic science expert evi-dence appears to be one of the leading causes oferroneous convictions, second only to eyewit-ness testimony (see Figure 1).

As discussed in regard to the ACE-V methodabove, forensic individualization science in-volves two fundamental steps. The first stepis to compare a questioned item of evidenceto an exemplar from a known source and tojudge whether they appear so alike that they canbe said to match. The second step is to assessthe meaning of that reported match: What is theprobability that the questioned and the knownoriginated from one and the same source? Dif-ferent risks of error are present at each step. Therisk of error in the first step is that a reportedmatch between a questioned and known sam-ple might not really match. Even if the methodused to compare questioned and known sampleswere flawless, an error could occur if, for ex-ample, one of the samples had been mislabeledor mixed up with a different sample. The riskof error associated with the second step is thatthe reported match may be accurate but arosethrough coincidence (the random match prob-lem) rather than because the samples share acommon source. The risks of error at both stepsaffect the ultimate inferences that can be drawnabout the identification evidence in a case. (For

more thorough discussion, see Thompson &Cole 2006.)

Both risks are significantly underresearched.As to the first step, existing standards and proce-dures do not provide sufficient protection fromerroneous conclusions that two marks are in-distinguishably alike—that is, that they matchwhen in fact they differ. Few, if any, criminal-istics subfields have objective standards for de-ciding whether two patterns match. That de-termination is left to the subjective judgment ofeach examiner. For example, consider Stoney’s(2001) discussion of fingerprint examinationstandards:

How much correspondence between two fin-gerprints is sufficient to conclude that they[are the same pattern] . . . ? An adequate an-swer . . . is not currently available. The best an-swer at present . . . is that this is up to the in-dividual expert fingerprint examiner to deter-mine, based on that examiner’s training, skill,and experience. Thus, we have an ill-defined,flexible, and explicitly subjective criterion forestablishing fingerprint identification. . . . Anyunbiased, intelligent assessment of fingerprintidentification practices today reveals that thereare, in reality, no standards (p. 329).

The lack of objective standards helps ex-plain some disturbing findings from the smallbody of research on pattern matching byforensic scientists that has been conducted todate. In some tests, examiners disagreed withone another about whether various imagesmatched (see proficiency studies summarized inFaigman et al. 2008). In other tests, examin-ers who agreed that two patterns matched dis-agreed (sometimes dramatically) on what con-stituted the match (Evett & Williams 1996).Examiners differ not only in their ability to per-ceive pattern similarity and differences, but alsoin their thresholds for calling matches.

As to certainty of forensic identifications,FBI forensic scientist Bunch (2000) has noted:“[I]t must be observed that there is no ra-tional or scientific ground for making claimsof absolute certainty in any of the traditional

160 Saks · Faigman


identification sciences, which include finger-print, document, firearms, toolmark, and shoeand tire-tread analysis.”

ILLUSORY JUDICIAL OVERSIGHT

The courts, and the legal profession moregenerally, have been remarkably ineffective inpolicing forensic science. With only a few no-table exceptions, courts have not closely eval-uated the scientific bases for forensic iden-tification and have not been responsible forexposing the flaws in any of the forensic sciencefields that have, over time, been abandoned ow-ing to their lack of validity. For reasons that willbecome clear in this section, the failures of thecourts are intimately connected to the failuresof the forensic sciences.

The reasons for the courts’ empirical inef-fectuality are likely manifold. Three systemicfactors, in particular, appear to explain muchregarding the courts’ failures in this area. Thefirst, and most basic, is the cultural divide thatseparates law and science. Outside of legal con-tent areas such as intellectual property, patents,and the environment, lawyers have little train-ing in the basic precepts of research method-ology and statistics. Although law school couldbe described as a glorified liberal arts educa-tion, it generally does not include courses in re-search methods and statistics. Even where suchcourses are available, they are taken by a smallpercentage of students. The second factor in-evitably affecting the law’s insouciance is thestandard by which courts judge the probativevalue of expert evidence. During most of thetwentieth century the courts used admissibilitytests that were either explicitly deferential to theprofessional fields that supplied the expertise oroperated deferentially as a practical matter. Al-though modern admissibility standards appearto mandate a less indolent review of profferedexpertise, little has changed on the criminalside of courts’ dockets. This may be a functioneither of courts’ inclination to grandfather inexpertise traditionally admitted or of some pre-disposition that favors evidence offered by pros-ecutors. The third factor is the ironic situation

that because nonscience forensic science is notscientific, there are few defense experts avail-able to testify. Most scientific fields are highlydynamic and often hotbeds of debate and dis-agreement about the meaning of their researchand the content and reach of their theories. Realscience teaches doubt. Good faith disagreementamong scientists spills over into the courtroomand provides fodder for the adversarial process.Although the courtroom is an imperfect forumfor resolving scientific disagreement, this di-alectic at least produces considerable informa-tion for courts to ponder. Nonscience foren-sic science creates no similar content in largepart because group consensus substitutes forscience.

These systemic challenges might combineto help make judicial ignorance of the meth-ods of science seem easy to understand. Yet itstill remains quite puzzling. Although law is adistinct institution with its own goals and objec-tives, it constantly interacts with the world andinstitutions around it. The law receives inputfrom a variety of sources, digests it through thelegal process, and applies the output with the ex-pectation of effecting some result. These stepsrequire judges to have extraordinarily broad un-derstanding of an assortment of professionaldisciplines. For instance, judges routinely relyon historical sources, particularly in constitu-tional cases, and it would be remarkable if theyeither eschewed responsibility for understand-ing it critically or employed it in plainly igno-rant ways. Yet when it comes to science, andparticularly statistics, judges pause and sput-ter, wondering whether it is truly part of theirresponsibility to know the details of scientificmethods. On occasions when judges are re-quired to think in a critically scientific way, theresults can be inane.

A particularly trenchant example of thisinanity can be found in the case of U.S. v.Havvard (2000). The Havvard court attemptedto justify its decision to admit fingerprint ex-pert testimony in a written opinion that inad-vertently does just the opposite. The court em-ployed the admissibility standard set forth inthe Federal Rules of Evidence and famously



explicated in the case of Daubert v. MerrillDow Pharmaceuticals, Inc. (1993). Daubert, asdiscussed further below, establishes a scientificthreshold for the admission of ostensibly sci-entifically based expert evidence. In particu-lar, the Daubert court instructed trial courts toconsider, among other things, whether the ba-sis for scientific expert opinion had been ad-equately tested and whether the testing hadbeen methodologically sound (with help in thatevaluation coming from peer review and pub-lication), had acceptable error rates, and wasgenerally accepted in the pertinent field. Re-markably, the Havvard court took these indica-tors of good science and converted them intocrime investigation and litigation markers. Onthe issues of falsification and whether the tech-niques are testable or have been tested, thecourt wrote that “[t]hey have been tested in ad-versarial proceedings with the highest possiblestakes,” as if the stakes and not the research de-sign, methods, and analysis are the key to soundempirical research. On the issue of “peer re-view and publication,” the court confused theprocess of fundamental knowledge building ina science with the practice of examiners check-ing each other’s everyday casework. The courtstated that “the methods of identification aresubject to peer review. [A]nother qualified ex-aminer can compare the objective informationupon which the opinion is based and may rendera different opinion if warranted. In fact, peer re-view is the standard operating procedure amonglatent print examiners.” Although there was noresearch to review, there still was peer review,the court suggested. The court excused the ab-sence of published research by saying again that“latent fingerprint identification has been sub-ject to adversarial testing for roughly 100 years,again in cases with the highest stakes possi-ble.” On the issue of error rates, the court wastold that fingerprint examiners never make er-rors. Although it disbelieved the government’s“breathtaking” claim that “the error rate for themethod is zero,” the court made what it consid-ered a “reasonable concession” that occasionalmistakes “can of course” occur. It admitted theevidence.

Thus, in U.S. v. Havvard (2000), the courtsubstituted courtroom debate for scientific re-search in an effort to justify admission. Fol-lowing this strange exercise, the judge declaredthis expert evidence to be “the very archetypeof reliable expert testimony under [Daubert]”(p. 855).

In another fingerprint case, U.S. v. Llera-Plaza II (2002), the court acknowledged thegaping absence of empirical research testing theclaims of fingerprint examiners, but it searchedfor some reason, any reason, to admit the ex-pertise. The judge in U.S. v. Cline (2002) tookthe view that the Supreme Court could nothave wanted the nonscience forensic sciences,such as asserted fingerprint identification, tobe excluded, and therefore, notwithstandingDaubert’s apparent commands, soundness (va-lidity, reliability) of the evidence could not bethe touchstone of admission. Consequently, inthe Cline court’s view, given the serious short-comings in the scientific foundations of finger-print expert evidence, alternative, less rigorous,criteria would have to be employed in order tofacilitate admission. Put differently: If the stu-dent cannot pass the test, the test has to be madeas easy as it needs to be to ensure that the stu-dent can pass it. A review of the many cases inwhich courts wrestled with challenges to finger-print expert testimony summed up those courts’opinions (in which they could not find basesfor admission that comported with the require-ments of Daubert and Kumho Tire, but never-theless found various ways to rule the testimonyadmissible) as amounting to “a catalog of eva-sions” (Faigman et al. 2008, chapter 32, § 32:3).

These cases, and the many others that mightbe cited (see, generally, Faigman et al. 2008),well illustrate courts’ limited understanding ofthe basics of science. These limitations, how-ever, were little on display until fairly recently.As noted above, in 1993 the Supreme Courtchanged the fundamental orientation federaljudges use for evaluating the admissibility ofexpert evidence. Prior to Daubert, courts thatused any articulated test at all used one that wasdeferential to the professionals from the perti-nent field, a test most famously associated with

162 Saks · Faigman


the case of Frye v. U.S. (1923). The Frye testqueried whether the knowledge or techniqueon which the scientific opinion depended was“generally accepted in the particular field” fromwhich it came. (For a detailed review of Fryeand its comparison with Daubert, see Saks &Faigman 2005.) Frye and similar tests seemed torequire no special scientific acumen. Daubert, incontrast, changed the primary focus from whatwas accepted by those who practiced in the fieldto whether the methodology and principles un-derlying the expert opinion were reliable andvalid. The Daubert analysis, which was subse-quently codified in amendments to the FederalRules of Evidence in 2000, requires a fair de-gree of scientific sophistication to be conductedwell. Courts lacking such sophistication, how-ever, have turned the test into something of acaricature.

When Daubert was first decided (see Saks& Faigman 2005), no consensus existed amongcourts or commentators about whether the newtest loosened or tightened the judicial filteringof such testimony. It is probably fair to say that,initially, most observers believed the test wasless stringent. But as courts applied the test, itturned out to be more stringent because manyexpert opinions that had been routinely admit-ted turned out, upon inspection, to have littledata and poor research methods behind them.As applied, therefore, Daubert seemed to raisethe threshold for scientific evidence. Courts,however, made an exception for the nonscienceforensic sciences, for which the test has usuallybeen applied so as to make it as undemandingas necessary to facilitate admission.

This strange mosaic can be understood onlyby considering two major planes, each of whichis itself somewhat paradoxical: what Daubertdoes (rather than what it says) and what judgesdo when they apply that law (something thatlegal realists conceived as the law on the booksversus the law in action). (For a more thor-ough review and analysis of these issues, seeSaks & Faigman 2005; Faigman et al. 2008,chapter 1.) On the plane of law, Frye requiredthat a proffered expertise be generally acceptedwithin its field, whereas Daubert requires that

the basis for proffered expertise be demonstra-bly sound, valid, and reliable. Thus, Daubertought more readily to admit expert testimonythat has a sound basis even if it has not yetgained general acceptance, whereas Frye wouldmore readily admit expert testimony that hadcome to be generally accepted even though ithas no demonstrably sound basis. No family ofasserted expertises falls more clearly into thecategory of simultaneously being generally ac-cepted (at least within their own orbit) and yetlacking any demonstrably sound basis than thenonscience forensic sciences. Thus, logically,these fields ought to have been more readilyadmitted by Frye and more readily excludedby Daubert. On the plane of judicial behav-ior, however, although the switch from Frye toDaubert led to more intense scrutiny and morefrequent exclusion of expert testimony in civilcases, something quite different occurred on thecriminal side.

Judges have, in general, bent over backwardsto evade the application of Daubert when con-scientious application would lead to the ex-clusion of any of the nonscience forensic sci-ences. For example, upon realizing that assertedhandwriting identification expertise (FDE, orforensic document examination) had no soundscientific basis, the court in U.S. v. Starzecpyzel(1995) declared:

Were the Court to apply Daubert to theproffered FDE testimony, it would have tobe excluded. This conclusion derives froma straightforward analysis of the suggestedDaubert factors—testability and known errorrate, peer review and publication, and generalacceptance—in light of the evidence adducedat the Daubert hearing (p. 1036).

Seeing where Daubert led, the court lookedfor a way to get off of Daubert’s path. In the end,the court reasoned that FDE lacked scientificvalidity; because it lacked scientific validity, itwas not subject to Daubert review; and becauseit was not subject to Daubert, it was admissible.Were it a sound science, it would have beenadmissible; because it was not a sound science,



it was admissible. This solution was followedby numerous federal courts until it was implic-itly overruled by the Supreme Court’s decisionin Kumho Tire v. Carmichael (1999), holdingthat all fields of asserted expertise, whether la-beled science or anything else, had to meet seri-ous and appropriate criteria of soundness or beexcluded.

One would have expected that after nearlya century of admission as evidence in courts,when finally called upon to defend the scien-tific foundations of their claims in court, fin-gerprint examiners would be able to roll wheel-barrows filled with studies and data into courtto prove the bases for their claims. But theyhad nothing. Their expert testimony had beenpresented in American courts since early in thetwentieth century, having been admitted firstby courts that applied no real test of validity,and then by courts that relied on those earliercourts as precedent of a sort (Saks 1998). Af-ter the Supreme Court’s decisions in Daubert(1993) and Kumho Tire (1999), challenges beganto be laid (see Saks & Faigman 2005). Amaz-ingly, the first empirical studies subjecting thebasic claims of fingerprint examiners to empir-ical testing did not occur until the midst of thepretrial phase of the first of those post-Daubertchallenges, U.S. v. Mitchell (2000). The judge inthat case simply ruled, without a written opin-ion, that fingerprints were admissible. The firstcourt to use the record developed for Mitchell ina formal opinion ruled fingerprint expert opin-ions on individual identity to be inadmissible(U.S. v. Llera-Plaza I 2002). The same court re-versed itself several months later, admitting thetestimony, though it continued to find that vir-tually no research existed supporting the claimsof fingerprint experts (U.S. v. Llera-Plaza II2002).

State courts have been similarly wide-eyedin their assessments of the nonscience forensicsciences. In a case involving a challenge to theadmission of expert testimony on microscopichair comparison, for example, the KentuckySupreme Court, in Commonwealth v. Johnson(1999), purported to apply a Daubert analysisto admissibility. Because the court had no

relevant data of any kind in the record beforeit, the proffered expertise could not meet anyof the Daubert criteria—except perhaps generalacceptance. But the court also could find no ev-idence of general acceptance, and so it relied onits own past cases from decades earlier for theconclusion that microscopic hair identificationexamination passes general acceptance muster.It turns out, however, that those earlier casessaid nothing about general acceptance. TheKentucky Supreme Court acknowledged thisfact but reasoned that those courts “must have”implicitly found general acceptance or theywould not have admitted the hair identificationevidence. The flaw in this reasoning was thatKentucky had not adopted Frye or any otherversion of general acceptance until after thosecases had been decided. Through this sleightof hand—forming grounds for admission outof thin air—the Kentucky Supreme Court heldthat microscopic hair identification satisfiedDaubert.

In another state case, involving asserted ex-pertise in voiceprint identification, Alaska v.Coon (1999)—the very case by which the AlaskaSupreme Court adopted Daubert as its test ofadmission—the court remanded the case to thetrial court to take additional evidence in antic-ipation of adopting and applying Daubert, butno court involved in the case at any level builtits holding on any relevant research studies orother data. Undaunted, in another magical ju-dicial opinion, the Alaska court conjured thebases for admission out of nothing.

The problem has not merely been judgeswho cannot bring themselves to follow the lawto its logical conclusion if doing so will re-sult in an outcome that makes them nervous.Judges in criminal cases are not likely to havethe opportunity to hear many challenges be-cause lawyers in criminal cases are much lesslikely than lawyers in civil cases to mount chal-lenges to the other side’s scientific expert ev-idence (Risinger 2000). This is a reflection ofthe familiar sociological and institutional dif-ferences between civil and criminal cases.

Some courts have accepted the command offederal or state versions of Daubert and ruled as

164 Saks · Faigman


required by those tests. This was true especiallyin regard to asserted handwriting identificationexpertise. Following Kumho Tire, most federalcourts that thoughtfully examined this evidencefollowed the path of U.S. v. Hines (1999), whichheld that experts could not testify regarding theidentity of the author of a questioned docu-ment. These courts limited the expert to “show-and-tell” descriptive statements regarding sim-ilarities and differences between the knownand unknown exemplars. Indeed, in some cir-cumstances, courts excluded the testimony offorensic document examiners entirely (U.S. v.Saelee 2001, U.S. v. Fujii 2000; for a more com-plete discussion, see Faigman et al. 2008, chap-ter 33). Several courts ruled that fingerprintidentification expert opinions on identity aremostly inadmissible (U.S. v. Llera Plaza I 2002)(though later reversed) or completely excluded(Maryland v. Rose 2007).

But at the end of the day, these cases aresmall islands of inadmissibility surrounded by asea of admission. As one commentator summedup this seascape: “It seems that the only stan-dard the courts are requiring of forensic scienceis that it be incriminating to the defendant”(Frisbie & Garrett, 1998, p. 56, quoting JohnMarshall Law School Professor Melvin B.Lewis). In essence, the nonscience forensicsciences are granted exemption from seriousscrutiny. Despite the hopes of the framers of ourConstitution, the decisions of individual judgesare not made in a vacuum insulated from pres-sures of various kinds. This is not to say that thepressures are improper or illegal. The spotlightfocused by the news media on an unusual (evenif unusually correct) decision, or the shrillnessof pleas from the government when it loses, orconsiderations internal to the judicial branchcan impel judges to decide in ways other thanwhat they believe to be correct. For a notewor-thy example, consider the opinion of the judgein U.S. v. Green (2005), a case involving firearmsidentification, where the proffered expert in-sisted he could identify the suspect weapon “tothe exclusion of all others in the world” despitebeing unable to offer any basis for believing that

to be true. In admitting the testimony, albeitonly partially, the court wrote:

I reluctantly [admit the proffered expert tes-timony] because of my confidence that anyother decision will be rejected by appellatecourts, in light of precedents across the coun-try, regardless of the findings I have made.While I recognize that the Daubert-Kumhostandard does not require the illusory perfec-tion of a television show (CSI, this wasn’t),when liberty hangs in the balance—and, in thecase of the defendants facing the death penalty,life itself—the standards should be higher thanwere met in this case, and than have been im-posed across the country. The more courtsadmit this type of toolmark evidence withoutrequiring documentation, proficiency testing,or evidence of reliability, the more sloppypractices will endure; we should require more(p. 109).

Indeed, we should.

WHAT IS TO BE DONE

In Hans Christian Andersen’s famous tale, “theEmperor walked in the procession under thelovely canopy, while all the crowds in the streetand all the people at their windows said, ‘Heav-ens, How marvelous the Emperor’s new clotheslook!’” The crowd, in fact, saw that the emperorhad no clothes, but feared saying so: “No onewanted it thought that he could not see any-thing, as that would make him somebody whowas either very stupid or badly fitted for his po-sition.” A little child, however, spoke up: “‘Buthe has nothing on!’” The crowd realized thatthe child was right and all began to shout: “‘Hehas nothing on!’” The emperor realized it, too.He “cringed inside himself, for it seemed to himthat they were right, but he thought like this: ‘Ishall have to go through with the procession’”(Andersen 1959, pp. 30–32).

Many nonscience forensic scientists are inthe same situation as the emperor. They have“nothing on.” At this point in time, of course,they continue in the procession as testifying



experts because, like the emperor, they cannotadmit the reality. To stop would reveal them tobe “either very stupid or badly fitted for [their]position.” Change, if it is to occur, must comefrom outside the nonscience forensic sciences.

There are two courses of action that mightlead to wholesale reform in the practice offorensic science: First, the courts need to be-come better consumers of expert testimony anddemand better methods than so far have beenforthcoming; second, mainstream academic sci-entists must join the effort to test forensic hy-potheses. Although many other reforms mightaccompany these two, both are essential to anytrue changes. Otherwise, these naked expertswill continue to march on, with their “chamber-lains . . . behind [them] carrying the train thatwas not there at all.”

Gatekeeping Courts

In some respects, the problems endemic in theforensic sciences must be understood in termsof market failure (Faigman 1999). The manu-facturers of forensic knowledge produce infe-rior products, and the buyers (largely courts)continue to purchase them. Reform, therefore,can be imagined in regard to both the sup-ply and the demand of the forensic product.Although it probably does not matter greatlywhich factor is revolutionized first, true changecannot take place unless the practices of bothsellers and buyers change. We begin with thebuyers.

To a large extent, the revolution has al-ready occurred in the law, but many of therevolutionaries still do not realize how perva-sive the reforms have been, at least in princi-ple. Daubert largely put in place the proceduralframework that would permit courts to see non-science forensic science for what it is. The keyparadigm shift wrought by Daubert was to re-quire courts to consider the methods and prin-ciples that supported expert evidence, not sim-ply inquire about its general acceptance amongsome insular group of true believers (Faigman2000, Saks & Koehler 2005). The gatekeepingmetaphor was first described in Daubert itself.

Largely explicit in the decision, the Court con-templated that judges were charged with the re-sponsibility to understand science well enoughto measure the scientific validity (i.e., “eviden-tiary reliability”) of proffered expert opinion.This responsibility led Chief Justice Rehnquistto complain that Daubert would require federaljudges “to become amateur scientists” (Daubert,p. 601). And indeed they must (Faigman 2006).

Science is such a pervasive force in modernsociety that judges are essentially failing theirobligations if they persist in their scientific ig-norance. This task, in fact, is not profoundlydifficult, at least in the forensics area. In mostcases, recognizing the weaknesses of nonscienceforensic science is not rocket science. Muchof the expert opinion propounded in court hasnot been tested beyond mere anecdotal experi-ence and is not supported by adequate researchdesigns or basic statistical testing. Moreover,texts and treatises are available that detail thenumerous and profound failings of these non-sciences (see Faigman et al. 2008, Giannelli &Imwinkelried 2007).

Just why the courts have so utterly failedto carry out their gatekeeping duties is diffi-cult to say. We can only speculate about possi-ble reasons for this dereliction. The two mainpossibilities are lack of knowledge or lack ofwill. We believe the latter is more likely, forseveral reasons. First, as noted, the failures ofthe nonscience forensic sciences are so pro-found and so elementary that they are just notthat difficult to understand. Second, the samejudges who demur from their gatekeeping obli-gations when prosecutors proffer bad forensicsare vigorous gatekeepers when considering badexpertise proffered by plaintiffs in civil cases.It is hard to believe that judges who minutelycriticize plaintiffs’ epidemiological proof can-not understand that prosecutors’ latent finger-print evidence lacks scientific validation. Fi-nally, so many of the judicial opinions dealingwith forensic evidence employ such torturedlogic that it is impossible to believe that courtsare blind to the empirical realities of this evi-dence. It appears, therefore, that lack of will isa better explanation than plain ignorance.

166 Saks · Faigman


There are, of course, many credible reasonswhy courts should lack courage in this context.Foremost, perhaps, they fear that excludingany forensic expertise that lacks a scientific ba-sis will result in wholesale exclusion of substan-tial portions of prosecutors’ cases. This couldlead to the disruption of large numbers ofcases and lead prosecutors not to pursue othercases. Moreover, it would potentially throw intodoubt hundreds, if not thousands, of past cases,possibly setting off a firestorm of habeas peti-tions seeking to overturn convictions based onnow-excluded forensic evidence. In individualcases, where courts are convinced from otherevidence that a defendant is guilty, they willsee no need to scrutinize the forensic evidenceclosely because (if it too inculpates) they willassume it is correct. Evaluating science, for itsown sake, is not the purpose of judicial proceed-ings. Finally, courts legitimately see no presentalternative to this proof. Whatever its weak-nesses, courts probably think it has to be ad-mitted until something better comes along.

From the perspective of courts, therefore,the current state of the art of forensic sciencecannot be jettisoned until a new state of the art isavailable. Yet, it also appears that without courtsraising serious doubts about current practices,there is little pressure for the forensics field tochange course. As a consequence, the status quowill persist until something intervenes. If nei-ther the courts nor the forensic experts are likelyto change things, the only group left is the main-stream scientific community. It is there that realreform in forensics can take place.

Scientific Gate-Makers

In light of the prevailing practice in forensicscience, there is little reason to believe that thefield will soon begin a comprehensive and rigor-ous research program. Indeed, even if the fieldgenerally recognized the need for such a pro-gram, it is not at all clear that it has the where-withal to even begin this work. Most profes-sional forensic experts are employed by stateor private laboratories that apply forensic tech-nology, not produce it. Existing laboratories are

not likely to have the time, resources, or incli-nation to vigorously test the technology theyemploy daily. Such an effort would certainly ap-pear to them as a waste of time and money. Whytest what already works, at least insofar as “whatworks” is measured by its acceptance in court?More problematic still, today’s forensic scien-tists are generally not trained as researchers;they are technicians. A study of crime labora-tory personnel found that fewer than 1% hadPhD-level research degrees and that only about3% had earned a master’s degree (Furton et al.1999). The overwhelming majority of forensicprofessionals simply do not have the statisticalor methodological training to do original re-search. Hence, if good science is to be broughtto bear on forensic hypotheses, mainstream aca-demic scientists will have to take the laboringoar.

At least two ingredients are necessary to in-terest academic scientists in the study of foren-sic issues: money and relevance. The first is rel-atively easily explained, if not readily obtained.Research, quite simply, follows the money.Historically, little government financing hasbeen available for basic forensic science re-search. Agencies such as the National Sci-ence Foundation and the National Institutes ofHealth do not include forensics as part of theirmission statements. Probably the single agencymost devoted to funding forensic research is theNational Institute of Justice (NIJ), an arm of theJustice Department. Thus far, NIJ has not beensuccessful in funding cutting-edge research.Finally, because academic scientists have notconsidered forensics to be within their baili-wick, they have not asked for money to dothis kind of research. Clearly, before academicscientists can be expected to reform forensic sci-ence, they must be convinced that the subject isrelevant and important to their day jobs.

Although the question of the relevance offorensics to various mainstream scientific com-munities is too large to canvass fully here, justbrief contemplation on the subject provides aclear answer. Forensics provides a treasure troveof research possibilities. Statisticians, in partic-ular, would find a surplus of hypotheses to test in



areas such as latent fingerprints, firearms iden-tification, and handwriting analysis. Indeed,conventional statistical subjects such as pattern-recognition and signal detection theory couldbe tested and applied to forensic problems.Moreover, many scientists would find foren-sics to be a fertile area to test general theoriesfrom their disciplines. The work of Dror andcolleagues—psychologists interested in study-ing cognitive biases—is a particularly good il-lustration of this. The list matching forensicsubjects to established academic departments islimited only by one’s scientific imagination.

In the short term, mainstream science hasthe potential to reform forensic science throughthe efforts of individual scientists (e.g., Zabell2005) and the contributions of select scien-tific organizations. In particular, the NationalResearch Council of the National Academies ofScience has played, and could continue to play, apivotal role in both evaluating the state of the artof forensic science and in making recommen-dations for the future (e.g., National ResearchCouncil 1992, 1996). In the long term, how-ever, only with substantial funding and the con-certed efforts of dedicated individuals will thenonscience forensic sciences become scientific.Perhaps one day forensic science will becomeanother subject in the vast scientific curriculumof major research universities.

CONCLUSION

We have explored the state of nonscienceforensic science—the absence of basic-scienceorigins, unsupported assumptions, exaggeratedclaims, lack of empirical testing, little use ofscientific method, pretermission of error. Andwe have discussed the interplay of nonscienceforensic science with illusory judicial oversight,as well as the indifference of normal science to-ward forensic science.

Let us conclude by imagining what foren-sic science would look like if it had grown upin the world of conventional basic and appliedscience, as typically found in the departmentsand laboratories of universities and industryand many government agencies. And, there-

fore, what might it look like if in the futureit could be set on a path toward that world,whether by its own initiative, under the incen-tives of courts conscientiously applying Daubertand its progeny, by the restructuring of the wayforensic services are organized (Koppl 2005), orby normal scientists becoming seriously inter-ested in the subject?

The underlying assumptions of forensicidentification would be subjected to intensequestioning and empirical testing. In the courseof this work, researchers would report whatthey tested, how they tested it, and what theyfound. Their results—whether good, bad, orindifferent—would be reported with equal can-dor. Because many of these areas (handwriting,firearms, tiremarks, etc.) had no basic sciencefrom which to draw, these researchers wouldbe building the science that would be the foun-dation of forensic science.

The individualization techniques thatemerged would probably resemble DNAtyping, with its measurement of attributes,sampling of variation in populations, andstatistical bases. Error rates, probability levels,confidence intervals, and so on would benatural parts of what developed. The elementsof subjectivity in forensic examination wouldthemselves be topics of research, the betterto understand their operation and how totame them. These scientists, like most, wouldrecognize problems of experimenter andobserver effects; their own research would useprocedures designed to prevent such effects, aswould the procedures they designed for use incrime labs.

Those doing this research would be doc-toral scientists and their graduate students, as-sisted by novices, some of whom would becomethe scientists and technicians who would laterstaff the crime labs applying the techniques be-ing developed. Incompetence, error, and fraudwould be minimized by the various cultural andorganizational features of the scientific enter-prise that have evolved, though sometimes suchproblems would still arise. Deliberate fraud is acareer-ending act. At the end of the day, errors(whatever their cause) are revealed by the failure

168 Saks · Faigman


of findings to replicate. All this work would pro-vide the science for forensic science.

As the knowledge became ready for “primetime” (and as newer knowledge and devel-opments improved upon older), it would beadopted for casework in crime labs. It is any-one’s guess how crime labs would be organizedin such a world, but they likely would inheritsome of the culture of the conventional sci-entific world along with the knowledge basedrawn from that world. This might includean ethos of candor, a dedication to data aboveall else, and therefore a hostility toward in-competence or fudging or fraud. It might alsoinclude closer and more reciprocal relation-ships with basic researchers, attentiveness to

new findings, and an alertness to problems inneed of further basic and applied research. Be-cause much of the knowledge probably will havebeen grown in universities, there would be morecollege courses in these subjects (indeed, therewould be a larger body of scientific knowledgeto teach, more than techniques and assump-tions). Personnel would arrive at crime labs withmore and better education in relevant subjects.A more highly trained doctoral- and master’s-level cadre of lab directors and managers wouldcome into being and would more readily movebetween the worlds of research and casework.These labs and their personnel would be bothmore welcoming of oversight, review, accredi-tation, and certification and less in need of it.

DISCLOSURE STATEMENT

Some might wonder whether the fact that the authors occasionally testify in hearings where theadmissibility of some particular forensic science is at issue creates a bias. In fact, however, theirresearch and publications have led to requests that they testify about that scholarship, just as it hasled to invitations to speak to audiences of academics, judges, forensic scientists, and governmentaland quasi-governmental bodies, as well as invitations to contribute papers to legal, social science,and forensic science publications.

LITERATURE CITED

Alaska v. Coon, 974 P. 2d 386 (Alaska 1999)Andersen HC. 1959. The Emperor’s New Clothes, transl. E. Blegvad. New York: HarcourtBiasotti A, Murdock J, Moran B. 2008. Firearms and toolmark identification. See Faigman et al. 2008, 4:571–

629Bowers CM. 2008. Identification from bitemarks. See Faigman et al. 2008, 4:649–743Budowle B, Buscaglia J, Schwartz Perlman R. 2006. Review of the scientific basis for friction ridge compar-

isons as a means of identification: committee findings and recommendations. Forensic Sci. Comm. 8(1).http://www.fbi.gov/hq/lab/fsc/backissu/jan2006/research/2006 01 research02.htm

Bunch SG. 2000. Consecutive matching striation criteria: a general critique. J. Forensic Sci. 45:955–63Champod C, Evett IW. 2001. A probabilistic approach to fingerprint evidence. J. Forensic Identif. 51:101–22Cole SA. 2001. Suspect Identities: A History of Fingerprinting and Criminal Identification. Cambridge, MA: Harvard

Univ. PressCole SA. 2005. More than zero: accounting for error in latent fingerprint identification. J. Crim. Law Criminol.

95:985–1078Cole SA. 2007. Forensic science: it’s the testimony stupid. Natl. Law J. 30(Dec. 10):22Cooley C. 2004. Reforming the forensic science community to avert the ultimate injustice. Stanford Law Policy

Rev. 15:399–446Cummins H, Midlo C. 1943 (1961). Finger Prints, Palms and Soles: An Introduction to Dermatoglyphics. New

York: DoverDaubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993)DeForest PR, Gaensslen RE, Lee HC. 1983. Forensic Science: An Introduction to Criminalistics. New York:

McGraw-Hill



Dror IE, Charlton D. 2006. Why experts make errors. J. Forensic Identif. 56:600–15Dror IE, Charlton D, Peron AE. 2006. Contextual information renders experts vulnerable to making erroneous

identifications. Forensic Sci. Int. 156:74–78Dror IE, Rosenthal R. 2008. Meta-analytically quantifying the reliability and biasability of forensic experts.

J. Forensic Sci. In pressEvett IW, Williams RL. 1996. A review of the sixteen points fingerprint standard in England and Wales.

J. Forensic Identif. 46:49–73Faigman DL. 1999. Legal Alchemy: The Use and Misuse of Science in the Law. New York: W.H. FreemanFaigman DL. 2000. The law’s scientific revolution: reflections and ruminations on the law’s use of experts in

year seven of the revolution. Washington Lee Law Rev. 57:661–84Faigman DL. 2006. Judges as “amateur scientists.” Boston Univ. Law Rev. 86:120–25Faigman DL, Kaye DH, Saks MJ, Sanders J. 2008. Modern Scientific Evidence: The Law and Science of Expert

Testimony, Vols. 1–5. St. Paul, MN: West/ThomsonFrisbie T, Garrett G. 1998. Victims of Justice. New York: AvonFrye v. U.S., 293 F. 1013 (D.C. Cir. 1923)Furton KG, Hsu YL, Cole MD. 1999. What educational background is required by crime laboratory directors?

J. Forensic Sci. 44:128–32Giannelli P, Imwinkelried E. 2000. Scientific evidence: the fallout from the Supreme Court’s decision in Kumho

Tire. Crim. Just. 14:12–40Giannelli PC, Imwinkelried EJ. 2007. Scientific Evidence. Newark, NJ: LexisNexis. 4th ed.Haber L, Haber RN. 2003. Error rates for human fingerprint examiners. In Automatic Fingerprint Recognition

Systems, ed. NK Ratha, R Bolle, pp. 339–60. New York: Springer-VerlagHarris J. 1958. How much do people write alike: a study of signatures. J. Crim. Law Criminol. 48:647–51Inman K, Rudin N. 2001. Principles and Practice of Criminalistics. Boca Raton, FL: CRC PressJohnson v. Commonwealth, 12 S.W. 3d 258 (Kentucky 1999)Kam M, Fielding G, Conn R. 1997. Writer identification by professional document examiners. J. Forensic Sci.

42:778–86Koppl R. 2005. How to improve forensic science. Eur. J. Law Econ. 20:255–86Kumho Tire Ltd. v. Carmichael, 526 U.S. 137 (1999)Lentini J. 2008. Fires, arsons, and explosions. See Faigman et al. 2008, 5:57–159Maryland v. Rose, No. K06-0545 (2007). http://www.barlib.org/BryanRoseFingerprintMemDecision2.pdfMoenssens A. 1998. Handwriting identification evidence in the post-Daubert world. U.K. Law Rev. 66:251–343Morange M. 1998. A History of Molecular Biology, transl. Matthew Cobb. Cambridge, MA: Harvard Univ. PressNational Institute of Justice. 2007. Focus Group on Scientific and Forensic Evidence in the Courtroom. Washington,

DC: NIJNational Research Council. 1992. DNA Technology in Forensic Science. Washington, DC: NASNational Research Council. 1996. The Evaluation of Forensic DNA Evidence. Washington, DC: NASOsterburg JW. 1969. The evaluation of physical evidence in criminalistics: subjective or objective process?

J. Crim. Law Criminol. 60:97–101Phillips V, Saks M, Peterson JL. 2001. The application of signal detection theory and decision-making in

forensic science. J. Forensic Sci. 46:294–308Popper K. 1959. The Logic of Scientific Discovery. New York: Basic BooksPretty I, Sweet D. 2001. The scientific basis for human bitemark analyses—a critical review. Sci. Justice 41:85–

92Pyrek KM. 2007. Forensic Science Under Siege: The Challenges of Forensic Laboratories and the Medico-Legal Death

Investigation System. Amsterdam/Boston: AcademicRisinger DM. 2000. Navigating expert reliability: Are criminal standards of certainty being left on the dock?

Albany Law Rev. 64:99–152Risinger DM, Denbeaux MP, Saks MJ. 1989. Exorcism of ignorance as a proxy for rational knowledge: the

case of handwriting identification “expertise.” Univ. Pa. Law Rev. 137:731–92Risinger DM, Saks MJ, Thompson WC, Rosenthal R. 2002. The Daubert/Kumho implications of observer

effects in forensic science: hidden problems of expectation and suggestion. Calif. Law Rev. 90:1–56

170 Saks · Faigman


Rudolph F. 1990 (1962). The American College & University: A History. Athens: Univ. Georgia Press/New York:Knopf

Saks MJ. 1998. Merlin and Solomon: lessons from the law’s formative encounters with forensic identificationscience. Hastings Law J. 49:1069–141

Saks MJ. 2007. Remediating forensic science. Jurimetrics J. 48:119–24Saks MJ, Faigman DL. 2005. Evidence after Daubert. Annu. Rev. Law Soc. Sci. 1:105–30Saks MJ, Koehler JJ. 2005. The coming paradigm shift in forensic identification science. Science 309:892–95Saks MJ, Koehler JJ. 2008. The individualization fallacy in forensic science. Vanderbilt Law Rev. 61:199–219Schwartz A. 2005. Toolmark and firearm identification. In Psychological and Scientific Evidence in Criminal Trials,

ed. J Campbell Moriarty, pp. 12–49. Deerfield, IL: West GroupStacey RB. 2004. Report on the erroneous fingerprint individualization in the Madrid train bombing case.

J. Forensic Identif. 54:706–18State v. Quintana, 103 P.3d 168 (Utah Ct. App. 2004)Stoney DA. 1991. What made us ever think we could individualize using statistics? J. Forensic Sci. Soc. 31:197–99Stoney DA. 2001. Measurement of fingerprint individuality. In Advances in Fingerprint Technology, ed. HC Lee,

RE Gaensslen, pp. 329–30. Boca Raton, FL: CRC Press. 2nd ed.Stoney DA. 2008. Fingerprint identification. See Faigman et al. 2008, 4:275–377Swets JA. 1973. The relative operating characteristic in psychology. Science 182:900–9Thelin JR. 2004. A History of American Higher Education. Baltimore, MD: Johns Hopkins Univ. PressThompson WC, Cole SA. 2006. Psychological aspects of forensic identification evidence. In Expert Psychological

Testimony for the Courts, ed. M Costanzo, Krauss DA, Pezdek K, pp. 31–68. Mahwah, NJ: ErlbaumThornton J. 1975. Criminalistics: past, present, future. Lex et Scientia 11:1–44Thornton J, Peterson J. 2008. The general assumptions and rationale of forensic identification. See Faigman

et al. 2008, 4:1–81U.S. v. Allen, 207 F. Supp. 2d 856 (2002)U.S. v. Cline, 188 F. Supp. 2d 1287 (D. Kan. 2002), aff ‘d, 349 F.3d 1276 (10th Cir. 2003)U.S. v. Fujii, 152 F. Supp. 2d 939 (N.D. Ill. 2000)U.S. v. Green, 405 F. Supp. 2d 104 (D. Mass. 2005)U.S. v. Havvard, 117 F. Supp. 2d 848, 849 (S.D. Ind. 2000), aff’d, 260 F.3d 597 (7th Cir. 2001)U.S. v. Hines, 55 F. Supp. 2d 62 (D. Mass. 1999)U.S. v. Llera-Plaza I, 2002 WL 27305 (2002)U.S. v. Llera Plaza II, 188 F. Supp. 2d 549, 565 (E.D. Pa. 2002)U.S. v. Mitchell, Crim. No. 96-407-1 (E.D. Pa., judgment entered February 2000)U.S. v. Starzecpyzel, 880 F. Supp. 1027 (S.D.N.Y. 1995)U.S. v. Saelee, 162 F. Supp. 2d 1097 (D. Alaska 2001)Wilson JQ. 1989. Bureaucracy: What Government Agencies Do and Why They Do It. New York: Basic BooksZabell SL. 2005. Fingerprint evidence. J. Law Policy 13:143–79


www.annualreviews.org ● Failed Forensics C-1

Figure 1

Postmortem analysis of causes of wrongful conviction in cases of DNA exonerations.

HI-RES-LS04-03-Saks.qxd 07/01/2008 05:42 PM Page C-1

AR359-FM ARI 26 September 2008 20:18

Annual Review ofLaw and SocialScience

Volume 4, 2008Contents

Home Away from Home: Collaborative Research Networks andInterdisciplinary Socio-Legal ScholarshipStuart A. Scheingold � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 1

Conditionality: Forms, Function, and HistorySarah L. Babb and Bruce G. Carruthers � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �13

Organizations, Regulation, and Economic Behavior: RegulatoryDynamics and Forms from the Nineteenth to the Twenty-FirstCenturyMarc Schneiberg and Tim Bartley � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �31

The Political Economy of American Indian GamingStephen Cornell � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �63

After InclusionDevon Carbado, Catherine Fisk, and Mitu Gulati � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �83

Divergent Paths: Conflicting Conceptions of EmploymentDiscrimination in Law and the Social SciencesRobert L. Nelson, Ellen C. Berrey, and Laura Beth Nielsen � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 103

Providing Expert Knowledge in an Adversarial Context: SocialCognitive Science in Employment Discrimination CasesSusan T. Fiske and Eugene Borgida � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 123

Failed Forensics: How Forensic Science Lost Its Way and How ItMight Yet Find ItMichael J. Saks and David L. Faigman � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 149

Convicting the InnocentSamuel R. Gross � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 173

The Psychology of ConfessionsSaul M. Kassin � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 193

Forecasting Methods in Crime and JusticeRichard Berk � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 219

v

AR359-FM ARI 26 September 2008 20:18

Undercover Policing and the Shifting Terms of Scholarly Debate:The United States and Europe in CounterpointJacqueline E. Ross � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 239

Jury Systems Around the WorldValerie P. Hans � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 275

Women in the Legal ProfessionFiona Kay and Elizabeth Gorman � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 299

The Reform of Legal Education in East AsiaSetsuo Miyazawa, Kay-Wah Chan, and Ilhyung Lee � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 333

The Countermajoritarian Difficulty: From Courtsto Congress to Constitutional OrderMark A. Graber � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 361

Toward a New Sociology of Rights: A Genealogy of “Buried Bodies”of Citizenship and Human RightsMargaret R. Somers and Christopher N.J. Roberts � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 385

Indexes

Cumulative Index of Contributing Authors, Volumes 1–4 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 427

Cumulative Index of Chapter Titles, Volumes 1–4 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 429

Errata

An online log of corrections to Annual Review of Law and Social Science articles may befound at http://lawsocsci.annualreviews.org

vi Contents

Documents

Failed Forensics: How Forensic Science Lost Its Way and ... · forensic identiﬁcation science is not really science at all. No Basic-Science Origins Although forensic science is