New paradigms in translational science researchin cancer biomarkers

PAUL D. WAGNER, and SUDHIR SRIVASTAVA

ROCKVILLE, MD

From the Cancer Biomarkers Res

Prevention, National Cancer Institu

Submitted for publication Decem

January 9, 2012; accepted for publ

Reprint requests: Sudhir Srivastav

search Group, Division of Cancer P

tute, 6130 Executive Boulevard, S

MD 20852; e-mail: srivasts@mail.

1931-5244/$ - see front matter

Published by Mosby, Inc.

doi:10.1016/j.trsl.2012.01.015

Despite significant investments in basic science by the US National Institutes ofHealth, there is a concern that the return on this investment has been limited in termsof clinical utility. In the field of biomarkers, translational research is used to bridgethe gap between the results of basic research that identify biomolecules involvedin or the consequence of carcinogenesis and their incorporation into medical ap-plication. The cultural separation between different scientific disciplines oftenmakes it difficult to establish the multidisciplinary and multi-skilled teams that arenecessary for successful translational research. The field of biomarker research re-quires extensive interactions between academic researchers and industrial devel-opers, and clinicians are needed to help shape the research direction that can beaddressed only by a multidisciplinary, multi-institutional approach. In this article, weprovide our perspective on the relatively slow pace of cancer biomarker transla-tion, especially those for early detection and screening. (Translational Research2012;159:343–353)

Abbreviations: AFP ¼ alpha-fetoprotein; BDL ¼ Biomarker Developmental Laboratory; BRL ¼Biomarker Reference Laboratory; CVC ¼ Clinical Validation Center; DMCC ¼ Data Manage-ment and Coordinating Center; EDRN ¼ Early Detection Research Network; FDA ¼ Food andDrug Administration; NCI ¼ National Cancer Institute; NIH ¼ National Institutes of Health;PPV ¼ positive predictive value; PSA ¼ prostate-specific antigen

D espite significant investments in basic scienceby the US National Institutes of Health(NIH), there is a concern that the return on

this investment has been limited in terms of clinical util-ity. It is frequently stated that translational research isa missing component between basic science and clinicalapplication. Numerous definitions exist for translationalresearch, and to make it relevant to this commentary, wedefine translational research as a discipline that makes

earch Group, Division of Cancer

te, Rockville, Md.

ber 4, 2011; revision submitted

ication January 13, 2012.

a, Chief, Cancer Biomarkers Re-

revention, National Cancer Insti-

uite 3142, MSC 7362, Rockville,

nih.gov.

the results of basic research applicable to clinical use.The term ‘‘translational medicine’’ has been applied toa research approach that seeks to move ‘‘from benchto bedside’’ or from laboratory experiments into clinicaltrials to actual point-of-care patient applications. Out-side of the medical domain, the term ‘‘translationalresearch’’ has been applied more generally where re-searchers seek to shorten the time-frame of the basicto applied continuum to translate fundamental researchresults into practical applications. In the field of bio-markers, translational research is used to bridge thegap between the results of basic research that identifybiomolecules involved in or the consequence of carci-nogenesis and their incorporation into medical applica-tion. The cultural separation between different scientificdisciplines often makes it difficult to establish the mul-tidisciplinary and multi-skilled teams that are necessaryfor successful translational research. The field of bio-marker research requires extensive interactions betweenacademic researchers and industrial developers, and cli-nicians are needed to help shape the research direction

343

Translational Research344 Wagner and Srivastava April 2012

that can only be addressed by a multidisciplinary, multi-institutional approach.

TRANSLATIONAL SCIENCE AND BIOMARKERRESEARCH

In this article, we provide our perspective on the rel-atively slow pace of cancer biomarker translation, espe-cially those biomarkers for use in early detection andscreening. We are members of the US National CancerInstitute’s (NCI) Cancer Biomarkers Research Groupand administer the NCI’s Early Detection Research Net-work (EDRN). The mission of this network is to dis-cover and validate biomarkers for the detection ofearly-stage cancers and to assess risk. We will also dis-cuss how the EDRN is working to accelerate the pacetranslation of biomarkers from discovery into clinicalapplication.A cancer biomarker refers to a substance or process

that is indicative of the presence of cancer in the body.It might be a molecule secreted by a malignancy itself,or it can be a specific response of the body to thepresence of cancer. The NIH Biomarkers DefinitionWorking Group1 provided a formalized definition ofbiomarker as cellular, biochemical, and molecular alter-ations by which a normal, abnormal, or simply biologicprocess can be recognized or monitored and are used toobjectively measure and evaluate normal biologicalprocesses, pathogenic processes, or pharmacologic re-sponses to a therapeutic intervention. The majority ofcancer biomarkers are measured in the tumor or inblood. To maximize usefulness and minimize cost forscreening or early detection, it is advantageous thata biomarker be measurable in a body fluid that can beobtained using minimally invasive methods, such asblood, urine, sputum or stool. Prostate-specific antigen(PSA) is an example of a blood-based protein markerfor prostate cancer, methylated vimentin is a stool-based DNA biomarker for colorectal cancer, and DNAfluorescence in situ hybridization assays are urine-based biomarkers for bladder cancer. The Papanicolaoutest is a cell-based biomarker for cervical cancer that hascontributed significantly to the 99% reduction in deathsdue to cervical cancer in the United States. Annualscreening using the fecal occult blood test reduces coloncancer mortality by 15% to 33%.Discovering cancer biomarkers is a relatively easy

process as judged by the number of articles published ev-ery year on this subject. A PubMed search for cancer andbiomarkers shows more than 15,000 publications in2010, and 1100 publications when limited to cancerbiomarkers and early detection. However, translatingthese discoveries into a useful clinical assay is difficult.To date, less than 30 cancer biomarkers have been ap-proved by the Food and Drug Administration (FDA),

andmost of these are formonitoring response to therapy.A number of explanations have been given for the rela-tively few cancer biomarkers being moved into clinicaluse. These include the high performance characteristicsneeded to a make a biomarker clinically useful, the biol-ogy of tumors, a flawed discovery process, a validationprocess that is cumbersome and expensive, regulatoryrequirements, and an academic system that does not re-ward translational research. Issues of credit, publicationpriority, and patent credit have also slowed progress intovalidation. Indeed, much of the biomarker research re-mains stuck at the discovery phase, and unfortunately,some investigators seem content to reiterate the discov-ery process, never proceeding beyond that point.Successful biomarker research requires a knowledge-

driven environment, in which investigators generate,contribute, manage, and analyze data from a variety ofsources and technology platforms. It is only by integrat-ing these diverse data types that the complex and under-lying causes of illness can be revealed and effectivestrategies for prevention, early detection and personal-ized treatments be realized. The translation of bio-marker discovery into clinical application is aniterative process with multiple feedback loops, that is,results from the verification of a biomarker’s perfor-mance can be used to inform additional discovery ef-forts. Collaboration, data sharing, data integration, andstandards are integral. Successful translational researchalso requires that information and data also flow fromhospitals, clinics, and participants of studies in an orga-nized and structured format to repositories and research-based facilities and laboratories.The scale, scope, and multidisciplinary approach of

translational research requires a new level of operation’smanagement capabilities within and across studies, re-positories, and laboratories. The increased operationalrequirements of these large studies, with large annotatedspecimen collections, large complex data sets, and gov-ernment regulations, necessitate an informatics ap-proach that enables the integration of both operationalcapabilities and clinical and basic data. Most informat-ics systems in use today are inadequate to handle thetasks of complicated operations and contextually indata management and analysis.

BUSINESS MODEL FOR TRANSLATIONAL RESEARCH INBIOMARKERS

Why do ‘‘big science’’ projects that have such greatpromise often take decades from concept to fruition(if they make it to practice at all)? A likely contributoris that the health care industry, while accounting formore than 13% of the US gross domestic product andgrowing at triple the rate of inflation, remains a frag-mented industry. Perhaps health care can learn from

Translational ResearchVolume 159, Number 4 Wagner and Srivastava 345

computer electronics, where ‘‘big science’’ is alwayspart of the equation, but so is a business model thatdrives the translation from ‘‘art to part.’’ Part of the prob-lem is the difference in the business models that drivethe respective systems.Health care organizations have generally grown or-

ganically, which typically results in structures that areorganized along functional ‘‘silos,’’ areas of expertisewhere depth of knowledge in one particular area is crit-ical. Although such a horizontal structure fosters excel-lent solutions for primary scientific problems, it oftengenerates barriers if knowledge must be shared be-tween silos. In contrast, computer chip manufacturersare organized in a more vertical structure. In this struc-ture, formal hand-off procedures have been designed toensure that discoveries in one aspect of chip design andconstruction are rapidly and efficiently conveyed toothers who require the information. This allows forrapid vetting of ideas, quickly culling out poor con-cepts, and fostering the rapid acceptance of goodconcepts.In a vertical design, there are a number of focused

experts within in a single organization, generatingeconomies of scale for the rapid discovery and devel-opment, and there is a focus on coordination of multi-ple entities, using shared resources and emphasizinghand-offs between entities. This is in contrast witha horizontal approach, which may result in rapiddiscoveries but limited advancement toward a usefuldeployed product. A horizontal design may also in-crease duplication and reduce potential synergiesacross disciplines.When deciding whether to adopt a horizontal or ver-

tical model, one must consider the influence and in-terests of constituents that can drive or hinder theprocess. The NCI’s EDRN has developed methods, pol-icies, and procedures for relating with each of the majorconstituent groups. The EDRN promotes a vertical ap-proach for conducting biomarker research, wherebybiomarkers are developed, refined, and analyticallyand clinically and validated all within one organization(Fig 1). A critical aspect of the EDRN is its focus on co-ordinating multiple resources with a goal of minimizingbarriers to the rapid and efficient ‘‘hand-off’’ betweenentities. One method used for achieving this is a struc-tured set of criteria for assessing the roles and clinicalsignificance of each newly discovered biomarker, alongwith criteria and strategies for judging the use of bio-markers in relationship to one another (Fig 2).

THE EARLY DETECTION RESEARCH NETWORK ASA MODEL FOR BIOMARKER RESEARCH

The NCI’s EDRN was formed to facilitate the dis-covery, development, and validation of biomarkers

for early cancer detection and risk assessment(http://edrn.nci.nih.gov/). This network includes ex-perts from academia, government, and industry andhas the goal of translating research discoveriesinto substantial benefits that can be implemented inthe clinic or the population setting. The EDRN isa vertically integrated network composed of 4 maincomponents.

� Biomarker Developmental Laboratories (BDLs)discover, develop, and characterize new biomarkersor refine existing biomarkers.

� Biomarker Reference Laboratories (BRLs) serve asnetwork resources for analytic and clinical valida-tion of biomarkers, including technologic develop-ment and refinement.

� Clinical Validation Centers (CVCs) conduct andsupport biomarker validation trials, provide high-quality biological specimens to other EDRN in-vestigators for use in biomarker discovery, andcontribute biospecimens for the formation ofEDRN reference sets (common sets of specimensfromwell-characterized andmatched cases and con-trols from specific disease spectra).

� The Data Management and Coordinating Center(DMCC) provides network coordination, data man-agement, and protocol development in support ofEDRN biomarker validation trials and conducts the-oretic and applied statistical research related to thegoals of the EDRN.

This collaborative arrangement allows for the rapidmovement of newly discovered or refined biomarkersfrom the laboratory into clinical validation. Whena BDL has obtained promising data on a biomarker orpanel of biomarkers, it works with a BRL to refine theassay and independently reproduce the results usingblinded specimens; these specimens may come froma CVC. If the assay is reproducible and has sufficientsensitivity and specificity, the BDL works witha CVC, the BRL, and the DMCC to conduct a multi-site biomarker validation trial. Investigators outsidethe EDRN who have promising biomarkers can accessthe EDRN infrastructure to validate their biomarkersor participate in an ongoing validation trial.Other goals of the EDRN are to establish scientific

criteria to evaluate biomarkers as indicators of earlycancer, prognostic factors of pre-invasive cancer,markers of risk, and surrogate end points. The EDRNhas developed a quality assurance program for bio-marker testing and evaluation.The EDRN follows an adaptive model in which all of

the players (stakeholders), investors, policy makers, andtechnology developers work coordinately toward thegoal of developing biomarkers for cancer diagnostics

Fig 1. Vertical approach business model followed by the EDRN (adapted from EDRN 4th Report, January 2008,

NIH Publication Number 07-6135). (Color version of figure is available online.)

Translational Research346 Wagner and Srivastava April 2012

that improve health and decrease mortality. The EDRNhas the requisite expertise and resources to create anevidence-based biomarker pipeline. There is no similarcollaborative or cooperative group that is capable ofconducting research projects from basic discovery toclinical validation. Without the pressure of venture cap-ital, ‘‘best science’’ rather than ‘‘best business’’ practicepermits development of novel concepts that otherwise,because of commercial needs, might never reach fru-ition.

WHY ARE MODELS LIKE THE EDRN NEEDED?

There is a perception that translational research re-ceives less support from the NIH than does basic re-search. The NCI’s Translational Research WorkingGroup (http://www.cancer.gov/researchandfunding/trwg)reported that in fiscal year 2004, $0.9 to $1.3 billionof the NCI’s $4.4 billion research budget fit the broadinclusion criteria for translational research. Despitethis substantial commitment, some investigators believethat translational research is of lower priority than basicor clinical research or that it is somehow less presti-gious, that is, results from translational research arepublished in journals with a lower impact. The very na-ture of translational research is to a certain extent atodds with the current system of academic advancement,which favors the independent investigator. Translationalresearch requires interdisciplinary research teams andthe involvement of multiple departments or multiple in-stitutions, and the time required to set up and conduct

the studies, especially if they involve a prospective trial,is long with few publications while the study is ongoing.

ROLE OF INDUSTRY

Industry plays a unique role in bringing the end-stageproducts of the Nation’s investment in research throughproduct development and into clinical use. Although ac-ademic scientists and clinicians are well suited for thediscovery, development, and initial validation of cancerbiomarkers, the final steps of obtaining FDA clearanceor Clinical Laboratory Improvement Amendment ap-proval require an industry involvement. It will be theprivate company that performs the assays on patientspecimens or sells the in vitro diagnostic. Thus, it is im-portant for academic investigators to interact with theirindustrial counterparts, preferably at an early stage ofdevelopment. The EDRN holds biannual industrial fo-rums that bring together academic investigators and in-dustrial representatives interested in commercializingcancer biomarkers.Kalorama Information reported that cancer diagnos-

tics sales totaled $4.1 billion in 2004 and projectedthat the worldwide market for in vitro diagnostic testsfor cancer by the end of 2012 will be $8 billion, an an-nual growth rate of approximately 11% (http://www.healthimaging.com/index.php?option5com_articles&view5article&id510094). The worldwide market forcancer in vitro diagnostic tests is comparable to thatfor cardiac diseases and approximately one third thatfor infectious diseases.

Fig 2. EDRN decision criteria for judging relationships and strategies

in biomarker development and fruition (adapted from EDRN 4th

Report, January 2008, NIH Publication Number 07-6135). (Color ver-

sion of figure is available online.)

Translational ResearchVolume 159, Number 4 Wagner and Srivastava 347

CHOOSING THE ‘‘WINNERS’’ AND ELIMINATING THE‘‘LOSERS’’ AMONG BIOMARKERS

Despite substantial advances in our understanding ofthe molecular basis of cancer, there continues to bea paucity of biomarkers for clinical use. Although bio-markers in biological fluids have great potential tohelp determine the risk of disease or to allow early de-tection, the rate of success in this area has been disap-pointing. To increase the likelihood of success, it isimperative for us to learn from those biomarkers thatfailed to progress or whose performance did not validatewhen analyzed in independent specimens. There hasbeen some criticism of the experimental approacheswith which biomarkers studies have been conducted.2

These studies have resulted in the thousands of publica-tions describing promising candidates, but few of thesecandidates have been pursued to support their clinicalutility, and most of those that have been pursued havefailed in subsequent validation studies. A case in point

is a recent validation study that encompassed 28 prom-ising candidate protein biomarkers for ovarian cancer.When assayed in prediagnostic serum specimens fromovarian cancer cases and controls from the NCI’s Pros-tate, Lung, Colorectal, and Ovarian Cancer screeningtrial, none of the individual markers exhibited perfor-mance characteristics that equaled those of CA-125,the current best marker for ovarian cancer,3 and theircombination into a panel did not outperform CA-125.This study has been cited as evidence that somethingis wrong in our approach to biomarkers. Rather than dis-missing this as failed experiment, the communityshould examine why the performances of these bio-markers did not hold up and use this information toguide future discovery.The lack of clinically useful biomarkers may be partly

attributed to inadequate resources and incentives to de-velop and maintain collaborative teams. As a result,studies are mostly conducted by a single laboratorywith limited means to accomplish objectives. Such stud-ies generally consist of comparisons using a limitednumber of convenience specimens, leading to the dis-covery of candidate biomarkers, which are not general-izable and need to be further pursued to determine theirrelevance. In retrospect, most such studies have not beensolidly designed to minimize the biological and molec-ular heterogeneity of cancer. Compounding this prob-lem is a considerable shortage of quality specimensfor discovery and validation studies that overcome thebiases inherent in retrospective samples. Unfortunately,biomarker discovery is too often performed using poor-quality or poorly matched specimens, resulting in theidentification of biomarkers that result from differencesin specimen handling or storage or differences in thesubjects other than cancer.The size of early-stage tumors and pre-neoplastic le-

sions and the heterogeneity of cancers contribute to thedifficulty of identify biomarkers for early cancer detec-tion or screening. Effective screening and early detec-tion biomarkers should be measurable in bodily fluidsthat can be obtained using minimally invasive tech-niques (eg, blood, urine, or stool). This requires thatthe biomarker be present at a sufficiently high concen-tration that it can be measured in these fluids. The inher-ent difficulty is that tumors, especially at early stages,are small, and consequently the amount of protein orother type of biomarker shed into the blood is low, mak-ing them difficult to detect. Indeed, the only FDA-approved blood-based biomarkers for screening arePSA and complex PSA for prostate cancer. However,the use of PSA for screening is a subject of debate,and the US Preventive Services Task Force recently is-sued a draft recommendation against using the PSA testto screen healthy men for prostate cancer, primarily

Translational Research348 Wagner and Srivastava April 2012

because this test, although detecting more cases of pros-tate cancer, resulted in little or no reduction in prostatecancer–specific mortality and harms associated withsubsequent evaluation and treatments. A number of in-vestigators are using more proximal fluids as a sourcefor biomarkers, for example, urine for prostate cancer,stool for colorectal cancer, and sputum for lung cancer.The FDA-approved protein marker NMP22 and fluores-cence in situ hybridization assays for chromosomes 3, 7,9, and 17 are urine-based biomarkers for screening forbladder cancer.Because most of the major epithelial cancers, includ-

ing lung, colorectal, breast, prostate, and ovarian, areheterogeneous, it is unlikely that a single biomarkerwill detect all tumors from a particular organ. Conse-quently, many investigators are attempting to developbiomarker panels to increase sensitivity. However, de-veloping biomarker panels requires balancing increasedsensitivity with decreasing specificity. Simply addingone biomarker to another may increase the sensitivityof the assay, but it is likely to decrease the specificity(each marker contributes to the number of false posi-tives), and thus one typically has to alter cutoff valuesof the individual biomarkers to maximize performanceor develop complex algorithms to interpret the results.The difficulty with simply lowering cutoff values ofthe individual markers in the panel to increase specific-ity is that this also lowers sensitivity.

IT TAKES A VILLAGE…

As pointed out by Samir Hanash, an EDRN investiga-tor,4 ‘‘It will ‘take a village’ to implement a paradigmshift in our approach to biomarkers, necessitating a part-nership among the various stakeholders, from academiawith multi-disciplinary contributions, to philanthropy,government and industry.… Initiatives aimed at contrib-uting to a paradigm shift in our approach to the deve-lopment of biomarkers are burgeoning. They areexemplified by the increased availability of biospeci-mens through cohort studies with reduced bias. Theyare also exemplified by consortia to assess and standard-ize technologies for discovery and assays of biomarkercandidates.’’

GUIDELINES FOR DEVELOPING AND VALIDATINGBIOMARKERS

The EDRN has developed a 5-phase approach for bio-marker development that provides a systematic ap-proach to discovery, development, and validation andcan be used to help identify ‘‘winners’’ or ‘‘losers’’among biomarkers.5 This phased approach has beenwidely accepted by the biomarker research community.

� Phase 1, discovery: Exploratory studies to identifypotentially useful biomarkers.

The majority of biomarkers do not progress beyondthis phase. Reasons for this include modest differencesin the concentration of the biomarker in cases comparedwith controls and large variability in the levels of thebiomarker in healthy subjects.

� Phase 2, clinical assay and validation: Studies to de-termine the capacity of biomarkers to distinguish be-tween peoplewith cancer from thosewithout cancer.

Specimens used in these studies are from patientswith cancer (cases), healthy subjects (controls), and pa-tients with confounding conditions (eg, men with be-nign prostate hyperplasia when evaluating biomarkersfor prostate cancer).Most biomarkers do not progress beyond this phase

primarily because the validation study shows that thebiomarker does not have sufficient sensitivity or speci-ficity to be clinically useful (see below).

� Phase 3, retrospective longitudinal: Determine howwell biomarkers detect preclinical disease by testingthe markers against specimens collected longitudi-nally from research cohorts.

These studies are usually retrospective and use spec-imens collected from apparently healthy individualswho were monitored for the development of cancer.However, phase 3 studies do not establish the stage ofthe disease when the biomarker is first detected.

� Phase 4, prospective screening: Identify the extentand characteristics of disease detected by the testand determine the false referral rate.

Asymptomatic people are screened, and those witha positive result are followed up to determine if theyhave cancer and if so its stage.

� Phase 5, cancer control: Evaluate both the role of thebiomarkers for detection of cancer and the overallimpact of screening on the population throughlarge-scale population studies.

These studies are designed to determine whether thescreen results in a reduction in morbidity and mortality.For a biomarker to have clinical utility (the likelihood

that the test will lead to an improved health outcome), itmust achieve a certain level of performance in its abilityto distinguish patients with cancer from those withoutcancer, to determine prognosis, or to predict responseto treatment. The performance required for a biomarkerdepends on the cancer and intended use. For example,biomarkers used to detect cancers with a high preva-lence may be clinically useful with lower specificity

Translational ResearchVolume 159, Number 4 Wagner and Srivastava 349

than those used to detect cancers with a low prevalence.The sensitivity of a biomarker is the proportion of indi-viduals with cancer who test positive for the biomarker,and specificity is the proportion of individuals withoutcancer who test negative for the biomarker. The lowera biomarker’s sensitivity the more people with cancerwill not be detected, and the lower the specificity themore people without cancer will be incorrectly identi-fied as having cancer. Sensitivity is also referred to asthe true-positive rate. The false-positive rate is the prob-ability that a subject without cancer has a positive testresult. None of the currently used biomarkers are100% sensitive and specific. For example, PSA ata cut-point of 4 ng/mL has a sensitivity of 70% to80% and a specificity of approximately 60%, that is,20% to 30% of men with cancer will be missed and40% of men without cancer will undergo needless biop-sies. Lowering the PSA cut-point will increase the sen-sitivity, resulting in fewer missed cancers but will lowerthe specificity, resulting in more needless biopsies.Thus, when choosing where to set the cut-point, it isnecessary to balance the impact of the number of missedcancers with the impact of the number of false positivesand the relative harms of each. Does a positive bio-marker test result trigger an imaging procedure, a bi-opsy, or surgery? Two other criteria by which to judgea biomarker’s performance are positive predictive value(PPV) and negative predictive value. PPV is the chancethat a person with a positive test has cancer and is de-pendent on the prevalence of the cancer. For a given sen-sitivity and specificity, the higher the prevalence, thehigher the PPV. If a cancer’s prevalence is 1% and thebiomarker has 100% sensitivity and 95% specificity, 1of 6 people with a positive test will actually have cancer(PPV 5 0.17). However, even when a biomarker hashigh specificity and sensitivity, it may not be usefulfor screening if the cancer has low prevalence. The prev-alence of pancreatic cancer is approximately 1 in 10,000people at age 50 years (0.01%) and 6 in 10,000 people atage 70 years (0.06%). To achieve 10% PPV (1 true pos-itive for every 9 false-negatives), a biomarker wouldneed to have 100% sensitivity and 99.9% specificityto be used to screen 50-year-olds for pancreatic cancerand 100% sensitivity and 99.4% specificity to be usedto screen 70-year-olds for pancreatic cancer. Likewise,a biomarker would need to have 100% sensitivity and99.6% specificity to be used to screen average-riskwomen for ovarian cancer.The EDRN focuses on phases 1 to 3. The NCI’s

Translational Research Working Group has also pub-lished a developmental pathway for biospecimen-based assays that overlaps with phases 2 to 4 and isfocused on enhancing the efficiency and effectivenessof the process.6

DISCOVERY PROCESS

Successful translation of biomarkers into clinicalapplication depends on the biomarkers entering intothe validation pipeline. No matter how well thoughtout and rigorous the translation process, if the bio-markers lack the requisite performance characteristics,they will not be successfully translated into clinicalapplication. Unfortunately, most of the biomarkers re-ported in the literature have insufficient sensitivity andspecificity or lack data using appropriate specimens,for example, late-stage cancers or mismatched casesand controls. Thus, the first step to enhancing the trans-lation of biomarkers is to improve the discovery process.Although a thorough discussion of shortcomings andproblems in biomarker discovery and potential solutionsis beyond the scope of this review, we list some of the re-current problems with study design and biospecimens.

STUDY DESIGN

� Biomarker discovery is undertaken without consid-eration of potential clinical use.

� Specimens used come from inappropriate kinds oftissues or stages of cancer. If interested in early de-tection, perform discovery using early-stage cancersor body fluids from patients with early-stage diseaseor, if possible, work with pre-neoplastic lesions orprediagnostic sera/plasma. If interested in detectingaggressive cancer (eg, prostate cancer), use speci-mens from patients with aggressive cancer.

� Failure to account for confounding conditions. Bio-marker levels are altered by factors other than cancer,for example, PSA is increased in benign prostatic hy-perplasia and in prostate cancer, and CA19-19 is ele-vated in pancreatitis and pancreatic cancer.

� Failure to account for lack of tissue specificity.Other tissues produce the biomarker resulting inhigh biomarker levels in healthy patients.

� Classifier results from overfitting. This occurs inbiomarker discovery when models are used that in-volve a large number of variables (eg, mass spec-trometry peaks) measured on a small number ofsamples. This can result in a statistical model thatdescribes random error or noise instead of the under-lying relationship. Completely independent trainingand test sets of specimens should be used to rule outthe possibility that the observed classification is a re-sult of overfitting.

BIOSPECIMENS

� Useof poorlymatched cases and controls (differencesin age, sex, confounding conditions, diet, ethnicity,

Translational Research350 Wagner and Srivastava April 2012

and exercise) can result in biomarkers that reflectthese differences and not the presence of cancer.

� Artifacts due to sample collections, processingtimes, and storage can result in biomarkers that re-flect these differences and not the presence of cancer.

� Cases and controls collected under different condi-tions: controls collected during a general check-upand cases collected immediately before surgery.These 2 different collection conditions can resultin the discovery of biomarkers for stress ratherthan for disease.

Whenever possible, discovery should be performedusing high-quality specimens from carefully matchedcases and controls that are collected under the same con-ditions using a standard operating procedure. It has beensuggested that discovery should be performed usingspecimens collected from asymptomatic patients wholater develop cancer. Several investigators within theEDRN are basing their discovery efforts on prediagnos-tic sera or plasma. However, because there are few col-lections of prediagnostic sera, it is unclear how widelythis approach can be used.Another approach the EDRN is taking to develop

panels of biomarkers or to combine biomarkers devel-oped by different investigators is to support collabora-tive team projects. EDRN investigators combineresources to evaluate their biomarkers using a commonset of well-characterized, high-quality biospecimens.This allows for a direct comparison of the performanceof the individual biomarkers and the possibility of com-bining them to form a more effective panel.There a number of reviews and commentaries that

discuss in detail the problems associated with study de-sign and biospecimen collection.7-12 The EDRN bringstogether investigators from discovery laboratories,clinicians, and biostatisticians to discuss issues relatedto clinical needs and study design and to shareresources, such as high-quality biospecimens that arecollected to address a specific clinical question. AnNCI sponsored workshop Development of BiospecimenReporting Criteria for Publication concluded that ‘‘hu-man biospecimens are subject to a number of differentcollection, processing, and storage factors that can sig-nificantly alter their molecular composition and consis-tency’’ and recommended guidelines for reporting dataelements for human biospecimens used in biomedicalstudies, the Biospecimen Reporting for Improved StudyQuality.13 These reporting guidelines are intended to al-low other researchers to better evaluate and reproduceexperimental results.Development of the Reporting Recommendations for

Tumor Marker Prognostic Studies14 was a major recom-mendation of the NCI–European Organization for

Research and Treatment of Cancer First InternationalMeeting on Cancer Diagnostics. These reporting guide-lines are intended to address the all too commonproblem of initial promising reports on prognostic bio-markers not being reproduced in subsequent studies.Guidelines are provided for reporting study design, pre-planned hypotheses, patient and specimen characteris-tics, assay methods, and statistical analysis methods.This information should help other investigators to bet-ter understand the reported results. Reporting Recom-mendations for Tumor Marker Prognostic Studiesguidelines did not make specific recommendation onstudy design or analysis methods. Although these guide-lines were developed for prognostic biomarkers, manyof the reporting guidelines are also applicable to screen-ing and diagnostic biomarkers.

VALIDATION PROCESS

When there are sufficient preliminary data to demon-strate the potential clinical use of a biomarker, the pro-cess of translation or validation begins. This process canbe divided into 2 steps: First a prevalidation or verifica-tion study is performed to determine if the biomarker isworth pursuing, followed by a validation trial; the latteroften requires a prospective multi-site trial. In the past,validation trials have been performed on biomarkersthat had not gone through a verification step, frequentlyresulting in a disappointing or ambiguous outcome. TheEDRN requires verification, preferably performed by anindependent laboratory, before conducting a biomarkervalidation trial. The process the EDRN follows for ver-ification and validation is described in detail in the fol-lowing sections.Verification of a biomarker or panel of markers in-

volves both analytic validation of the assay and in-dependent evaluation of its clinical performance.Analytic validation or assay validation includes optimi-zation and determination of the accuracy, reproducibil-ity, and reliability of the assay for the intendedapplication. Because most investigators conducting bio-marker discovery have little experience in this area, theinvolvement of experts in assay development is recom-mended. Even if the discoverer has the expertise, it ishighly recommended that the assay be reproduced ina completely independent laboratory before initiatinga biomarker validation trial. The EDRN BRLs are ex-perts in assay development and meet the requirementsof the Clinical Laboratory Improvement Amendmentfor testing patient specimens. These laboratories workwith the discoverers to optimize and independently val-idate the assays. Equally important is the verification ofthe clinical performance of the biomarker. This can beachieved by performing the assay on a new set of

Table I. Verification of colon cancer biomarkers

using EDRN colon cancer reference set

Biomarker Sensitivity Specificity

Galectin-3 ligand 72% 80%K-ras in urine 77% 65%K-ras on FOBT card 14% 65%GOS 77% 49%GOS 1 FOBT 27% 95%Proteomics-A 78% 88%Proteomics-B 70% 66%Proteomics-SELDI-TOF 19% 98%Proteomics-MALDI-TOF 63% 48%p53 40% 70%CEA 40% 70%Topoisomerase II 35% 70%Cathepsin D 50% 70%Cyclin B 40% 70%IGF binding protein 2 35% 70%TRAILR2 10% 96%CIN248 12% 92%P108 27% 94%TRAILR2 and CIN248 and P108 29% 60%FOBT TRAILR2 and CIN248 and P108 42% 97%

Abbreviations:CEA, carcinoembryonic antigen; FOBT, fecal occultblood test; GOS, galactose oxidase-Schiff test; IGF, insulin-likegrowth factor; MALDI-TOF, matrix-assisted laser desorption/ioniza-tion–time of flight; SELDI-TOF, surface-enhanced laser desorption/ionization–time of flight.

Translational ResearchVolume 159, Number 4 Wagner and Srivastava 351

specimens for which the investigators are blinded. Ifpossible, it is recommended that the assays be conduct-ed by an independent laboratory using specimens froman independent source.Within the EDRN, the BRLs canact as an independent laboratory to perform the assaysand the CVCs can often supply the needed blindedspecimens.To help expedite the verification process, the EDRN

has established reference sets for many of the major ep-ithelial cancers, including breast, lung, colon, prostate,pancreas, and liver. These sets of specimens are fromwell-characterized and matched cases and controls.The specimens are collected using standard operatingprocedures, and common data elements are collectedon all patients. For example, the EDRN colon cancerreference set consists of multiple replicate aliquots ofsera, plasma, and urine from 50 subjects with colorectaladenocarcinoma, 50 subjects with adenomas confirmedby pathology, and 50 subjects with normal colons aftercolonoscopy. Any investigator with a promising bio-marker or panel of biomarkers can apply to the EDRNto receive these reference sets to verify the clinical per-formance of their markers on blinded specimens (http://edrn.nci.nih.gov/resources/sample-reference-sets). Thesereference sets also allow for the comparison of individ-ual candidate biomarkers from different investigators onthe same samples and for the complementary or additiveperformance of these biomarkers to be assessed. The re-sults of biomarkers or panel of biomarkers in the coloncancer reference set are shown in Table I.Results from the prevalidation/verification studies

should be used to determine whether the biomarker orpanel of biomarkers has sufficient performance to war-rant a full validation trial to determine clinical validity:Does the biomarker accurately and reliably distinguishthose with cancer from those without? Most biomarkersevaluated in a prevalidation study do not progress toa validation trial, primarily because of lack of perfor-mance when assayed using independently collectedspecimens that represent the broad spectrum of the dis-ease and its confounding conditions (eg, benign pros-tatic hyperplasia for prostate cancer).A biomarker validation trial should be undertaken

only after the assay has been analytically validatedand the clinical performance of the biomarker hasbeen verified using an independent set of blinded spec-imens. This trial may require a prospective specimencollection, or if appropriate specimens exist, it can beretrospective. Whether the trial is retrospective or pro-spective, the study must be well designed, conductedfollowing a standard operating procedure, and suffi-ciently powered.EDRN investigators at the DMCC have developed

a study design for phase 2/3 biomarker validation trials.13

They have termed this the ‘‘PRoBE design forProspective-specimen-collection-Retrospective-Blinded-Evaluation.’’ A key feature of this design is the avoidanceof bias by collecting all specimens before diagnosis, suchthat cases and controls are enrolled under the same con-ditions and all specimens are collected and processedidentically. This design can require large numbers of sub-jects because the incidence of most cancers are low; mostof those enrolled will not have cancer. The design has 4main components:

1) Clinical context and population

� Define the clinical context in which the bio-

marker will be used.� Define the clinically relevant outcome.

2) Biomarker performance criteria

� Define what the biomarker will do.� Define discriminatory accuracy and minimally

acceptable performance.� Compare with existing biomarker if one exists

or determine if it is additive.

3) Biomarker test

� Biomarker or panel of biomarkers are fixed,based on discovery and verification studies.

� Define protocol for specimen collection, pro-cessing, and storage.

� Define assay and fix procedure.� Assay performed blinded to outcome.

Translational Research352 Wagner and Srivastava April 2012

4) Study size

� Sample size based onminimally acceptable per-

formancevalues and the performancevalues ob-tained from discovery and verification studies.

These4 componentsof thePRoBEdesignare describedin detail by Pepe and colleagues,15 and the EDRN has be-gun using this as the basis for the design of its biomarkervalidation trials and reference set collections.When the EDRN conducts a prospective biomarker

validation trial, it collects multiple aliquots of seraand plasma and, if applicable, urine from each subject.In some studies, lymphocyte DNA is also collected. Forexample, the EDRN conducted a phase 2 biomarker val-idation for the early diagnosis of hepatocellular carci-noma; the biomarkers tested were alpha-fetoprotein(AFP), the currently used biomarker, AFP-L3%, a spe-cific glycoform of AFP, and des-gamma carboxypro-thrombin.16 A total of 424 cirrhotic controls and 422cases of hepatocellular carcinoma (208 early-stage can-cers) were enrolled. Serum and plasma were collectedand stored from all patients, and genomic DNAwas col-lected from the majority of the patients. After comple-tion of the trial, the samples were divided into 2 sets;a prevalidation reference set composed of 50 early-stage cases and 50 cirrhotic controls, and a validationset composed of 372 cases, 158 of which are early stage,and 374 cirrhotic controls. These specimens are avail-able to any investigator with promising biomarkers.The reference set is provided to investigators whohave candidate biomarkers, with preliminary data indi-cating that the markers perform as well as or comple-ment AFP. If the biomarker performs as well as orcomplements AFP, AFP-L3%, or des-gamma carboxy-prothrombin in the reference set, the validation set ismade available to the investigator. The investigatorsare blinded as to whether the specimens are case or con-trol, and the EDRN DMCC performs the data analysis.The reference set has been used to test candidate bio-markers from 6 different investigators, and 3 of these in-vestigators have been approved for access to the largervalidation set. This 2-step procedure allows the efficienttesting of potential biomarkers on a smaller prevalida-tion reference set to determine performance characteris-tics before access to the larger validation set.In 2011, the FDA cleared the Risk of Ovarian Malig-

nancy Algorithm (Fujirebio Diagnostics, Inc, Malvern,PA), which uses the CA-125 and HE4 blood markers todetermine the likelihood that an ovarian pelvic mass ismalignant. Stephen Skates and colleagues of EDRNhelped to validate CA-125 and HE4 in preclinical sam-ples received from the NCI’s Prostate, Lung, Colon,and Ovarian Screening Trial.17 Prostate, Lung, Colon,and Ovarian data and biospecimens are available to all

qualified researchers though a peer-review process(http://prevention.cancer.gov/plco/biospecimens).

CONCLUSIONS

The translation of biomarkers into clinical use is a longand difficult process, and themajority of biomarkers willnever make it beyond verification because they lack suf-ficient sensitivity and specificity to be clinically useful.Successful translation requires close collaborationamong investigators involved in discovery research, cli-nicians, assay developers, and statisticians, but the in-vestigators have to be willing to have their assays andbiomarkers verified by an independent laboratory.Some investigators have expressed reluctance to handover their biomarkers or to have them independently ver-ified, asking ‘‘what’s in it for me?’’ The EDRN is de-signed to foster collaboration among investigators withdifferent expertise and to allow for the biomarker discov-erer to actively participate in the validation process.EDRN investigators Zen Zhang and Daniel Chan18

have described lessons learned during the 6 years ittook from biomarker discovery to FDA approval forthe in vitro diagnostic OVA1. As they noted, ‘‘From dis-covery of biomarkers to their use for a specific clinicalindication, it requires the resolution of many interwovenissues and knowledge and expertise from diverse areas.’’Investigators work with the EDRN because of its vi-

sion for translational research. Such a vision, althoughchallenging to implement, allows investigators accessto expertise, technologies, and resources that wouldnot otherwise be available to them. Investigators havebuilt strong collaborative ties with other first-rate labo-ratories across the country that have enabled implemen-tation of a translational research paradigm. The EDRNsystem and resources attract outstanding investigatorswho make their scientific resources available to othernetwork investigators.The EDRN is a model for translational research that

links strong basic scientists with translational clinicalinvestigators in formal ways. Although it is sometimeschallenging, the resources and collaborative environ-ment have made a major difference in the quality and in-novation in biomarker research. The biomarkercommunity foresees extension of the EDRN model toother aspects of biomarker application for cancer prog-nosis and therapeutic monitoring.

REFERENCES

1. Biomarkers Definitions Working Group. Biomarkers and surro-

gate endpoints: preferred definitions and conceptual framework.

Clin Pharmacol Ther 2001;69:89–95.

2. Diamandis EP. Cancer biomarkers: can we turn recent failures

into success? Natl Cancer Inst 2010;102:1462–7.

Translational ResearchVolume 159, Number 4 Wagner and Srivastava 353

3. Zhu CS, Pinsky PF, Cramer DW, et al. A framework for evaluating

biomarkers for early detection: validation of biomarker panels for

ovarian cancer. Cancer Prev Res 2011;4:375–83.

4. Hanash SM.Why have protein biomarkers not reached the clinic?

Genome Med 2011;3:66–7.

5. Pepe MS, Etzioni R, Feng Z, et al. Phases of biomarker develop-

ment for early detection of cancer. J Natl Cancer Inst 2001;93:

1054–61.

6. Srivastava S, Gray JW, Reid BJ, Grad O, GreenwoodA, Hawk ET.

Translational Research Working Group developmental pathway

for biospecimen-based assessment modalities. Clin Cancer Res

2008;14:5672–7.

7. Baker SG, Kramer BS, McIntosh M, Patterson BH, Shyr Y,

Skates S. Evaluating markers for the early detection of cancer:

overview of study designs and methods. Clin Trials 2006;3:

43–56.

8. Ransohoff DF. How to improve reliability and efficiency of re-

search about molecular markers: roles of phases, guidelines, and

study design. J Clin Epidemiol 2007;60:1205–19.

9. Brenner DE, Normolle DP. Biomarkers for cancer risk, early de-

tection, and prognosis: the validation conundrum. Cancer Epide-

miol Biomarkers Prev 2007;16:1918–20.

10. Baker SG. Improving the biomarker pipeline to develop and

evaluate cancer screening tests. J Natl Cancer Inst 2009;101:

1116–9.

11. Dunn BK, Wagner PD, Anderson D, Greenwald P. Molecular

markers for early detection. Semin Oncol 2010;37:224–42.

12. Ransohoff DF, Gourlay ML. Sources of bias in specimens for re-

search about molecular markers for cancer. J Clin Oncol 2010;28:

698–704.

13. Moore HM, Kelly AB, Jewell SD, et al. Biospecimen Reporting for

Improved Study Quality (BRISQ). J Proteome Res 2011;10:

3429–38.

14. McShane LM, Altman DG, Sauerbrei W, et al. Reporting recom-

mendations for tumor marker prognostic studies (REMARK).

J Natl Cancer Inst 2005;97:1180–4.

15. Pepe MS, Feng Z, Janes H, Bossuyt PM, Potter JD. Pivotal eval-

uation of the accuracy of a biomarker used for classification or

prediction: standards for study design. J Natl Cancer Inst 2008;

100:1432–8.

16. Marrero JA, Feng Z,WangY, et al. Alpha-fetoprotein, des-gamma

carboxyprothrombin, and lectin-bound alpha-fetoprotein in early

hepatocellular carcinoma. Gastroenterology 2009;37:110–8.

17. CramerDW, Bast RC Jr, Berg CD, et al. Ovarian cancer biomarker

performance in prostate, lung, colorectal, and ovarian cancer

screening trial specimens. Cancer Prev Res 2011;4:365–74.

18. Zhang Z, Chan DW. The road from discovery to clinical diagnos-

tics: lessons learned from the first FDA-cleared in vitro diagnostic

multivariate index assay of proteomic biomarkers. Cancer Epide-

miol Biomarkers Prev 2010;9:2995–9.