Elightened Experimentation - The New Imperative for Innovation

8/7/2019 Elightened Experimentation - The New Imperative for Innovation

1/11


2/11

The high cost qf experimentation has long puta damper on companies'attempts to creategreat new products. But new technologies aremaking it easier than ever to conduct complexexperiments quickly and cheaply Companiesnow have an opportunity to take innovation toa who le new level-if they're willing to rethinktheir R&D from the ground up .

EnlightenedExperimentationThe New Imperative fornnovation

by Stefan ThomkeE XPER IM ENTATION LIES AT THE HEAR T OF EVERYcompany's ability to innovate. In other words, thesystematic testing of ideas is what enables companiesto create and refine their products. In fact, no product canbe a product w ithout having first been an idea that wasshaped, to one degree or another, tbrough the process ofexperimentation. Today, a major development project canrequire literally thousands of experiments, all with thesame objective; to leam whether the product concept orproposed technical solution holds promise for address-ing a new need or problem , then incorporating that in-formation in the next round of tests so that the bestproduct ultimately results.


3/11

Enlightened ExperimentationIn the past, testing was relatively expensive, so comp a-nies had to be parsimonious with the number of experi-me ntal iterations. Today, however, new technologies sucbas computer simulation, rapid prototyping, and combina-torial chemistry allow comp anies to create mo re learn ingmore rapidly, and that knowledge, in tum, can be incor-porate d in more experim ents at less expense. Indeed, new

information-based technologies have driven down themarginal costs of experimentation, just as they have de-creased the m arginal costs in some produ ction and distri-bution systems. Moreover, an experimental system thatintegrates new information-based technologies doesmo re tha n lower costs; it also increases the opp ortu nitiesfor innovation. That is, some technologies can make ex-isting experim ental activities more efficient, while oth ersintroduce entirely new ways of discovering novel con-cepts and solutions.Millennium Pharmaceuticals in Cambridge, Massachu-setts, for instance, incorporates new techno logies such asgenom ics, bioinformatics, and combinatorial chem istryin its technology platform for conducting experiments.The platform enables factory-like automation that cangenerate and test drug candidates in minutes or seconds,compared w ith the days or more tba t traditional method srequire. Gaining information early on abou t, say, tbe tox-icological profile of a drug candidate significantly im-proves M illennium 's ability to predict th e drug 's success inclinical testing and, ultimately, in the marketplace. Un-promising candidates are eliminated before hundreds ofmillions of dollars are invested in their development. Inaddition to reducing th e cost and time of traditional drugdevelopment, tbe new technologies also enhance Millen-nium 's ability to innovate, according to Cbief TechnologyOfficer Michael Pavia. Specifically, the company hasgreater opportunities to experiment with more diversepotential drugs, including those tbat may initially seemimprobable but might eventually lead to breakthroughdiscoveries.This era of "enlightened experimentation" has thus faraffected businesses with higb costs of product develop-ment, sucb as the pharmaceutical, automotive, and soft-ware industries. By studying them , I have learned several

valuable lessons tbat I believe have broad ap plicability toother industries. As the cost of computing continues tofall, making all sorts of complex calculations faster andcheaper, and as new technologies l ike combinatorialchemistry emerge, virtually all companies will discoverthat they have a greater capacity for rapid experimenta-tion t o investigate diverse concepts. Financial institutions,for example, now use computer simulations to test newfinancial in strum ents. In fact, the develop men t of spread-Stefan Thomke is an associate professor of technology an doperations m anagement at Harvard Business School inBoston.

sheet software has forever changed financial modeling;even novices can perform many sophisticated what-if ex-per iments that were once prohibitively expensive.

A System for Exp erime ntationUnderstanding enlightened experimentation requires anappreciation of tbe process of innovation. Namely, prod-uct and tecbnology innovations don't drop from tbe skj^they are nurtured in laboratories and developm ent orga-nizations, passing throug b a system for experimentation.All development organizations have such a system inplace to belp them narrow the num ber of ideas to pursueand tben refine that group into what can become viableproducts. A critical stage of the process occurs when anidea or concept becomes a workin g artifact, or prototy pe,which can tbe n be tes ted, discussed, shown to customers,and learned from.

Perhaps the m ost famous example of the experimentalsystem at work comes from the laboratories of ThomasAlva Edison. When Edison noted that inventive genius is"99% perspiration and i% inspiration," be was well aw areof tbe importance of an organization's capability and ca-pacity to ex periment. That 's why he designed his opera-tions in Menlo Park, New Jersey, to allow for efficient andrapid experimental iterations.Edison knew that the various components of a systemfor exper imentat ion- including personnel , equ ipment ,libraries, and so o n- a l l function interde pendent I y. Assuch, they need to b e jointly optimized, for together theydefine tbe system's performance: i ts speed (the t imeneeded to design, build, test, and analyze an experiment) ,cost, fidelity (the accuracy of the e xperim ent and t he con-ditions under which it is cond ucted), capacity (the num-ber of experiments that can be performe d in a given rimeperiod), and th e learning gained (the amount of new in-formation generated by the experiment and an organiza-tion's ability to benefit from it). Tbus, for example, bighiyskilled machinists worked in close proximity to lab per-sonnel at Menlo Park so tbey could quickly make im-provements when researchers had new ideas or learnedsomething new from previous experiments. This systemled to landmark inventions, including the electric light-bulb, which required more than i,ooo complex experi-men ts with filament materials and shapes, electromechan-ical regulators, and vacuum technologies.Edison's objective of achieving great innovationthrough rapid and frequent experimentation is especiallypertinent today as the costs (both financial an d time) ofexperimentation plunge. Yet many companies mistakenlyview new technologies solely in terms of cost cutting,overlooking tbeir vast potential for innovation. Worse,com panies witb that limited view get bogged down in theconfusion that occurs when they try to incorporate new

technologies. For instance, com puter simulation d oesn't


4/11

Enlightened Experimentationsimply replace physical prototyp es as a cost-saving measure; it introduces an entirelydifferent way of experimenting that invitesinnovation. Just as the Intemet offers enor-mous opportunities for innovat ion-far sur-passing its use as a low-cost substitute forp h o n e or catalog t ransact ions - so doesstate-of-the-art experimentation. But realiz-ing that potent ial requires companies toadopt a different mind-set.

Indeed, new technologies affect every-thing, from the development process itself,including the way an R&D organization isst ructured, to how new k n o w l e d g e - a n dhence learning- is created. Thus, for com-panies to be more innovative, the challengesare man agerial as well as technical, as thesefour rules for enlightened experimentationsuggest:I.Organizefor rapid experimentation.The ability to expe rimen t quickly is integralto innovation: as developers conceive of amul t i tude of diverse ideas, experimentscan provide the rapid feedback necessaryto shape those ideas by reinforcing, modi-fying, or complement ing exist ing knowl-edge. Rapid exper ime ntat ion, however ,often requires the complete revamping ofentrenched rout ines. When, for example,certain classes of experiments become an

order of magni tude cheaper or faster, orga-nizational incentives may suddenly becomemisaligned, and the activities and routinesthat were once successful might becomehindrances. (See the sidebar "The PotentialPitfalls of New Technologies.")Consider the major changes that BMWrecently underwent. Only a few years ago,experimenting with novel design concepts -to make cars withstand crashes better, forinstance - required expensive physical pro-to types to be built . Because that process

took months, it acted as a barrier to innova-tion because engineers could not get timelyfeedback on their ideas. Furtherm ore, datafrom crash tests arrived too late to signifi-cantly influence decisions in the early stagesof product development. So BMW had toincorporate the informat ion far down-stream, incurring greater costs. Neverthe-less, BMW's R&D organization, structuredaround this traditional system, developedaward-winning automobiles, cementing th ecompany's reputation as an industry leader.But its success also made change difficult.

The Essentials forEnlightened Experimentat ionNew technologies such as computer simulations not only make experi-mentation faster and cheaper, they also enable companies to be moreinnovative. But achieving that requires a thorough understanding ofthe link between experimentation and learning. Briefly stated, innova-tion requiresthe right R&D systems for performing experiments thatwill generate the information needed to develop and refine productsquickly. The challenges aremanagerial as well as technical:1) Organize for rapid experimentation Examine and, if necessary, revamp entrenched routines,

organizational boundaries, and incentives to encourage rapidexperimentation.

Consider using small development groups that contain keypeople (designers, test engineers, manu facturing engineers)with all the knowledge required to iterate rapidly.

Determine what experiments can be performed in parallelinstead of sequentially. Parallel expe riments are most effectivewhen time matters most, cost is not an overriding factor, anddevelopers expect to learn little that w ould guide them inplanning the next round of experiments.

2) Fail early and often, but avoid mistakes Embrace failures th at occur early in the development process

and advance knowledge significantly. Don 't forget the basics of experi mentation. Well-designed testshave clear objectives (what doyou anticipate learning?) and

hypotheses (what do you expect to happen?). Also, mistakes oftenoccur when you don't control variables that could diminish yourability to learn from theexperiments. When va riability can't becontrolled, allow for multiple, repeated trials.

3) Anticipate and exploit early information Recognize the full value of front-loading: identifying problems

upstream, where they areeasier and cheaper to solve. Acknowledge the trade-off between cost and fideliiy. Experimentsof lower fidelity (generally costing less) are best suited in theearly exploratory stages of developing a pro du ct High-fidelityexperiments (typically more expensive) are best suited later toverify the product.

4) Combine new and traditional technologies Do not assume that a new technology will necessarily replace

an established one. Usually, new and traditional technologiesare best used in concert.

Remember that new technologies emerge and evolve contin u-ally. Today's new technology might eventually replace itstraditional counterpart, but it could then be challenged bytomorrow's new technology.


5/11

Enlightened ExperimentationToday, thanks to virtual experiments-crashes simu-

lated by a high-performance computer rather thanthrough physical prototypes-some of the informationarrives very early, before BMW has made major resourcedecisions. The costs of experimentation (both financialand time) are therefore lower because BMW eliminatesthe creation of physical prototypes as well as the expenseof potentially reworking bad designs after the companyhas committed itself to them. (Physical prototypes arestill required much further downstream to verify the finaldesigns and meet safety regulations.) In addition, the

rapid feedback and the ability to see and manipulate highquality computer images spur greater innovation: manydesign possibilities can be explored in "real time" yet virtually, in rapid iterations.

To study this new technology's impact on innovationBMW performed the following experiment. Several designers, a simulation engineer, and a test engineer formeda team to improve the side-impact safety of cars. Primarily using computer simulations, the team developed andtested new ideas that resulted from their frequent brainstorming meetings.

The Potential Pitfal ls of New TechnologiesNew technologies can stash the costs(both financial and time) of experimen-tation and dramatically increase acompany's ability to develop innova-tive produ cts. To reap those ben efits,thoug h, organizations m ust preparethemselves for the f ull effects of suchtechnologies.

Computer simulations and rapidprototyping, forexample, increasenot only a company's capacity to con-duct experiments but also the w ealthof inform ation generated by thosetests. That, however, can easily over-load an organization that lacksthe ca pability to process infor-ma tion from each round of ex-periments quickly enough to in-corporate it into the next round .In such cases, the result is w aste,confusion, and frustration. Inother words, with out carefuland thorough planning, a newtechnology might not onlyfail todeliver on itspromiseof lower cost, increased speed,and greater innovation, itcouidactually decrease the overa llperformance of an R&D organi-zation, or at aminimum dis-rupt its operations.

Misaligned objectives areanother common problem.Specifically, some managers donot fully appreciate the trade-off

between response time and resourceutilization . Consider what happenswhen companies establish centraldepartments to oversee co mp utingresources for performing simulations.Clearly, testing ideas and concepts vir-tually can provide developers with therapid feedback they need to shape newproducts. At the same tim e, computersare costly, so people m anag ing th emas cost centers are evaluated by howmuch those resources are being used.

The busier a central computer is,however, the longer it takes for devel-

Waitingfor a ResourceAccording to queuing theory, the waiting time fora resource such as a central mainframe computerincreases gradually as more of the resource isused. But w ben the utilization passes 70%, delaysincrease dramatically

40 50 60 70 80 90 100Percent of Resource Utilization

opers to get the feedback they need.In fact, the relationship between wa it-ing time and util ization is not l i near-queuing the ory has shown that thewa iting tim e typically increases gradu-ally until a resource is utilized around70%, and then the length of the delayssurge. (See the exh ibit "W aiting for aResource.") An organization tryingto shave costs may become a v ict imof its own myopic objective. That is,an annual savings of perhaps a fewhundred thousand dollars achievedthrough increasing util ization from

70% to 90% may lead to verylong delays for dozens ofdevelopment engineers wait-ing for critical feedback fromtheir tests.

A huge negative conse-quence is that the excessivedelays no t only affect develop-ment schedules but also dis-courage people fro m experi-me nting,thus squelching theirability to innovate. So in thelong term , running additionalcomputer equipment at a lowerutil ization level might wellbe worth the investment. Analternative solution is to movethose resources away from costcenters and under the controlof developers, who have strongincentives for fast feedback.


6/11

Enlightened ExperimentationBecause all the knowledge required about safety, de-sign, simulation, and testing resided within a small group,the team was able to iterate experiments and developsolutions rapidly. After each round of simulated crashes,the team analyzed the results and developed new ideasfor the next roun d of experim ents. As expected, the te ambenefited greatly from the rapid feedback: it took them

only a few days to accept, refine, or reject new design so-lut ions-som ething tha t had once taken m onths .As the trials accrued, the group members greatly in-creased their knowledge of the underlying mechanics,which enabled them to design previously unimaginableexperiments. In fact, one test completely changed theirknowledge about the complex relationship between ma-terial strength and safety. Specifically, BMW's engineershad assumed that the stronger the area next to the bot-tom of a car's pillars (the structures tha t connect t he roofof an auto to its chassis), the better the vehicle wouldbe able to withstand crashes. But one member of the de-velopment team insisted on verifying this assumptionthroug h an inexpensive computer simulation.The results shocked the team: strengthening a particu-lar area below one of the pillars substantially decreasedthe vehicle's crashworthiness. After more experiments andcareful analysis, the engineers discovered th at strength-ening the lower part of the center pillar would make thepillar prone to folding high er up, above tbe stre ngth enedarea. Thus, the passenger compartm ent would be m orepen etrable at the part of the car closer to the midsection,chest, and head of passengers. The solution was to

weaken, not strengthen, the lower area. This counterin-tuitive knowledge-that purposely weakening a part ofa car's structure could increase the vehicle's safety-hasled BMW to reevaluate ail the reinforced areas of itsvehicles.In summary, this small team increased the side-impactcrash safety by abou t 30%. It isworth no ting that two crashtests of physical prototypes at the end of the project con-firmed the sim ulation results. It should also be noted th atthe physical prototypes cost a total of about $3(Xt,ooo,which was more t ba n t he cost of all 91 virtual crashescombined. Furthermore, the physical prototypes took

longer to build, prep are, and test than the en tire series ofvirtual crashes.But to obtain the full benefits of simulation technolo-gies, BMW had to u ndertake sweeping changes in process,organiza t ion, and a t t i tude-changes tha t took severa lyears to accom plish. Not only did the co mpan y have to re-organize the way different groups worked together; it alsohad to change habits that had worked so well in the oldsequential development process.Previously, for example, engineers were often loath torelease less-than-perfect data. To some extent, it was ineach group's interest to hold back and mo nitor the ou tpu tfrom oth er g roups. After all, the group that submitted its

information to a central database first would qu ite likelyhave to make the most changes because it would havegoften tb e least feedback from o th er area s. So, for in-stance, the door development team at BMW was accus-tomed to-and rewarded for-releasing nearly flawlessdata (details about the material strength of a proposeddoor, for example), which could take many months togenerate. Tbe idea of releasing rough information veryearly, an integral part of a rapid and parallel experimen-tation process, was unthinkable-and not built into theincentive system. Yet a six-month delay while data werebeing perfected could derail a deve lopm ent programpredicated on rapid iterations.

Tbus, to encourage the early sharing of information,BMW's managers had to ensure that each group under-stood and appreciated the needs of other teams. Thecrash simulation g roup, for examp le, needed t o make thedoor designers aware of the information it required inorder to build rough models for early-stage crash simula-t ions. That transfer of knowledge had a ripple effect,changing how the door designers worked because someof the requested information demanded that they payclose attention to the needs of oth er group s as well. Theystarted to understand that withholding information aslong as possible was counterp rodu ctive. By making th esekinds of organizational changes, BMW in Germany sig-nificantly slashed develop men t time and costs and b oostedinnovation.

2. Fail early and often, but avoid mistakes. Experi-menting with many diverse-and sometimes seeminglyabsurd - ideas is crucial to innovation. When a novel con-cept fails in an experiment, the failure can expose impor-tant gaps in knowledge. Such experiments are particu-larly desirable when th ey are performed early on so thatunfavorable options can be eliminated quickly and peo-ple can refocus their efforts on more promising alterna-tives. Building the capacity for rapid experimentation inearly development means rethinking the role of failurein organizations. Positive failure requires having a thickskin, says David Kelley, foun der of IDEO, a leadin g designfirm in Palo Alto, California.

IDEO encourages its designers "to fail often to succeedsooner," and the company understands that more radicalexperimen ts frequently lead to more spectacular failures.Indeed, IDEO has developed numerous prototypes thathave bordered on the ridiculous (and were later rejected),such as shoes with toy figurines on t he shoelaces. At thesame time, IDEO's approach has led to a host of best-sellers, such as the Palm V handheld computer, which hasmade the company tbe subject of intense media interest,including a Nightline segment with Ted Koppel and cov-erage in Serious Play, a book by Michael Schrage, a co-director of the e-markets initiative at the MIT Media Lab,tha t describes the crucial imp ortance of allowing innova-tors to play with p rototypes.


7/11

Enlightened ExperimentationRemoving the stigma of failure, though, usually re-quires overcoming ingrained attitudes. People wh o fall inexperiments are often viewed as incompetent, and thatattitude can lead to counterprod uctive behavior.AsKelleypoin ts out, developers w ho are afraid of failing and look-ing bad to manag eme nt will sometimes huild expensive,sleek prototypes that they become committed to before

they know any of the answers. In other words, the sleekprototype might look impressive, but it presents th e falseimpression that the produ ct is farther along than it reallyis, and that perception subtly discourages people fromchanging the design even though bet ter a l ternat ivesmight exist. That's why IDEO advocates the developmentof cheap , rough pro totypes that people are invited to crit-icize-a process that eventually leads to better products."You have to have the guts to create a straw man," assertsKelley.

To foster a cu lture in which people aren't afraid of fail-ing, IDEO has created a playroomlike atmosphere. OnMondays, the different branches hold show-and-tells inwhich employees display and talkabout their latest ideas and products.IDEO also maintains a giant "tech box"of hundred s of gadgets and curiositiesthat designers rout inely rummagethrough, seeking inspiration amongthe switches, button s, and various oddmaterials and objects. And brain-storming sessions, in which wild ideasare encouraged and participants deferjudgment to avoid damping the dis-cussion, are a staple of the differentproject groups.

3M is another company wi th ahealthy att i tude toward failure. 3M'sproduc t groups often have skunk-works teams that investigate the op-portunities (or difficulties) that a po -tential product might pose. The teams,consisting primarily of technical peo-ple , including manufactur ing engi-neers, face little repercus sion if an ideaflops-indeed, sometimes a failure iscause for celebration. When a teamdiscovers that a po ten t i a l p roduc tdoesn't work, the group quickly dis-bands and its members move on toother projects.

Failures, however, should not beconfused with mistakes. Mistakes pro-duce little new or useful informationand are therefore wi thout value. Apoorly plan ned or badly conducted ex-periment, for instance, might result inambiguous data, forcing researchers

to repeat the experiment. Another com mon mistake is re -pea ting a prior failure or being una ble to learn from tha texperience. Unfortunately, even the best organizationsoften lack the management systems necessary to care-fully distinguish be twee n failures and m istakes.3. Anticipate and exploit early information. Whenimp ortan t projects fail late in the game , the consequenc escan be dev astating. In the pharm aceutical industry, for ex-amp le, mo re than 80% of drug candidates are discontin-ued during the clinical development phases, where moretha n half of total project expenses can be incu rred. Yet al-though companies are often forced to spend millions ofdollars to correct problems in the later stages of productdevelopment, they generally underestimate the cost sav-ings of early prohlem solving. Studies of software devel-opment, for instance, have shown that late-stage prob-lems are more tha n l oo times as costly as early-stage ones.For other environments that involve large capital invest-ments in production equipment, the increase in cost can

be orders of magnitude higher.

The Benefits ofFront-Loaded DevelopmentIn the 1990S,Toyota made a ma jor push to accelerate its pr oduc t developnnentcycle. The objective was to shorten the time from the approval of a body style tothe first reta il sales, thereby increasing the likelih ood t hat Toyota kept up w iththe rapidly changing tastes of consumers.

Toyota made a concerted effort to identify and solve design-related problem searl ier in product development-a concept known affront-loading. To accom-plish that, the company imp lemented a num be ro f in it iat ives, such as involv ingmore m anufacturing engineers during the p roduct-engineering stage, increas-ing the transfer of knowledge between projects, investing substantially in com-puter-aided design and engineering tools, and developing rapid-prototypingcapabilities.

To measure the benefits of these in it la t ive s-a nd to m onitor the company'sevolv ing capabili t ies for early problem solv ing- Toyota tracked problems overmultiple development projects. (See the exhibit "Solving Problems Earlier")The knowledge that a higher percentageof problems were being solved atearlier stages reassured Toyota's managers that they could aggressively reduceboth development t im e and cost with out r isking product quali ty. In particular,between the first and third front-loading initiatives, Toyota slashed the cost{including the number of full physical prototypes needed) and time of develop-ment by between 30% and 40%.

It should be noted that in the early 1990s Toyota substantially reorganizedits development activ it ies, result ing in more effective com mun ication andcoo rdina tion between the different groups. This change most likely accountedfor some of the performance improvemen ts observed, particularly dur ing thefirst front-loading initiatives.


8/11

Enlightened ExperimentationIn addition to financial costs, com panies need to con-sider the value of time whe n those late-stage problem s areon a project's cri tical p at h- as they often are. In pharma -ceuticals, shaving six mon ths off drug dev elopm ent me anseffectively extending patent protection when the producthits the market. Similarly, an electronics company might

easily find that six m onth s acco unt for a qu arte r of a prod-uct's life cycle and a third of all profits.New technologies, then, can provide some of theirgreatest leverage by identifying and solving problem s up-stream -be st described asfront-baded development. In theautom otive industry, for exam ple,"quick-and-dirty"crashsimulations on a computer can help companies avoidpotential safety problems dow nstream. Such sim ulationsmay no t be as complete or as perfect as late-stage proto-types will be, but they can force organizational problemsolving and comm unication at a t ime w hen m any down-stream groups are not participating directly in devel-

opment. (See the sidebar "The Benefits of Front-LoadedDevelopment.")

Several years ago, Chrysler (now Daim lerChrysler) dis-covered the power of three-dimensional computer mod-els, kno wn interna lly as digital mock-ups, for identifyingcer tain problems in ear ly development s tages. WhenChrysler developed the 1993 Concorde and Dodge In-trepid m odels, the process of decking - placing the p owe rtrain and related components like the exhaust and sus-pension in the prototype auto mo bi le- to ok more thanthree weeks and required many attem pts before the pow-ertrain could be inserted successfully. By contrast, theearly use of digital mock -ups in the 1998 Con corde a nd In-trepid models allowed the com pany to simulate deckingto identify (and solve) numerous interference problemsbefore th e physical decking took place. Instead of takingweeks, decking was completed in 15 minutes because allobstruction problems had been resolved earlier-when itwas relatively inexpensive an d fast to do so.

Of course, it is nei ther pragm atic n or econom ically fea-sible for companies to obtain all the early informationthey would like. So IDEO follows the principle of three

Solving Problems Earlier010^ 6 0E


9/11

Enlightened ExperimentationR's: rough, rapid, and right. The final R recognizes thatearly prototy pes may be incomp lete but can still get spe-cific aspects of a product right. For example, to design ateleph one receiver, an IDEO team carved dozens of piecesof foam and cradled them betwe en their h eads and shoul-ders to find the best possible shape for a handset. Whileincomp lete as a teleph one , the m odel focused o n gettingioo% of the shape right. Perhaps the main advantage ofthis approach is that it forces people to decide judiciouslywhich factors can initially be rough and which must beright. With its three R's, IDEO has established a processthat generates important information when it is mostvaluable: the early stages of develop ment.

In addition to saving tim e and money, exploiting earlyinformation helps product developers keep up with cus-tomer preferences that might evolve over the course ofa project. As many companies can attest, customers willoften say about a finished product: "This is exactly whatI asked you to develop, but it is not w hat I want." Leadingsoftware bu sinesses typically show incomplete prototypesto custom ers in so-called beta tests, and throu gh t hat pro-cess they often discover changes an d problem s wh en theyare still fairly inexpensive to handle.

4. Combine new and traditional technologies. Newtechnologies that are used in the innovationprocess itself are designed t o he lp solve prob-lems as part of an experimentation system. Acompany must therefore understand how touse and manage new and traditional tech-nologies together so that they complementeach other. In fact, research by Marco Iansitiof Harvard Business School has found th at, inmany industries, the ability to integrate tech-nologies is crucial to developing superiorproducts.A new technology often reaches the samegeneral performance of its traditional coun-terpart much more quickly and at a lowercost. But the new technology usually per-forms at only 70% to 80% of th e establishedtechnology. For exam ple, a new chemical syn-

thesis process might be able to obtain a puritylevel that is just three-q uarters tha t of a ma-ture technique. Thus, by combining new andestablished technologies, organizations canavoid the performance gap while also enjoy-ing the benefits of cheaper and faster experi-mentation. (See the exhibit "Combining theNew with the Traditional.")Indeed, the true potential of new tech-nologies lies in a company's ability to recon-figure its processes and organization to usethem in concert with traditional technolo-

gies. Eventually, a new technology can re-place its traditional counterpart, but it then

might be challenged by a newer technology that mus t beintegrated. To understand this complex evolution, con-sider what has happened in the pharmaceutical industry.In the late nineteenth century and for much of thetwentieth century, drug development occurred throug h aprocess of systematic trial-and-error exp eriments. Scien-

tists would start w itb little or no knowledge about a par-ticular disease and try out numerous molecules, manyfrom their com pany's chem ical libraries, until they foundone th at h appe ned to work. Drugs can be likened to keysthat need to fit the locks of targets, such as th e specificnerve cell receptors associated with central nervous dis-eases. Metaphorically, then, chem ists were once blind, orat least semiblind, locksmiths who have had to make upthou sand s of different keys to find the one that m atched .Doing so entailed synthesizing com pou nds , one at a t ime,each of which usually required several days at a cost from$5,000 to $10,000.Typically, for each successful d rug t ha t m akes it to mar-ket, a company investigates roughly 10,000 starting can-didates. Of those, only 1,000 com poun ds m ake it to m oreextensive trials in vitro (that is, outside living organisms insettings such as test tubes), 20 of which are tested evenmore extensively in vivo (that is, in the bo dy of a living or-

Com bining the New with the TraditionalA ne w technology (blue curve) wilt reach perhaps just 70% to 80 %of the performance of an established technology (red curve). A newcomputer model, for instance, might be able to represent real-worldfunction ality that is just three-quarters that of an advanced prototypemodel. To avoid this performance gap - and potentially create ne wopportunities for innovation - companies ca n use tbe new and tradi-tional technologies in concert (dotted red curve). Tbe optimal timefor switching between tbe two occurs wben tbe rates of Improveme ntbetween tbe new and mature technologies are about tbe same - thatis, when the slopes of tbe two curves ar e equal.

a V;-'Combined

Traditional and New Potentialfor. _ Increased Innovation4 PerformanceiCap

Savings from CombiningTraditional an d New Technologies

Effor t (Elapsed Time, Cost)


10/11

Enlightened Experimentationganism such as a mo use), and ten of which ma ke it to clin-ical trials with humans. The entire process represents along and costly com mitm ent.

But in the last ten years, new technologies have signif-icantly increased the efficiency and speed at which com-panies can generate a nd screen chemical com pounds. Re-searchers no longer need to painstakingly create onecom pound at a t ime . Instead, they can use com binatorialchemistry, quickly generating numerous variations si-mu ltaneously around a few building blocks, just as today'slocksmiths can make thousands of keys from a dozenbasic shapes, tbereby reducing the cost of a comp oundfrom th ous and s of dollars to a few dollars or less.

In practice, however, combinatorial chemistry has dis-rupted well-established routines in laboratories. For onething, the rapid synthesis of drugs has led to a new prob-lem: how to screen those compounds quickly. Tradition-ally, potential drugs were tested in live animals-an ac-tivity fi-aught w ith logistical difficulties, high expe nse, andconsiderable statistical variation.

So laboratories developed test- tube-based screeningmethodologies that could be automated. Cal led high-thro ugh put screening, this technolog y requires significantinnovations in equipment (such as high-speed precisionrobotics) and in the screening process itself to let re-searchers condu ct a series of biological tests, or assays, onmembers of a chemical library virtually simultaneously.The large pharmaceutical corporations and academicchemist ry depar tments ini t ia l ly greeted such "combi-chem " technologies (combinatorial chem istry and high-throughput screening) with skepticism. Among tbe rea-sons cited was that th e purity of com poun ds gene rated viacombichem was relatively poor compared to traditionalsynthetic chemistry. As a result, many advances in thetechnolog y were made by small biotechnology comp anies.But as the tech nology ma tured , it caught the interest oflarge corporations like Eli Lilly, which in 1994 acquiredSphinx Pharmaceuticals, one of the start-ups developingcom bichem . Eli Lilly took a few years to transfer the newtechnologies to its drug discovery division, which usedtraditional synthesis. To overcome the internal resistance,

senior management implemented var ious mechanismsto control how the new technologies were being adopted.For example, it temporarily limited the in-house screen-ing available to chemists, leaving them no choice but touse some of the high-throughput screening capabilitiesat the Sphinx subsidiary and interact with the staff there.Until now, pharmaceutical giants like EH Lilly haveused combinator ial chemist ry pr imar i ly to opt imizepromising new drug candidates that resulted from an ex-haustive search through chemical libraries and other tra-dit ional sources. But as combinatorial chemistry i tselfadvances and achieves levels of purity and diversity com-

parable to th e com poun ds in a library, com panies will in-creasingly use it at the earlier phases of drug discovery. In

fact, ail major p harm aceu tical com panies have had to usecombichem and traditional synthesis in concert, and th ecompanies that are best able to manag e the new and ma-ture technologies together so that they fully comp lementeach other will have the greatest opportunity to achievethe highest gains in productivity and innovation.

Enlightened ImplicationsNew technologies reduce the cost and time of experi-mentation, allowing companies to be more innovative.Automotive companies, for example, are currently ad-vancing the performance of sophisticated safety systemstha t meas ure a passenger's position, weight, and height toadjust the force and speed at which airbags deploy. Theavailability of fast and inexpensive simulation enables th emassive and rapid experimentation necessary to developsuch complex safety devices.

But it is imp ortant to no te that the increased autom a-tion of routine experiments will not remove the humanelem ent in innovation. On the contrary, it will allow peo-ple to focus on area s where th eir value is greatest: gener-ating novel ideas and concepts, learning from experi-me nts, and ul t imately m aking decisions that requirejudgm ent. For example, although Millennium's R& D fa-cilities look more and more like factories, the value ofknow ledge w orkers has actually increased. Instead of car-rying out routine laboratory experiments, they now focuson the early stages (determining which experiments toconduct, for instance) and making sense of the informa-tion generated by the experimen tation.

The im plications for industries are eno rmo us. The elec-tronic spreadsheet has already revolutionized financialprob lem solving by driving dow n the margina l cost of fi-nancial experimentation to nearly zero; even a small start-up can perform complex cash-fiow analyses on an inex-pensive PC. Similarly, com pute r simulation and oth ertechnologies have enabled small businesses and individ-uals to rapidly experiment with novel designs of cus-tom ized inte grated c ircuits. The result has been a massivewave of innovation, ranging from sma rt toys to electron icdevices. Previously, the high cost of integrated-circuit cus-tomizat ion made such exper imentat ion economical toonly the largest companies.Perhaps, thoug h, this era of enlightened experimenta-tion is still in its bare infancy. Indeed, the u ltima te tech-nology for rapid experimentation might turn out to bethe Intemet, which is already turning countless users intofervent innovato rs. 9

Re print R0102DTo order reprints, see the last page of Executive S umm aries.

To further explore the topic of this article, go towww.hbr.org/explore.


11/11

Documents

Elightened Experimentation - The New Imperative for Innovation