IEEE ENGINEERING IN MEDICINE AND BIOLOGY MAGAZINE JANUARY/FEBRUARY 2007 91

FDA-regulated validation in clinical and nonclinical environments

System validation is a multiphaseprocess which demonstrates thata laboratory instrument or pieceof medical equipment functions

as intended, is “fit” for use per accep-tance criteria, and meets the Food andDrug Administration’s (FDA’s) guide-lines and user expectations. It is fieldtesting from the end user’s perspective,above and beyond the detailed testingperformed by developers and manufac-turers. The clinical environment (i.e.,involving humans) has more validationrequirements than the preclinical envi-ronment (i.e., involving animals or in-vitro) due to the different usage, yet thefoundation is the same [1]–[5]. Systemvalidation is performed at the end-usercompany’s site, except for devices iden-tified by the FDA that are to be tested atthe manufacturer’s site.

Per the FDA regulations, system vali-dation must occur before the system orsoftware is put into service by the enduser [1]–[5]. Hardware and softwareengineers are often recruited to performvalidation services. Engineers may rec-ognize the various validation phaseslisted below because the same projectprocesses are performed during a sys-tem’s development and manufacturing,prior to its release for sale to an end-user company. When devices are soldfrom a manufacturer directly to an enduser, the manufacturer has the regulato-ry burden of final system validationfrom the user’s perspective (e.g., single-user insulin pump).

System validation of a purchased sys-tem is similar to the software develop-ment lifecycle (SDLC), or waterfallmethod, of software development. TheFDA refers to its method as the soft-ware life-cycle (SLC) [6].

FDA RegulationsThe FDA governs the activities of med-ical and pharmaceutical research and

healthcare organizations through regula-tions including, but not limited to, goodlaboratory practices (GLPs), currentgood manufacturing practices (CGMPs),good clinical practices (GCPs), andelectronic records and electronic signa-tures (ERESs). These good practicesand ERESs cite various requirementsthat comprise “validation” [1]–[5].These regulations cover both the clinicaland nonclinical environments.

The FDA requires many differenttypes of validation, such as methodsand data validations. This article pre-sents an overview of two validationtypes: a system validation and computersystem validation (CSV).

A system validation includes testingand documentation of a system’s hard-ware and software with all componentsconfigured and connected as they willbe used when placed into service. Asystem validation includes both instru-ments and equipment.

CSV includes testing and documenta-tion of only software, with its configu-rations and interfaces as they will beused when placed into service by theend-user company. One typical CSVproject is validation of a laboratoryinformation management system(LIMS), a large software applicationthat may connect to several instrumentsto import and analyze the data.

System validation and CSV have thesame deliverable documents for a vali-dation project. End-user companies thatpurchase laboratory instruments, bio-medical equipment, and medical ser-vices must comply with systemvalidation and CSV to ensure the systemor software operates as expected and canperform its intended functions in theenvironment in which it will be used.

With insufficient or no validation, theFDA can stop use of the instrument andrefuse to accept results for any tests per-formed prior to validation, or possibly

shut down a laboratory facility until theproblem is resolved (a worst-case sce-nario that is extremely rare).

System Validation and CSV ActivitiesFormal training is required to fullyunderstand each activity’s prerequisites,requirements, and variations foundamong systems and uses. Some activi-ties occur simultaneously or are ongo-ing through a major portion of theproject.

The system validation and CSVactivities typically performed include:➤ creating a validation master plan➤ assigning roles and responsibilities➤ defining system requirements➤ defining required security➤ showing traceability ➤ performing risk assessment ➤ performing a vendor assessment➤ writing a system description ➤ testing

• creating and executing an installa-tion qualification (IQ)

• creating and executing an opera-tional qualification (OQ)

• creating and executing a perfor-mance qualification (PQ)

➤ writing a validation summary➤ establishing support and mainte-

nance infrastructure➤ writing a retirement/decommission-

ing plan➤ writing a disaster recovery plan, if

one is not already in place➤ ensuring a business continuity plan

is in place.Approvals by appropriate managers

and authors are obtained throughout thevalidation process [1]–[5].

The validation master plan is essen-tially a project plan. It defines theapproach to the overall process. It mayinclude the assignment of roles andresponsibilities, a preliminary list ofrequirements, and a preliminary risk

Connie CurtsRegulatory Affairs

92 IEEE ENGINEERING IN MEDICINE AND BIOLOGY MAGAZINE JANUARY/FEBRUARY 2007

Regulatory Affairs (continued)

assessment. At this point, the list ofrequirements would be used to evaluateinstruments on the market and to makea purchasing decision.

For an end-user company, therequirements cite the functionalityneeded by the end user, including regu-latory requirements. Depending on thesystem, an end-user company mayrequire installation of extra electrical,water, or other utility capacity. For sys-tems that include a computer, therequirements would include➤ computer hardware and software

minimum specifications issued bythe instrument/equipment/devicemanufacturer, and

➤ additional software applications andversion numbers which would beused with the system or its raw data(e.g. Microsoft Office 2002, Excelfor systems that export data toExcel, or proprietary software infor-mation).

Security requirements commonlyinclude physical facility security, suchas campus guards or key cards to accessa room. Requirements also include soft-ware security, such as mandatory loginsand passwords with a minimum numberof characters, or data protections to con-form to privacy requirements per theHealth Insurance Portability andAccountability Act (HIPAA).

The requirements document may beupdated during the project to includeadditional functions or features notoriginally envisioned. For example, thepurchase of a data acquisition systemmay have an audit trail report with mul-tiple sort functions; this type of sortingfunctionality may become a require-ment for future system purchases. Withthe new function listed in the require-ments document, an end-user companycould issue the same document to mul-tiple vendors for future purchases. Notall companies stay with their initial sys-tem manufacturer.

The traceability aspect is the crossreference for each system requirementto its test and documentation in the OQand/or PQ, which are discussed below.A requirement may be traced to a stan-dard operating procedure (SOP) or to an

action in the IQ (discussed below),depending on the system environmentand corporate situation. In the event arequirement cannot be met at all, thebusiness management may choose toschedule its usage for a later time, orthey may deem it unnecessary.

The risk assessment may be conduct-ed as part of the validation master planand is often an ongoing activity duringthe entire validation process. It docu-ments anticipated problems and optionsto fix or avoid these problems. Risks tosaving original data, to privacy, or topatients are of utmost importance.

The higher the risk, the more end-user company testing is required to ver-ify operation and accuracy of thefunctions to be used and to resolve asmany problems as necessary prior tousage of the system. Also needed is theuse of a change control procedure toresolve all problems and provide docu-mentation that the system’s validatedstate is maintained. One high-risk situa-tion is an instrument or medical devicethat has proprietary software with mini-mal or no documentation from the man-ufacturer. This would require a detailedIQ and a very detailed OQ/PQ test plan.System documentation, such as a sys-tem description and manuals, must becreated by the end-user company whenit is not provided by the manufacturerand/or vendor.

Another high-risk situation is thecase of statistical analysis software thathas not been used previously in a regu-lated laboratory environment. High-risk software typically does notconform to the ERES requirements,such as having an audit trail or ensur-ing that original data is not overwritten.Or it may not have a separate login andpassword to ensure that only authorizedindividuals access the software. Therisk could be reduced by adding securi-ty through an SOP, installing a cus-tomized software addition, or limitingaccess to a room. High-risk softwareneeds to be tested thoroughly for itscalculation accuracy and the compa-ny’s ability to save the original data,even if that raw data is only on a paperprint out. An example of a low-risk sit-

uation is a laboratory-designated net-work printer being offline.

The vendor assessment is the end-user company’s audit of the vendor’smethodologies, quality, and integrity.Its purpose is to provide high confi-dence in the vendor, its products, and itsservices. To accomplish the audit, cus-tomers may visit the vendor’s site orconduct the audit via telephone confer-ences, e-mails, or letters. The FDAexpects due diligence. End-user compa-nies look for standards that have beenused in the development and manufac-ture of the equipment (e.g. IEEE andISO standards), and they often look forany valid certifications, such as a cer-tificate of ISO 9000:2000 or its subsec-tion ISO 9001:2000. This certificateindicates that the vendor company fol-lows an international standard for quality management systems with amanagement method that aims to pre-vent problems and provide quality andcustomer satisfaction from the initialconcept of the product to its retirement[7]–[9].

The system description defines thesystem’s final environment. Usually,diagrams are used to show how the sys-tem is connected to other equipment orto an animal or human. If a computer isused as part of the system, a final sys-tem description would include the actu-al computer’s specifications, whichmight exceed the manufacturer’s speci-fications.

The installation qualification (IQ) is atest plan or protocol that demonstratesthat the installation has been performedcorrectly and in a reproducible manneraccording to the manufacturer’srequirements. It also indicates specificconfigurations needed for the actualusage at the end-user company. It is notuncommon to have the IQ refer to themanufacturer’s instructions within thedocument so as not to rewrite a well-written set of instructions.

The operational qualification (OQ) isthe test plan or protocol that checks thesystem’s functionality when things goright or wrong with the usage. It veri-fies that the system or software consis-tently operates within established limits

IEEE ENGINEERING IN MEDICINE AND BIOLOGY MAGAZINE JANUARY/FEBRUARY 2007 93

and tolerances. Tests usually includethe printing and verification of contentaccuracy of all the reports that end userswill print. The OQ includes significanttesting of error handling and limita-tions, such as how the system handlesinvalid data entries or blocks unautho-rized attempts to change or delete data.The OQ may be written or executed bya technical person other than the enduser, such as an information technology(IT) person or an engineer.

The OQ is not to be confused withthe more detailed unit testing performedby hardware and software engineers toprove every nuance of the systembefore it is sold. There may be morethan 2,000 unit tests, whereas the OQmay have only 50 tests to demonstratethe bulk of functionality the end userswill employ. Also, unit testing is con-ducted prior to release of the system orsoftware for sale to customers. OQ test-ing is performed at the end-user’s site,after the sale and before release to endusers. The OQ testing may be per-formed by an end user, an IT person, aquality assurance person, or a manufac-turer’s representative. Sometimes acombination of these individuals per-forms the testing.

The performance qualification (PQ)is a test plan or protocol that checks theend user’s intended usage to provideconfidence that the business processesare effective and reproducible. It isintended to test when things go right.The PQ may have some security teststhat were previously qualified in theOQ; however, the PQ is not as detailedas the OQ. Additionally, it is commonlyexecuted by an actual end user.

Over the years, the term “qualifica-tion” has been confused with, and usedinterchangeably with, the term “valida-tion.” There is a difference. A qualifica-tion is a single protocol to accuratelyassess the quality, reliability, and repro-ducibility of a system’s usage. The testplan’s execution must meet designatedacceptance criteria to “qualify.” A vali-dation is a final conclusion, based onthe successful qualifications, that a sys-tem meets the needs for the intendedusage. In the trenches, however, the

term qualification usually refers to thehardware’s performance, while the termvalidation is used for software perfor-mance. Rely on each business environ-ment’s context for guidance.

Before the validation can be complet-ed and the system released to the endusers, a support and maintenance infra-structure must be in place. Provision oftraining, user manuals, SOPs, contactinformation for technical support, databack-up procedures, and more comprisethe infrastructure. Critical to the infra-structure is change control.

Change control is the management ofall modifications to the hardware orsoftware after it has been qualified orvalidated. It provides assurance that thesame system environment can be repro-duced if necessary. The change controlprocedure should define what approvalsare needed and when they are needed.

A retirement or decommissioningplan should indicate how to archive rawdata for long-term storage and how toretrieve that data. Documentation shouldalso indicate how long the raw data willbe retained and determine a final dispo-sition for the data and its media.

A disaster recovery plan and a busi-ness continuity plan differ in theiremphasis. The former focuses on gettinga company back into business followinga catastrophic event. The latter focuseson keeping the company in operationthroughout a catastrophic event.

Finally, there is a formal summary ofthe validation. This brief statement indi-cates that the validation master planwas carried out. If there were deviationsor problems, it states why they occurredand how they were handled. This state-ment is the final conclusion that thecomponent or multicomponent systemis “validated.” With this conclusion, anend-user company may release themechanism for usage.

The FDA does not require specifictitles for the various documents, and itpermits some documents to be com-bined into one project document. Forexample, the final system descriptionmay be detailed in the IQ; and the OQand PQ may be combined into one testplan called an OQ/PQ.

A system validation can take a fewdays, weeks, or months, depending onthe system’s complexity, intendedusage, and available resources. Amulticomponent data acquisition sys-tem that connects to animals mayrequire three months, whereas a sin-gle high-performance liquid chro-matograph (HPLC) may require lessthan a week. However, both systemswill have gone through the same sys-tem validation process.

Impact on EngineersThe services most often being soughtfrom vendors, manufacturers, and con-tractors are the creation and/or execu-tion of the IQ, OQ, and PQ. Somehardware and software engineers arealready engaged in these system valida-tion or CSV services because the engi-neers either know the system or canlearn it quickly. An engineer can writeand/or execute these documents moreeasily for end-user companies than acontracted technical writer who has noknowledge of instrumentation or a bio-medical environment.

The validation process requirespatience and a lot of documentation, yetit can be a very satisfying experience.Currently, it is very difficult to find abook that discusses the entire systemvalidation process, but help is on thehorizon. The Drug InformationAssociation (DIA) based in Horsham,Pennsylvania, organized and hosted anupdate of the original Red AppleConference–the Red Apple IIConference. In 1987, the Office ofRegulatory Affairs of the FDA, theNational Center for ToxicologicalResearch (NCTR), and the NCTRAssociated Universities, Inc. conducteda conference at the Red AppleConference Center in Heber Springs,Arkansas, on quality assurance of com-puterized systems used in nonclinicalsafety assessment. The Red AppleConference, as it became known, was amilestone event in establishing bestpractices in the design and validation of

(continued on page 101)

IEEE ENGINEERING IN MEDICINE AND BIOLOGY MAGAZINE JANUARY/FEBRUARY 2007 101

Book Reviews (continued)

and 12 describe medical imagingprinciples and applications. Chapters13 and 14 describe two companies(Medtronic and Neurotech) and pro-vide insight into the business modeland plans of the private sector. Theremaining chapters provide anoverview of tissue engineering andthe future of the field, which theauthor describes as an effort tounderstand genes and explore thegenome and protein production, andan advancement of nanotechnology.

The book is well written and eventhough there are many names andexamples included of researchers, stu-dents, health care providers, and insti-tutions, it is easy to follow all thestories. The author visited severalsites throughout the country and inter-

viewed many researchers, studentsand patients and is to be commendedfor his thorough investigation.Perhaps one could argue that the solefocus on US-based academic researchand training institutions does notaccurately capture all trends anddevelopments of the field of biomed-ical engineering as it excludes cuttingedge research in other countries.

The chapters provide an overviewof the field of biomedical engineer-ing for a lay audience and as such thebook does not provide an in depthdiscussion of principles or technicalbackground behind the applicationsnor does it require any prior back-ground in engineering or knowledgeof the field. While this book wouldnot be the right choice as a textbook

of an undergraduate or graduatecourse in biomedical engineering dueto its superficial coverage of materialcustomized for a broader audience, itis an excellent resource for peoplewho want to learn more about thefield in general, but also for studentsin medicine, nursing and other healthprofessions who need an overview ofways with which technology is revolu-tionizing health care delivery. The com-pelling stories of pioneering engineers,patients and health care providers inthis book have the potential to recruitstudents into the field and inspire futureresearchers, and to increase the public’sknowledge and appreciation of biomed-ical engineering.

—George Demiris University of Washington

computerized systems in nonclinicallaboratories. The outcome of the confer-ence was a reference published by theDIA, Computerized Data Systems forNonclinical Safety Assessment: CurrentConcepts and Quality Assurance, whichis used throughout the world [10].

For the Red Apple II Conference, aselect group of international participantsfrom industry, government, and acade-mia was selected to participate on thevarious committees of the conference.The participants represented therequired mixture of skills and a crosssection of pertinent organizational affil-iations. The international conference,which was held 22–24 March 2006, hadas its objective the creation of a singleupdated book with significant insightinto current system validation processesfor preclinical instruments. The publica-tion date has been scheduled for early2007. Validations performed in clinicalenvironments may refer to this book, asmany of the concepts and illustrationsare GxP. The book may be supplement-ed by research to cover the additional

clinical details required (e.g., data pro-tection, HIPAA) [10].

ConclusionsAs baby boomers age, the demand willincrease for more drugs and betterhealth care. In turn, the demand forinstrumentation, biomedical equipment,and medical devices will increase,along with the demand for qualifiedpeople to provide validation services.Due to the nature of validation, hard-ware and software engineers can suc-cessfully expand into the end-usercompany validation niche.

References[1] “Good laboratory practice for nonclinical labo-ratory studies,” U.S. Government Printing Office,Washington DC, 21 Code of Federal RegulationsPart 58, Apr. 1, 2005.[2] “Current good manufacturing practice in manu-facturing, processing, packing, or holding of drugs,”Government Printing Office, Washington DC, U.S. 21Code of Federal Regulations Part 210, Dec. 6, 2005.[3] “Current good manufacturing practice for fin-ished pharmaceuticals,” U.S. Government PrintingOffice, Washington DC, 21 Code of FederalRegulations Part 211, Dec. 6, 2005.[4] “General principles of software validation; Finalguidance for industry and FDA staff ,” U.S.

Department of Health and Human Services, Foodand Drug Administration, Center for Devices andRadiological Health, Center for Biologics Evaluationand Research, U.S. Government Printing Office,Washington DC, Jan. 11, 2002.[5] “Guidance for industry, Part 11, electronic records;electronic signatures—Scope and application,” U.S.Government Printing Office, Washington, DC, 21 Codeof Federal Regulations Part 11, Aug. 2003. [6] Stephen H. Kan, Metrics and Models in SoftwareQuality Engineering , 2nd ed. Reading, MA:Addison-Wesley, 2003.[7] “What is ISO 9001:2000?” BSI Management SystemsWeb site, July 14, 2006 [Online]. Available: http://www.bsiamericas.com/QualityGateway/index.xalter[8] J. Ketola and K. Roberts, ISO 9000 in a Nutshell,2nd ed. Chico, CA: Paton Press, 2001.[9] D. Hoyle, ISO 9000 Quality Systems Handbook,3rd ed. Woburn, MA: Butterworth-Heinemann, 1998.[10] E. Hulihan, private communication, July 17, 2006.[11] J. Chen, private communication, “Qualification ofLaboratory Instruments” seminar, New York, July 2003.[12] L. Milum, private communication, July 29, 2006.[13] R. Temple, private communication, August 1,2006. [14] L. Ouderback, private communication, August2, 2006. [15] “Guidance for industry, FDA reviewers andcompliance on off-the-shelf software use in medicaldevices,” U.S. Department of Health and HumanServices, Food and Drug Administration, Center forDevices and Radiological Health, Center forBiologics Evaluation and Research, U.S. GovernmentPrinting Office, Washington, DC, Sept. 9, 1999.[16] “Food and Drug Administration glossary”[Online]. Available: http://www.fda.gov/ora/inspect_ref/igs/gloss.html

Regulatory Affairs (continued from page 93)