Validation of Excel an FDA - Welcome to QCSV Inc. … Spreadsheets in WL’s Drager Medizintechnik GmbH, 1999 WL-M2848N • “Failure to validate comppputer software used as part

Proper Use & Validation of Excel Spreadsheets in an FDA Regulated Environment

October 30th, 2008

Sponsored By: Regulatory Compliance CommitteeMedical Device ClusterMedical Device Cluster

PRTEC

Speakers• Juan Oscar Pérez Rodríguez President QCSV Inc :• Juan Oscar Pérez Rodríguez, President QCSV Inc.:Mr. Pérez, started his professional career while still in college with the Central IntelligenceAgency. He then joined Johnson & Johnson where he worked for over 13 years in diverseprojects in Puerto Rico, United States and Brazil. He founded QCSV Inc in 2004 with the goal inmind to help IT and QA communicate with each other and help IT to understand their regulatoryresponsibility with FDA laws and regulations. Mr. Pérez is also a speaker and has had

t Ph IT F I t h d th U i it f P t Ri H i lappearances at Pharma IT Forum, Interphex and the University of Puerto Rico. He is alsoMember of the Editorial Review Board for Pharma IT Journal, which is an internationalpublication based in England; such board is formed by seasoned professionals in IT andRegulatory Compliance from all over the world.

• Dennis Cantellops QA Manager FDA San Juan DistrictDennis Cantellops, QA Manager, FDA San Juan DistrictMr. Cantellops is the QA Manager for the FDA San Juan District. He is currently supportingforeign inspections auditing laboratory operations, as well as API Manufacturing and FinishedProducts. He has been very active in regards to the use of spreadsheets in the laboratory andhas a wide variety of publications (FDA Laboratory Information Bulletins as well as ScientificJournals) such as on spreadsheet software validation, fatty acid in food, statistic andelectrochemistry among others His articles has been referenced by international top speakerselectrochemistry among others. His articles has been referenced by international top speakerssuch as Dr. Ludwig Huber. Mr. Cantellops will also be a speaker at the next GaMP Forum event totake place in November and he is in the middle of a new article about the validation of ExcelSpreadsheets.

• Ty Mew President Ofni Systems IncTy Mew, President Ofni Systems Inc.He is president of Ofni Systems Inc., a software provider and Part 11 consulting firm. He has astrong background in software development and computer system validation, and has spoken atmany conferences, webinars, audio conferences and workshops on FDA requirements forelectronic records and signatures. Ty regularly conducts training sessions for organizations toeducate employees about the specific requirements of 21 CFR Part 11 and Computer SystemsValidation He is an active member of ISPE DIA SQA and the Triangle PEERS groupValidation. He is an active member of ISPE, DIA, SQA and the Triangle PEERS group.

1

October 30th, 2008October 30th, 2008

Welcome to the Medical Device Cluster

October 30th, 2008October 30th, 2008

IT Compliance & Spreadsheets in the Medical Device Industry

MD Cluster Mission

IT Responsibilities 10 Years Ago…

Compliance

2

TODAY!

BusinessSecurity

MD Cluster Background

Compliance

BusinessSecurity


When IT Operations are in Compliance, Control is ensured and Computerized Systems are

properly implemented…

ProcessesProcesses

Decision Making / ROI

When notProblems

Time/Propagation/CriticalityCost

3

• 21 CFR 210, 211, 820, GLP,…• August 1997 – 21 CFR Part 11

– Electronic Records

IT Compliance Scope

– Electronic Signatures

• July 2002 – Sarbanes-Oxley Act• ITIL, ICH, PICS• ISO 17025, 17799, …• Other international regulations as

applicableIT is now one of the most heavily regulated operations

IT OperationsGxP

Applications, Spreadsheets Internet/

Email

IT Network Infrastructure & Processes

Email

Why is IT Compliance so important?

“The role of IT now plays helping to ensure compliance in a GxP environment is being


compliance in a GxP environment is being acknowledge, and IT practices are being scrutinized during FDA inspections. This new role for IT requires establishment of a

sound IT infrastructure…” Timothy Fields

President, Drumbeat Dimensions

4

“It ain't what you don't know that gets you into trouble. It's what you know for sure that

just ain't so.”Mark Twain

Most common threads in 483’s and WL’s for Computerised

Systems• Lack of knowledge or

understanding of the regulations

• Inadequacy of computerised system validations

5

FDA Hiring Initiative

Common IT Compliance Issues• 21 CFR 820.22 - Internal quality audits must be conducted

properly to prevent major problems from developing and to assure the adequacy of your quality system.

• 21 CFR Part 11• 21 CFR Part 11• Spreadsheets management• Resources / Training (820.25)• Weak procedures / policies• Risk Based Approach• CPG 7132a.11• Vendor Management (820.50a, CPG 7132a.12)• Change Control <> Configuration Management

Dependence on Computerized Systems

Greater performance and compliance expectations for ITFace to Face events between IT and Inspectors

Regulatory Scrutiny

Inspector’s knowledge in IT

IT’s knowledge of regulations

6

• Typical Use of Spreadsheets– Laboratory (Sample Results)– Scheduling (Production / Maintenance)

Manufacturing (Lot # Assignment)– Manufacturing (Lot # Assignment)– QA (SOP # Assignment)– Management Review Metrics– Other uses

– Excel Spreadsheets can fall into any of the following categories as per GaMP Model:

• Category 3 – Non Configured• Category 4 – Configured• Category 5 - Custom

7

Spreadsheets in WL’sDrager Medizintechnik GmbH, 1999 WL-

M2848N• “Failure to validate computer software used as part of the p p

quality system for its intended use according to an established protocol as required by 21 CFR 820.70(i). For example, the data in the Excel spreadsheet identified as a “Hit List” of top non-conforming components contains 16 record counts for Part number 8601618 DC converter failures compared to 18 record counts for Part number 860168 DC converter failures in the dbase database. The spreadsheet is used for management review of component suppliers for all components.”

Spreadsheets in WL’sCordis, 2004 – WL-G4601D (not excel)

“ fi li d th t t lid t d t f• “…your firm relied on the not yet validated system for automated calculations, obtained by using custom-made formula fields, in making release decisions without manual verification.”

Spreadsheets in WL’sNational Genetics Institute, 2006 WL-G5699D

“There are no data to demonstrate that the quality control/quality assurance spreadsheets used for tracking q y p gand trending various quality metrics have been properly validated (installation qualification, operational qualification, and performance qualification) and are performing as intended. Examples of these spreadsheets include: [redacted]”

8

Spreadsheets in WL’sQuantimetrix Corp, 2007 WL-S6381C“Software used as part of the production quality system p p q y ywas not validated for its intended use according to an established protocol [21 C.F.R. 820.70(i)]. Specifically,

(a) Spreadsheets intended to check for outliers and calculate mean, SC, % CV, value assignments for finished devices.”

“Failure to see the writing on the wall is likely to end in a close and unpleasant encounter with regulatory inspectors.”

Juan O. PérezIT and Compliance in the Regulated Life-Science Industry

Pharma IT Journal – January 2007

Terrible Truth

9

Thank you very much for yourmuch for your

attention

Mr. Juan O. PérezPrincipal Consultant

Quality Computer System Validation Inc.(787)549-9052

[email protected]://www.qcsv-inc.com

11/17/2008

1

October 30October 30thth, 2008 , 2008

Use of Excel Spreadsheets in a Regulated Environment Dennis Cantellops, QAM - FDA

MD Cluster Mission

DisclaimerThe views presented do not necessarily

reflect those of the Food and Drug

22

gAdministration

Any mention of product or references to organizations isintended only to convey information; it does not implyrecommendation or endorsement by the US Food and DrugAdministration, nor does it imply that the products mentionedare necessarily the best available for the purpose.

SpreadsheetsSpreadsheet structure is similar to a computer

program

3

Formulas in a spreadsheet are basically a form of computer programming

Therefore, basic application of software engineeringtechniques and principles shall be applied in theconstruction of spreadsheet models.

3

11/17/2008

2

Typical Use of Spreadsheets– Laboratory (Sample Results)– Scheduling (Production / Maintenance)

Manufacturing (Lot # Assignment)

44

– Manufacturing (Lot # Assignment)– QA (SOP # Assignment)– Management Review Metrics– Other uses

Typical issues found during inspections– No validation or control procedures– Rounding-off errors (use of Excel round-off

function) USP 31, page 4 ExamplesF l i l ti l d d t f ll

5

– Formulas in analytical procedures do not follow Excel equations

– Conversion factors not expressed in analytical procedures.

– Formulas used for reviewing using manual calculations were not the same as those used in the spreadsheets. 5

Typical issues found during inspections

– No pre-determined specifications or limits– Spreadsheets are poorly documented. For

6

example, the product declaration was not indicated, replaced by only a number in a cell

– Units expressed as numbers, without descriptive labels such as mg/mL or mg/g.

– Sample weights described as sample volume and areas as ratios

6

11/17/2008

3

Typical issues found during inspections

– No provisions for security and data integrity– No protection against unauthorized changes;

analysts could freely change labels and formulas

7

analysts could freely change labels and formulas. – Regression analysis calculated with the y and x

axis inverted in the Excel formula, which generated erroneous slope and intercept results

– No Change Control– No data validation used: data-entry and drop-

down-list cells not color coded 7

In Other Words:

Overconfidence tends to blind people to the need

8

Overconfidence tends to blind people to the need for taking steps to reduce risk and regulatory

non compliance; the ability to catch errors instills confidence.

8

Spreadsheet DesignA spreadsheet model is a set of worksheets andmacro sheets designed to evaluate and organizedata or to solve a particular problem. Such a

99

model should meet two design objectives:• Guaranteed Correctness - If you cannot guarantee that your model always yields the right answers, Regulatory Compliance might be jeopardized. Design your model to make errors immediately apparent. • Adaptability - Most likely, the task you want your model to perform will change in time. Design the model so that you or others can easily improve it.

11/17/2008

4

Recommendations• Validate – Risk Based Approach• Document spreadsheet functionality and instructions,

service packs, add-ons, Operating system used, date of installation and version number etc

10

of installation and version number etc. • Use color-coding for data-entry• Describe mathematical formulas used • Document the relationship between the formulas in

the analytical method and the Excel equations• List macro programs – make sure that they are tested

10

Recommendations• Document acceptance criteria (including product

declarations and specifications or USP limits) • Test sheets with anticipated and actual results, signed

and reviewed that have been verified by manual

11

and reviewed, that have been verified by manual calculations.

• Also use abnormal results (beyond boundaries, numeric/alphanumeric mix up, etc.)

• Security and password maintenance. • Include Controls and Checks• Use drop-down list and menus (when possible) 11

Recommendations• Introduce Units in appropriate cells• Format Numbers• Use supplemental cell comments

1212

• Provide error messages• Add validation to data-entry cells• Establish backup/restore mechanisms• Maintain Change Control & Configuration

Management

11/17/2008

5


Supply Chain

RecommendationsUpgrading to Excel 2007? Perform testing assessment!Excel 97 - 2003 Excel 200765,536 rows / worksheet 1,048,576 rows / worksheet256 columns / worksheet 16,384 columns / worksheet(16 777 216 cells) / orksheet (17 179 869 184 cells) / orksheet

13

MD Cluster

(16,777,216 cells) / worksheet (17,179,869,184 cells) / worksheet4 thousand types of formatting per workbook

Unlimited number of types of formatting per workbook

8 thousand cell references per cell Cell references per cell limited by available memory per workbook

Memory management 1 GB Memory management 2 GB

Not support multiple processorsSupports multiple processors and multithreaded chipsets - faster calculations in large, formula-intensive worksheets Supports up to 16 million colors


Supply ChainExcel 97-2003 features that are not supported in Office Excel 2007

http://office.microsoft.com/en-us/excel/HA101988951033.aspx

Recommendations

14

MD Cluster

• Unsupported worksheet features • Unsupported table features • Unsupported formula and function features • Unsupported charting features • Unsupported PivotTable features • Unsupported developer features • Unsupported file formats


Supply Chain

Policies on spreadsheet development

Spreadsheet PoliciesWhen policies on analytical spreadsheet development are applied,

improvement was noticed in both: formatting properties and reduced time in its development and reviewing Additionally the

1515

MD Cluster

reduced time in its development and reviewing. Additionally, the end users are more familiar with the consistency of the worksheets, reducing time in data-entry and training.

When spreadsheet software technology is improved the established policies in spreadsheet development will facilitates the

application of new technology as it become available.

11/17/2008

6

Supply Chain

Policies for Development of Spreadsheets – FDA San Juan District

• Only the front side of the form will be used.• Has exactly 18 columns (width 5.00 (35 pixels)) and 55 rows.• Page Setup (General Continuation Sheet):

• Page: Orientation = Portrait and Scaling = 100% normal size•Margins: Left: 0 75" Top & bottom = 0 2" Right = 0 2"

1616

MD Cluster

•Margins: Left: 0.75 , Top & bottom = 0.2 , Right = 0.2 , Header & footer = 0", •Center on page: Horizontally & vertically. Sheet: Print = Black & white

• Custom Toolbar disabled when form 431 is active. • Short cut menu only functions when appropriate.• Font : Times New Roman, Size 10• File Identification & Path as footer• Protection attributes: Sheets are protected, as appropriate.

Supply Chain

Policies for Development of Spreadsheets – FDA San Juan District

• New columns can not be inserted and any column present can notbe deleted

• To maintain the form’s pre-set space and avoid distortion

• New rows can be inserted or deleted

1717

MD Cluster

• New rows can be inserted or deleted


Supply Chain

483 Items: Radio Chemical Active Pharmaceutical Ingredient Manufacturer - Nov2007

“ Your firm failed to validate the Excel spreadsheet used to calculate the monthlyverification of the Gamma Analyzer Spectrometry Detectors Efficiency. These Excelspreadsheets are also used as a primary QA tool to verify the radioactivitycalculations/measurements entered in the batch production records. Discrepancies

b d b t th d h t lt d th l l ti lt

483 Observations

18

MD Cluster

were observed between the spreadsheets results and the calculation resultsmanually entered in the batch records.”

“Failure to assure that when computers or automated data processing systems areused as part of the production or quality system the manufacturer shall validatecomputer software for its intended use according to an established protocol, asrequired by 21 CFR 820 .70(i). For example, electronic records are used, but therewas no software validation. No procedures are established to validate for itsintended purpose the Microsoft Word or Microsoft Excel software used increating and maintaining nonconformance records, product return records,internal audit corrective action records, or preventive action records.”

11/17/2008

7


Supply Chain

Sub Chapter 425 - Computerized Drug Processing

Sec. 425.100 Computerized Drug Processing; CGMP Applicability To Hardware and Software (CPG 7132a.11)

Sec. 425.200 Computerized Drug Processing; Vendor Responsibility (CPG

FDA Compliance Policy Guides Manual , Chapter 4 Human Drugs - Contents

1919

MD Cluster

http://www.fda.gov/ora/compliance_ref/cpg/cpgdrg/default.htm#420%20-%20Compendia

Issued: 10/19/84 Revised: 9/4/87

p g g p y (7132a.12)

Sec. 425.300 Computerized Drug Processing; Source Code for Process Control Application Programs (CPG 7132a.15)

Sec. 425.400 Computerized Drug Processing; Input/Output Checking (CPG 7132a.07) As required by 21 CFR 211.68 the extent and frequency of input/output checking will be assessed on an individual basis


Supply Chain

Good Laboratory Practice (GLP) Regulations

The use of spreadsheets by FDA regulated industries to judge the quality of a product is covered by cGMPs, GLP, regulations and ISO 17025 accreditation. The following are some of the requirements established by GLP regulations:

2020

MD Cluster

requirements established by GLP regulations: Configuration ManagementWritten Standard Operating Procedures: 21 CFR 58.81 (a) (2)Completeness of Data: Compliance Policy Guide (CPG)

7132a.07 (21 CFR 211.68) defines the program for input/output (I/O) checking

Raw/Secondary Data: 21 CFR 58.3 (k) (2)Electronic Records / Security: 21 CFR Part 11Training: 21 CFR 58.29 (a) (2)


Supply Chain

GMP Predicate Rule Requirement for Validation

21 CFR 211.68 (a)Automatic, mechanical, or electronic equipment or othertypes of equipment, including computers, or relatedsystems that will perform a function satisfactorily, may

2121

MD Cluster

y p y, ybe used in the manufacture, processing, packing, andholding of a drug product. If such equipment is so used,it shall be routinely calibrated, inspected, or checkedaccording to a written program designed to assureproper performance. Written records of those calibrationchecks and inspections shall be maintainedPredicate rules are: The Federal Food, Drug, and Cosmetic Act, the Public Health Service Act, any other statute FDA enforce, and any rules implementing those statutes.

11/17/2008

8


Supply Chain

Medical Device Predicate rule Requirement forValidation

21 CFR 820.70 (i):‘When computers or automated data processing

2222

MD Cluster

When computers or automated data processingsystems are used as part of production or thequality system, the manufacturer shall validatecomputer software for its intended use according toan established protocol. All software changes shallbe validated before approval and issuance. Thesevalidation activities and results shall bedocumented.


Supply Chain

Part 11 CSV Requirement Subpart B--ElectronicRecords Sec.

11.10 (a) ‘Validation of systems to ensure accuracy,

2323

MD Cluster

0 (a) a da o o syste s o e su e accu acy,reliability, consistent intended, performance, and theability to discern invalid or altered records.’

Note: Part 11 requirements include, for example, certain provisions of cGMP regulations (21 CFR Part 211), the Quality System regulation (21 CFR Part 820), and the Good Laboratory Practice for Nonclinical Laboratory Studies regulations (21CFR Part 58)

FDA will apply enforcement discretion for the following areas

o Validation

2424

o Audit trails o Copies of records o Record retention o All Part 11 requirements for legacy systems (pre 8/20/97)

Note: However, records must still be maintained or submitted in accordance with the predicate rules – Agency can take regulatory action for noncompliance with such predicate rules.

11/17/2008

9


Supply Chain

CFR 21 Part 11 – San Juan District Laboratory

San Juan Lab is evaluating a paperless work environment and id i t l th t h l d h t hil

2525

MD Cluster

we are considering tools that help us use spreadsheets while complying and better understanding of 21 CFR Part 11 is

achieved.


Supply Chain

Use of Excel Spreadsheets at FDA Laboratory- San Juan

Some concerns we have are:

1. File accountability

2626

MD Cluster

2. Control of templates3. Verification & Validation 4. Backup of data5. File formats6. Reviewing process7. Time of spreadsheet development


Supply Chain

Possible Process improvements: 1. Analytical worksheet packages need not to be printed (saving a

considerable amount of money)2. Analyst printing time will be reduced improving laboratory

efficiency


2727

MD Cluster

efficiency3. Review time of worksheet packages will be reduced due to the

review of the actual electronic file where excel formulas and all of the formatting properties can be verified.

4. Accountability for all analytical workbook files generated by the analysts as well as accountability for all official templates.

5. Permits sending analytical worksheet packages by electronic means instead of FedEx

6. Spreadsheet policies improve standardization and reviewing time

11/17/2008

10


Supply ChainProject Limits:

This project covers the process of data integrity from thegeneration of original raw-data to the generation and completion


2828

MD Cluster

of the analytical workbook files and its accountability thrumanagement of files.

Instrument raw data (HPLC/GC chromatograms, UV/Visspectrums, etc.) will be considered with the process of changingExcel worksheets and instrument print out to “PDF” documents.


Supply ChainProject Goals:

To transform the analytical laboratory into a paperlessenvironment under Part 11 Rules utilizing a tool that permits


2929

MD Cluster

electronic signatures and audit trails. Reduce time on worksheetdevelopment during analysis and to reduce time on the reviewingprocess.

Laboratory Data Integrity is maintained.

Instrument raw-data (chromatograms, spectrums, etc.) files to beconverted to “PDF” files.

ConclusionExcel spreadsheets are a useful tool in the

3030

Excel spreadsheets are a useful tool in the regulated environment. We will enforce

regulatory scrutiny on spreadsheet use whenever they are used for compliance with cGMP’s.

11/17/2008

11

Dennis CantellopsQA / Manager

3131

U.S. Food & Drug AdministrationSan Juan District(787)474-4795

[email protected]

Thanks for your attention

1

Title Page

How to Validate Excel SpreadsheetsTyson M. Mew

Ofni Systems, Inc.

October 30October 30thth, 2008 Meeting, 2008 Meeting

In the News

FDA Awards up to $2.5 Billion to Modernize Information Technology

• September 30, 2008 - The U.S. Food and Drug Administration today announced the selection of ten contractors to receive up to a total of $2.5 billion for

Ofni Systems 2

contractors to receive up to a total of $2.5 billion for information technology (IT) and data center management services over the next ten years. The contract is the cornerstone of the FDA’s Information Technology for the 21st Century (ICT21) bioinformatics initiative, an extensive IT modernization program encompassing data management, data warehousing, IT infrastructure and IT security.

In the News

• FDA hires 1,300 new doctors and scientists– The FDA's enforcement branch, which has

lost many field inspectors in recent years will

Ofni Systems 3

lost many field inspectors in recent years, will get 245 new staffers.

2

FDA Expectations for Compliance

• All computer systems which control Quality and Production data must be validated and the data within must be controlled.

21 CFR 11.10 • Persons who use closed systems to create, modify, maintain, or

transmit electronic records shall employ procedures and controls designed to ensure the authenticity integrity and when appropriate

Ofni Systems 4

designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and to ensure that the signer cannot readily repudiate the signed record as not genuine. Such procedures and controls shall include the following:

– (a) Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.

Spreadsheet Warning Letters

• Dated 5/17/07: Software used as part of the production quality system was not validated. Specifically, spreadsheets intended to check for outliers and calculate mean, SC, % CV.

Ofni Systems 5

• Dated 1/17/06: “There is no data to demonstrate that the quality control/quality assurance spreadsheets used for tracking and trending various quality metrics have been properly validated.”

Selected 483 Observations

• The following deficiencies were found in the software used in the laboratory to control instrument operation and store raw test data for method validation studies

Ofni Systems 6

test data for method validation studies– The software was not validated– The software did not secure data from

alteration, inadvertent erasures, or losses– The software did not provide a working audit

trail to track changes made

3

Selected 483 Observations

• Computer input and output not checked for errors

• Software used in monitoring program not validated

Ofni Systems 7

validated• Notable Observation

– Original laboratory data were found on scrap paper with a note stating:

“Do you want this to be recorded in note book?”

Results of Non-Compliance

Ofni Systems 8

• http://www.fda.gov/opacom/Enforce.html

Validation Strategy Overview

• We define our basic documentation practices and methodology in a single document, the Spreadsheet Validation Master Plan. This document can be referred to for all individual spreadsheet validation projects, and will not need to be reviewed or approved for each validation effort

Ofni Systems 9

validation effort.• We define the requirements for each sheet or chart in

the workbook, then focus our testing on verifying these requirements.

• A heavy emphasis is placed on defining and testing formulas, and also on the security for each sheet by limiting the parts of each sheet that the users are allowed to edit.

4

Requirements Specification

• List all sheets and charts in the workbook.• For each Sheet, define:

– What users should be allowed to enter into the sheet. These are your Input Cells.

•• HintHint You may find it helpful here and for later use to label h i t ll th b l b l d i th d i

Ofni Systems 10

each input cell, or they can be labeled in the design specification document.

– What formulas or calculations exist on the sheet.– What cells represent the output or final calculation of

the sheet.• For each Chart, define key properties (Title, axis

labels and units), and the datasets used to create the chart.

Design Specification

• The purpose of this document is to describe how the requirements have been implemented.

• This document should include enough information so that a developer could create the

Ofni Systems 11

pentire software project based on the information contained within this document

• We like to break each sheet into four sections: Inputs, Processing, Outputs and Security.

Design Spec - Inputs

• Document the cells users are expected to enter or update data. – In an automated system, you should also

define the source or the input data or

Ofni Systems 12

define the source or the input data or instructions.

• If any validation rules are used to enforce proper data entry, these should be documented here as well.

5

Design Spec - Processing

• Processing is mostly about documenting the formulas that are used on the sheet. Any custom macros or code is also documented here.

• The majority of errors that we find when

Ofni Systems 13

j yvalidating sheets is in the formulas.

• The best way we have found to catch these errors is to define each formula using the actual names of the variables represented as input cells.

Ex: Documenting Formulas

• It may be easy to document that the range of cells F10:F20 contains the formula =(A10*$C$5)/($D$5*B10). However, it is difficult to verify if this formula is correct

Ofni Systems 14

difficult to verify if this formula is correct. • We recommend writing the formulas out

like this:

Example of Documenting Macros

• The other kind of processing is the validation of macros and code that is used in the spreadsheet. Here, the design specification is a good place to copy the code and annotate as

Ofni Systems 15

needed to describe the purpose of each macro or function.

• Note: If you have good coding standards and user proper headers and comments in your code, this step may be already done for you.

6

Design Spec - Outputs

• Outputs usually fall under one of the following three main categories:– The cell or range of cells that contain the final

result of all previous calculations

Ofni Systems 16

result of all previous calculations– Charts – many times these are printed and

saved with external reports– Data that is copied into a final result sheet or

exported to a separate file or database.

Design Spec - Security

• Examples of Security Settings– “All non-input cells should be locked to prevent

changes.” – “Only members of the group ‘Labs’ shall be able to

th d h t f diti All th

Ofni Systems 17

open the spreadsheet for editing. All other users may open the spreadsheet in read-only mode.”

– You can also include additional security settings if you are using either custom code or a third party add-on to implement multiple levels of security to control who can edit certain cells, run a macro or function, etc.

Test Protocols

• The testing of any spreadsheet should prove that the requirements were properly implemented according the design specifications.

• Installation Qualification (IQ) testing is usually limited to making sure the file is in a location where users can

th fil l th kb k i t f l

Ofni Systems 18

access the file, unless the workbook is part of a larger automation project.

• Operational Qualification (OQ) testing is mostly about verifying formulas, macros, and also to test the security of each sheet to verify that all non-input cells are locked to prevent changes. The IQ and OQ can be combined into a single IOQ protocol as needed.

7

Input Testing

• Some examples of the types of test cases to write for input testing.– What data can be entered into each input

cell?

Ofni Systems 19

cell?– Are validation rules being enforced?

Process & Output Testing

• These are usually easier to combine when writing test cases.

•• FOCUS ON TESTING THE FORMULAS!FOCUS ON TESTING THE FORMULAS!• Are formulas correct? There are several

Ofni Systems 20

Are formulas correct? There are several methods for verifying and testing formulas. – Visual inspection of each formula or range of

formulas– Verification of each numerical calculation

using a calculator.

Verify Formula Consistency

• Testing for consistent formulas in a certain range. – Put the same value in all input cells to verify that all formulas

report the same output value.– Alter the first input cell to verify that the first calculated value and

the final result changes.– For critical application, continue altering each subsequent input

cell to verify that all formulas report the same value For less

Ofni Systems 21

cell to verify that all formulas report the same value. For less critical sheets, you may just want to test altering the first, last and a third input cell in the middle to spot-test formulas.

• Continue testing until all input cells have had data entered on changed, and that result has been verified.

• The key point is to look for formulas that are incorrect or charts that are not using the correct or complete set of data, or are pointing to the wrong columns of data.

8

Testing Macros and Charts

• Macros can be tested by entering a range of data and comparing the results with a hand calculator, or by visual inspection that the function performed as expected

Ofni Systems 22

that the function performed as expected.• Charts can be tested by a combination of

visual inspection or verification of the properties, including the dataset used as the basis for the charts.

Security Testing

• The type and amount of security testing that you can do for spreadsheets is largely based on how security has been implemented.

• At a bare minimum, you should test that: li it d t t i d t i t th d fi d

Ofni Systems 23

– users are limited to entering data into the defined input cells only

– that they do not have the ability to alter any other part of the spreadsheet.

• Failure to do this can compromise the integrity of the validation effort and of any data or information generated by the workbook.

Conclusion

• This methodology will result in a User/Functional Requirements Specification, a Software Design Specification, and an IOQ Protocol ready for approval and execution.

• Any deviations found during testing can be

Ofni Systems 24

• Any deviations found during testing can be handled according to existing validation practices

• The Summary Report should show that all the activities and deliverables specified in the Validation Master Plan or SOP have been accomplished or generated.

9

Thanks for Coming!

Ty Mew(919) 844-2494Ofni Systems Inc

Ofni Systems 25

Ofni Systems [email protected]

References and additional materials will be available at www.OfniSystems.com/References at the conclusion of the conference.

Documents

Validation of Excel an FDA - Welcome to QCSV Inc. … Spreadsheets in WL’s Drager Medizintechnik GmbH, 1999 WL-M2848N • “Failure to validate comppputer software used as part