Auditing & Quality Assurance

Auditing & Quality Assurance

Igor Espinoza-Delgado, M.D.Gary Smith M.Sc.Cancer Therapy Evaluation Program

Health CareDelivery Research

Good ClinicalPractice

Clinical Trials

Good Clinical Practice (GCP)•An internationally recognized standard

for the design, conduct, performance, monitoring, auditing, recording,

analyses, and reporting of clinical trials. • GCP provides assurance

that the data and reported results are credible and accurate, and that the

rights, integrity, and confidentiality of trial subjects are protected.

Regulatory Basis forMonitoring and Auditing

1962 Harris-Kefauver amendments to the food, drug, and cosmetic act required that FDA regulate the testing of new drugs as an investigational new

drug (IND) prior to drug approvalDemonstration of efficacy and safety in well-

controlled clinical investigations

Regulatory Basis forMonitoring and Auditing (Cont.)

FDA IND regulations (21CFR 312.50) requires sponsors to ensure proper monitoring of

investigation(s), ensure that investigations are conducted in accordance with approved protocols as contained in the IND, and ensure that the FDA

and all participating investigators are promptly informed of significant new adverse effects or risks

with respect to the drug.

All those planned and systematic actions that are established to ensure that the

trial is performed and the data are generated, documented (recorded), and

reported in compliance with GCP and applicable regulatory requirements.

Quality Assurance

Patient Safety/Ethical Considerations-Respect for Persons

-Beneficence: maximize possible benefits and minimize possible harms

Regulatory Considerations-Delay in product approval

-Approval withdrawals-Sanctions

Why Be Concerned With Quality?

Monetary Considerations-$802 million: total average preclinical and

clinical costs up to the time of receiving FDA marketing approval

-$897 million fully capitalized cost to develop a new drug, including cost of conducting post

marketing surveillance studies after receiving regulatory approval

(Tufts Center for the Study of Drug Development)


Public Trust


Staff training/mentoringQuality control

Data safety and monitoringStudy monitoring

Auditing

Components of Quality Assurance

The act of overseeing the progress of a clinical trial, and of ensuring that it is conducted, recorded, and reported in accordance with the

protocol, standard operating procedures (SOPs), GCP, and applicable regulatory

requirements.

Monitoring

A systematic and independent examination of trial related activities and

documents to determine whether the evaluated trial related activities were conducted, and the data recorded, analyzed and accurately reported

according to the protocol, sponsor’s standard operating procedures, GCP, and

the applicable regulatory requirements.

Audit

Auditing: Snapshot in time On-site process Subset of patients

on a trial

Monitoring: Continuous process On-site and/or off-site Includes all patients on

a trial

Quality AssuranceAuditing vs. Monitoring

Objectives:

Assure accuracy and quality of dataAssure compliance with Federal regulations

Assure compliance with sponsor’s policies and procedures

Assure compliance with GCPServes as an educational tool

On-Site Audit Program

Phase of Study

Study Monitoring ReportingMechanism

Auditing

Phase 1 &Select Phase 2

CTMS + CTEP Bi-weekly CRFs to CTMSMonthly CTMS reports, downloads to CDUS

CTMS Pharm.D./M.D. once per yearCTMS/CRATwice per year

Phase 2 & 3 Cooperative Group

Cooperative Group + CTEP

CDUS quarterly

Group physicians, CRAs, nurses once every 3 years

Phase 2 & 3 Cancer Center/Single Inst.

CTEP CDUS quarterly

CTEP + CTMS + peer physicians once every 3 years

CTEP Monitoring and Auditing

Special audits:

-For cause audits

-Response audits

Other Types of Audits

NCI model:

Phase 1 CTMS monitored trials

Phase 2 & 3 trials:

- Monitoring via electronic submission of data quarterly via the clinical data update system (CDUS) and adverse event expedited reporting system (AdEERS).

Industry Model:

Similar to NCI model Phase 2 & 3 trials:

- Sites undergo on-site monitoring visits every 6 weeks with 100% of Case Report Forms included in source data verification during monitoring visit.

Key Differences Between NCI and Industry Model

NCI model: Audit unit is institution

based >10% of patient cases

are audited Scientific incentives

Industry Model: Audit unit is protocol

based >10% of sites are

audited Financial incentives

Key Differences Between NCI and Industry Model (Cont.)

On the day of the audit:

Provide overview of research procedures (scientific review committee, IRB)

Provide overview of organization of the medical records and research charts

Have data management staff available throughout the process if questions arise

Audit Preparation

Audit Preparation

On the day of the audit (cont.):

Meeting with principal investigator to review status of studyPharmacy inspection

Exit interview with P.I. and staff

Regulatory:

-Documentation of initial IRB approval-Documentation of continuing reviews-Documentation of IRB approval of all

amendments-Documentation of IRB review of reportable

adverse events and IND safety reports-Documentation of other IRB

correspondence pertaining to protocol

Audit PreparationWhat Is Needed?

Regulatory (cont.):

-1572s for Principal Investigator and Co-Investigator*-Investigator CV/Medical License*-IRB Membership List*-Office of Human Research Protections (OHRP) Assurance Number-Laboratory Certification* and Lab Normal Ranges

*primarily industry requirement


Pharmacy:

-Drug Accountability Records as provided by the sponsor along with

shipping receipts, return receipts, and transfer forms when applicable-Appointment with pharmacy to

conduct inspection


Patient medical records:

-ORIGINAL signed and dated informed consent (at time of enrollment as well as at time of re-

consent, if applicable)-Baseline history and physical exam

-Prior therapies including start and stop dates-Pathology report

-Protocol required parameters (labs, x-rays, scans, EKG, etc.)-Physician orders


Patient medical records (cont.):-Medication administration records

-Nursing and physician progress notes-Off-study note

Research Records:-Eligibility checklist

-Confirmation of registration including arm and/or dose assignment

-Documentation of collection/submission of research tests (pharmacokinetics, marker studies, etc.)

-Documentation of tumor measurements


Research Records (cont.):-Outside labs and other study parameters

-Outside physician records and correspondence -Appointment books

-Subject diaries/calendars-Study flow sheets and other research records that are signed and dated on a real time basis by the

health practitioner evaluating the patient-Protocol or study road maps


Classification of deficiencies:

Major: any variance from protocol-specified procedures that makes the resulting data

questionableLesser: any variance that is judged to not have a significant impact on the outcome or interpretation

of the study data

Cooperative Group On-siteAudit Program

Possible Actions:

Re-auditSuspension of patient registrationSuspension of Investigator’s 1572Problematic audit findings may be referred to other agencies, such as the FDA, the Office for Human Research Protections (OHRP), or the Office of Research Integrity (ORI) when warranted

Unacceptable Audits

Reapproval delayed less than 30 daysReapproval delayed greater than 30 days but

less than one yearLack of documentation of full IRB approval of a

protocol amendment that affect more than minimal risk

Delayed reapprovals for protocols closed to accrual for which all patients have completed

therapyMissing reapproval

Most Frequently Occurring Deficiencies: IRB

Most Frequently Occurring Deficiencies: Consent Content

Lack of disclosure of all risks or side effects contained in model informed consent approved by NCILack of disclosure of approximate number of

participantsLack of disclosure of extent of confidentiality of recordsLack of disclosure of the contact person for research

questions, information regarding subject’s rights, and/or contact for research-related injury

Failure to disclose circumstances in which subject’s participation may be terminated by investigator without

subject’s consent

NCI drug accountability record forms (DARFs) incomplete or inaccurate

NCI DARFs not protocol and drug specificSatellite NCI DARFs not accounted for

NCI DARFs not kept as primary transaction record

Most Frequently Occurring Deficiencies: Pharmacy

Consent form does not contain all required signatures or dates

Consent form does not include updates or information required by IRB

Consent form used was not current IRB-approved version at time of patient registrationConsent form not signed and dated by patientConsent form signed after patient started on

treatment

Most Frequently Occurring Deficiencies: Informed Consent

Documentation missing; Unable to confirm eligibility

Review of documentation confirms patient did not meet all eligibility criteria as specified by the protocol

Most Frequently Occurring Deficiencies: Eligibility

Treatment doses incorrectly administered, calculated or documentedDose deviations incorrect

(greater than +/- 10%)Dose modifications unjustifiedUnjustified delays in treatment

Additional agent(s)/ treatment given prohibited by protocol

Most Frequently Occurring Deficiencies: Treatment

Tumor measurements/evaluation of status or disease not performed according to protocol

Claimed response (partial response, complete response, etc.) cannot be verified

Protocol-directed response criteria not followedInaccurate documentation of initial sites of involvementFailure to detect cancer (as in a prevention study) or

failure to identify cancer progression

Most Frequently Occurring Deficiencies: Response

Follow-up studies necessary to assess toxicities not performed

Grades, types or dates/duration of serious toxicities inaccurately recorded

Recurrent under or over reporting of toxicitiesFailure to report a toxicity that would require

filing an adverse event reportReported toxicities cannot be substantiated

in source documents

Most Frequently Occurring Deficiencies: Toxicity

Most Frequently Occurring Deficiencies: General Data Quality

Errors in submitted dataDelinquent data submission

Recurrent missing documentation, e.g., chartsProtocol-specified laboratory tests

not documented

Quality Improvement

General Record Keeping:

Remember, if it is not documented, it cannot be verified as being done

Follow standard procedures for documentation and error correction, including dating

and initialing entries

Quality Improvement

“The myth of perfect performance of the health care worker is unrealistic. We must change to an

engineering mode of thinking that things will always go wrong. We need to protect the patient from results of errors, or failure of safe design. We not only need a

‘fail-safe’ design, but redundancy in the system, because a backup system protects the process. The

system must possess two independent redundant steps, not interdependent steps.”

(Quote from Richard J. Croteau, M.D., JCAHO executive director for strategic initiatives)9

Documents

Auditing & Quality Assurance