What’s new in Anatomic Pathology Quality Assurance? Raouf E. Nakhleh, MD Mayo Clinic Florida

What’s new in Anatomic Pathology Quality Assurance? Raouf E. Nakhleh, MDMayo Clinic Florida

Disclosure InformationRaouf E Nakhleh, MD

• I have no financial relationships to disclose

Objectives

• Discuss evidence based guideline development and their impact on the future from a QA perspective

• Discuss issues related to immunohistochemistry validation

• Discuss ongoing assessment of practicing pathologists

Why these topics

• Not new

• Subject to regulation (USA)

• Formalization of issues not previously addressed

Evidence Based Guidelines

• How does one judge good medical practice?

• Much easier in retrospect, particularly when there is a negative outcome.

• Literature is vast with conflicting knowledge

• In the past, informal evolution of practice

• Emphasis on local standards

• Today national standards


• In particular situation: • Guidelines provide a road map on how to

proceed.• Guidelines help us judge appropriateness of

care• From a practitioner’s perspective, easier to

know if you are meeting your obligations• From a quality assurance perspective, easier

to create measures for comparison

Evidence Based Guidelines – Examples

• In pathology• Cancer protocols and checklists

• Standard set by physicians• Reporting standards• Accreditation by Commission on Cancer

• Clinical Laboratory Improvement Act • Standard set by legal decree• Procedural and quality standards


• Cancer reporting

• 1980 report• Breast, right, mastectomy:

• Infiltrating ductal carcinoma• 1 of 20 lymph nodes positive • Margins free

Evidence Based Guidelines• INVASIVE BREAST CANCER• • Macroscopic• 1. Specimen (partial breast, total breast including nipple and skin, etc)• 2. Procedure (excision without wire‐guided localization, total mastectomy, etc)• 3. Lymph Node Sampling (no lymph nodes present, sentinel lymph nodes, etc)• 4. Specimen Integrity (single intact specimen, multiple designated specimens, etc)• 5. Specimen Size (greatest dimension in cm)• 6. Specimen Laterality (right, left, not specified)• Microscopic• 7. Tumor Size (size of largest invasive cancer, may also be based on macroscopic)• 8. Tumor Focality (single focus, multiple foci, etc)• 9. Skin extent (not present, not involved, invades dermis, etc)*• 10. Nipple extent (not present, not involved, DICS, Paget’s disease)• 11. Skeletal muscle extent (not present, present, etc)• 12. Ductal Carcinoma In Situ (DCIS not present, present, etc)• 13. Lobular Carcinoma In Situ (LCIS not present, present, etc)• 14. Histologic Type of Invasive Carcinoma (ductal NOS, mucinous, papillary, etc)

• 15. Nottingham Score Overall Grade (1, 2, 3, not graded)• 16. Tubular Differentiation (1, 2, 3, not graded)• 17. Nuclear Pleomorphism (1, 2, 3, not graded)• 18. Mitotic Count (1, 2, 3, not graded)• 19. Margins (involved by invasive cancer, involved by DCIS, etc)• 20. Lymph Nodes ‐ total number examined*• 21. Lymph Nodes ‐ number with micrometastases*• 22. Lymph Nodes ‐ number with macrometastases*• 23. Lymph Nodes 0 size of largest metastatic deposit*• 24. Tumor pT Stage (pTX, pT0, etc)• 25. Regional Lymph Nodes pN Stage (pNX, pN0, pN1a, etc)• 26. Distant Metastasis M Stage (not applicable, pM1, etc)• Ancillary Studies• 27. Estrogen Receptor Disposition (performed, not performed, etc)• 28. Progesterone Receptor Disposition (performed, not performed, etc)• 29. HER2 Disposition (performed, not performed, etc)• * If not applicable to the specimen, code as if the element was present in the report

Evidence Based Guidelines – Laws

• Physician Quality Reporting System• Tax Relief and Health Care Act of 2006• Incentive pay (up to 2% of eligible Part B) for

physician who report quality measures (pay-for-reporting)

• Voluntary program• 2015 becomes mandatory

Physician Quality Reporting System

• Colon Cancer grade and stage (2008)

• Breast Cancer grade and stage (2008)

• Barrett’s esophagus (2012)

• Prostate grade and stage (2012)

• HER2 determination in breast cancer (2012)

Accountable Care Organization (ACO)

• Payment of ACO’s rewards quality and efficiency rather than volume

• Monitor practice patterns and use performance data to improve the quality of care• CMS establishes performance measures• If met, then eligible for shared savings• Over time - higher standards

• Use evidence-based guidelines

Overview of Potential Quality System

Evidence based medicine

Practice Guidelines

Addition of standardsInspection, QA review and PT of an institutions

National or local QA review

Accreditation

Improvement

Clinical studies


• What’s in our present and future?

College of American Pathologists Pathology and Laboratory Quality Center

• The Center develops evidence-based guidelines and consensus statements related to the practice of pathology and laboratory medicine. Through them, we continually improve the quality of diagnostic medicine and patient outcomes.

College of American Pathologists Pathology and Laboratory Quality Center

• HER2, ER and PR before official Center creation• CAP/ADASP Consensus Statements on Effective

Communication of Urgent Diagnoses and Significant Unexpected Diagnoses in Surgical Pathology and Cytopathology (published)

• Validating Whole Slide Imaging Systems for Diagnostic Purposes in Digital Pathology(complete, not published)

• CAP/ASCCP Lower Anogenital Squamous Terminology (LAST) Standardization Project for HPV-associated Lesions (complete, not published)

• CAP/IASLC/AMP Molecular Testing Guidelines for Selection of Lung Cancer Patients for EGFR and ALK Tyrosine Kinase Inhibitors(near completion)

Guidelines in the works• Immunohistochemistry (IHC) Assay Analytic Validation

Principles• CAP/ASH Algorithm for Initial Work-up of Acute Leukemia• ASCP/CAP/AMP/ASCO Molecular Markers for the Evaluation of

Colorectal Cancer• CAP-ADASP Interpretive Diagnostic Error Reduction in

Surgical Pathology and Cytopathology• Bone Marrow Synoptic Reporting for Hematologic Neoplasms• CAP-NSH Uniform Labeling Requirements for Blocks and

Slides in Surgical Pathology• ASCO/CAP Guideline Recommendations for HER2 Testing in

Breast Cancer – Update to 2007 Edition• ASCO/CAP Guideline Recommendations for HER2 Testing in

Gastric Cancer

Selection Criteria for Center Topics1. Patient risk, patient safety and quality; issues that

affect patient care and quality outcomes2. Performance characteristics of assay and ability to

reach consensus. Do problems exist including high false positive/false negative rates and lack of uniformity in practice with significant ability to reach consensus?

3. Amount of available evidence to predict a clinical response to an agent or change a patient outcome

4. Guidelines are lacking but regulatory bodies or other professional organizations are trying to preempt specialty development of guidelines or approval/disapproval or show unusual interest

5. Adoption momentum of a particular assay. Consider the number of pathologists affected.

18

Selection Criteria for Center Topics6. Feasibility of significantly changing practice

of pathology or medicine (e.g., using Ultra Sound Guided FNA to change cytology practice).

7. Ability to collaborate with the right partner(s) in development of a Center product to facilitate development and acceptance, and improve overall probability of success.

8. The public's perception of the issue negatively affects the image of pathologists and the pathologists' role in medicine.

9. Evaluation of existing methods is obsolete or does not bring value in patient care

19

Submit and Select Ideas

Determine Scope and Form Workgroup

Research and Review

Solicit Public Comment

Complete Recommendations

Review and Approve

Publish and Implement

Maintain

The Center Process for Developing Guidelines and Consensus Statements

IHC Antibodies and FDA Classification

• Antibodies are medical devices • Classification base on patient safety• Class I

• Adjunctive diagnostic information • Class II

• Prognostic or predictive data • Separately reported• More stringent quality assurance

• Class III• Require FDA premarket approval

HER2 Guideline

• Provide the appropriate HER2 assay validation procedure• Validation with FISH or with another IHC lab

• Prescribe appropriate fixation time and documentation: 6 – 48 Hours

• Modifies scoring criteria to enhance specificity

• Ongoing competency assessment for pathologists

• Enroll in HER2 proficiency testing service

HER2

• Guideline introduced in late 2006

• Mandated in 2008

• LAP checklist standards related to HER2 effective in late 2008

• CAP sponsored survey to determine how well labs are able to comply (late 2008)

• Leads to insight of effectiveness of guideline (2010)

LAP HER2 Standards

• Properly validate HER2 assays

• Ensuring appropriate fixation

• use the ASCO/CAP scoring criteria

• Enforce HER2 proficiency testing

2008 HER2 Survey

• Arch Pathol Lab Med 2010;134: 728-734

• HER2 assay validation

• Concordance with FISH• 81% of Labs achieved 95% concordance for(-)• 73% of Labs achieved 95% concordance for(+)

• Concordance with another IHC LAB• 72% of Labs achieved 95% concordance for(-)• 68% of Labs achieved 95% concordance for(+)

2008 HER2 Survey

• Fixation time and documentation

• 86% made changes to address fixation

• 60% process specimens on the weekend

• Documentation is addressed in reports in 70% of laboratories

2008 HER2 Survey

• ASCO/CAP scoring criteria

• 84% using criteria

• Ongoing pathologists competency assessment

• 91% have a program in place

2008 HER2 Survey – Summary

• Most labs are meeting HER2 guideline

• Gaps still exist, particularly validation

• Deficiencies hopefully corrected over time with inspection cycle

• Follow up survey conducted at the end of 2011, currently under analysis

Validation

• What about other antibodies?• Gap in understanding of Validation in AP• Few documents address IHC antibody

assay validation

• Definition• Confirmation through a defined process

that a method performs as intended

31

Principles of Test Validation

• Use manufacturer’s instructions• Use high-quality test materials with

known target values• Test materials should be of similar type

as patient samples• Complete documentation, including

procedure and results• Lab director review/approve results

IHC Validation

• For cell markers, (e.g. keratin, actin, Melan A etc.) how many cases should be performed in validation study?

• Should you validate with another lab or another technique?

• What level of concordance is acceptable?

• Are there situations where validation is not necessary?

• What about cytology material?

IHC Validation – Example • New antibody introduction – Prognostic • Example: MIB-1 (Ki-67)

• Proliferation marker (estimate of mitotic count)• Antibody needs to work at spectrum of findings (0 –

100%)• Include tissues with various levels of proliferation

• Carcinoid tumors of lung (0-10%)• Typical carcinoid (0-3%)• Atypical carcinoid (2-10%)• Small cell carcinoma (25-90%)

• Validation cases should cover the spectrum of findings

Immunohistochemistry Validation Procedures and Practice: A College of American Pathologists Survey of 727 Laboratories

Lindsay Hardy, MD (Boston, MA)Raouf Nakhleh, MD (Jacksonville, FL) Jeffrey Goldsmith, MD (Boston, MA)Patrick Fitzgibbons, MD (Fullerton, CA) Richard Eisen, MD (Greenwich, CT)Mary Beth Beasley, MD (New York, NY) Rhona Souers, MS (Northfield, IL)

Methods• September 2010: Questionnaire

• 1064 surveys distributed with CAP proficiency test mailing (HER-2)

• 754 returned (Oct. 2010)• 27 excluded• 727 included

• 2 major sections• Non-FDA approved, non-predictive• Non-FDA approved, predictive other than

HER-2

Results: Non-predictive

Validation Procedures Percent

Laboratory has a procedure for validation of new antibodies? 68

Procedure specifies the number of cases to be used? 54


• 68% had a written procedure

• 86% validated the most recently introduced non-predictive marker (despite lack of procedures for some)


Validation Procedures 10th Pctl Median 90th PctlMinimum # Cases to validate new antibody: Positive 2 8 20 Negative 0 5 15 Total Cases 4 13 30

Validation Procedures 10th Pctl Median 90th PctlMinimum # Cases to validate new antibody: Positive 2 8 20 Negative 0 5 15 Total Cases 4 13 30Acceptable concordance (%): Positive 80 95 99 Negative 50 95 99

Validation Procedures 10th Pctl Median 90th PctlMinimum # Cases to validate new antibody: Positive 2 8 20 Negative 0 5 15 Total Cases 4 13 30Acceptable concordance (%): Positive 80 95 99 Negative 50 95 99

# Cases used in most recent validation: Positive 2 7 20 Negative 1 5 15


Most recent validation Percent

Weakly or focally positive cases were included 40

Results confirmed by running parallel at another lab (same tissue) 18

Cases were tested on multiple days (between-run precision) 53

Results: Predictive, excluding HER2

Predictive Markers:o ER o PRo ER/PRo CD117/C-KITo KI-67

o P53o P63o PMS2o EGFRo Other

Validation Procedures Non-Predictive

Percent PercentLaboratory has a procedure for validation of new antibodies? 46 68



Procedure specifies the number of cases to be used? 65



Procedure specifies the number of cases to be used? 65 54

Results: Predictive, excluding HER2

• 46% had a written procedure

• 75% validated the most recently introduced predictive marker other than HER-2

Results: Predictive, excluding HER2Validation Procedures 10th Pctl25th Pctl Median 75th Pctl90th PctlMinimum # cases to validate new Ab:

Positive 3 5 11 20 25 Negative 1 5 10 15 25 Total Cases 5 10 25 35 50Acceptable concordance (%): Positive 90 90 95 95 99 Negative 50 90 95 95 99

# Cases used in most recent validation:

Positive 2 5 10 20 30 Negative 1 3 5 13 20

Revalidation


Percent Percent

Procedure specifies repeat validation for:

New lot of antibody 64 66 Antigen retrieval 80 71 Detection system 81 74 Instrumentation 74 74 Fixative 78 65 Tissue processor 55 49


Percent Percent


New lot of antibody 64 66 Antigen retrieval 80 71 Detection system 81 74 Instrumentation 74 74 Fixative 78 65 Tissue processor 55 49Procedure includes specifications for use with cytology? 42


Percent Percent


New lot of antibody 64 66 Antigen retrieval 80 71 Detection system 81 74 Instrumentation 74 74 Fixative 78 65 Tissue processor 55 49Procedure includes specifications for use with cytology? 42 37

Summary• IHC validation: 727 labs

• Validation was performed at higher rates than the availability of written procedures

• Labs are generally following procedures• Most meet designated minimum number of cases • Trend to slightly fewer

• Guideline is needed to assist in Predictive and non-predictive IHC

• Uncertainty as to the # of cases needed• Uncertainty as to when to revalidate vs. verification• Uncertainty RE validation in cytology

Ongoing assessment of practicing pathologists

• Why now?

American Board of Pathologists

• Time limited certification (10 yrs)

• The four part MOC process requires diplomates to submit documentation in the following areas:

• Part I – Professional Standing

• Part II – Lifelong Learning and Self-Assessment

• Part III – Cognitive Expertise

• Part IV – Practice Performance Assessment

Joint Commission Standards

• Ms.08.01.01 Monitoring Performance; The organized medical staff defines the circumstances requiring monitoring and evaluation of a practitioner's performance (FPPE)

• Ms.08.01.03 Use of Monitoring Information: Ongoing professional practice evaluation information is factored into the decision to maintain existing privileges, to revise existing privileges, or to revoke an existing privilege prior to or at the time of renewal

Professional Practice Evaluation

• Use of objective performance data in the granting and/or maintenance of practice privileges

Ongoing Professional Practice Evaluation – OPPE • On an ongoing basis for privileged practitioners

Focused Professional Practice Evaluation – FPPE • When a practitioner is first privileged• When new privileges are first granted to an already privileged

practitioner• When insufficient activity or “performance issues” are identified

for a privileged practitioner

Concepts

• Professional competency assessment• Is part of the peer review process

(protected)• Is not publicly reported or stored in

credentialing files• Is a tool for evaluation, not an adverse

action• Produces some data points for

assessment • Thresholds are minimum bars, not goals

OPPE – Six Core Competencies

• Patient care

• Medical/Clinical knowledge

• Practice-based learning and improvement

• Interpersonal and communication skills

• Professionalism

• Systems-based practice

OPPE - Ongoing Professional Practice Evaluation

REQUIREMENTS

• Evaluation must be ongoing – More frequently than annually

• Process must be clearly defined • What data (“metrics”) will be monitored• Who will be responsible for review of data• How often data will be reviewed• How performance data will be used for decision

making• How data will be archived and made available


POSSIBLE OUTCOMES

• Privilege being evaluated may be continued, limited, or revoked

POSSIBLE ACTIONS

• Continue privilege – no action required

• Suspend or revoke privilege because it’s no longer performed or required

• Initiate FPPE – Due to insufficient activity or identified “performance issues” with privilege


• WHAT METRICS ARE APPROPRIATE FOR PATHOLOGISTS?

• Metrics for core vs. delineated privileges• Core privileges – Privileges broadly applicable to the

majority of practitioners in a discipline• e.g. anatomic pathologists, clinical pathologists,

etc.• Delineated privileges – Privileges specific to a

subset of practitioners within a discipline• e.g. surgical pathologists, cytopathologists,

hematopathologists, microbiologists, etc,


• EXAMPLES OF METRICS FOR CORE PRIVILEGES IN AP OR CP• Patient care – Turnaround time for key reports• Medical knowledge – General CME credits, successful MOC• Practice-based learning and improvement – Participation in

SAMs• Interpersonal and communication skills – 360° evaluation

provided by clinicians, lab staff, etc.• Professionalism – Attendance at medical staff meetings• System-based practice – Membership on hospital or other

organizational committees


• EXAMPLES OF METRICS FOR DELINIATED PRIVILEGES IN SPECIFIC AREAS

• Patient care• Surgical pathology – Turnaround time for biopsy reports

Number or % amended reports IOC vs. permanent dx

discrepancies• Cytopathology – FNA success rate• Clinical pathology subspecialties – timeliness of

administrative and clinical reports (transfusion reaction)• All subspecialties – Peer review of new method validations

in relevant lab section(s)• Medical knowledge

• Each subspecialty – CME credits relevant to subspecialty

FPPE - Focused Professional Practice Evaluation

• DIFFERENCES FROM OPPE• More restricted in scope – “focused”• Episodic rather than continuous, with finite endpoint

• SIMILARITIES TO OPPE• Basic process must be pre-defined and consistent• Similar monitoring methodologies (review of records,

direct observation, monitoring practice techniques, discussion with others involved in care)

FPPE - Focused Professional Practice Evaluation

• CIRCUMSTANCES WHEN NEEDED1. When a practitioner is first privileged2. When new privileges are first granted to an

already privileged practitioner3. When insufficient activity or “performance

issues” are identified for a privileged practitioner

Thank You!

Documents

What’s new in Anatomic Pathology Quality Assurance? Raouf E. Nakhleh, MD Mayo Clinic Florida