Update on ISPE Quality Metrics Project Michael Davidson,26 March 2015 ISPE Quality Metrics Project ISPE Program Background Why a Quality Metrics Pilot Program? Status Future ISPE, Quality Metrics Summit, April 2 I SSUES/ CHALLENGESRAI SED AT I SPEJUNE2013BALTI MORE WORKSHOP How to achieve consistency of metrics Does the metric truly reflect quality and/or compliance? Are metrics by site, product or company? Will quality metrics lead to unforeseen outcomes? Will introduction of metrics lead to unacceptable behaviour, either by industry or regulators? How to clarify and advance comments made by industry at the Baltimore meeting? ISPE Quality Metrics Program To produce a white paper acceptable to industry on quality metrics,which could be reportable to the FDA to support a risk-based inspectionprogram included in Sections 704 to 706 of US FDASIA. The white paper should: Identify proposals targeted for discussion with the FDA and potentially with other agencies.Justify why certain metrics are proposed, considered or rejected Justify the choice of site based and/or product-based metrics Comment on the relevance of and/or relationship to data already provided to regulatory agencies, e.g., Field Alert Reports Be published on ISPE website FIRST GOAL GUI DI NG PRI NCI PLES Clearly defined to allow consistent reporting across sites Objective and Meaningful Easy to Capture Easy to Report Normalized based on factors such as process differences and technical complexity Drive acceptable, not unwanted behaviors ISPE Quality Metrics Program I NI TI ALMETRI CSPROPOSED ISPE Quality Metrics Project 1. Batch Rejection Rate 2. Rework and Reprocessing Rate 3. Confirmed Out of Specification Rate 4. Unconfirmed Out of Specification Rate 5. Critical Complaints Rate 6. % Annual Product Quality Reviews Completed on Time 1. Quality System Effectiveness 2. Process Capability 3. Quality Culture Index 4. Right First Time 5. % GMP Training on Time 6. Unplanned Down Time 7. On Hold Batch Rate Other Metrics Considered WHI TEPAPER RECOMMENDATI ONS ISPE Quality Metrics Program Conduct a Pilot to flesh out the definitions and approach Start with site metrics, move to product metrics later >9,000 reportable sites (by FEI #) >100,000 reportable product (by Application April 16th 2014 ISPE announces Pilot Program Partner with McKinsey for operational support Confidentiality of data Allow blinded comparison with industry average and technology platform peers Responsive to FDAs recommendation for industry to initiate activities Quality Metrics Industry Pilot8 Primary Objectives for the Pilots InitialPhase Test a set of quality performance metrics based on industry and FDA input to date (as of June 2014) Harmonize a set of metric definitions Test feasibility of data collection across different companies Explore industry practices in quality culture and use of process capability Inform continued industry input to the FDA 9 Input from Industry and FDA (Brookings Meeting, May 2014) http://www.brookings.edu/events/2014/05/01-measuring-pharmaceutical-quality, The Brookings Institution 10 Technology Specific metrics Quantitative metrics Media fill (for sterile aseptic sites) failures B Environmental monitoring (for sterile aseptic sites) B Lot acceptance rate B Complaints rate (total B and critical Confirmed OOS rate B US recall events (total and by class) B Stability Failure rate B Invalidated (unconfirmed)OOS rate B Right first time (Rework / Reprocessing) rate B APQR reviews completed on time Recurring deviations rate CAPA effectiveness rate Additional survey-based metrics Process capability Quality culture MEASURI NGQUALI TYPERFORMANCE Quality Metrics Industry PilotConsensus Industry Metrics B - metric proposed in Brookings meeting - Product- and site-based metric Metric set represents both leading and lagging indicators June 2nd ISPE-FDA CGMP Conference Workshop to explain and launch pilot approach Formal check-inwith FDA Timeline Quality Metrics Industry Pilot Pilot: Wave 1 Confirm pilot participants Launch data collection Complete data collection, clean/validate data Share initial findings and correlations Individual report-outs to all pilot participants Issue ISPE report on Wave 1 Plan for Wave 2 June 2014 - Mar. 20152015+ PotentialWave 2 Participate in Brookings meeting to discuss FDA direction Finalize metrics and definitions Prepare data collection templates and surveys Outline analysis methodology Announce pilot details Finalize metrics and timelines April-June 2014 12 13 Diverse sample: 18 participating companies with 44 sites/technologies846818Bio DSSterileSolidsOtherAPI 56528CMO, Labs Cons. health GxRx 731618NAAsiaEMEALA By technologyBy type of product By regionBy company size 395SmallLarge Note: If a site has more than 1 technology we count the number of separate templates they will fill, usually 1 per technology Initial Quality Metrics Pilot Debrief with FDA Sub teams continue to work on Quality Culture Index and Process Capability Readout to Wave 1 participants 1Q 2015 Participants are inputting to ISPE Report Discussions with FDA on going Draft ISPE Report in 1Q 2015 Inform Wave 2 Issue Report for ISPE/FDA Quality Manufacturing Conference, 1 3 June 2015 14 Wave 1 of Pilot Program Next Steps Basis for ISPE Quality Metrics Summit Next Steps Continue to expand participation Assess a potentially adjusted metric set based on learnings from Wave 1 and aligned with FDA technical regulatory agenda Incorporate input from sub-teams working on Quality Culture and Process Capability More focus on current data collection and analysis Continue to expand cooperation with other groups and associations Thoughts on Wave 2 of Pilot Program 15 16 Quality Metrics Summit Pilot Data Available 17 Learn directly from ISPEs Quality Metrics Pilot Program leaders. Hear participating companies experiences to understand the benefits and challenges of your own quality metrics program. Understand what metrics and definitions lead to the most uniform interpretation, least gaming and best quality practices. Recognize how a metrics program can best balance value verses reporting burden Provide your perspectiveinput from the Summit will be incorporated into the final report to be shared with the FDA. Form proposals for implementing your own quality metrics program to take back to colleagues and management for discussion and execution. Quality Metrics Summit 18 Themes FDA presence and listening Learnings from Pilot Quality Culture can it be measured? What would Wave 2 look like? Next Steps for ISPE Summary Industry Pilot contributing valuable insights Major change transformation for industry and FDA Critical to maintain dialogue amongst all stakeholders Recognize call for industry to step up to lead on Quality Metrics 19 I SPEQUALI TYMETRI CSCORETEAM MEMBERS Diane Hagerty (Team Leader) and Matt Pearson, Genentech/Roche Laura Cannon and Lorraine McClain, Teva Pharmaceuticals Michael Davidson, Pfizer Mairead Goetz and Lorraine Thompson, Novartis Peggy Speight, Bristol Myers Squibb Nuala Calnan, Dublin Institute of Technology Steve Lynn, Mylan Paul Weninger, Perrigo Katy George, Vanya Telpis, Lorenzo Positano, Paul Rutten, McKinsey Chris Potter, ISPE Advisor Carol Winfield, ISPE Regulatory Director