Pharmaceutical Manufacturing Research Project Final ...apps.olin.wustl.edu/faculty/nickerson/results/PMRPFinalReportSept2… · Research Project Final Benchmarking Report By Jeffrey

Pharmaceutical Manufacturing Research Project

Final Benchmarking Report

By

Jeffrey Macher McDonough School of Business

Georgetown University Washington, DC 20057

(202) 687-4793 [email protected]

Jackson Nickerson John M. Olin School of Business

Washington University in St. Louis Campus Box 1133, One Brookings Drive

St. Louis, Missouri 63130-4899 (314) 935-6374

[email protected]

September 2006

i

Executive Summary The Pharmaceutical Research Manufacturing Project (PRMP) was launched in 2002. It investigates the effects of product, process, manufacturing site location, firm reputation and experience, and organizational structure and incentives of pharmaceutical manufacturing on manufacturing performance and the likelihood and type of FDA enforcement efforts.

Two phases comprised the PMRP. The first phase involved working with the Food and Drug Administration (FDA) to collect and compile data from FDA internal databases. Based on this data, the project developed risk-based statistical models to assess the probability of a facility being chosen for inspection, to evaluate the effect of investigator training and experience on the probability of investigational outcomes as well as individual investigator effects on the probability of investigational outcomes, and to identify characteristics of facilities and firms that correlate with the likelihood of noncompliance. A preliminary report was delivered to the FDA on January 28, 2005. This report is publicly available from the authors.

The second phase of the PMRP focused on collecting and analyzing data from pharmaceutical manufacturing facilities. The results from this second phase are reported herein. This report summarizes the data collected from 42 manufacturing facilities owned by19 manufacturers. The data collected is presented in this report in two ways. Benchmarking charts compare the data collected from oral and topical (O&T) manufacturing facilities, active pharmaceutical ingredients (API) manufacturing facilities, and injectable (I) manufacturing facilities. These “benchmarking” charts are provided in Appendices A, B, and C, respectively. This report also presents and discusses results from 27 statistical analyses exploring how organizational practices impact various manufacturing performance metrics. Statistical analyses focus on those factors correlated with cycle time yield performance, deviation management outcomes, product unavailability, and process development.

While results from all 27 statistical analyses are presented in the report, we identify five findings that are generally consistent across these analyses.

1. Information technology—electronically and automatically reporting deviations, tracking deviations by lot, tracking deviations by type of issue, tracking people assigned to resolving the deviation, and centrally storing data—universally corresponds to superior manufacturing performance metrics.

2. The locus of decision rights, especially with respect to deviation management, lot failure, lot review, and process validation, matters. The locus of decision rights impacts performance metrics.

3. Facilities engaged in contract manufacturing generally, although not in all instances, correspond to inferior performance metrics.

4. The use of process analytic technology tools generally, although not in all instances, corresponds to worse performance metrics. This correspondence, however, does not imply causation, which means that these tools may be adopted for good reason.

5. Scale and scope of the manufacturing facility have a complex interplay with manufacturing performance. Scale and scope can be both a benefit and a detriment to performance depending on the metric of interest and the type of production process.

ii

Table of Contents

1. Introduction _______________________________________________________________ 1

2. Data Collection: PMRP Questionnaire__________________________________________ 2

3. Data Verification ___________________________________________________________ 6

4. Use and Interpretation of Benchmarking Data ___________________________________ 6

5. Benchmarking Data_________________________________________________________ 7

6. Manufacturing Performance: OT&I ___________________________________________ 8 6.1. Overview of Analysis___________________________________________________________ 8 6.2. Data Description and Variable Definitions _________________________________________ 8 6.3. Analytical Methodology _______________________________________________________ 15 6.4. “Levels” Analytic Results ______________________________________________________ 16

6.4.1. Batches Failed _____________________________________________________________________16 6.4.2. Actual Yield ______________________________________________________________________17 6.4.3. Cycle Time _______________________________________________________________________17

6.5. “Rates of Change” Analytic Results _____________________________________________ 18 6.5.1. Batches Failed _____________________________________________________________________19 6.5.2. Actual Yield ______________________________________________________________________20 6.5.3. Cycle Time _______________________________________________________________________21

6.6. Discussion of Results __________________________________________________________ 21 7. Manufacturing Performance: API ____________________________________________ 22

7.1. Overview of Analysis__________________________________________________________ 22 7.2. Data Description and Variable Definitions ________________________________________ 23 7.3. Analytical Methodology _______________________________________________________ 29 7.4. “Levels” Analytic Results ______________________________________________________ 30

7.4.1. Batches Failed _____________________________________________________________________30 7.4.2. Actual Yield ______________________________________________________________________30 7.4.3. Cycle Time _______________________________________________________________________31

7.5. “Rates of Change” Analytic Results _____________________________________________ 32 7.5.1. Batches Failed _____________________________________________________________________32 7.5.2. Actual Yield ______________________________________________________________________33 7.5.3. Cycle Time _______________________________________________________________________33

7.6. Discussion of Results __________________________________________________________ 34 8. Deviation Management Performance: OT&I____________________________________ 35

8.1. Overview of Analysis__________________________________________________________ 35 8.2. Data Description and Variable Definitions ________________________________________ 36 8.3. Analytical Methodology _______________________________________________________ 41 8.4. Product Unavailability Analytic Results __________________________________________ 43 8.5. Deviation Number Analytic Results______________________________________________ 44

iii

8.5.1. Raw Materials Deviations ____________________________________________________________44 8.5.2. Production Component (Equipment) Deviations __________________________________________44 8.5.3. Product and Process Specification Deviations ____________________________________________46

8.6. Deviation Rate Analytic Results_________________________________________________ 46 8.6.1. Raw Material Deviation Rate _________________________________________________________46 8.6.2. Production Component (Equipment) Deviation Rate _______________________________________46 8.6.3. Product and Process Specification Deviation Rate _________________________________________47

8.7. Discussion of Results __________________________________________________________ 49 9. Deviation Management Performance __________________________________________ 50

9.1. Overview of Analysis__________________________________________________________ 50 9.2. Data Description and Variable Definitions ________________________________________ 51 9.3. Analytical Methodology _______________________________________________________ 56 9.4. Product Unavailability Analytic Results __________________________________________ 56 9.5. Deviation Number Analytic Results______________________________________________ 57

9.5.1. Raw Materials _____________________________________________________________________57 9.5.2. Production Component (Equipment)____________________________________________________57 9.5.3. Product and Process Parameter ________________________________________________________57

9.6. Deviation Rate Analytic Results_________________________________________________ 58 9.6.1. Production Component (Equipment) Deviation Rate _______________________________________59 9.6.2. Product and Process Parameter Deviation Rate ___________________________________________60

9.7. Discussion of Results __________________________________________________________ 60 10. Process Development ______________________________________________________ 61

10.1. Overview of Analysis _________________________________________________________ 61 10.2. Data Description and Variable Definitions _______________________________________ 62 10.3. Analytical Methodology ______________________________________________________ 63 10.4. Process Development Analytical Results _________________________________________ 64 10.5. Discussion of Analysis Results _________________________________________________ 64

11. Conclusions _____________________________________________________________ 66 11.1. Extent and Use of IT _________________________________________________________ 66 11.2. Decision Rights______________________________________________________________ 66 11.3. Contract Manufacturing______________________________________________________ 67 11.4. Process Analytic Technology Tools _____________________________________________ 67 11.5. Scale and Scope _____________________________________________________________ 67 11.6. Concluding Remarks_________________________________________________________ 68

iv

List of Figures Table 6.1: Manufacturing Performance Measures ____________________________________ 8 Table 6.2: Dependent Variable Definitions _________________________________________ 9 Table 6.3: Independent Variable Definitions _______________________________________ 10 Table 6.4: Summary Statistics __________________________________________________ 12 Table 6.5: Correlation Statistics _________________________________________________ 14 Table 6.6: Batches Failed and Yield and Cycle Time SUR Levels Analyses ______________ 18 Table 6.7: Batches Failed and Yield and Cycle Time SUR Rate of Change Analyses _______ 19 Table 7.1: Manufacturing Performance Measures ___________________________________ 23 Table 7.2: Dependent Variable Definitions ________________________________________ 23 Table 7.3: Independent Variable Definitions _______________________________________ 24 Table 7.4: Summary Statistics __________________________________________________ 25 Table 7.6: Batches Failed and Yield and Cycle Time SUR Levels Analysis_______________ 31 Table 7.7: Batches Failed and Yield and Cycle Time SUR Rate of Change Analysis________ 34 Table 8.1: Deviation and Regulatory Performance Measures __________________________ 36 Table 8.2: Dependent Variable Definitions ________________________________________ 37 Table 8.3: Independent Variable Definitions _______________________________________ 38 Table 8.4: Summary Statistics __________________________________________________ 40 Table 8.5: Correlation Statistics _________________________________________________ 42 Table 8.6: Product Unavailability and SUR Deviation Levels Analyses __________________ 45 Table 8.7: SUR Analysis of RM, PC, and PP Deviations Rate of Change_________________ 48 Table 9.1: Deviation and Regulatory Performance Measures __________________________ 50 Table 9.2: Dependent Variable Definitions ________________________________________ 51 Table 9.3: Independent Variable Definitions _______________________________________ 52 Table 9.4: Summary Statistics __________________________________________________ 54 Table 9.5: Correlation Statistics _________________________________________________ 55 Table 9.6: Product Unavailability and SUR Deviation Levels Analyses __________________ 58 Table 9.7: SUR Analysis of RM, PC, and PP Deviations Rate of Change_________________ 59 Table 10.1: Variable definitions for process development analysis ______________________ 62 Table 10.2: Summary statistics and correlations for process development variables_________ 63 Table 10.3: Process development SUR analysis_____________________________________ 64

1

1. Introduction The Pharmaceutical Research Manufacturing Project was launched in 2002. The project’s goals are to investigate the effects of production technology, product technology, manufacturing site location, firm reputation and experience, and organizational structure and incentives of pharmaceutical manufacturing on the likelihood and type of enforcement efforts utilized by the FDA. By studying these relationships, the project's desire is to generate new insights into the strategic management of pharmaceutical manufacturing, as well as to offer new insights into strategies for improving product and workplace safety in this and other industries.

The project was implemented in two stages. The first stage focused on FDA oversight of pharmaceutical manufacturing. The second stage focused on manufacturing performance of pharmaceutical manufacturing facilities. Throughout, this report will refer to the former as the “FDA Study” in the latter as the “Pharmaceutical Manufacturing Study”. While this report focuses on the Pharmaceutical Manufacturing Study, we nonetheless provide a brief overview of the FDA study.

Working with the FDA, the project team collected a wide range of FDA confidential data. While a confidentiality agreement prohibits the release of these data, results from statistical analyses are publicly available. Data collected came from the FDA’s Field Alerts, Inspections (FACTS), Product Listing, Facility Registration, and ORA training databases. (Additional data involving product recalls, product shortages, and warning letters also were collected but have not been fully integrated into the statistical analyses.) With this information, the team developed statistical models that predict the probability of a facility being chosen for inspection. Models were developed to evaluate the effect of investigator training and experience on the probability of investigational outcomes as well as individual investigator effects on the probability of investigational outcomes. Finally, the project identified characteristics of facilities and firms that correlate with the likelihood of noncompliance. For instance, the statistical analyses show that some facilities were over inspected while other facilities were under inspected. Preliminary results of the FDA Study were presented to the FDA on January 28, 2005. At present, results are publicly available in the form of a PowerPoint presentation.

Working with 19 manufacturers, the project team collected data on 42 pharmaceutical manufacturing facilities for the Pharmaceutical Manufacturing Study. Data collection included information about the firm and the manufacturing facilities; human resource management, the management of deviations, the use of various teams, shop floor performance metrics, process development metrics, and regulatory performance. Types of facilities include oral and topical (OT) manufacturing facilities (22 in all), injectable (I) manufacturing facilities (eight in all), active pharmaceutical ingredients (API) manufacturing facilities (15 in all), and biologic manufacturing facilities (five in all). As only one biologic facility provided complete performance metric information, we are unable to provide benchmarking data on performance for biologics. Nonetheless, data from this facility and other biologic facilities will be used in several of the statistical analyses.

2

Manufacturers spent substantial effort collecting these data and entering it into a secure web site. We thank all participants for their generous efforts in collecting and entering these data. It is these data that are summarized and analyzed herein as a benchmarking report.

For the purposes of this preliminary report, all data are presented in graphical format (i.e., what we call benchmarking charts). Confidentiality agreements prevent us from disclosing any firm- or facility-specific information such that the reader can identify the firm or facility. Thus, each manufacturing facility is identified by a unique number, which is meant to allow the reader to make comparisons across different responses while maintaining firm and facility anonymity.

The volume of data collected and presented in this report is immense. The project team has given its best effort in portraying the data appropriately. There nonetheless may be questions unanswered by the way in which the data is presented or in the way that the statistical analyses are undertaken. We therefore encourage all readers to communicate such questions to us so that we can evaluate if there are better ways in which to present the data.

The following pages provide a description of the data collected, efforts undertaken to verify data entry, a brief discussion of how to use and interpret benchmarking charts, and a large number of benchmark comparisons. The bulk of the remaining portion of this report presents statistical models that allow us to quantitatively evaluate relationships among various technological and organizational alternatives and their effects on shop-floor productivity metrics such a cycle time, yield, product unavailability, deviation management, and process development.

2. Data Collection: PMRP Questionnaire The PMRP questionnaire consists of nine sections. Each section and the information it requested is described below. A copy of the PMRP questionnaire is provided in Appendix D.

Section 1 asks for information about the plant liaison that was responsible for filling out the requested information.

Section 2 asks for general company and business unit information including name, address, whether the Corporation was publicly traded or not, and basic annual financial data for the Company or strategic business unit (SBU). The financial information included annual sales revenue, R&D expense, market and sales expense, net profit, the value of plant and equipment assets, total assets, and total employees. The same financial data was requested for the Corporation. Finally, four additional questions about whether or not the Company/SBU sells FDA-regulated devices, whether or not the Corporation or its affiliates sell FDA-regulated devices, whether or not the Company/SBU has a chief information officer, and whether or not the Corporation has a chief information officer.

Section 3a asks for financial information about the manufacturing facility including sales revenue, research and development expense, capital expenditures, total assets, and total full-time employees. Section three also asked about a variety of employee data. For

3

instance, participants entered the number of employees in each of nine functional categories—manufacturing facilities, process development, operations/production, quality assurance, quality control, regulatory compliance, regulatory affairs, information technology services, and plant management—and classified these workers into one of six job classifications—operators, craft workers, technicians, professionals, officials and managers, and officials and clerical. Additional information on workers included the level of education—high school diploma or equivalent, bachelor's degree or equivalent, master's degree or equivalent, and Ph.D. or equivalent— for each job classification. Other manufacturing facility information included the age of the facility, its size by floor space as well as the plant's largest reactor/mixing vat/fermenter, the number of times the facility had been acquired since 1990, the date of the most recent acquisition, and a list of all the types of manufacturing processes located at the facility. Regulatory information included the number of DMS/NDA/ANDA/BLA designations manufactured at the facility, warning letters and consent decrees, and which regulatory authorities inspected the facility. Participants identified whether or not their facility provided contract manufacturing. The final data collected in this section entailed the total manufacturing plant operating hours on a monthly basis from 1999 to 2003.

Section 3b asks participants to list all product and substance names, the first year they were produced, the last year they were produced (if production ceased), therapeutic area, as well as the type of process used to produce the product or substance. For APIs, this section asks for the number of chemical reactions, whether or not the compound was sold to branded manufacturers, and the number of buyers of this compound if the compound or substance was an API. For biologics, this section asks for the number of purification steps at the time of the BLA filing, the number of microbiologic assays, the number of chemistry assays, whether or not this product involves lyophilization, and whether or not the product involves cryogenic storage. Other questions include whether or not the product is a modified release pharmaceutical and the number of NDC numbers associated with it. Finally, participants are asked to identify up to, but no more than, five representative products/compound produced at their facility.

Section 4 inquires about human resources at the facility. This section investigates employee mobility in terms of new hires, rehires, quits, retires, terminations, and involuntary layoffs with respect to the job classifications listed in Section 3a above. Additional information about employee demographics including the average years employed, average age, and the percent of employees that are female are entered for each of the six job classifications. Section 4 requests information on various types of employee training for operators, craft workers, technicians, and engineers. Training includes: basic skills, basic science, statistical process control, machine operation, machine maintenance, teamwork and communication skills, problem-solving methods, design of experiments to test hypotheses, safety procedures, clean room procedures, and cGMP training. Participants provide information about employee appraisal and promotion for each of the six job classifications. Types of compensation included: stock options, profit sharing, employee stock ownership plan, knowledge/skill-base pay, pay for suggestions, individual performance bonus/incentives, team performance bonus/incentives. Participants also provide information on whether or not different positions received bonuses for achieving targets/goals associated with NDA/ANDA, supplements, and deviation management. These positions include: pilot plant manager, plant manager,

4

plant engineering manager, plant operations manager, plant quality manager, plant regulatory affairs manager, plant information technology manager, plant human resources manager, operators, craft workers, technicians, and professionals. Finally, the survey requests employee pay levels for operators, craft workers, and technicians in terms of salary plus benefits.

Section 5 requests information about product and process development. The survey collects information on where the products were first manufactured, whether all of the compounds had been manufactured at this facility or other facilities, and the amount of production of the compounds at prior facilities. Section five also has a series of questions on where different activities were performed. These activities are discovery, process research, pilot development, commercial plant transfer and start up, and assay development. For each activity, participants were asked to identify whether this activity took place at the facility, some other location within the SBU, some other location within the Corporation, or at another Corporation. For the same activities, participants provide information on the country in which the activity took place and the approximate physical distance between the activities.

The organization of development is investigated for all of these activities. Organizational features are illuminated by asking participants to report the number of vertical reporting relationships between each activity. The survey also explores the extent to which personnel for one activity shifted to another activity to manage the handoff. Contract manufacturers are asked to identify when work first began for this particular customer, the number of products being produced for this customer, the number of products that have ever been produced this customer, and the frequency of interaction with a typical customer. API facilities are asked to identify the number of customers that purchase the API. Several questions pertained to process validation. Does process validation report to plant management or corporate management? To what extent is process validation part of other activities such as process development, operation/production, engineering, quality assurance, quality control, regulatory compliance, and regulatory affairs? The timing of the various development steps along with the number of man-hours for each activity and the number of people involved in each activity were provided for each process stage. Regulatory filing and approving dates are requested. Additional manufacturing quantities and cumulative quantities are also requested.

Section 6 requests performance metrics for each compound/product. Depending on the type of product, participants enter monthly quantities from Jan-1999 through Dec-2003. These metrics, which are requested on a monthly basis, include batches started, batches reworked, batches failed, theoretical manufacturing yield, actual manufacturing yield and manufacturing cycle time. For Biologics, participants enter manufacturing yield, manufacturing batches, purification yields, purification batches, cycle time between biologic manufacturing and fill, storage and reconstitution yield, number of biologic product units initially bottled, and cycle time between the sale and delivery. Other performance metrics include whether or not the product was unavailable (stock out), the number of yield alerts or biologic product deviation reports, whether or not the finished product was recalled, the number of deviations that arose from raw materials purchased, the number of deviations that arose from production components, the number of deviations that arose from production process and product specifications, and the percent

5

of deviations from repeat occurrences for each one of these deviation categories. The monthly person-hours expended on resolving deviations and the monthly person hours expended on deviation prevention programs and practices also are requested. The number of FDA pre-approval inspections, general cGMP inspections, cGMP inspections for cause, and inspections from other countries are collected on a monthly basis.

Section 7 collects information on teams. For each type of job classifications the survey asks to what extent are personnel involved in teams and to how many teams do they belong? The survey asks for when teams of various types were first introduced, the number of teams of that type in the facility of each year, the average hours per month dedicated to formal team meetings, whether or not managers or supervisors are part of the team, the percentage of team members who are operators, the percent of team members who are technicians, the percent of team members who are craft workers, the percent of team members are who engineers, the approximate size of the team, the extent to which teams use formal structured problem-solving techniques, and whether or not a formal department organization exists to support the activities of these teams. These questions are asked with respect to quality improvement teams/quality circles, continuous improvement teams, self-directed work teams, and total preventative maintenance teams, with the opportunity to describe any other type of team.

Section 8a investigates deviation and manufacturing management. Participants provide data on the extent of information technology use. For instance, participants are asked whether deviations are electronically and automatically reported to an IT network? Does the IT system track by lot, track by deviation issue, track by people assigned resolving deviations, and collect equipment operating data, which is then stored in a central data center? Several questions focus on process analytic technology. Are tools available for the statistical designer experiments, response surface methodologies, process simulation, and pattern recognition? Are process analyzers for process analytic chemistry tools available for simple process measurement, chemical composition management, and physical attribute measurement? Are process monitoring, control, an endpoint tools available to measure critical material and process attributes related to product quality, for real-time or near-real-time monitoring of critical elements, for adjustments to ensure control of all critical elements, to assess mathematical relationships between product quality attributes and measurements of critical material and process attributes, and process endpoint monitoring and control? Have PAT tools been effective in acquiring information to facilitate understanding, developing risk mitigation strategies, achieving continuous improvement, and sharing information and knowledge? The organization of deviation management is investigated by identifying the number of vertical reporting relationships between operations, engineering, quality, and regulatory affairs. Other questions include whether the plant’s quality organization reports to the plant manager or to corporate management, how conflicts among groups are resolved, and how frequently conflicts in the area of deviation management and resolution arise among different functional groups.

Section 8b investigates several additional questions about deviation management. For instance, the section explores who has final responsibility for failing a lot. Is a QA member assigned to the deviation, the QA manager, head of QA at the plant, plant manager, or corporate QA? Which functional group typically takes the lead in responding

6

to deviations? Who has final responsibility for reviewing and approving for release a lot with a major deviation? Responses include operations, engineering, quality, regulatory affairs, and the group from which the deviation originated. Which groups must review and approve deviations? Which groups typically participated in reviewing and approving deviations even though they are not required to review and approve them? Responses include operations, engineering, quality, and regulatory affairs.

Section 9 explores supplement management. This section requests information on supplements filed with the FDA for the particular product/compound. The information collected includes the type of supplement, the date development work began, initial filing date, FDA's initial response, FDA's final response, the date of FDA's approval, the date of implementation of the change, whether or not stability data was submitted with the supplement, whether or not the supplement involved a change in batch/reactor size or manufacturing equipment, whether or not the supplement involved a change in starting material or a change in process synthesis, and whether or not the supplement involved a change that impacted formulation. Additional supplement data was collected on the number of development personnel involved in developing the process change as well as the amount of time involved in preparing the FDA's submission. Several other questions about changes in the process specific to API process synthesis and Biologics were asked as well as issues particular to contract manufacturers. Another set of questions investigated which groups within the manufacturing facility as well as the Corporation were involved in assembling the supplement and had primary responsibility for managing the supplement.

3. Data Verification A critical aspect of the study is that data are collected from a large number of firms and facilities. With different respondents from different segments of the pharmaceutical industry and from different countries, interpretations of questions may vary. In order to verify consistency across the data several procedures were followed. First, definitions in the on-line questionnaire were provided. Second, the project team interacted with all facility liaisons to identify potential points of confusion and to respond to them. Third, the data for each facility was reviewed and evaluated for inconsistencies. All inconsistencies were reported back to the facility, which led to adjustments to several data elements. Fourth, all benchmarking charts were reviewed to identify inconsistencies and these inconsistencies were reported to and discussed with participants, which led to further corrections.

While these procedures cannot guarantee consistency in every element of data across all facilities, they nonetheless provide some assurance that obvious inconsistencies were identified and corrected.

4. Use and Interpretation of Benchmarking Data Benchmarking data, which is presented in Appendices A, B, and C, provides several sources for developing meaningful conclusions for managers. First, managers may find it useful to assess how their manufacturing facility compares to other manufacturing

7

facilities on individual categories of collected data. For instance, a formulation facility can compare itself to other facilities of similar size and product variety on performance dimensions such as yield, cycle time, etc. Such performance differences provide insight into the relative competitiveness of a particular manufacturing facility. A related opportunity is to evaluate the extent to which a facility engages in certain practices that others do not. For instance, some facilities utilize teams extensively while others utilize few or no teams. Other differences can be found in the extent to which facilities provide training in various knowledge areas or the use of process analytic technology. These comparisons by themselves provide value because they highlight whether or not a given facility is more or less advanced in its capabilities vis-à-vis other facilities.

Second, managers can begin to explore trade-offs among variables. For instance, some facilities experienced high rates of employee turnover, which might translate into performance differences, compared to other facilities which experienced low turnover. The same type of analysis can be made regarding the extent of human capital manifested in the distribution of educational degrees. An area where such comparisons may be particularly fruitful is new process development. Facilities demonstrated wide-ranging differences in how processes are developed with respect to the location of development, utilization of development resources, and time utilized to develop processes. The benchmarking data contained within this report can be used to identify such differences. Benchmarking charts are described in Section 5 and presented in Appendices A, B, and C.

Third, the data can be analyzed statistically to address a wide variety of questions. Such analysis allows for the investigation of how various organizational decisions and features affect manufacturing performance, deviation management, and process development. Of interest is not only how various organizational decisions and features affect the level of manufacturing performance, deviation management, and process development, but also the rate of change of manufacturing performance and deviation management. Statistical analyses and results are presented in Sections 6 through 10.

5. Benchmarking Data All benchmarking data is presented in graphs that allow comparison across facilities. Given the extensive number of graphs, the data is presented in three separate appendices.

Appendix A presents benchmarking comparisons for formulation processes that produce oral and topical products. Each chart in Appendix A presents data collected from a question in the PMRP questionnaire. A unique facility identification number has been randomly assigned to each manufacturing facility so that readers can compare facilities across different data elements. Unfortunately, given the large number of O&T processes for which there is data, data is frequently spread across two charts in the appendix. These charts are placed next to each other wherever possible.

Appendix B presents benchmarking comparisons for API products. In this case, all observations for the participants are recorded in each graph and graphs are presented for each question posed.

8

Appendix C presents benchmarking comparisons for Injectable products.

An index is provided at the beginning of each appendix to facilitate finding benchmarking graphs of interest.

6. Manufacturing Performance: OT&I

6.1. Overview of Analysis

This section of the report provides a statistical analysis of manufacturing performance for the data collected in the Pharmaceutical Manufacturing Research Project (PMRP). It examines only Oral, Topical and Injectable Manufacturers—26 unique manufacturing facilities from 14 unique pharmaceutical firms.

The manufacturing performance section of the PMRP collected monthly data on (1) the number of batches started; (2) the number of batches failed; (3) the number of batches reworked but ultimately failed; (4) theoretical manufacturing yield; (5) actual manufacturing yield; and (6) cycle time. Table 6.1 provides definitions of these performance measures.

Using data collected in the PMRP questionnaire about the manufacturing facility (sections 3a and 3b), human resources (section 4), performance metrics (section 6), and deviation management (sections 8a and 8b), the analysis explores those factors that correlate with various manufacturing performance metrics, including batches failed, actual yield, and cycle time. The interested reader is referred to Appendix D, for a complete list of data collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire.

The text below describes the data used in the empirical analyses, summarizes the methodology through which the data is analyzed, presents the empirical results, and discusses these results.

6.2. Data Description and Variable Definitions

The data input by pharmaceutical firms on manufacturing performance required extensive data entry. The PMRP collected performance information for oral, topical and injectable manufacturers. These data represent 4,252 monthly observations for 71 products manufactured in 26 distinct manufacturing facilities by 14 distinct pharmaceutical firms.

Table 6.1: Manufacturing Performance Measures Measure Definition Batches Started The number of batches begun per month. For pharmaceutical

products, this item refers to the number of batches started once the API is received.

Batches Failed The number of batches per month that failed and were not able to be successfully reworked.

Batches Reworked The number of batches per month that were reworked but

9

ultimately failed. In other words, the number of batches that initially failed inspection, were reworked, but then ultimately failed inspection.

Theoretical Yield The ratio of the theoretical amount of output that would be produced at any appropriate phase of production of a particular drug product to the quantity of components (bulk quantity) used, in the absence of any loss or error in production, stated as a percentage.

Actual Yield The ratio of the actual yield (at any appropriate phase of production of a particular drug product) to the theoretical yield (at the same phase), stated as a percentage.

Cycle Time The average number of days between batch start and those batches either accepted or rejected during the month.

While a number of different statistical analyses were undertaken, this section examines only those empirical analyses that were sensible and yielded significant findings. In terms of batch manufacturing performance, we are interested in the number of batches failed and reworked compared to the number of batches started, as the latter could strictly be correlated with the size of the facility. The number of batches reworked did not provide enough heterogeneity in performance, however, so we only present results for batches failed. In terms of more traditional manufacturing performance measures, we present results for both actual yield and cycle time. Besides the performance level of batches failed, actual yield and cycle time, we are interested in the rates of change of these performance metrics. Table 6.2 provides definitions for the dependent variables used in the econometric analyses.

Table 6.2: Dependent Variable Definitions Actual Yield Monthly actual manufacturing yield in percentage terms. Change in Actual Yield Ratio of current month’s actual manufacturing yield to the

prior month’s actual manufacturing yield. Cycle Time Monthly manufacturing cycle time in days. Change in Cycle Time Ratio of current month’s cycle time to the prior month’s

cycle time. Batches Failed Monthly number of batches failed. Change in Batches Failed Ratio of current month’s number of batches failed to the

prior month’s number of batches failed.

As mentioned above, we utilized a number of independent variables collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire in the empirical analyses. Most of these variables are collected yearly from 1999 to 2003.

Table 6.3 provides definitions only for those variables that were retained in the empirical analyses that yielded statistically significant results.

10

Table 6.3: Independent Variable Definitions Total Employees Natural logarithm of the total number of employees at the

manufacturing facility. Facility Size Natural logarithm of the manufacturing facility size occupied

by operational equipment expressed in square meters. Number of Processes The number of different types of manufacturing processes

located at the manufacturing facility. Number of Products The number of different products produced at the

manufacturing facility. Contract Manufacturer Dummy variable equal to 1 if the manufacturing facility

engages in contract manufacturing and 0 otherwise. Employee Experience A measure of the interim correlation of the average number

of years of experience for operators, craft workers, technicians, professionals and managers. Scale Reliability = 0.87.

Employee Training A measure of the interim correlation of the extent to which operators, craft workers, technicians, and engineers receive on the job and class room training in the manufacturing facility across the following skills:

• Basic Skills (e.g., math, reading, language) • Basic Science (e.g., chemistry, physics) • Statistical Process Control • Machine Operation • Machine Maintenance • Teamwork & Communication Skills • Problem Solving Methods • Design of Experiments • Safety Procedures • Clean Room Procedures • Good Manufacturing Practice (cGMP) Training

Scale Reliability = 0.85. IT Usage A measure of the interim correlation of the extent to which

information technology is utilized in the manufacturing facility to:

• Electronically and automatically report deviations • Track deviations by lot • Track deviations by type of issue • Track people assigned to resolving the deviation • Centrally store data

Scale Reliability = 0.93. PAT Data Analysis Tools A measure of the interim correlation of the extent to which

the manufacturing facility uses the following multivariate data acquisition and analysis tools:

• Statistical design of experiments • Response surface methodologies

11

• Process simulation • Pattern recognition tools

Scale Reliability = 0.74. PAT Process Analytic Tools A measure of the interim correlation of the extent to which

the manufacturing facility uses the following process analyzer or process analytic chemistry tools:

• Simple process measurement • Chemical composition measurement • Physical attribute measurement

Scale Reliability = 0.73. PAT Monitoring Tools A measure of the interim correlation of the extent to which

the manufacturing facility uses the following process monitoring, control and endpoint tools:

• Critical material in process attributes related to product quality

• Real-time or near real-time (e.g., on-, in- or at-line) monitoring of critical elements

• Adjustments to ensure control of all critical elements • Mathematical relationships between product quality

attributes and measurement of critical material and process attributes

• Process endpoint monitoring and control Scale Reliability = 0.84.

Lot Failure Responsibility A measure of the interim correlation of the extent to which a QA manager has responsibility to fail a lot compared to the head of QA at the manufacturing facility. Scale Reliability = 0.80.

Deviation Participation

A measure of the interim correlation of the extent to which operations and quality participate in reviewing and approving deviations even though they are not required to review and approve in the manufacturing facility. Scale Reliability = 0.81.

The manufacturing performance analyses ultimately employed nine different dependent variables and 13 independent variables. The logarithm of total employees and facility size is taken because the distributions of these data are skewed.

The last eight variables are constructed. Constructed variables derive from factor analysis to reduce the number of independent variables for our econometric analyses. Doing so improves the number of degrees of freedom for our econometric analysis. A factor analysis of all relevant variables for the set of questions in each survey area was used for each one of these constructed variables. For instance, the factor analysis for Employee Experience included variables representing the number of years of experience for each category of worker collected in the survey: operators, craft workers, technicians, professionals and managers. Based on results of the factor analyses we first constructed

12

Cronbach’s alpha for each eigenvalue greater than one, and then included only those variables with a factor loading greater than 0.4. Each constructed variable represents the corresponding alpha score. Scale reliability is presented in Table 6.3 for each constructed variable. All scale reliability coefficients surpass the expected norm of 0.7.

Table 6.4 reports summary statistics for these dependent and independent variables.

Table 6.4: Summary Statistics MEAN S.D. MIN MAX Dependent Variables Batches Failed 0.13 0.74 0.00 19.00 Actual Yield 96.38 6.64 24.00 124.00 Cycle Time 27.14 28.76 1.00 187.00 Change in Batches Failed -0.60 1.35 -1.00 15.00 Change in Actual Yield 0.00 0.11 -0.71 2.96 Change in Cycle Time 0.24 1.60 -0.97 49.33 Independent Variables Total Employees 6.15 0.80 2.83 7.50 Facility Size 9.91 0.95 8.20 14.51 Number of Processes 1.75 0.77 1.00 4.00 Number of Products 18.83 24.12 0.00 109.00 Contract Manufacturer 0.53 0.50 0.00 1.00 Employee Experience -0.01 0.79 -1.62 1.57 Employee Training 0.04 0.63 -1.18 1.10 IT Usage 0.07 0.84 -1.13 0.91 PAT Data Analysis Tools -0.02 0.73 -0.58 1.86 PAT Process Analytic Tools -0.03 0.80 -1.79 0.69 PAT Monitoring Tools 0.05 0.77 -1.94 0.51 Lot Failure Responsibility 0.04 0.95 -1.40 0.71 Lot Review -0.04 0.92 -1.67 0.61

In terms of the dependent performance measures, mean level of batch failed is 0.13 per month, with a range from 0 to 19. The data indicate 93.5% of the sample has 0 batches failed in any month. The mean actual yield is roughly 96.4% with a standard deviation of 6.6%, while the mean cycle time is just over 27 days, with a large standard deviation. The data also indicate that the mean change in batches failed declines over time, while the change in actual yield has been negligible and the change in cycle time has increased.

In terms of independent variables, the mean level of total employees (logged) within a manufacturing facility in the sample is 6.15, which is equivalent to just over 600 employees. Similarly, the mean facility size (logged) is 9.91 square meters, which is equivalent to 55,000 square meters.

Each manufacturing facility has about two distinct manufacturing processes on average, and produces almost 19 distinct compounds. Both of these variables show substantial

13

variance among the manufacturing facilities in the sample. About half of the sample manufacturing facilities are also engaged in contract manufacturing.

As mentioned above, the rest of the independent variables are based on constructed variables derived from factor analyses to reduce the number of independent variables for our econometric analyses. The items in the scale are standardized (mean 0, variance 1) so discussion of their summary statistics is uninteresting. While constructed variables are standardized, their mean and standard deviation may not be reported as exactly zero and one, respectively in Table 6.4 because of dropped observations.

Table 6.5 presents correlations for these dependent and independent variables. Pair-wise correlations in bold are statistically significant at 95% confidence intervals. The dependent variables—both levels and rates—are significant with each other and several of the independent variables.

The number of batches failed varies positively with the number of employees, number of distinct processes, and number of distinct products made in the manufacturing facility. The number of batches failed varies negatively with the greater amount of employee experience and training. These results are not surprising – more employee experience is associated with fewer batches failed. Perhaps more surprising is the number of batches failed varies with the extent of Process Analytic Technology (PAT) tools. For both data acquisition and analysis tools and process analytic tools, the number of batches failed increases. We explore this result in more detail in the analytic results section.

Actual yield varies negatively with the number of employees, the size of the facility, the number of distinct processes, and the number of products made in the manufacturing facility. These results suggest that complexity in the manufacturing facility—in terms of size (employees and square footage) or technologies (processes or products) has a detrimental effect on yield. The results of Table 6.5 also indicate that employee experience and training are negatively associated with actual yield. In other words, manufacturing facilities with more experienced and more highly trained employees also possess lower manufacturing yields, all else held equal. Finally, data acquisition and analysis, process analytic and monitoring tools associated with PAT vary negatively with actual yield. We explore these results in more detail in the analytic results section.

Cycle time also varies negatively with the number of employees, the size of the facility, the number of distinct processes, and the number of products made in the manufacturing facility. These results again suggest that complexity in the manufacturing facility—in terms of size (employees and square footage) or technologies (processes or products) has a detrimental effect on cycle time. Table 6.5 also indicates that employee experience and training are positively associated with actual yield. In other words, manufacturing

14

Table 6.5: Correlation Statistics

Bat

ches

Fai

led

Act

ual Y

ield

Cyc

le T

ime

Cha

nge

in B

atch

es F

aile

d

Cha

nge

in A

ctua

l Yie

ld

Cha

nge

in C

ycle

Tim

e

Tot

al E

mpl

oyee

s

Fac

ility

Siz

e

Num

ber o

f Pro

cess

es

Num

ber o

f Pro

duct

s

Con

trac

t Man

ufac

ture

r

Em

ploy

ee E

xper

ienc

e

Empl

oyee

Tra

inin

g

IT U

sage

PAT

Dat

a An

alys

is T

ools

PAT

Pro

cess

Ana

lytic

Too

ls

PAT

Mon

itori

ng T

ools

Lot

Fai

lure

Res

pons

ibili

ty

Lot

Rev

iew

Batches Failed 1 Actual Yield -0.17 1 Cycle Time 0.07 -0.21 1

Change in Batches Failed 0.85 -0.09 0.04 1 Change in Actual Yield -0.03 0.27 -0.02 -0.02 1 Change in Cycle Time -0.01 0.00 0.12 -0.04 -0.04 1

Total Employees 0.11 -0.16 0.18 -0.02 0.04 0.01 1 Facility Size 0.01 -0.22 0.33 -0.11 0.02 -0.06 0.45 1

Number of Processes 0.07 -0.15 0.07 0.08 0.05 0.04 0.25 0.07 1 Number of Products 0.08 -0.26 0.21 0.04 0.06 -0.03 0.41 0.48 0.22 1

Contract Manufacturer 0.04 0.14 0.29 -0.03 -0.03 0.00 0.22 0.00 0.06 0.21 1 Employee Experience -0.05 -0.05 0.26 -0.10 0.00 -0.03 0.08 0.23 -0.03 0.14 -0.03 1

Employee Training -0.07 -0.10 0.16 -0.04 0.00 -0.05 0.00 0.07 -0.05 0.13 0.20 0.28 1 IT Usage 0.02 -0.04 -0.07 -0.01 0.03 0.05 0.43 0.31 0.01 0.22 0.13 -0.11 0.03 1

PAT Data Analysis Tools 0.07 -0.22 0.03 0.03 0.04 -0.04 0.33 0.34 0.11 0.28 -0.13 -0.14 0.14 0.26 1 PAT Process Analytic Tools 0.10 -0.22 0.07 0.13 0.00 -0.09 0.44 0.31 0.17 0.26 0.10 0.00 0.17 0.09 0.42 1

PAT Monitoring Tools 0.00 -0.12 0.05 -0.02 -0.01 -0.08 0.27 0.25 0.14 0.12 0.07 0.28 0.56 0.01 0.37 0.55 1 Lot Failure Responsibility -0.08 0.29 -0.25 -0.02 -0.03 0.03 -0.20 -0.49 0.11 -0.25 0.31 -0.59 0.03 -0.11 -0.15 -0.24 -0.16 1

Lot Review 0.03 0.01 0.00 0.13 0.02 0.04 0.13 -0.05 -0.10 0.14 0.28 -0.15 -0.02 0.25 -0.15 -0.21 -0.26 0.28 1 Bold indicates pair-wise significance at 95% confidence interval.

15

facilities with more experienced and more trained employees exhibit higher cycle times. Finally, the use of process analytic and monitoring tools associated with PAT increases cycle time.

Two other correlations are worth mentioning. The first relates cycle time to actual yield, and shows a negative and statistically significant correlation. This result not surprisingly indicates that actual yield and cycle time vary inversely with each other. The second relates contract manufacturers to the various measures of manufacturing performance examined. Those manufacturing facilities that provide contract manufacturing services have correlations that vary positively with batches failed, actual yield and cycle time.

6.3. Analytical Methodology

Several different analytic methods are used to evaluate the effect of covariates on the dependent variables.

The first performance metric examined is the number of batches failed in a given month, which represents a count variable. As Table 6.3 indicates the counts are small in number in any given month, which suggests that a count model methodology is appropriate. A negative binomial model is used to explore the number of batches failed because the number of batches failed produces values greater than or equal to zero (e.g., nonnegative integers). We also employ the Huber/White/sandwich estimator of variance to achieve robust standard errors, as well as correct the variance-covariance matrix of the estimators for clustering (by manufacturing facility). Clustering accounts for the fact that observations are independent across manufacturing facilities, but not necessarily within manufacturing facilities.

Our analysis of the rate of change in the number of batches failed per month and per year employs ordinary least squares. We again utilize robust standard errors and clustering for this analysis.

Our second set of performance metrics involves two related dependent variables: Actual Yield and Cycle Time. Both measures are essentially continuous variables that can be analyzed using linear regression methodology. As indicated in Table 6.5, however, these variables are negatively correlated with each other. In other words, higher Actual Yield is associated with lower Cycle Time, and vice versa. At the same time, organizational practices may simultaneously affect each measure of shop-floor performance.

The analysis of Actual Yield and Cycle Time is therefore undertaken with a statistical method called Seemingly Unrelated Regression (SUR). Many econometric models (including the one examined here) contain a number of linear equations of different dependent variables. It is often unrealistic to expect that the errors are uncorrelated between and among these different equations. Seemingly Unrelated Regression gets its name because at first glance, the equations seem unrelated, but in fact are related through correlation in the errors. A set of equations that has contemporaneous cross-equation error correlation is thus called an SUR system, and the SUR method makes corrections to the standard errors of the estimates in each equation to adjust for this correlation.

We are interested not only in the levels of Batches Failed, Actual Yield and Cycle Time, but also in their rate of change. For instance, do some organizational practices or factors

16

allow pharmaceutical manufacturers to reduce the number of Batches Failed over time, or improve Actual Yield or Cycle Time over time? Or, do these factors reduce performance measures? To explore the possibility of such relationships covariates are regressed against Change in Batches Failed, Change in Actual Yield, and Change in Cycle Time. Because each of these variables is a ratio of the current month’s value divided by the last month’s value, no change equates to 1. More batches failed, higher actual yields and longer cycle times are greater than 1, while fewer batches failed, lower actual yields and shorter cycle times are less than one.

Given that several of the constructed independent variables are standardized (a mean of zero and a variance of one), interpretation of results should instead focus on the signs of the coefficients rather than on their magnitudes. It also should be noted that these models provide an initial lens with which to view the data. Given the time series nature of the data collected by the PMRP the data can be analyzed using other econometric techniques and methodologies. In some instances, the use of these other econometric methodologies may offer superior approaches with which to assess and interpret the data. Nonetheless, the analyses presented below offer an appropriate first glimpse of the relationships between various organizational practices and performance measures.

6.4. “Levels” Analytic Results

Table 6.6 provides the results of the econometric analysis for the levels of Batches Failed, Actual Yield, and Cycle time. All of the regression results for each performance metric use the same set of explanatory variables, where appropriate. All of the regressions employ robust standard errors and clustering. We explore each performance metric in turn.

6.4.1. Batches Failed Negative binomial regression results for the number of batches failed is presented in the first column of Table 6.6. Although the R2 (a measure of the explained variance of the regression) is small in this model (0.08), R2 should not be relied on as the only measure of fit. More importantly, the χ2 statistic for each model is statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

The results indicate that more batches fail with more products manufactured in the facility—a result that is not surprising given the complexity argument discussed above. More batches fail if the manufacturing facility engages in contract manufacturing, evidenced by the negative and statistically significant coefficient for Contract Manufacturer.

The coefficient for Employee Training is negative and significant, indicating that higher levels of employee training correspond to lower levels of Batches Failed. In other words, there appears to be a real payoff to the types of training indicated in Table 6.3.

The negative and statistically significant coefficient for Lot Failure Responsibility indicates that assigning lot responsibility to a QA manager instead of the head of QA for the facility lowers the number of batches failed.

17

6.4.2. Actual Yield The second column of Table 6.6 provides the coefficient estimates for the level of Actual Yield. Recall that the analysis of Actual Yield and Cycle Time is undertaken using an SUR model, which accounts for the possibility that Actual Yield and Cycle Time may be correlated and adjusts the standard errors accordingly.

The model fit the data reasonably well. The R2 of 0.19 indicates that nearly twenty per cent of the variance is explained by the included coefficients. The χ2 statistic for each model is also statistically significant and indicates rejection of the null hypothesis that the set of coefficients is random.

The coefficient for Number of Processes is significant and negative, which implies that Actual Yield is lower for those manufacturing facilities that manage a larger number of processes.

The results indicate that Actual Yield increases with greater employee experience, evidenced by the positive and statistically significant coefficient for Employee Experience. Surprisingly, however, the negative and significant coefficient for Employee Training indicates that actual yield is lower for those manufacturing facilities that engage in high amounts of employee training.

The coefficient for Lot Failure Responsibility is positive, large, and highly significant. This result can be interpreted as assigning lot failure responsibility to a QA manager instead of the head of QA for the facility corresponds to a substantially higher yield.

6.4.3. Cycle Time The third column of Table 6.6 provides the coefficient estimates for the level of Cycle Time. The model fit the data well. The R2 of 0.34 indicates that 34 per cent of the variance is explained by the included coefficients. The χ2 statistic for each model is also statistically significant, indicating rejection of the null hypothesis that the set of coefficients is random.

The large, positive, and highly statistically significant coefficient for Contract Manufacturer suggests that cycle times are longer for those facilities that engage in contract manufacturing.

The coefficient for Employee Experience is large, negative and significant, and indicates that employee experience corresponds to lower cycle times. However, the coefficient for Employee Training is large, positive and significant, and indicates that greater training is associated with higher cycle time.

The large, negative and statistically significant coefficient for IT Usage indicates that the extent and use of information technology in the manufacturing facility is associated with lower cycle times.

The negative and highly statistically significant coefficient for Lot Failure Responsibility indicates that assigning lot responsibility to a QA manager instead of the head of QA for the manufacturing facility is associated with lower cycle times.

18

Table 6.6: Batches Failed and Yield and Cycle Time SUR Levels Analyses Batches Failed Actual Yield Cycle Time

Number of Processes 0.312(0.262)

-1.674(0.675)

* 3.683 (5.338)

Number of Products 0.016(0.009)

† 0.012(0.017)

-0.169 (0.228)

Contract Manufacturer 0.954(0.559)

† -0.685(1.728)

32.412 (5.647)

**

Employee Experience -0.138(0.430)

3.484(2.004)

† -9.741 (5.141)

†

Employee Training -0.914(0.346)

** -2.503(1.217)

* 14.631 (7.653)

†

IT Usage -0.026(0.178)

0.672(0.888)

-8.237 (4.038)

**

PAT Data Analysis Tools 0.064(0.322)

-0.307(1.007)

4.786 (4.020)

PAT Process Analytic Tools -0.064(0.380)

-0.181(0.590)

-6.770 (5.661)

PAT Monitoring Tools 0.226(0.315)

0.571(1.136)

-8.446 (7.469)

Lot Failure Responsibility -0.855(0.400)

* 4.196(1.541)

** -21.194 (5.842)

**


0.189(0.208)

-0.982(0.798)

3.356 (3.966)

Constant -3.801(0.557)

** 99.276(1.730)

*** 10.316 (8.336)

N 2293 2293 2293 R2 0.08 0.19 0.34 χ2 105.98** 51.32** 108.47 ** ** p<0.01, * p<0.05, † p<0.10; Standard errors of coefficients in parentheses.

6.5. “Rates of Change” Analytic Results

Table 6.7 provides the results of the econometric analysis for the rates of Change in Batches Failed, Change in Actual Yield, and Change in Cycle Time.

Recall that each of these dependent variables is a ratio of the current month’s value divided by the last month’s value, with no change in value equating to 1. An increase in batches failed, higher actual yields and longer cycle times are greater than 1, while a reduction in batches failed, lower actual yields, and shorter cycle times are less than one.

Regression results for each performance metric use the same set of explanatory variables, as before and where appropriate. We also utilize robust standard errors and clustering for this analysis. We explore each performance metric in turn.

19

6.5.1. Batches Failed Negative binomial regression results for the rate of change in batches failed is presented in the first column of Table 6.7. The R2 achieved in this model is 0.07, which indicates a poor fit. The χ2 statistic for is statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

The coefficient for the Number of Products is negative and significant, which indicates that the Change in Batches Failed is lower for those manufacturing facilities that produce a larger number of products.

The results indicate that Change in Batches Failed decreases with greater employee experience, evidenced by the negative, large and statistically significant coefficient for Employee Experience.

Table 6.7: Batches Failed and Yield and Cycle Time SUR Rate of Change Analyses Change in Batches Failed

Change in Actual Yield

Change in Cycle Time

Number of Processes -0.057(0.092)

0.006(0.002)

** 0.145(0.044)

**

Number of Products -0.003(0.002)

† 0.000(0.000)

** -0.004(0.002)

*

Contract Manufacturer -0.124(0.117)

-0.005(0.002)

* -0.021(0.104)

Employee Experience -0.145(0.040)

** 0.000(0.002)

0.038(0.067)

Employee Training 0.083(0.066)

0.001(0.002)

-0.082(0.069)

IT Usage -0.102(0.027)

** 0.002(0.003)

0.034(0.049)

PAT Data Analysis Tools 0.055(0.116)

0.005(0.001)

** 0.059(0.062)

PAT Process Analytic Tools 0.210(0.083)

* -0.002(0.001)

* -0.194(0.107)

†

PAT Monitoring Tools -0.109(0.082)

-0.001(0.002)

-0.198(0.114)

Lot Failure Responsibility -0.299(0.156)

† -0.003(0.002)

-0.028(0.057)


0.013(0.073)

0.004(0.001)

** 0.017(0.043)

Constant 0.491(0.147)

** 0.993(0.002)

** 1.076(0.103)

**

N 1987 1987 203 R2 0.04 0.05 0.07 χ2 66.92*** 1.62† 4.66** ** p<0.01, * p<0.05, † p<0.10; Standard errors of coefficients in parentheses.

20

The coefficient for IT Usage is negative, large and statistically significant, and indicates that higher levels of information technology within the manufacturing facility reduce the Change in Batches Failed. In other words, the extent and use of IT as indicated in Table 6.3 appears to provide value to the manufacturing facility.

The coefficient for PAT Process Analytic Tools is positive and statistically significant, which suggests that the Change in Batches Failed increases with the implementation of this practice.

The negative and statistically significant coefficient for Lot Failure Responsibility indicates that assigning lot responsibility to a QA manager instead of the head of QA for the facility is associated with a lower rate of increase in the number of batches failed.

6.5.2. Actual Yield The second column of Table 6.7 provides the coefficient estimates for the Change in Actual Yield. The analysis of Change in Actual Yield and Change in Cycle Time is undertaken using SUR analysis.

The R2 of 0.05 indicates that five per cent of the variance is explained by the included coefficients. The χ2 statistic for each model is only modestly statistically significant, but does allow for rejection of the null hypothesis that the set of coefficients is random at a 90 per cent confidence interval.

The coefficients for the Number of Processes and Number of Products are significant and positive, which imply that the change in actual yield is higher for those manufacturing facilities that manage a larger number of processes and produce a larger number of products, respectively.

The coefficient for Contract Manufacturer is negative and significant, and indicates that the change in actual yield is lower for those manufacturing facilities that provide contract manufacturing services.

The positive and significant coefficient for PAT Data Analysis Tools indicates that the change in actual yield is higher for those manufacturing facilities that implement multivariate data acquisition and analysis tools. At the same time, however, the negative and significant coefficient for PAT Process Analytic Tools indicates that the change in actual yield is lower for those facilities that implement process analyzers or process analytic chemistry tools. Nevertheless, the implementation of both of these tools does have a positive effect on the change in actual yield.

The coefficient for Lot Failure Responsibility is positive and significant. This result can be interpreted as assigning lot failure responsibility to a QA manager instead of the head of QA for the facility corresponds to a higher Change in Actual Yield. Also, the coefficient for Deviation Participation is positive and significant. The more that operations and quality participate in reviewing and approving deviations, even though they are not required to review and approve, the greater will be the increase in actual yield.

Finally, we note that the constant term is positive and highly statistically significant. The coefficient’s magnitude indicates that actual yield decreased slightly on a month-to-month basis.

21

6.5.3. Cycle Time The third column of Table 6.6 provides the coefficient estimates for Change in Cycle Time.

The R2 of 0.07 indicates that seven per cent of the variance is explained by the coefficients included. The χ2 statistic is modestly statistically significant.

The coefficient for the Number of Processes is positive and statistically significant, while the coefficient for the Number of Products is negative and statistically significant. These results imply that the change in cycle time increases with the number of processes in the manufacturing facility and decreases with the number of products produced.

The coefficient for PAT Process Analytic Tools is negative and statistically significant, which suggests that Change in Cycle Time decreases with the implementation of this practice.

Finally, the constant term is positive and highly statistically significant. This coefficient indicates that cycle times tend to increase from one month to the next.

6.6. Discussion of Results

Several interesting conclusions can be drawn from the statistical results presented above. One conclusion pertains to complexity in OT&I manufacturing facilities. More complex OT&I facilities, in terms of the number of unique processes or products, tend to perform worse than those manufacturing facilities with less complexity. Manufacturing facilities with more products fail more batches, but do lower over time the number of batches failed, do increase over time actual yield, and do lower over time cycle time. OT&I manufacturing facilities with more unique processes have lower actual yields but higher year-to-year increases in cycle time and higher rates of actual yield improvement.

The effect of the number of processes in an OT&I manufacturing facility therefore appears to generate diseconomies of scope. The higher the number of processes, the lower the actual yield and the more cycle time increases year-to-year. That said, the number of processes produced by the manufacturing facility does correspond to increasing actual yield over time; but, only minimally so.

By contrast, the number of products produced in an OT&I manufacturing facility appears to generate economies of scope. The more products manufactured in a facility, the more actual yield increases over-time as does cycle time. Surprisingly, the number of products has its greatest positive impact on the rate of change of cycle time. The number of products produced does, however, increase with the number of batches failed.

OT&I contract manufacturers appear to suffer on numerous performance dimensions. Contract manufacturers have a higher number of batches failed and longer cycle times than those facilities that do not engage in contract manufacturing. At the same time, contract manufacturers decrease actual yields over-time.

OT&I manufacturing facilities that utilize information technology to electronically and automatically report deviations or tack deviations by lot, type of issue, or people assigned

22

to resolving the deviation have lower cycle times and month-to-month reductions in the number of batches failed.

The extent and use of process analytic technology appear to impact performance in a complicated way. The use of multivariate data analysis and acquisition tools do not affect performance levels per se, but the use does correlate the month-to-month changes in these performance measures. PAT data analysis tools are associated with an increase in the rate of actual yield improvement. By contrast, PAT process analytic tools are associated with an increase in the month-to-month change in batches failed, decrease the month-to-month change in actual yield, and increase the month-to-month change in cycle time.

Assigning lot review responsibility to a QA manager instead of the head of QA for the manufacturing facility universally is associated with performance improvements. Such an assignment decreases batches failed, increases actual yield, and lowers cycle time. This managerial practice also corresponds to a reduction of batches failed from month-to-month. Also noteworthy is the finding that allowing operations and quality to participate in reviewing and approving deviations even though they are not required to do so corresponds with higher rates of improvement in actual yield.

Finally, it is important to note that all of these results describe correlations and do not determine causation. Rather than higher batches failed, lower actual yields or higher cycle times being the “result” of implementing a given managerial or organizational practice, it may be that worse performance “causes” manufacturing facilities to adopt these practices to improve these performance measures. Different causations can be attributed to all of the findings described above.

7. Manufacturing Performance: API


This section of the report provides a statistical analysis of manufacturing performance for the data collected in the Pharmaceutical Manufacturing Research Project (PMRP). It examines only Active Pharmaceutical Ingredient Manufacturers—15 unique manufacturing facilities from 11 unique pharmaceutical firms.

The manufacturing performance section of the PMRP collected monthly data on (1) the number of batches started; (2) the number of batches failed; (3) the number of batches reworked but ultimately failed; (4) theoretical manufacturing yield; (5) actual manufacturing yield; and (6) cycle time. Table 7.1 provides definitions of these performance measures.

Using data collected in the PMRP questionnaire about the manufacturing facility (sections 3a and 3b), human resources (section 4), performance metrics (section 6), and deviation management (sections 8a and 8b) sections, the analysis explores those factors that correlate with various manufacturing performance metrics, including batches failed, actual yield, cycle time. The interested reader is referred to Appendix D, for a complete list of data collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire.

23

The text below describes the data used in the empirical analysis, summarizes the methodology through which the data is analyzed, presents the empirical results, and finally discusses the results.

Table 7.1: Manufacturing Performance Measures Measure Definition Batches Started The number of batches begun per month. Batches Failed The number of batches that failed and were not able to or

successfully reworked per month. Batches Reworked The number of batches that were reworked but ultimately

failed per month. In other words, the number of batches that initially failed inspection, were reworked, but then ultimately failed inspection.

Theoretical Yield The ratio of the theoretical amount of output that would be produced at any appropriate phase of production of a particular drug product to the quantity of components (bulk quantity) to be used, in the absence of any loss or error in production, stated as a percentage. It is likely that the theoretical yield changes little unlike accompanied by changes in equipment, piping, etc.

Actual Yield The ratio of the actual yield (at any appropriate phase of production of a particular drug product) to the theoretical yield (at the same phase), stated as a percentage.

Cycle Time The average number of days between batch start and those batches either accepted or rejected during the month.


The data input by pharmaceutical firms on manufacturing performance required extensive data entry. The PMRP collected performance information for API manufacturers. These data represent 2,160 monthly observations for 36 active pharmaceutical ingredients manufactured in 15 distinct manufacturing facilities by 11 distinct pharmaceutical firms.

Table 7.2: Dependent Variable Definitions Actual Yield Monthly actual manufacturing yield in percentage terms. Change in Actual Yield Ratio of current month’s actual manufacturing yield to the

prior month’s actual manufacturing yield. Cycle Time Monthly manufacturing cycle time in days. Change in Cycle Time Ratio of current month’s cycle time to the prior month’s

cycle time. Batches Failed Monthly number of batches failed. Change in Batches Failed Ratio of current month’s number of batches failed to the

prior month’s number of batches failed.

24

While a number of different statistical analyses were undertaken, this section focuses attention only on those empirical analyses that were sensible and yielded significant findings. In terms of batch manufacturing performance, we are more interested in the number of batches failed and reworked than the number of batches started, as the latter could be strictly correlated with the size of the manufacturing facility. The number of batches reworked did not provide enough heterogeneity in performance, however, so we only present results for batches failed. In terms of more traditional manufacturing performance measures, we present results for both actual yield and cycle time.

Besides the performance level of batches failed, actual yield and cycle time, we also are interested in the rates of change of these performance metrics. Table 7.2 provides definitions for the dependent variables used in the statistical analyses.

As mentioned above, we utilized a number of independent variables collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire in the empirical analyses. Most of these variables are collected yearly from 1999 to 2003.


Table 7.3: Independent Variable Definitions Number of Processes The number of different types of manufacturing processes





Employee Age A measure of the interim correlation of the average age for operators, craft workers, technicians, professionals and managers. Scale Reliability = 0.77.

IT Usage A measure of the interim correlation of the extent to which information technology is utilized in the manufacturing facility to:


Scale Reliability = 0.74.

25

Lead Deviation Responsibility A measure of the extent to which operations, engineering, quality, regulatory affairs and the group from which the deviation originated (if applicable) all typically take the lead in responding to deviations in the manufacturing facility.

Deviation Review & Approval A measure of the extent to which operations, engineering, quality and regulatory affairs must all review and approve deviations in the manufacturing facility.

The manufacturing performance analyses for API manufacturers ultimately employed six different dependent variables and nine independent variables.

Several of the independent variables are based on constructed variables derived from factor analysis to reduce the number of independent variables for our analyses. Doing so improves the number of degrees of freedom for our econometric analysis. A factor analysis of all relevant variables for the set of questions in each survey area was used for each one of these constructed variables. For instance, the factor analysis for Employee Experience included variables representing the number of years of experience for each category of worker collected in the survey: operators, craft workers, technicians, professionals and managers. Based on results of the factor analysis we first constructed a Cronbach’s alpha for each eigenvalue greater than one, and then included only those variables with a factor loading greater than 0.4. The constructed variable represents the corresponding alpha score. Scale reliability is presented in Table 7.3 for each constructed variable. All scale reliability coefficients surpass the expected norm of 0.7.

Two variables related to deviation management are constructed based on the extent to which operations, engineering, quality and regulatory affairs work together.


Table 7.4: Summary Statistics MEAN S.D. MIN MAX Dependent Variables Batches Failed 0.10 1.07 0.00 34.00 Actual Yield 75.31 22.52 5.00 124.00 Cycle Time 29.28 30.16 0.21 194.00 Change in Batches Failed 0.91 3.21 0.00 17.00 Change in Actual Yield 1.01 0.25 0.47 7.40 Change in Cycle Time 1.10 1.19 0.03 31.00 Independent Variables Number of Processes 2.42 0.96 1.00 5.00 Number of Products 28.22 32.25 2.00 101.00 Contract Manufacturer 0.36 0.48 0.00 1.00 Employee Experience -0.12 0.77 -1.14 1.66 Employee Age 0.03 0.74 -1.30 0.95 IT Usage 0.14 0.59 -1.37 0.74 Lead Deviation Responsibility 1.87 0.88 1.00 3.00 Deviation Review & Approval 1.91 0.91 1.00 4.00

26

In terms of the dependent performance measures, mean level of batch failed is 0.10 per month, with a range from 0 to 34. The data indicate 96 per cent of the sample has 0 batches failed in any month. The mean actual yield is roughly 75 per cent with a large standard deviation, while the mean cycle time is just over 29 days, again with a large standard deviation.

The data also indicate that the mean month-to-month Change in Batches Failed is declining overall, while the month-to-month Change in Actual Yield is slightly positive and the month-to-month Change in Cycle Time is positive.

In terms of the independent variables, manufacturing facilities in the sample have between two and three distinct manufacturing processes on average, and produce slightly more than 28 distinct active pharmaceutical ingredients. Both of these variables show substantial variance among the manufacturing facilities in the sample.

Slightly more than one-third of the API manufacturing facilities in the sample indicate that they are also engaged in contract manufacturing.

As mentioned above, three of the independent variables—Employee Experience, Employee Age and IT Usage—are based on constructed variables derived from factor analysis to reduce the number of independent variables for our econometric analyses. The items in the scale are standardized (mean 0, variance 1) so discussion of their summary statistics is uninteresting.

The two other independent variables—Lead Deviation Responsibility and Deviation Review and Approval—are based on the extent to which operations, engineering, quality, and regulatory groups take the lead in responding to deviations and must review and approve deviations, respectively.

Table 7.5 presents correlations for these dependent and independent variables. Pair-wise correlations in bold are statistically significant at 95% confidence intervals. The dependent variables—both levels and rates—are significant with each other and several of the independent variables.

The number of Batches Failed varies negatively with whether the manufacturing facility indicates it is a contract manufacturer. This performance metric varies positively with the greater amount of employee experience. Finally, the number of batches failed varies negatively with the extent of process analytic technology tools.

The level of Actual Yield varies negatively with the number of distinct processes and positively with the number of products made in the manufacturing facility. These results suggest that complexity in the manufacturing facility—in terms of technologies (processes or products) has an indeterminate effect on Actual Yield. The results of Table 7.5 also indicate that Employee Age is negatively associated with Actual Yield. In other words, manufacturing facilities with older employees also possess lower manufacturing yields, all else held constant. The use of information technology in the manufacturing facility is positively associated with Actual Yield. Finally, the two managerial practices associated with deviation management are also positively associated with Actual Yield. We explore these correlations in more detail in the analytic results section.

Cycle Time varies negatively with the number of distinct processes and positively with the number of products made in the manufacturing facility. These results again suggest

27

that complexity in the manufacturing facility in terms of technologies (processes or products) has an effect on cycle time performance. Table 7.5 also indicates that Employee Age is negatively associated with Cycle Time. In other words, manufacturing facilities with older employees exhibit lower cycle times, all else held constant. The more extensive use of Information Technology in the manufacturing facility is negatively associated with Cycle Time, as are the deviation management managerial practices related to lead responsibility and review and approval, respectively.

Two other correlations are worthy of mention. The first relates Cycle Time to Actual Yield, and shows a negative and statistically significant correlation. This result not surprisingly indicates that actual yield and cycle time vary inversely with each other. The second relates contract manufacturers to the various measures of manufacturing performance examined. Those manufacturing facilities that provide contract manufacturing services have correlations that vary negatively with Batches Failed and Actual Yield and positively with Cycle Time. We explore these correlations in more detail in the analytic results section.

28


Bat

ches

Fai

led

Act

ual Y

ield

Cyc

le T

ime

Cha

nge

in B

atch

es F

aile

d

Cha

nge

in A

ctua

l Yie

ld

Cha

nge

in C

ycle

Tim

e

Num

ber o

f Pro

cess

es

Num

ber o

f Pro

duct

s

Con

trac

t Man

ufac

ture

r

Em

ploy

ee E

xper

ienc

e

Em

ploy

ee A

ge

IT U

sage

Lea

d D

evia

tion

Resp

onsi

bilit

y

Dev

iatio

n Re

view

& A

ppro

val

Batches Failed 1 Actual Yield -0.11 1 Cycle Time 0.13 -0.49 1

Change in Batches Failed 0.92 -0.17 0.11 1 Change in Actual Yield 0.00 -0.04 0.04 -0.06 1 Change in Cycle Time 0.01 0.06 0.11 0.07 0.06 1 Number of Processes -0.01 -0.08 -0.10 -0.09 -0.02 -0.02 1 Number of Products -0.05 0.16 0.15 -0.05 -0.02 0.09 -0.01 1

Contract Manufacturer -0.05 -0.19 0.06 -0.16 -0.01 0.07 0.03 0.21 1 Employee Experience 0.08 -0.12 0.03 0.13 0.05 0.02 0.15 0.30 -0.11 1

Employee Age 0.04 0.15 -0.14 0.09 0.03 0.02 0.30 0.43 -0.30 0.62 1 IT Usage 0.03 0.12 -0.17 0.05 0.01 0.05 0.08 0.52 0.01 0.51 0.71 1

Lead Deviation Responsibility -0.05 0.30 -0.30 -0.16 -0.05 -0.04 0.36 0.08 -0.62 0.01 0.37 -0.02 1 Deviation Review & Approval -0.04 0.25 -0.41 -0.15 -0.04 -0.04 -0.18 -0.25 -0.37 -0.32 0.02 0.01 0.33 1

Bold indicates pair-wise significance at 95% confidence interval.

29


Several different analytic methods are used to evaluate the effect of the covariates on the manufacturing performance metrics.

In every regression, we employ the Huber/White/sandwich estimator of variance to achieve robust standard errors, as well as correct the variance-covariance matrix of the estimators for clustering (by manufacturing facility). Clustering accounts for the fact that observations are independent across manufacturing facilities, but not necessarily within manufacturing facilities.

The first performance metric examined is the number of batches failed in a given month, which represents a count variable. As Table 7.3 indicates the counts are small in number in any given month, which suggests that count model methodologies are appropriate. A negative binomial model is used to explore the number of batches failed because the number of batches failed produces values greater than or equal to zero (e.g., nonnegative integers).

Our second set of performance metrics involves two related dependent variables: Actual Yield and Cycle Time. Both measures are essentially continuous variables that are analyzed using linear regression methodology. As indicated in Table 7.5, however, these variables are negatively correlated with each other. In other words, higher actual yields are associated with lower cycle times, and vice versa. At the same time, organizational practices may simultaneously affect each measure of shop-floor performance.

The analysis of Actual Yield and Cycle Time is therefore undertaken with a statistical method called Seemingly Unrelated Regression (SUR). SUR gets its name because at first glance, the equations seem unrelated, but are in fact related through correlation in their errors. Many econometric models (including the one examined here) contain a number of linear equations of different dependent variables. It is often unrealistic to expect that the errors are uncorrelated between and among these different equations. A set of equations that has contemporaneous cross-equation error correlation is thus called an SUR system, and makes corrections to the standard errors of the estimates in each equation to adjust for this correlation.

We are also interested in Change in Batches Failed, Change in Actual Yield, and Change in Cycle Time. For instance, do some organizational practices or factors allow pharmaceutical manufacturers to improve batches failed, actual yield, or cycle time over time, or do these factors reduce these performance measures? To explore the possibility of such relationships, covariates are regressed against Change in Batches Failed, Change in Actual Yield, and Change in Cycle Time, respectively. Because each of these variables is a ratio of the current month’s value divided by the last month’s value, no change equates to 1. Changes in batches failed and cycle time that are greater than 1 and changes in actual yield that is less than 1 indicate worse performance, while changes in batches failed and cycle time are less than 1 and changes in actual yield that is greater than 1 indicate better performance.

Given that several of the constructed independent variables are standardized (a mean of zero and a variance of one), interpretation of results should be based on the signs of the

30

coefficients rather than on their magnitudes. However, for the two deviation management practices examined, the results can be interpreted based on the signs and magnitudes of the coefficients.

It also should be noted that these models provide an initial lens with which to view the data. Given the time series nature of the data collected by the PMRP the data can be analyzed using other econometric techniques and methodologies. In some instances, the use of these other econometric methodologies may offer superior approaches with which to assess and interpret the data. Nonetheless, the analyses presented below an appropriate first glimpse of the relationship between various organizational practices in the performance measures identified above.

7.4. “Levels” Analytic Results

Table 7.6 provides the results of the econometric analysis for the levels of Batches Failed, Actual Yield, and Cycle Time. All of the regression results for each performance metric use the same set of explanatory variables, where appropriate. All of the regressions employ robust standard errors and clustering. We explore each performance metric in turn.

7.4.1. Batches Failed Negative binomial regression results for the number of batches failed are presented in the first column of Table 7.6. The model fit the data reasonably well. The R2 (a measure of the explained variance of the regression) is 0.19, but R2 should not be relied on as the only measure of fit. More importantly, the χ2 statistic for each model is statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

The coefficients for Number of Processes and Number of Products are both negative and highly statistically significant. These results indicate that a lower number of batches fail with more processes and more products produced in the manufacturing facility, respectively—results that are somewhat surprising given the complexity argument discussed above.

Contract manufacturers fail more batches than manufacturers not engaged in contract manufacturing, evidenced by the positive and statistically significant coefficient for Contract Manufacturer.

The coefficients for Employee Experience and Employee Age are positive and significant, indicating that older and more experienced employees correspond to more Batches Failed.

The positive and highly statistically significant coefficient for Deviation Review and Approval indicates that more groups who take part in the review and approval of deviations corresponds to a larger number of batches failed.

7.4.2. Actual Yield The second column of Table 7.6 provides the coefficient estimates for the level of Actual Yield. Recall that the analysis of Actual Yield and Cycle Time is undertaken using SUR

31

analysis, which accounts for the possibility that Actual Yield and Cycle Time may be correlated and adjusts the standard errors accordingly.

The model fit the data reasonably well. The R2 of 0.21 indicates that more than twenty per cent of the variance is explained by the included coefficients. The χ2 statistic for each model is also statistically significant and indicates rejection of the null hypothesis that the set of coefficients is random.

Table 7.6: Batches Failed and Yield and Cycle Time SUR Levels Analysis Batches Failed Actual Yield Cycle Time


** 15.775 (10.139)

-43.800 (11.125)

**


** 0.228 (0.169)

-0.132 (0.163)

Contract Manufacturer 2.154(0.950)

* -24.534 (18.219)

37.527 (15.861)

*

Employee Experience 2.462(0.234)

** -28.076 (9.603)

** 39.556 (11.721)

**

Employee Age 0.989(0.427)

* -4.935 (7.577)

32.647 (6.743)

**

IT Usage -0.516(0.480)

20.009 (8.531)

* -68.692 (8.351)

**

Lead Deviation Responsibility 0.520(0.408)

0.596 (7.178)

-3.633 (5.980)

Deviation Review & Approval 0.772(0.194)

** -4.009 (4.713)

-2.012 (3.654)

Constant 0.280(0.564)

43.792 (19.961)

* 145.279 (20.700)

**

N 714 794 794 R2 0.19 0.21 0.57 χ2 97.14 ** 27.67 ** 21.27 **** p<0.01, * p<0.05, † p<0.10; Standard errors of coefficients in parentheses. The results indicate somewhat surprisingly that level of actual yield decreases with employee experience, evidenced by the negative and highly statistically significant coefficient for Employee Experience.

The large, positive and statistically significant coefficient for IT Usage indicates that the extent and use of information technology in the manufacturing facility is associated with higher actual yields.

7.4.3. Cycle Time The third column of Table 7.6 provides the coefficient estimates for the level of Cycle Time. The model fit the data well. The R2 of 0.47 indicates that almost half of the variance is explained by the included coefficients. The χ2 statistic for each model is also

32

statistically significant, indicating rejection of the null hypothesis that the set of coefficients is random.

The large, negative and statistically significant coefficient for Number of Processes suggests that cycle times are lower for those facilities with a greater number of manufacturing processes.

The large, positive and significant coefficient for Contract Manufacturer suggests that cycle times are longer for those facilities that engage in contract manufacturing.

The coefficients for Employee Experience and Employee Age are large, negative and statistically significant, and indicate that employee experience and age corresponds with higher cycle times.

The large, negative and statistically significant coefficient for IT Usage indicates that the extent and use of information technology in the manufacturing facility is associated with better cycle times.

7.5. “Rates of Change” Analytic Results

Table 7.7 provides the results of the econometric analysis for Change in Batches Failed, Change in Actual Yield, and Change in Cycle Time.

Recall that each of these dependent variables is a ratio of the current month’s value divided by the last month’s value, with no change equating to 1. More batches failed, higher actual yields and longer cycle times are greater than 1, while less batches failed, lower actual yields and shorter cycle times are less than one.

All of the regression results for each performance metric use the same set of explanatory variables and where appropriate. We also utilize robust standard errors and clustering for this analysis. We explore each performance metric in turn.

7.5.1. Batches Failed Linear regression results for the rate of change in batches failed is presented in the first column of Table 7.7. The model fits the data poorly. The R2 achieved in this model is 0.05. Yet, the χ2 statistic for is highly statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

The coefficients for the Number of Products and Number of Products are negative and statistically significant, indicating that Change in Batches Failed is lower for those manufacturing facilities with a larger number of processes and produce a larger number of products.

The results indicate that Change in Batches Failed increases with employee experience, evidenced by the positive and highly statistically significant coefficient for Employee Experience.

The coefficient for Lead Deviation Responsibility is negative and weakly statistically significant, which suggests that Change in Batches Failed decreases with more groups assigned to lead deviation responsibility within the manufacturing facility.

33

7.5.2. Actual Yield The second column of Table 7.7 provides the coefficient estimates for the rate of Change in Actual Yield. The analyses of Change in Actual Yield and Change in Cycle Time are undertaken using SUR analysis.

The model fits the data poorly. The R2 of 0.02 indicates that only two per cent of the variance is explained by the included coefficients. Moreover, the χ2 statistic is not significant. We therefore cannot reject the null hypothesis that the set of coefficients is random. Thus, the results of its analysis must be interpreted with caution.

The coefficient for the Number of Processes is highly statistically significant and negative, which implies that actual yield declines over time for those manufacturing facilities that manage a larger number of processes.

The coefficient for Contract Manufacturer is positive and significant, and indicates that the month-to-month change in actual yield increases for those manufacturing facilities that provide contract manufacturing services.

The results indicate that the month-to-month change in actual yield increases with employee experience evidenced by the positive and highly statistically significant coefficient for Employee Experience, but decreases with employee age, evidenced by the negative and highly statistically significant coefficient for Employee Age.

The positive and highly statistically significant coefficients for Lead Deviation Responsibility and Deviation Review and Approval indicate that actual yield increases from month-to-month for those manufacturing facilities that engage in these deviation management practices.

Finally, we note that the constant term is positive and highly statistically significant. The coefficient’s magnitude indicates that actual yield an average increases slightly on a month-to-month basis.

7.5.3. Cycle Time The third column of Table 7.7 provides the coefficient estimates for Change in Cycle Time.

The regression model fits the data poorly. The R2 of 0.02 indicates that only two per cent of the variance is explained by the coefficients included. The χ2 statistic is not statistically significant. Thus, results must be interpreted with caution.

The coefficient for the Number of Products is positive and significant, which implies that the cycle time increases from month-to-month with the number of products produced in the manufacturing facility.

The results also indicate that cycle time decreases month-to-month with high-levels of employee experience evidenced by the negative and highly statistically significant coefficient for Employee Experience.

Finally, the constant term is positive and highly significant. This coefficient indicates that cycle times tend to increase from one month to the next.

34

Table 7.7: Batches Failed and Yield and Cycle Time SUR Rate of Change Analysis

Change in Batches Failed

Change in Actual Yield

Change inCycle Time


† -0.026(0.005)

** -0.027 (0.026)


** 0.000(0.000)

0.003 (0.000)

**

Contract Manufacturer -0.245(0.243)

0.040(0.009)

** 0.106 (0.056)

†

Employee Experience 0.821(0.239)

** 0.042(0.011)

** -0.054 (0.021)

**

Employee Age -0.463(0.341)

-0.019(0.008)

** 0.012 (0.023)

IT Usage 0.357(0.309)

0.000(0.004)

0.013 (0.031)

Lead Deviation Responsibility -0.436(0.220)

† 0.009(0.003)

** -0.030 (0.022)

Deviation Review & Approval 0.148(0.145)

0.013(0.004)

** -0.002 (0.013)

Constant 1.735(0.245)

* 1.007(0.012)

** 1.085 (0.035)

**

N 54 585 585 R2 0.05 0.02 0.02 χ2 318.97** 1.02 0.72

** p<0.01, * p<0.05, † p<0.10; Standard errors of coefficients in parentheses.


Several interesting conclusions can be drawn from the statistical results presented above.

API contract manufacturers appear to achieve worse performance on numerous performance dimensions. In particular, contract manufacturing has a higher number of batches failed and longer cycle times than manufacturers not engaged in contract manufacturing. At the same time, contract manufacturers have actual yields that are decreasing month-to-month. API contract manufacturers do, however, achieve decreasing rates of batches failed from month-to-month.

API manufacturing facilities that employ older and more experienced employees generally perform worse than those facilities with younger and less experienced employees. These results are surprising and may be related to the age or vintage of the facility or equipment or that older and more experienced employees are less likely to change their behavioral routines. This result requires further analysis.

API manufacturing facilities that utilize information technology to electronically and automatically report deviations or tack deviations by lot, type of issue, or people assigned

35

to resolving the deviation have higher actual yields and lower cycle times. These results support the argument that investment in information technology may provide significant payoffs in performance improvement.

Assigning deviation review and approval to several groups appears to increase the number of batches failed, but also significantly improves actual yield from month-to-month.

Finally, it is important to note that all of these results describe correlations and do not determine causation. Rather than higher batches failed, lower actual yields or higher cycle times being the “result” of implementing a given managerial or organizational practice, it may be that worse performance “causes” manufacturing facilities to adopt these practices to improve these performance measures. Different causations can be attributed to all of the findings described above.

8. Deviation Management Performance: OT&I


This section of the report provides a statistical analysis of deviation management performance for the data collected in the Pharmaceutical Manufacturing Research Project (PMRP). It examines only Oral, Topical and Injectable Manufacturers—26 unique manufacturing facilities from 14 unique pharmaceutical firms.

The performance section of the PMRP collected monthly data on (1) product unavailability; (2) the number of field alerts; (3) the percentage of finished product recalls; (4) the number of deviations that arise from raw materials, production components (equipment) and production product and process specifications, respectively; and (5) the percentage of repeat deviations that arise from raw materials, production components (equipment) and production product and process specifications, respectively.

Table 8.1 provides definitions of these performance measures.

Using data collected in the PMRP questionnaire about the manufacturing facility (sections 3a and 3b), human resources (section 4), performance metrics (section 6), and deviation management (sections 8a and 8b) sections, the analysis explores those factors that correlate with various deviation management performance metrics described above. The interested reader is referred to Appendix D, for a complete list of data collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire.

The text below describes the data used in the empirical analysis, summarizes the methodology through which the data is analyzed, presents the empirical results, and finally discusses results.

36

Table 8.1: Deviation and Regulatory Performance Measures Measure Definition Product Unavailability An indicator as to whether to product was unavailable

(stock-out) during the month due to manufacturing problems.

Field Alerts The number of field alerts for a given product in a given month.

Finished Product Recalls The percentage of finished product recalls during the month, expressed as a percentage.

Raw Material Deviations The number of deviations that arise from raw materials purchased per month.

Production Component Deviations

The number of deviations that arise from production components (equipment) per month.

Product/Process Specification Deviations

The number of deviations that arise from the product and process specifications per month.

Repeat Raw Material Deviations

The percentage of deviations that arise from raw materials purchased that are repeat occurrences per month.

Repeat Production Component Deviations

The percentage of deviations that arise from production components (equipment) that are repeat occurrences per month.

Repeat Product/Process Specification Deviations

The percentage of deviations that arise from product/process specifications that are repeat occurrences per month.


The data input by pharmaceutical firms on deviation and regulatory performance required extensive data entry. The PMRP collected performance information for oral, topical and injectable manufacturers. These data represent roughly 4,000 monthly observations for 71 products manufactured in 26 distinct manufacturing facilities by 14 distinct pharmaceutical firms.

While a number of different statistical analyses were undertaken, this section focuses attention only on those empirical analyses that were sensible and yielded significant findings.

One aspect of deviation management performance examines product availability, field alerts, and finished product recalls. Our sample unfortunately did not provide enough heterogeneity in the dependent variable for field alerts and finished product recalls. For field alerts, there were just 14 instances and 18 in total in the roughly 4,000 monthly observations. For finished product recalls, there were just 3 instances and 5 in total for the 4,000 monthly observations gathered. We did, however, find sufficient heterogeneity in product unavailability to undertake statistical analyses. There were 113 instances in which a given product was unavailable (stock out) during the month due to manufacturing problems.

37

Another aspect of deviation management performance is concerned with the number of raw material, production component (equipment) and product/process specification deviations in a given month, as well as the percentage of deviations that repeat from these categories, respectively. In this report, we provide only analytic results for the number of deviations, but plan to report the repeat deviation analysis at a latter date.

Besides the performance level of deviations reported, we also are interested in the rates of change (or ratio) of these performance metrics.

Table 8.2 provides definitions for the dependent variables used in the econometric analyses.

Table 8.2: Dependent Variable Definitions Product Unavailability Monthly indicator as to whether the product was

unavailable (stock-out) during the month due to manufacturing problems.

RM Deviations The number of deviations that arise from raw materials purchased per month.

PC Deviations The number of deviations that arise from production components (equipment) per month.

PP Deviations The number of deviations that arise from the product and process specifications per month.

RM Deviation Rate Ratio of current month’s number of raw material deviations to the prior month’s raw material deviations.

PC Deviation Rate Ratio of current month’s number of production component (equipment) deviations to the prior month’s raw material deviations.

PP Deviation Rate Ratio of current month’s number of product and process specification deviations to the prior month’s raw material deviations.

As mentioned above, we utilized a number of independent variables collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire in the empirical analyses of regulatory and deviation performance. Most of these variables are collected yearly from 1999 to 2003.


The deviation management performance analysis ultimately employed seven dependent variables and up to 13 independent variables, depending upon the performance metric. The logarithms of total employees and facility size are taken because the distributions of these data are skewed.

38










Scale Reliability = 0.85. IT Usage A measure of the interim correlation of the extent to which

information technology is utilized in the manufacturing facility to:



PAT Data Analysis Tools A measure of the interim correlation of the extent to which the manufacturing facility uses the following multivariate data acquisition and analysis tools:

• Statistical design of experiments

39

• Response surface methodologies • Process simulation • Pattern recognition tools

Scale Reliability = 0.74. PAT Process Analytic Tools A measure of the interim correlation of the extent to which

the manufacturing facility uses the following process analyzer or process analytic chemistry tools:

• Simple process measurement • Chemical composition measurement • Physical attribute measurement

Scale Reliability = 0.73. PAT Monitoring Tools A measure of the interim correlation of the extent to which

the manufacturing facility uses the following process monitoring, control and endpoint tools:

• Critical material in process attributes related to product quality

• Real-time or near real-time (e.g., on-, in- or at-line) monitoring of critical elements

• Adjustments to ensure control of all critical elements • Mathematical relationships between product quality

attributes and measurement of critical material and process attributes

• Process endpoint monitoring and control Scale Reliability = 0.84.

Lot Failure Responsibility A measure of the interim correlation of the extent to which a QA manager has responsibility to fail a lot compared to the head of QA at the manufacturing facility. Scale Reliability = 0.80.

Lead Deviation Responsibility A measure of the interim correlation of the extent to which operations and quality take the lead in responding to deviations in the manufacturing facility. Scale Reliability = 0.66.

Deviation Review & Approval A measure of the interim correlation of the extent to which operations, engineering and regulatory affairs must review and approve deviations in the manufacturing facility. Scale Reliability = 0.66.

Deviation Participation A measure of the interim correlation of the extent to which operations and quality participate in reviewing and approving deviations even though they are not required to review and approve in the manufacturing facility. Scale Reliability = 0.81.

Several of the independent variables are based on constructed variables derived from factor analysis to reduce the number of independent variables for our econometric analyses. Doing so improves the number of degrees of freedom for our econometric analysis.

40

A factor analysis of all relevant variables for the set of questions in each survey area was used for each of the constructed variables. For instance, the factor analysis for Employee Experience included variables representing the number of years of experience for each category of worker: operators, craft workers, technicians, professionals and managers. Based on results of the factor analysis we first constructed a Cronbach’s alpha for each eigenvalue greater than one, and then included only those variables with a factor loading greater than 0.4. The constructed variable represents the corresponding alpha score. Scale reliability is presented in Table 8.3 for each constructed variable. All scale reliability coefficients are near or exceed the expected norm of 0.7.


Table 8.4: Summary Statistics MEAN S.D. MIN MAX Dependent Variables Product Unavailability 0.03 0.17 0.00 1.00 RM Deviations 0.27 1.76 0.00 45.00 PC Deviations 0.47 1.47 0.00 37.00 PP Deviations 1.95 9.93 0.00 156.00 RM Deviation Rate 0.57 1.67 0.00 22.00 PC Deviation Rate 0.81 1.48 0.00 18.50 PP Deviation Rate 1.04 1.61 0.00 21.00 Independent Variables Total Employees 6.15 0.80 2.83 7.50 Facility Size 9.90 0.95 8.20 14.51 Number of Processes 1.75 0.77 1.00 4.00 Number of Products 18.83 24.12 0.00 109.00 Contract Manufacturer 0.53 0.50 0.00 1.00 Employee Experience -0.01 0.79 -1.62 1.57 Employee Training 0.04 0.63 -1.18 1.10 IT Usage 0.07 0.84 -1.13 0.91 PAT Data Analysis Tools -0.02 0.73 -0.58 1.86 PAT Process Analytic Tools -0.03 0.80 -1.79 0.69 PAT Monitoring Tools 0.05 0.77 -1.94 0.51 Lot Failure Responsibility 0.04 0.95 -1.40 0.71 Lead Deviation Responsibility -0.04 0.87 -1.09 0.95 Deviation Review & Approval 0.00 0.76 -0.61 2.29 Deviation Participation -0.04 0.92 -1.67 0.61

In terms of the performance measures, the mean level of Product Unavailability is 0.03 per month, or roughly 2.8 per cent likelihood in any month. The data indicate product is available in more than 97 per cent of the sample in any given month.

The mean RM Deviations is a 0.27 with a large standard deviation, which is due to the fact that deviations cannot be negative and the maximum of RM Deviations, 45, is large. The mean PC Deviations is 0.47 with a standard deviation of 1.47, and a maximum of 37.

41

The means of these two deviations indicate that they arise on average less than once a month. In contrast, PP Deviations has a mean of 1.95 and a standard deviation of 9.93, indicating that on average PP Deviations arise almost twice a month; although, this average is skewed because the maximum monthly number of PP Deviations is 156 and the minimum is constrained to be zero.

In terms of the independent variables, the mean level of total employees (logged) within a manufacturing facility in the sample is 6.15, which is equivalent to just over 600 employees. Similarly, the mean facility size (logged) is 9.90 square meters, which is equivalent to about 20,000 square meters.

Each manufacturing facility has almost two distinct manufacturing processes on average, and produces almost 19 distinct compounds. Both of these variables show substantial variance among the manufacturing facilities in the sample. About half of the sample manufacturing facilities in the sample are contract manufacturers.

As mentioned above, the rest of the independent variables are based on constructed variables derived from factor analysis to reduce the number of independent variables for our econometric analyses. The items in the scale are standardized (mean 0, variance 1) so discussion of their summary statistics is uninteresting. Note that with the constructed variables neither is the mean 0 nor the standard deviation 1. The difference lies in the fact that these independent variables are constructed prior to dropping observations with missing data.

Table 8.5 presents correlations for these dependent and independent variables. Pair-wise correlations in bold are statistically significant at 95% confidence intervals. Because of these dense and complicated set of relationships indicated by the pair-wise correlation analysis, we interpret the data through various regression analyses.



The first deviation management performance metric examined is Product Unavailability in a given month. Because Product Unavailability is an indicator variable (zero or one), a Logit analysis is used to estimate the relationship between the independent and dependent variables. We also employ the Huber/White/sandwich estimator of variance to achieve robust standard errors, as well as correct the variance-covariance matrix of the estimators to account for clustering (by manufacturing facility). Clustering accounts for the fact that observations are independent across manufacturing facilities, but not necessarily within manufacturing facilities.

42


Prod

uct U

nava

ilabi

lity

RM D

evia

tions

PC D

evia

tions

PP D

evia

tions

RM D

evia

tion

Rate

PC D

evia

tion

Rate

PP D

evia

tion

Rate

Tota

l Em

ploy

ees

Faci

lity

Size

Num

ber o

f Pro

cess

es

Num

ber o

f Pro

duct

s

Con

trac

t Man

ufac

ture

r

Empl

oyee

Exp

erie

nce

Empl

oyee

Tra

inin

g

IT U

sage

PAT

Dat

a An

alys

is T

ools

PAT

Proc

ess A

naly

tic T

ools

PAT

Mon

itori

ng T

ools

Lot F

ailu

re R

espo

nsib

ility

Lead

Dev

iatio

n Re

spon

sibi

lity

Dev

iatio

n Re

view

& A

ppro

val

Product Unavailability 1 RM Deviations 0.09 1 PC Deviations 0.17 0.13 1 PP Deviations 0.08 0.60 0.15 1 RM Deviation Rate 0.13 0.60 0.18 0.20 1 PC Deviation Rate 0.07 0.04 0.69 0.01 0.15 1 PP Deviation Rate 0.04 0.02 0.06 0.19 0.20 0.10 1 Total Employees 0.24 0.14 0.30 0.14 0.17 0.18 0.06 1 Facility Size 0.14 0.03 0.08 0.06 -0.01 0.04 0.17 0.46 1 Number of Processes 0.18 -0.02 0.04 0.03 0.08 0.02 0.02 0.25 0.07 1 Number of Products 0.31 -0.02 0.06 -0.04 0.05 0.04 0.03 0.41 0.49 0.22 1 Contract Manufacturer -0.18 -0.11 -0.08 -0.06 -0.02 -0.06 0.03 0.22 -0.01 0.06 0.21 1 Employee Experience 0.05 0.03 0.10 -0.02 -0.02 0.03 0.04 0.08 0.24 -0.03 0.14 -0.03 1 Employee Training 0.04 -0.08 0.01 -0.13 -0.11 0.04 -0.03 0.00 0.07 -0.05 0.13 0.20 0.28 1 IT Usage 0.14 0.11 0.03 0.12 -0.01 -0.06 0.06 0.43 0.31 0.01 0.22 0.13 -0.11 0.03 1 PAT Data Analysis Tools 0.32 0.04 0.08 0.07 0.08 0.00 -0.02 0.33 0.35 0.11 0.28 -0.13 -0.14 0.14 0.26 1 PAT Process Analytic Tools 0.10 0.11 0.20 0.15 0.14 0.11 0.12 0.44 0.32 0.17 0.26 0.10 0.00 0.17 0.09 0.42 1 PAT Monitoring Tools 0.06 0.02 0.10 0.00 -0.01 0.04 0.02 0.27 0.26 0.14 0.12 0.07 0.28 0.56 0.01 0.37 0.55 1 Lot Failure Responsibility -0.22 -0.12 -0.17 -0.12 -0.02 -0.03 -0.13 -0.20 -0.50 0.11 -0.25 0.31 -0.59 0.03 -0.11 -0.15 -0.24 -0.16 1 Lead Deviation Responsibility 0.13 0.02 -0.04 0.11 0.05 -0.01 0.03 0.16 0.11 0.14 0.11 0.27 0.06 0.41 0.24 0.41 0.27 0.28 -0.01 1 Deviation Review & Approval -0.02 -0.10 -0.14 -0.13 -0.08 -0.11 -0.14 -0.43 -0.29 0.08 0.36 -0.01 -0.01 0.10 -0.16 -0.15 -0.11 -0.21 0.25 -0.02 1 Deviation Participation 0.07 0.02 0.05 -0.05 0.01 0.19 0.02 0.13 -0.05 -0.10 0.14 0.28 -0.15 -0.02 0.25 -0.15 -0.21 -0.26 0.28 0.18 -0.08 Bold indicates pair-wise significance at 95% confidence interval.

43

Our analyses of RM Deviations, PC Deviations, and PP Deviations employ a linear regression methodology. These different types of deviations may be correlated; therefore, we employ a statistical method called Seemingly Unrelated Regression (SUR). Many econometric models (including the one examined here) contain a number of linear equations of different dependent variables. It is often unrealistic to expect that the errors are uncorrelated between and among these different equations. SUR gets its name because at first glance, the equations seem unrelated, but in fact are related through the correlation in the errors. A set of equations that has contemporaneous cross-equation error correlation is thus called an SUR system, and makes corrections to the standard errors of the estimates in each equation to adjust for this correlation. We again employ robust standard errors and clustering for this analysis.

We are unable to investigate change in product unavailability because of the limited number of observations. By contrast, we do have sufficient data to evaluate those factors that impact the change in deviations. Our third set of analyses investigates the relationship between our independent variables and RM Deviation Rate, PC Deviation Rate, and PP Deviation Rate. For reasons described above, we again employ an SUR methodology for analyzing this data.

Given that several of the constructed independent variables are standardized (a mean of zero and a variance of one), interpretation of results should be based on the signs of the coefficients rather than on their magnitudes. It also should be noted that these models provide an initial lens through which to view the data. Given the time series nature of the data collected by the PMRP, the data can be analyzed using other econometric techniques and methodologies. In some instances, the use of these other econometric methodologies may offer superior approaches with which to assess and interpret the data. Nonetheless, the analyses presented below are an appropriate first glimpse of the relationship between various organizational practices in the performance measures identified above.

8.4. Product Unavailability Analytic Results

The first column Table 8.6 provides the results of the logit analysis for Product Unavailability. The model fit the data well with an R2 of 0.52 and a highly statistically significant χ2 statistic, indicating rejection of the null hypothesis that the set of coefficients is random.

The results indicate that Product Unavailability increased with the total number of employees, the number of products produced in the facility, and information technology usage. While the coefficient for Total Employees is weakly statistically significant, the coefficients for Number of Products and for IT Usage are both highly statistically significant.

Results also indicate that Product Unavailability declines the larger is the facility, when PAT monitoring tools are used in the facility, and when deviation review and approval of deviations involves operations, engineering, and regulatory affairs. The coefficient for Facility Size is highly statistically significant and the coefficients for PAT Monitoring Tools and Deviation Review and Approval are statistically significant.

44

8.5. Deviation Number Analytic Results

Table 8.6 provides the results of the econometric analysis for the levels of RM Deviation, PC Deviation, and PP Deviation. All of the regression results for each performance metric use the same set of explanatory variables, where appropriate. All of the regressions employ robust standard errors and clustering and are the result of an SUR estimation procedure. We explore each performance metric in turn.

8.5.1. Raw Materials Deviations Regression results for the number of RM Deviations are presented in the second column of Table 8.6. The model fit the data poorly. Although the R2 (a measure of the explained variance of the regression) is small in this model (0.069), R2 should not be relied on as the only measure of fit. More importantly, the χ2 statistic for the model is statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

That said, the results indicate no coefficient is statistically significant. None of the factors in our model appear to be statistically related to the level of raw material deviations.

8.5.2. Production Component (Equipment) Deviations The third column of Table 8.6 provides the coefficient estimates for the level of PC Deviations. The model fit the data modestly. The R2 of 0.176 indicates that nearly 18 per cent of the variance is explained by the included coefficients. The χ2 statistic for the model is highly statistically significant and indicates rejection of the null hypothesis that the set of coefficients is random.

The coefficients for Total Employees, Employee Experience, IT Usage, and Lot Failure Responsibility are all negative and highly statistically significant. These coefficient estimates indicate that production component deviations is lower with a higher number of employees, is lower with a higher level of employee experience, and is lower when lot failure responsibility is assigned to a QA manager as opposed to the head QA at the manufacturing facility.

The coefficients for Facility Size, Number of Processes, and PAT Data Analysis Tools are positive. While the coefficient for Facility Size is highly statistically significant, the two other coefficients are weakly statistically significant. Larger facilities have a higher number of production component deviations. Similarly, facilities with a large number processes also have a high number of production deviations. Surprisingly, facilities that use PAT data analysis tools also have a higher level of deviations; although, as mentioned below, PAT data analysis tools might be adopted because of high production component deviations as opposed to causing such deviations. It also might be the case that PAT data analysis tools reveal more production component deviations than would otherwise be identified.

45

Table 8.6: Product Unavailability and SUR Deviation Levels Analyses

Product

UnavailabilityRM

Deviations PC

Deviations PP

DeviationsTotal Employees 3.866† -0.058 -0.277 ** 1.154 (2.306) (0.091) (0.059) (1.088) Facility Size -1.565** 0.117 0.651 ** -0.229 (0.484) (0.140) (0.130) (1.273) Number of Processes -0.047 0.132 + -0.282 (0.092) (0.068) (1.026) Number of Products 0.056** -0.005 0.002 -0.101

(0.020) (0.007) (0.003) (0.076) Contract Manufacturer -0.435 0.058 -3.349 (0.399) (0.164) (4.173) Employee Experience 1.107 0.135 -0.338 ** 2.119 (1.044) (0.292) (0.107) (3.115) Employee Training 0.196 -0.205 0.194 -1.592 (0.592) (0.262) (0.141) (2.454) IT Usage 1.437** 0.212 -0.372 ** 2.543 (0.524) (0.243) (0.141) (2.158) PAT Data Analysis Tools -0.047 0.139 † -1.198 (0.117) (0.084) (1.265) PAT Process Analytic Tools -0.920 0.372 -0.144 5.710 (0.767) (0.412) (0.095) (4.545) PAT Monitoring Tools -1.400* -0.096 0.078 -2.895 (0.676) (0.219) (0.079) (2.745) Lot Failure Responsibility -0.041 -0.303 ** -0.088 (0.141) (0.098) (1.701) Deviation Participation 0.124 0.066 0.899 (0.096) (0.059) (0.969) Lead Deviation Responsibility 0.682 (0.420) Deviation Review & Approval -3.287* (1.530) Constant -16.384 0.551 -1.140 -2.831 (15.728) (1.247) 0.961 10.968 N 3658 3586 3517 3532 R2 0.52 0.07 0.18 0.12 χ2 156.14** 21.5** 58.94 ** 36.61**** p<0.01, * p<0.05, † p<0.10; Standard errors of coefficients in parentheses.

46

8.5.3. Product and Process Specification Deviations The fourth column of Table 8.6 provides the coefficient estimates for the level of PP Deviations. The model fit the data modestly. The R2 of 0.12 indicates that 12 per cent of the variance is explained by the included coefficients. The χ2 statistic for the model is highly statistically significant, indicating rejection of the null hypothesis that the set of coefficients is random.

Like the results for raw material deviations, this statistical model indicates that no coefficient is statistically significant. None of the factors in our model appear to be statistically related to the level of product and process parameter deviations.

8.6. Deviation Rate Analytic Results

Table 8.7 provides the results of the econometric analysis for the rates of change in raw material deviations, production component deviations, and product and process specification deviations.

Recall that each of these dependent variables is a ratio of the current month’s value divided by the last month’s value, with no change equating to 1. Increases in RM Deviation Rate, PC Deviation Rate, and PP Deviation Rate are represented by rates of change that are greater than 1, while decreases in these measures are less than one.

All of the regressions employ robust standard errors and clustering and are the result of an SUR estimation procedure. We explore each performance metric in turn.

8.6.1. Raw Material Deviation Rate Linear regression results for the rate of change in raw material deviations, RM Deviation Rate, and presented in the first column of Table 8.7. The model fits the data modestly. The R2 achieved in this model is 0.11. The χ2 statistic for is statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

The coefficients for Facility Size, Employee Experience, IT Usage, and Lot Failure Responsibility all are negative and highly statistically significant. These findings indicate that the rate of raw material deviations for API production decline the larger is the facility, the more experienced are employees, the greater the use of information technology in the facility, and when lot failure responsibility resides with a QA manager as opposed to the head of QA for the manufacturing facility.

In contrast, the coefficients for Total Employees, and the constant term are positive and highly significant. The coefficients for the former indicates that the rate of raw material deviations increases the more employees there are at the facility. Also, the positive constant term indicates that on average raw material deviations increase over time.

8.6.2. Production Component (Equipment) Deviation Rate The second column of Table 8.7 provides the coefficient estimates for the rate of change in PC Deviations. The model fits the data modestly. The R2 of 0.09 indicates that only 8 to 9 percent of the variance is explained by the included coefficients. The χ2 statistic for

47

model is highly statistically significant, which allows us to reject the null hypothesis that the set of coefficients is random.

The coefficients for Facility Size, Employee Experience, IT Usage, and Lot Failure Responsibility are negative with the first two coefficient estimates highly statistically significant and the latter two coefficient estimates weakly statistically significant. These results suggest that the rate of production component deviations decline over time the more those facilities are large and the more that employees have experience. Also, the rate of production component deviations declines the more that information technology is used in the facility and when lot failure responsibility resides with a QA manager instead of the head of QA for the manufacturing facility.

The rate of production component deviations appears to increase with the number of employees at the facility and with employee training. The coefficient for Total Employees is positive and highly statistically significant whereas the coefficient for Employee Training is positive and statistically significant. Please note that this latter result does not imply causality. For instance, the more that employees are trained the more likely they may be able to identify production component deviations as opposed to employee training causing production component deviations.

8.6.3. Product and Process Specification Deviation Rate The third column of Table 8.7 provides the coefficient estimates for PP Deviation Rate. Again, the regression fits the data modestly. The R2 of 0.09 indicates that eight to nine percent of the variance is explained by the coefficients included. The χ2 statistic for the model is highly statistically significant, which allows us to reject the null hypothesis that the set of coefficients is random.

The coefficients for Number of Products, Contract Manufacturer, Employee Training, and the constant term are negative and highly statistically significant except for the coefficient for Employee Training, which is statistically significant. These coefficients indicate that the rate of product and process specification deviation declines the more products produced in the facility, if the facility engages in contract manufacturing, and with higher levels of employee training. The negative coefficient for the constant term indicates that, on average, facilities are reducing the number of product and process specification deviations.

The coefficients for Facility Size, Employee Experience, PAT Process Analytic Tools, and Deviation Participation all are positive and highly statistically significant except for the coefficient for Employee Experience, which is statistically significant. These coefficients indicate that the rate of product and process specification deviations increases the larger is the manufacturing facility, the more experience employees have, the more facilities use PAT processed analytic tools, and when operations and quality participate in reviewing and approving deviations even though they are not required to do so.

48

Table 8.7: SUR Analysis of RM, PC, and PP Deviations Rate of Change

RM

Deviations PC

Deviations PP

Deviations Total Employees 0.238** 0.714** -0.059 (0.089) (0.119) (0.063) Facility Size -1.120** -0.495** 0.435** (0.241) (0.115) (0.070) Number of Processes -0.107 0.077 0.079 (0.134) (0.062) (0.074) Number of Products -0.001 0.000 -0.006** (0.003) (0.002) (0.001) Contract Manufacturer -0.099 0.026 -0.201** (0.130) (0.093) (0.074) Employee Experience -0.763** -0.261** 0.122* (0.178) (0.079) (0.061) Employee Training 0.261 0.304* -0.161* (0.163) (0.121) (0.068) IT Usage -0.247** -0.145† 0.018 (0.089) (0.087) (0.062) PAT Data Analysis Tools 0.056 -0.082 -0.046 (0.152) (0.069) (0.032) PAT Process Analytic Tools 0.594** 0.017 0.533** (0.058) (0.106) (0.070) PAT Monitoring Tools -0.155 -0.003 -0.135 (0.139) (0.091) (0.114) Lot Failure Responsibility -0.769** -0.168† 0.061 (0.202) (0.086) (0.055) Deviation Participation 0.040 -0.066 0.186** (0.056) (0.053) (0.041) Constant 10.139** 0.643 -3.240** (2.206) (0.651) (0.564) N 238 632 1054 R2 0.11 0.09 0.09 χ2 3.25** 5.49** 8.60**


49


Several interesting conclusions can be drawn from the statistical results presented above. One conclusion pertains to manufacturing facility complexity. More complex OT&I manufacturing facilities—in terms of the number of employees, facility size, number of processes manufactured, and number of products—lead to higher levels of product unavailability, high levels of production component deviations, and increases in the level raw material production component, and product and process specification deviations over time. Yet, this complexity is not entirely associated with negative outcomes. A large number of employees corresponds to a lower level of production component deviations and a large facility size corresponds to more product availability and reductions in the rate of raw material and production component deviations. The more products in the manufacturing facility the lower will be the number of product and process specification deviations; although, the impact is rather small.

OT&I contract manufacturing has no impact on production unavailability or the level of deviations. It does however lead to lower product and process specification deviations over time.

Employee experience is beneficial from the standpoint of having lower levels of component deviations and lower raw material and production component deviations over time. We do see, however, an increase in the rate of product and process specification deviations over time; but, this may be due to experience being better able to recognize such deviations.

Employee training had no impact on product unavailability or the level of deviations. It did correspond to an increasing rate of production component deviations and declining rates of product and process specification deviations. These mixed results deserve further investigation.

The results for information technology usage and the various process analytic technology measures also yield mixed results. The use of information technology corresponds to low levels of production component deviations and reductions in raw material and production component deviations. Yet, the use of IT corresponds to higher levels of product availability. PAT data analysis tools correspond to higher levels of production component deviations and process analytic tools lead to higher rates of raw material and product and process specification deviations. Only monitoring tools, among the PAT measures, led to lower levels of product unavailability. These results need to be interpreted with caution because causality is not examined in these analyses. As mentioned above, these tools may be adopted because deviations are high or increasing. Alternatively, these tools may be better able to identify and detect deviations, which allows facilities to narrow process specification specifications that, in turn, leads to finding new deviations. Such causal relationships cannot be determined simply by interpreting estimated coefficients.

Finally, various decision rights appear to have a strong impact on our findings. Lot failure responsibility residing at the level of the QA manager instead of the QA head for the manufacturing facility corresponds to lower levels of production component deviations, greater reductions in raw material deviations, and greater reductions in production component deviations. Allowing operations and quality to participate in reviewing and

50

approving deviations even though they’re not required to appears to increase the number of product and process specification deviations.

9. Deviation Management Performance


This section of the report provides a statistical analysis of deviation management performance for the data collected in the Pharmaceutical Manufacturing Research Project (PMRP). It examines only API Manufacturers—15 unique manufacturing facilities from 11 unique pharmaceutical firms.

The performance section of the PMRP collected monthly data on (1) product unavailability; (2) the number of field alerts; (3) the percentage of finished product recalls; (4) the number of deviations that arise from raw materials, production components (equipment) and production product and process specifications, respectively; and (5) the percentage of repeat deviations that arise from raw materials, production components (equipment) and production product and process specifications, respectively.

Table 9.1 provides definitions of these performance measures.

Table 9.1: Deviation and Regulatory Performance Measures Measure Definition Product Unavailability An indicator as to whether to product was unavailable

(stock-out) during the month due to manufacturing problems.

Field Alerts The number of field alerts for a given product in a given month.

Finished Product Recalls The percentage of finished product recalls during the month, expressed as a percentage.

Raw Material Deviations The number of deviations that arise from raw materials purchased per month.


The number of deviations that arise from production components (equipment) per month.


The number of deviations that arise from the product and process specifications per month.


The percentage of deviations that arise from raw materials purchased that are repeat occurrences per month.

Repeat Production Component Deviations

The percentage of deviations that arise from production components (equipment) that are repeat occurrences per month.


The percentage of deviations that arise from product/process specifications that are repeat occurrences per month.

51

Using data collected in the PMRP questionnaire about the manufacturing facility (sections 3a and 3b), human resources (section 4), performance metrics (section 6), and deviation management (sections 8a and 8b) sections, the analysis explores those factors that correlate with various deviation management performance metrics described above.

The interested reader is referred to Appendix D, for a complete list of data collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire.

The text below describes the data used in the empirical analysis, summarizes the methodology through which the data is analyzed, presents the empirical results, and finally discusses results.


The data input by pharmaceutical firms on manufacturing performance required extensive data entry. The PMRP collected performance information for API manufacturers. These data represent 2,160 monthly observations for 36 active pharmaceutical ingredients manufactured in 15 distinct manufacturing facilities by 11 distinct pharmaceutical firms.

While a number of different statistical analyses were undertaken, this section focuses attention only on those empirical analyses that were sensible and yielded significant findings.

One aspect of deviation management performance examines product availability, field alerts, and finished product recalls. Our sample unfortunately did not provide enough heterogeneity in the dependent variable for field alerts and finished product recalls.

Table 9.2: Dependent Variable Definitions Product Unavailability Monthly indicator as to whether the product was

unavailable (stock-out) during the month due to manufacturing problems.

RM Deviations The number of deviations that arise from raw materials purchased per month.

PC Deviations The number of deviations that arise from production components (equipment) per month.

PP Deviations The number of deviations that arise from the product and process specifications per month.

PC Deviation Rate Ratio of current month’s number of production component (equipment) deviations to the prior month’s raw material deviations.

PP Deviation Rate Ratio of current month’s number of product and process specification deviations to the prior month’s raw material deviations.

Another aspect of deviation management performance is concerned with the number of raw material, production component (equipment) and product and process specification deviations in a given month, as well as the percentage of deviations that repeat from these

52

categories, respectively. In this report, we provide only analytic results for the number of deviations, but plan to report the repeat deviation analysis at a latter date.

Besides the performance level of deviations reported, we are also interested in the rates of change (or ratios) of these performance metrics. Unfortunately, our data does not permit us to provide a useful estimate for rate of change for raw material deviations. Table 9.2 provides definitions for the dependent variables used in the econometric analyses.

As mentioned above, we utilized a number of independent variables collected in sections 3a, 3b, 4, 6, 8a, and 8b of the PMRP questionnaire in the empirical analyses of regulatory and deviation performance. Most of these variables are collected yearly from 1999 to 2003.

Table 9.3 provides definitions only for those variables that were retained in the empirical analyses that yielded statistically significant results. The number of variables available is much smaller than the end prior analyses.




located at the manufacturing facility. Contract Manufacturer Dummy variable equal to 1 if the manufacturing facility






53

IT Usage A measure of the interim correlation of the extent to which information technology is utilized in the manufacturing facility to:


Scale Reliability = 0.74. PAT Data Analysis Tools A summation of binary responses to whether or not the

manufacturing facility uses the following multivariate data acquisition and analysis tools:

• Statistical design of experiments • Response surface methodologies • Process simulation • Pattern recognition tools

The deviation management performance analysis ultimately employed six dependent variables and up to eight independent variables, depending upon the performance metric. The logarithm of total employees and facility size is taken because the distributions of these data are skewed.

Several of the independent variables are based on constructed variables derived from factor analysis to reduce the number of independent variables for our econometric analyses. Doing so improves the number of degrees of freedom for our econometric analysis.

A factor analysis of all relevant variables for the set of questions in each survey area was used for each one of these constructed variables. For instance, the factor analysis for Employee Experience included variables representing the number of years of experience for each category of worker collected in the survey: operators, craft workers, technicians, professionals and managers. Based on results of the factor analysis we first constructed a Cronbach’s alpha for each eigenvalue greater than one, and then included only those variables with a factor loading greater than 0.4. The constructed variable represents the corresponding alpha score. Scale reliability is presented in Table 9.3 for each constructed variable. All scale reliability coefficients are near or exceed the expected norm of 0.7.


In terms of the performance measures, the mean level of Product Unavailability is 0.06 per month, or roughly 6 percent likelihood in any month. The data indicate product is available in more than 94 per cent of the sample in any given month.

The mean RM Deviations is a 0.10 with a large standard deviation, which is due to the fact that deviations cannot be negative and the maximum of RM Deviations, 8, is large. The mean PC Deviations is 0.43 with a standard deviation of 1.23, and a maximum of 21. PP Deviations has a mean of 0.89 and a standard deviation of 2.00, indicating that on average PC Deviations arise almost twice a month; although, this average is skewed because the maximum monthly number of PC Deviations is 27 and the minimum is

54

constrained to be zero. The means of these three deviations indicate that they arise on average less than once a month. On average, API deviations arise less frequently than OT&I deviations.

Table 9.4: Summary Statistics MEAN S.D. MIN MAXDependent Variables Product Unavailability 0.06 0.24 0.00 1.00RM Deviations 0.10 0.51 0.00 8.00PC Deviations 0.43 1.23 0.00 21.00PP Deviations 0.89 2.00 0.00 27.00PC Deviation Rate 0.65 1.03 0.00 8.00PP Deviation Rate 0.90 1.21 0.00 9.00 Independent Variables Total Employees 6.38 0.83 4.54 7.30Facility Size 10.42 1.14 8.07 12.55Number of Processes 2.42 0.96 1.00 5.00Contract Manufacturer 0.36 0.48 0.00 1.00Employee Experience -0.12 0.77 -1.14 1.66Employee Training -0.04 0.68 -0.82 1.40IT Usage 0.14 0.59 -1.37 0.74PAT Data Analysis Tools 1.47 1.07 0.00 3.00

In terms of the independent variables, the mean level of total employees (logged) within a manufacturing facility in the sample is 6.38, which is equivalent to over 600 employees. Similarly, the mean facility size (logged) is 10.42 square meters, which is equivalent to 33,500 square meters.

Each manufacturing facility has almost two and a half distinct manufacturing processes on average with a substantial variance among the manufacturing facilities in the sample. About a third of the sample manufacturing facilities are also contract manufacturers.

As mentioned above, the rest of the independent variables except for PAT Data Analysis Tools are based on constructed variables derived from factor analysis to reduce the number of independent variables for our econometric analyses. The items in the scale are standardized (mean 0, variance 1) so discussion of their summary statistics is uninteresting. Note that with the constructed variables neither the mean is 0.0 nor is the standard deviation 1. The difference lies in the fact that these independent variables are constructed prior to dropping observations with missing data.

PAT Data Analysis Tools is the summation of binary responses to questions about the use of process analytic data analysis tools (see Table 9.3). OT&I manufacturing facilities on average use about half the tools available with some facilities using none and other facilities using all tools.

Table 9.5 presents correlations for these dependent and independent variables. Pair-wise correlations in bold are statistically significant at 95% confidence intervals. Because of

55


Prod

uct U

nava

ilabi

lity

RM D

evia

tions

PC D

evia

tions

PP D

evia

tions

PC D

evia

tion

Rate

PP D

evia

tion

Rate

Tota

l Em

ploy

ees

Faci

lity

Size

Num

ber o

f Pro

cess

es

Con

trac

t Man

ufac

ture

r

Empl

oyee

Exp

erie

nce

Empl

oyee

Tra

inin

g

IT U

sage

PAT

Dat

a An

alys

is T

ools

Product Unavailability 1 RM Deviations 0.03 1 PC Deviations 0.03 0.10 1 PP Deviations -0.03 0.29 0.32 1 PC Deviation Rate 0.01 0.13 0.65 0.17 1 PP Deviation Rate -0.07 0.10 0.22 0.60 0.19 1 Total Employees 0.22 -0.08 0.04 0.07 -0.06 0.10 1 Facility Size 0.37 0.02 0.09 0.08 0.03 0.13 0.62 1 Number of Processes 0.11 0.13 -0.12 0.07 0.00 0.02 0.31 0.08 1 Contract Manufacturer 0.05 -0.15 -0.01 -0.07 -0.05 -0.01 0.42 0.24 0.03 1 Employee Experience 0.02 -0.02 0.02 -0.08 0.03 0.03 0.10 0.49 0.15 -0.11 1 Employee Training 0.41 0.10 -0.13 -0.03 -0.05 -0.02 0.30 0.52 0.51 0.07 0.17 1 IT Usage 0.14 -0.05 -0.02 -0.01 -0.06 0.04 0.50 0.65 0.08 0.01 0.51 0.20 1 PAT Data Analysis Tools 0.00 -0.09 0.18 0.04 0.02 0.10 0.48 0.43 -0.60 0.00 0.06 -0.30 0.41 1

Bold indicates pair-wise significance at 95% confidence interval.

56

these dense and complicated set of relationships indicated by the pair-wise correlation analysis, we interpret the data through various regression analyses.



The first deviation management performance metric examined is Product Unavailability in a given month. Because Product Unavailability is an indicator variable (zero or one), a Logit analysis is used to estimate the relationship between the independent and dependent variables. We also employ the Huber/White/sandwich estimator of variance to achieve robust standard errors, as well as correct the variance-covariance matrix of the estimators to account for clustering (by manufacturing facility). Clustering accounts for the fact that observations are independent across manufacturing facilities, but not necessarily within manufacturing facilities.

Our analyses of RM Deviations, PC Deviations, and PP Deviations employ a linear regression methodology. These different types of deviations may be correlated; therefore, we employ a statistical method called Seemingly Unrelated Regression (SUR). Many econometric models (including the one examined here) contain a number of linear equations of different dependent variables. It is often unrealistic to expect that the errors are uncorrelated between and among these different equations. SUR gets its name because at first glance, the equations seem unrelated, but in fact are related through the correlation in the errors. A set of equations that has contemporaneous cross-equation error correlation is thus called an SUR system, and makes corrections to the standard errors of the estimates in each equation to adjust for this correlation. We again employ robust standard errors and clustering for this analysis.

We are unable to investigate change in product unavailability because of the limited number of observations. In contrast, we do have sufficient data to evaluate those factors that impact the change in deviations; but, only for process component and product and process parameter deviations. Thus, our third set of analyses investigates the relationship between our independent variables and PC Deviation Rate and PP Deviation Rate. For reasons described above, we again employ an SUR methodology for analyzing this data.

Given that several of the constructed independent variables are standardized (a mean of zero and a variance of one), interpretation of results should be based on the signs of the coefficients rather than on their magnitudes. It also should be noted that these models provide an initial lens through which to view the data. Given the time series nature of the data collected by the PMRP, the data can be analyzed using other econometric techniques and methodologies. In some instances, the use of these other econometric methodologies may offer superior approaches with which to assess and interpret the data. Nonetheless, the analyses presented below are an appropriate first glimpse of the relationship between various organizational practices in the performance measures identified above.

9.4. Product Unavailability Analytic Results

The first column Table 9.6 provides the results of the logit analysis for Product Unavailability. The model fit the data well with an R2 of 0.47 and a highly statistically significant χ2 statistic, indicating rejection of the null hypothesis that the set of coefficients are random.

57

The results indicate that Product Unavailability increased with the total number of employees and employee experience. The coefficient for Total Employees is statistically significant and the coefficient for Employee Experience is weakly statistically significant.

Results also indicate that Product Unavailability declines with the more processes manufactured in the facility. In this case, the coefficient for Number of Processes is negative and highly statistically significant.

9.5. Deviation Number Analytic Results

Table 9.6 provides the results of the econometric analysis for the levels of RM Deviation, PC Deviation, and PP Deviation. All of the regression results for each performance metric use the same set of explanatory variables, where appropriate. All of the regressions employ robust standard errors and clustering and are the result of an SUR estimation procedure. We explore each performance metric in turn.

9.5.1. Raw Materials Regression results for the number of RM Deviations are presented in the second column of Table 9.6. The model fit the data poorly. Although the R2 (a measure of the explained variance of the regression) is small in this model (0.024), R2 should not be relied on as the only measure of fit. More importantly, the χ2 statistic for each model is statistically significant indicating rejection of the null hypothesis that the set of coefficients are random.

The coefficient estimates indicate that RM Deviations are lower when the facility engages in contract manufacturing, when employees receive a high level of training, and when the facility uses information technology extensively. The coefficient for Contract Manufacturer is negative but weakly statistically significant. The coefficients for Employee Training and IT Usage are both negative and highly statistically significant. The results also indicate that RM Deviations are higher when the facility size is large. In this case, the coefficient for Facility Size is positive and highly statistically significant.

9.5.2. Production Component (Equipment) The third column of Table 9.6 provides the coefficient estimates for the level of PC Deviations. The model fit the data poorly. The R2 of 0.07 indicates that 7 percent of the variance is explained by the included coefficients. The χ2 statistic is highly statistically significant and indicates rejection of the null hypothesis that the set of coefficients are random.

The coefficients for Employee Training and IT Usage are both negative and highly statistically significant. These coefficient estimates indicate that PC Deviations are lower when employees receive a high level of training and facilities use information technology extensively.

Coefficient estimates for Facility Size and PAT Data Analysis Tools are positive and statistically significant and highly statistically significant, respectively. These estimates indicate that PC Deviations are higher the larger facilities and the more that facilities use data analysis tools.

9.5.3. Product and Process Parameter The fourth column of Table 9.6 provides the coefficient estimates for the level of PP Deviations. The model fit the data moderately to poorly. The R2 of 0.08 indicates that eight percent of the variance is explained by the included coefficients. The χ2 statistic is highly statistically significant, indicating rejection of the null hypothesis that the set of coefficients are random.

58

Table 9.6: Product Unavailability and SUR Deviation Levels Analyses

Product

Unavailability RM

Deviations PC

Deviations PP

Deviations Total Employees 4.024* -0.025 -0.119 -0.148 (1.615) (0.031) (0.114) (0.298) Facility Size 0.497 0.051** 0.302 * 0.550† (0.781) (0.018) (0.132) (0.291) Number of Processes -3.906** -0.015 0.140 0.363 (1.295) (0.029) (0.121) (0.311) Employee Experience 2.611† 0.028 0.027 -0.301 (1.584) (0.031) (0.191) (0.397) Contract Manufacturer -0.061† -0.110 -0.348 (0.035) (0.182) (0.398) Employee Training -0.047** -0.382 ** -0.734** (0.014) (0.082) (0.161) IT Usage -0.048** -0.367 ** -0.279 (0.016) (0.108) (0.192) PAT Data Analysis Tools -0.018 0.257 ** 0.179 (0.024) (0.083) (0.172) Constant -29.490** -0.237* -2.569 † -5.016† (4.706) (0.115) (1.410) (2.702) N 1644 1644 1644 1644 R2 0.47 0.02 0.07 0.08 χ2 60.71** 6.13** 15.88 ** 18.44**** p<0.01, * p<0.05, † p<0.10; Standard errors of coefficients in parentheses. In this model only two coefficients yield statistical significance. The coefficient for Facility Size is positive and weakly statistically significant. This estimate indicates that product and process specification deviations are greater when facilities are large in size. In contrast, PP Deviations are lower when employees receive high levels of training because the coefficient for Employee Training is negative and highly statistically significant.

9.6. Deviation Rate Analytic Results

Table 9.7 provides the results of the econometric analysis for the rates of change in production component deviations and product and process specification deviations. The data is insufficient for estimating a model of the rate of change of raw material deviations.

Recall that each of these dependent variables is a ratio of the current month’s value divided by the last month’s value, with no change equating to 1. An increase in PC Deviations and PP Deviations are represented by rates of change that are greater than 1, while decreases in these measures are less than one.

Regressions employ robust standard errors and clustering and are the result of an SUR estimation procedure. We explore each performance metric in turn.

59

9.6.1. Production Component (Equipment) Deviation Rate The first column of Table 9.7 provides the coefficient estimates for the rate of change in PC Deviations. The model fits the data poorly. The R2 of 0.034 indicates that no more than four percent of the variance is explained by the included coefficients. The χ2 statistic for model is highly statistically significant, which allows us to reject the null hypothesis that the set of coefficients are random.

The coefficients for Facility Size, Employee Experience, Contract Manufacture and PAT Data Analysis Tools are positive and displays some degree of statistical significance. The coefficients for Facility Size, Employee Experience, and Contract Manufacturer are all positive and highly statistically significant. The coefficients for PAT Data Analysis Tools is positive and weekly statistically significant. These results indicate that production component deviations increase over time in large facilities, in facilities with high levels of employee experience, if the facility engages in contract manufacturing, and when PAT data analysis tools are employed. Please note that these findings do not indicate causation. For instance, it may be the case that PAT data analysis tools lead to the identification of more production component deviations or that firms use PAT data analysis tools when production component deviations are increasing rather than PAT data analysis tools causing increase rates of production component deviations. Our regression results cannot determine causality.

Table 9.7: SUR Analysis of PC and PP Deviations Rate of Change

PC

Deviation RatePP

Deviation Rate Total Employees -0.049 -0.232 * 0.057 0.103 Facility Size 0.111** 0.579 ** 0.029 0.190 Number of Processes -0.288** 0.553 ** 0.083 0.159 Employee Experience 0.386** -0.388 0.082 0.249 Contract Manufacturer 0.345** -0.571 * 0.081 0.235 Employee Training -0.318** -0.517 ** 0.025 0.074 IT Usage -0.271** -0.401 ** 0.037 0.075 PAT Data Analysis Tools 0.139+ 0.220 * 0.073 0.088 Constant -0.101 -5.161 * 0.334 2.053 N 358 465 R2 0.034 0.044 χ2 2.59** 2.63 **


60

The coefficients for Number of Processes, Employee Training, and IT Usage all are negative and highly statistically significant. These estimates indicate that production component deviations decrease over time the larger is the number of processes in the facility, when employees are highly trained, and when facilities extensively use information technology.

9.6.2. Product and Process Parameter Deviation Rate The second column of Table 9.7 provides the coefficient estimates for PP Deviation Rate. The regression fits the data poorly. The R2 of 0.044 indicates that only slightly more than four percent of the variance is explained by the coefficients included. The χ2 statistic for the model is highly statistically significant, which allows us to reject the null hypothesis that the set of coefficients is random.

The coefficients for Facility Size and Number of Processes are both positive and highly statistically significant. These estimates indicate that the number of product and process specification deviations increase over time for large facilities and facilities engaged a large number of production processes. The coefficient for PAT Data Analysis Tools also is positive and statistically significant indicating that the use of data analysis tools corresponds to increasing rate of product and process specification deviations.

In contrast, the coefficients for Total Employees, Contract Manufacturer, Employee Training, and IT Usage all are negative and statistically significant with the exceptions of Employee Training and IT Usage for which the coefficient estimates are highly statistically significant. Process and product specification deviations appear to decline with higher numbers of employees at the facility, if the facility engages in contract manufacturing, if employees receive high levels of training, and if the facility extensively uses information technology. In this estimation, the constant term is negative and statistically significant indicating that API produces on average reduce the process and product specification deviations over time.


Several interesting conclusions can be drawn from the statistical results presented above. API facilities experience diseconomies of size in deviation management. Larger API facilities correspond to higher levels of all types of deviations as well as correspond to increasing deviations over time. Relatedly, large numbers of employees correspond to higher levels of product unavailability. It should be noted, that large numbers of employees also correspond to lower levels of product and process specification deviations over time. While API facilities appear to suffer from diseconomies of size, they also appear to benefit from economies of scope. The larger the number of processes within the facility the lower its product unavailability and the more that production component deviations decline over time. On the downside, a large number of processes within facility corresponds to an increasing rate of product and process specification deviations.

Employee training appears to universally benefit API production facilities. Employee training reduces all types of deviations and reduces these deviations over time. In contrast, high levels of employee experience correspond to high levels of production component deviations and higher levels of product unavailability. One possibility that might explain this negative outcome is that more experienced employees may be less willing to change their behavior. Such routinization of behavior has been known to cause low-quality or failures in other industries.

API contract manufacturers appear to have lower raw material deviations and improve on product and process specification deviations over time but also experience increasing rates of production component deviations.

61

The extensive use of information technology appears to universally benefit API manufacturers. The use of information technology corresponds to lower levels of raw material and production component deviations. It also corresponds to reducing the rate of production component and product and process specification deviations over time.

Curiously, the use of PAT data analysis tools corresponds to higher levels of production component deviations and increasing rate of production component and product and process specification deviations. Process analytic technology may enable the discovery of deviations or may be invested in by facilities because the experience high rates of deviation or increases in deviations. Our analysis does not reveal the causal linkage between deviations and PAT data analysis tools.

10. Process Development


This section of the report provides a statistical analysis of process development for the data collected in the Pharmaceutical Manufacturing Research Project (PMRP).

The process development section of the PMRP collected data is used to investigate the extent to which the location of development activities, the organization of development activities, and the timing of development activities impacts development hours and development days. The development activities examined include discovery, process research, pilot development, commercial plant transfer and startup, and assay development.

The location of development activities analysis examines (1) location and (2) approximate distance (if not co-located within the same facility) between these development activities. Location differs according to whether the development activities are (a) in the same manufacturing facility; (b) in some other location within the strategic business unit; (c) in some other location within the corporation; and (d) at another corporation.

The organization of development activities examines (1) the number of vertical reporting relationships (i.e., levels of management); (2) how hand-offs from one activity to another are managed; and (3) how process validation is organized. Vertical reporting relationships measure the number of management levels whereby the lowest rank employees in each development “pair” of activities share a common manager. Handoffs measure the proportion of time personnel whose primary responsibility is in one development activity were involved in other development activities. Process validation simply investigates how process validation is organized in terms of whether it operates as a standalone group, or is part of process development, operations/production, engineering, quality assurance/control, or regulatory compliance/affairs.

The timing of development activities examines the length of time required for process development. Development hours and development days are both used as dependent variables in the econometric analysis because the former represents the total amount of development effort while the latter represents the total number of calendar days. The reader is referred to Appendix D, for a complete list of data collected in section five of the PMRP survey. The subsections below describe the data used in the analysis, summarizes the methodology through which the data is analyzed, presents the empirical results, and discusses results.

62


The process development data was perhaps the most difficult data in the PMRP survey for manufacturers to collect. Perhaps because of this, our database consists of just 43 process development observations. These 43 observations come from 17 different facilities owned by 10 different companies. While a number of different statistical analyses were undertaken, this section focuses attention on the analysis that yielded the most significant findings. We thus provide definitions for only those variables retained in the empirical analysis that yield statistically significant findings.

The process development analysis ultimately employed eight variables defined below in Table 10.1. Table 10.1: Variable definitions for process development analysis Dependent Variables ln(Development Hours) Natural logarithm of the sum of development hours for

Process Development, Scale up, Tech Transfer/Relocation, and Assay Development.

ln(Development Days) Natural logarithm of the number of days between the beginning of process development and the production of three successful batches.

Independent Variables Originated at Another Site A dummy variable equal to 1 if the production process

originated at another facility and 0 otherwise. Chemical Reactions The number of chemical reactions that occur at the

manufacturing facility to produce the substance. API Sold to Branded Man. A dummy variable equal to 1 if the compound is sold to a

branded pharmaceutical manufacturer and 0 otherwise. Plant Design A dummy variable equal to 1 if the manufacturing facility is

designed specifically to produce the compound and otherwise.

Generic Compound A dummy variable equal to 1 if compound is a generic and 0 otherwise.

Corp. Process Validation A dummy variable equal to 1 if process validation reports to corporate and 0 otherwise.

The logarithm of each dependent variable—Development Hours and Development Days—is taken because the distribution of these data is skewed.

Table 10.2 reports summary statistics and correlations for the dependent and independent variables. Correlations of 0.30 or greater are statistically significant at a 95% confidence interval. The natural logarithm of Development Hours (hereafter referred to as Development Hours) is positively and significantly correlated with the natural logarithm of Development Days (hereafter referred to as Development Days) and Chemical Reactions. Development Days is negatively correlated with Originated at Another Site. No other variable is correlated in a statistically significant sense with our two dependent variables.

The high level of correlation between these two dependent variables justifies the use of Seemingly Unrelated Regression, which is discussed in the next section. An examination of the other correlations shows that Plant Design is highly correlated with Originated at Another Site, which may explain why Development Days may be negatively correlated with Originated at Another Site, since much of the development activity may have already taken place. Generic Compound is positively and significantly correlated with Chemical Reactions

63

and API Sold to Branded Man; which suggests that some plants specialize in generic compounds with a large number of chemical reactions that are then sold to other firms. Finally, Corp. Process Validation is highly correlated with Chemical Reactions, which suggests that more complex processes are more likely to have process validation report to a corporate organization than to any other organizational approach. Nevertheless, the lack of clarity in these relationships leads us to use regression analysis to interpret our data.

Table 10.2: Summary statistics and correlations for process development variables

1 2 3 4 5 6 7 8 1 ln(Development Hours) 1 2 ln(Development Days) 0.68 1 3 Originated at Another Site -0.02 -0.34 1 4 Chemical Reactions 0.35 0.04 -0.01 1 5 API Sold to Branded Man. 0.13 -0.13 0.15 0.63 1 6 Plant Design 0.26 0.11 0.55 -0.05 0.19 1 7 Generic Compound -0.07 -0.11 -0.07 0.44 0.43 0.12 1 8 Corp. Process Validation -0.13 -0.21 -0.02 0.42 -0.02 0.03 0.24 1

MEAN 8.70 6.85 0.16 1.07 0.16 0.14 0.51 0.30S.D. 1.18 1.12 0.37 2.09 0.37 0.35 0.51 0.46MIN 5.89 4.09 0 0 0 0 0 0MAX 10.67 8.79 1 9 1 1 1 1

Correlations are significant at p<0.05 when ρ>0.30; N = 43.


The analytical method used to analyze process development has two important features. First, the model assumes a linear relationship between the dependent variables and independent variables. Nonlinear relationships are not assessed using this methodology. Second, the statistical method allows for correlation between the two dependent variables, which accounts for the possibility that the dependent variables (Development Hours and Development Days) may be correlated.

The statistical method employed in the analytic methodology is called Seemingly Unrelated Regression (SUR). Many econometric models (including the one examined here) contain a number of linear equations of different dependent variables. It is often unrealistic to expect that the errors are uncorrelated between and among these different equations. SUR gets its name because at first glance, the equations seem unrelated, but are in fact related through the correlation in the errors. A set of equations that has contemporaneous cross-equation error correlation is thus called an SUR system, and makes corrections to the standard errors of the estimates in each equation to adjust for this correlation.

A much larger number of variables than what appears in Table 10.1 were analyzed in many unreported regression analyses. Because of the relatively small number of observations, however, the full range of variables afforded by the PMRP data is not included in the econometric analysis. In particular, variables were omitted from the analysis if they had little or no statistical significance. That is, only those variables that had a statistically significant impact on our dependent variables are included in the econometric analysis.

64

10.4. Process Development Analytical Results

SUR results are presented in Table 10.3. The first column of coefficients reports the estimated effect of the covariates on the dependent variable Development Hours. All coefficient estimates are statistically significant at a p-value of 0.05 or better. Nevertheless, some independent variables display a positive relationship with this dependent variable while other variables display a negative relationship. In particular, the number of development hours decreases if the compound Originated at Another Site, if it is an API is Sold to Branded Manufactures, if it is a Generic Compound, and if Process Validation reports to Corporate. By contrast, the number of development hours increases with the number of Chemical Reactions and if the Plant Design is specific for the production of this compound.

The direction of these coefficient estimates is identical for the second dependent variable Development Days. The only qualitative difference between the coefficient estimates for these two dependent variables is that the coefficient for Generic Compound is not statistically significant for Development Days.

Overall, the data fits the model well. Although the R2 (a measure of the explained variance of the regression) is large in each model (0.55 and 0.49, respectively), R2 should not be relied on as the only measure of fit with such a small number of observations. More importantly, the χ2 statistic for each model is statistically significant indicating rejection of the null hypothesis that the set of coefficients is random.

Table 10.3: Process development SUR analysis log(Development hours) log(Development days)

Originated at Another Site -0.98 * -1.83 ** (0.40) (0.40) Chemical Reactions 0.56 ** 0.35 ** (0.09) (0.09) API Sold to Branded Man -1.39 ** -1.46 ** (0.49) (0.50) Plant Design 2.05 ** 1.92 ** (0.43) (0.44) Generic Compound -0.69 * -0.42 (0.28) (0.29) Corp. Process Validation -1.29 ** -1.14 ** (0.32) (0.32) Constant 8.94 ** 7.31 ** (0.19) (0.19) R2 0.55 0.49 χ2 52.31 ** 40.97 ** ** p<0.01, * p<0.05, † p<0.10

10.5. Discussion of Analysis Results

Our interpretation of the results focuses on the direction of estimated coefficients (positive or negative) rather than the magnitude of the coefficients. We do this because with such a small number of observations, the magnitude of the coefficient estimates varies depending on which variables are included in the econometric model. The sign and significance of the coefficients, however, are robust across various specifications.

65

Many of the results of this econometric analysis are not surprising. In particular, compounds that originated at other facilities take fewer hours and fewer days to develop in the focal facility, presumably because the compound received prior development work not accounted for in the PMRP data. As the complexity of the process increases as indicated by the number of chemical reactions, so too does the number of development hours and development days needed to develop an operational process. Neither of these findings is surprising nor controversial.

A more intriguing finding, however, is that a manufacturing facility selling an active pharmaceutical ingredient (API) to a branded manufacturer compared to utilizing this compound internally takes fewer man-hours and fewer development days to develop an operational process, even when controlling for whether the compound is generic or not. This finding suggests that pharmaceutical manufacturers find ways to reduce process development time when selling an API compound into the market, perhaps because the high-powered incentives of the market outweigh the incentives that exist within a given firm.

Manufacturing facilities designed to produce a particular compound correlate with longer development hours and more development days. These correlations may be due to interactions between process design and plant design. As might be expected, the development of a new process for generic compounds requires fewer development hours than for compounds that represent branded products. The coefficient for Generic Compound is not statistically significant for the Development Days dependent variable, however, which suggests that process development for generic compounds is similar to process development for branded compounds in this measure of performance.

Perhaps the most interesting result of the process development analysis is that both development hours and development days are shorter when process validation reports to a corporate entity. Centralized management of process validation appears to speed development time. A number of explanations might account for this finding. First, the estimation results do not account for the possibility that management matches the way in which it organizes process validation to the type of compound. For instance, management may centralize process validation for those compounds that are moving to market quickly rather than centralize process validation in order to reduce development times. Our statistical model does not account for this choice.

Assuming that the reporting relationship is not chosen specifically with respect to each compound, several factors might explain the relationship between process validation reporting to corporate and reduced development hours and days. One benefit may be that it is easier for one production facility to learn from another production facility by centralizing process validation. Thus, centralized process validation may offer learning or experience accumulation benefits that derive from developing a variety of compounds. Another possibility is that centralized process validation may have more ancillary resources available to it for advancing the compound to market that would not appear in the data reported in our study under person hours or development days. Yet another possibility is that reporting to a centralized corporate activity may increase the incentives for plant level personnel to accelerate development. Unfortunately, the limited number of observations in our sample makes it difficult to statistically identify whether one explanation or another is more valid in explaining why the Corporate Process Validation coefficient is negative and significant.

Besides the results reported above in Table 10.3, many “non-results” are worthy of mention. In our 43 observations we found no consistent evidence that co-location of discovery, process research, process development, commercial plant transfer and start up, or assay development had a statistically significant impact on development hours or development days.

66

Similarly, the distances between these development activities—in terms of miles or kilometers—and differences in the countries where these activities take place had no consistent statistically significant effect. Thus, the location of development had no enduring statistically significant impact on development hours and development days.

We also found no evidence that the flatness or verticality of an organization had a significant impact on development hours or days. The absence of any of these relationships is surprising. It may be due to our small sample size of only 43 processes. Further analyses will be undertaken to explore each of these non-result outcomes.

11. Conclusions This report provided a preliminary benchmarking study of 42 manufacturing facilities from 19 pharmaceutical manufacturers. The report presented data in the form of benchmarking charts and statistical analyses summarizing the relationship between various parameters and performance metrics for formulation facilities (OT&I), active pharmaceutical ingredient (API) facilities, and process development. While each analysis yielded its own conclusions detailed in the sections above, here we focus on broad conclusions that cut across all of the analyses.

11.1. Extent and Use of IT

The most consistent statistical result of the Pharmaceutical Manufacturing Study is that the extensive use of information technology corresponds to higher levels of performance for both API and OT&I products. The use of information technology includes electronically and automatically reporting deviations, tracking deviations by lot, tracking deviations by type of issue, tracking people assigned to resolving the deviation, and centrally storing data. In OT&I processes, the use of IT corresponds to lower cycle times, reductions in batches failed over time, lower levels of production component deviations, and reductions in raw material and production component deviations over time. In API facilities the use of IT corresponds to lower cycle times, higher yields, lower levels of raw material and production component deviations, and reductions in production component and product and process parameter deviations over time. Pharmaceutical manufacturing, whether producing OT&Is or APIs benefits from the extensive use of information technology.

11.2. Decision Rights

The second most consistent finding of the Pharmaceutical Manufacturing Study is that the locus of decision rights matters. For OT&I facilities, assigning responsibility for lot failure to a QA manager instead of the head of QA at the manufacturing facility corresponds to a lower number of failed batches, higher actual yields, lower cycle times, reductions in failed batched over time, lower production component deviations, and reductions in raw material and production component deviations over time. The findings also indicate that yield increases over time when operations and quality participate in reviewing and approving deviations even though they are not required to review and approve in the manufacturing facility. Also, product unavailability is substantially lower when operations, engineering, and regulatory affairs must review and approve deviations.

Although the same metrics are not available for our estimates of API production, our findings nonetheless indicate that the locus of decision rights matters. The number of failed batches decreases over time and the actual yield increases over time when operations, engineering, quality, regulatory affairs, and the group from which the deviation originated (if applicable)

67

all typically take the lead in responding to deviations in the manufacturing facility. Yet, the level of batches failed is higher and actual yield declines over time when operations, engineering, quality, and regulatory affairs must all review and approve deviations in the manufacturing facility. The former organizational structure implies that these different functional groups work together as a team where the latter structure implies that they work separately.

With respect to process development, our statistical analysis indicates that both development hours and development days are shorter when process validation reports to a corporate entity instead of to plant management.

11.3. Contract Manufacturing

The study’s third most consistent finding is that facilities engaged in contract manufacturing generally display poorer manufacturing performance. OT&I facilities engaged in contract manufacturing display a higher level of batches failed, longer cycle times, and declining yields over time. To their benefit, OT&I contract manufacturers reduce the number of process and product parameter deviations over time. API contract manufacturers suffer from a higher level of batches failed, longer cycle times, increasing cycle time over time, and increases in process component deviations over time. That said, API contract manufacturers improve actual yield over time, have lower levels of raw material deviations, and reduce product and process parameter deviations over time.

11.4. Process Analytic Technology Tools

The use of process analytic technology tools has mixed, although a generally negative, correspondence with manufacturing performance. For instance, process analytic tools correspond to increases in the number of batches failed, increases in raw material deviations and product and process parameter deviations, and decreases in yield over time for OT&I manufacturers. However, data analysis tools do increase yield over time. Similarly, data analysis tools correspond to higher levels of production component deviations and increasing amounts of production component and process and product parameter deviations over time for API manufacturing facilities. On the surface, these results might indicate that process analytic tools harm manufacturing performance. Such a conclusion may be incorrect. Because our analysis does not determine causality, it may be the case that these tools are adopted when products face worsening performance metrics. It also could be the case that these tools are superior at identifying deviations and failed batches, which, over time, will lead to problem-solving efforts that eventually translate into fewer deviations, fewer failed batches, greater product availability, higher yields, and shorter cycle times.

11.5. Scale and Scope

Our regression results also highlight a complex interplay between the scale and scope of a manufacturing facility and its performance metrics. In many instances, increasing scope in terms of the number of processes and the number of products and increasing scale in terms of the number of employees and facility size correspond to poor performance—a type of diseconomy of scale. Scale and scope can lead to a higher number of failed batches, lower yield, increasing cycle times, higher product availability, higher levels of deviations and increasing deviations. Yet, in other instances, scale and scope also correspond to production learning; reduced failed batches, increasing yield, shortening cycle times, and reducing deviations. The specific relationships depend on whether a facility produces OT&Is or APIs.

68

11.6. Concluding Remarks

While the data summarized herein provides one source of value to PMRP participants, the pharmaceutical industry, regulators, and academics, the data also provides another source of value. The PMRP provides a database that can be used to investigate future questions of interest to all of the stakeholders. We hope this future value will be realized.

We once again thank all who made this study possible. Thank you.

69

Index of Appendices

Appendix A: Pharmaceutical – Oral and Topical.............................................................. 72 I. Manufacturing Facilities .................................................................................................. 72

i. General Facility ........................................................................................................................ 72 ii. Products and Processes............................................................................................................. 78 iii. Regulatory ................................................................................................................................ 81 iv. Other Services .......................................................................................................................... 83

II. Human Resource Management .................................................................................... 84 i. Facility Employment ................................................................................................................ 84 ii. Employee Demographics ......................................................................................................... 97 iii. Employee Training................................................................................................................. 109 iv. Teams ..................................................................................................................................... 131

III. Product/Process Development.................................................................................... 140 i. General Product Overview ..................................................................................................... 140 ii. New Product Development .................................................................................................... 144

IV. Performance ............................................................................................................... 162 i. Manufacturing ........................................................................................................................ 162 ii. Deviation Management .......................................................................................................... 174 iii. FDA Inspection ...................................................................................................................... 192 iv. Electronic Tracking ................................................................................................................ 208 v. Process Analytical Technology .............................................................................................. 210 vi. Organization ........................................................................................................................... 214

Appendix B: Pharmaceutical – API .................................................................................. 219 V. Manufacturing Facility................................................................................................... 219

i. General Facility ...................................................................................................................... 219 ii. Products and Processes........................................................................................................... 223 iii. Regulatory .............................................................................................................................. 224 iv. Other Services ........................................................................................................................ 225

VI. Human Resource Management .................................................................................. 226 i. Facility Employment .............................................................................................................. 226 ii. Employee Demographics ....................................................................................................... 233 iii. Employee Training................................................................................................................. 239 iv. Teams ..................................................................................................................................... 250

VII. Product/Process Development.................................................................................... 255 i. General Product Overview ..................................................................................................... 255 ii. New Product Development .................................................................................................... 258

70

VIII. Performance ........................................................................................................... 267 i. Manufacturing ........................................................................................................................ 267 ii. Deviation Management .......................................................................................................... 273 iii. FDA Inspection ...................................................................................................................... 282

IX. Deviation Management .............................................................................................. 290 i. Electronic Tracking ................................................................................................................ 290 ii. Process Analytical Technology .............................................................................................. 291 iii. Organization ........................................................................................................................... 293

Appendix C: Pharmaceutical – Injectables ...................................................................... 296 X. Manufacturing Facilities ................................................................................................ 296

i. General Facility ...................................................................................................................... 296 ii. Products and Processes........................................................................................................... 299 iii. Regulatory .............................................................................................................................. 301 iv. Other Services ........................................................................................................................ 302

XI. Human Resource Management .................................................................................. 303 i. Facility Employment .............................................................................................................. 303 ii. Employee Demographics ....................................................................................................... 310 iii. Employee Training................................................................................................................. 316 iv. Teams ..................................................................................................................................... 327

XII. Product/Process Development.................................................................................... 331 i. General Product Overview ..................................................................................................... 331 ii. New Product Development .................................................................................................... 334

XIII. Performance ........................................................................................................... 343 i. Manufacturing ........................................................................................................................ 343 ii. Deviation Management .......................................................................................................... 349 iii. FDA Inspection ...................................................................................................................... 358

XIV. Deviation Management .............................................................................................. 366 i. Electronic Tracking ................................................................................................................ 366 ii. Process Analytical Technology .............................................................................................. 367 iii. Organization ........................................................................................................................... 369

Appendix D: PMRP Survey Instrument ........................................................................... 372 1. Plant Liaison Information 371 2. Company and Business Unit Information 373 3A. Manufacturing Facility 375 3B. Manufacturing Facility 379

71

4. Human Resources 382 5. Product Development 386 6. Performance Metrics 393 7. Teams 427 8A. Deviation and Manufacturing Management 440 8B. Deviation Management 445 9. Supplement management 452

72

Appendix A: Pharmaceutical – Oral and Topical I. Manufacturing Facilities

i. General Facility Facility Size

8

9

10

11

LN(S

quar

e M

eter

s)

1999 2000 2001 2002 2003Years

11 1718 2129 3842 4849 5054

1999-2003Oral & Topical - Facility Size

73

8

10

12

14

16LN

(Squ

are

Met

ers)

1999 2000 2001 2002 2003Years

06 1420 2227 2832 3647 5157

1999-2003Oral & Topical - Facility Size

74

Average Facility Size

02

46

810

LN(S

quar

e M

eter

s)

11 17 18 21 29 38 42 48 49 50 54

1999-2003Oral & Topical - Average Facility Size

05

1015

LN(S

quar

e M

eter

s)

6 14 20 22 27 28 32 36 47 51 57

1999-2003Oral & Topical - Average Facility Size

75

Reactor/Mixing Vat/Fermentor Size

0

5000

10000

Lite

rs

1999 2000 2001 2002 2003Years

11 1718 2129 3842 4849 5054

1999-2003Oral & Topical - Max Reactor/Vat/Fermentor

76

0

10000

20000

30000

40000Li

ters

1999 2000 2001 2002 2003Years

06 1420 2227 2832 3647 5157


77

Largest Reactor/Mixing Vat/Fermentor Size

05,

000

10,0

00Li

ters

11 17 18 21 29 38 42 48 49 50 54


010

,000

20,0

0030

,000

40,0

00Li

ters

6 14 20 22 27 28 32 36 47 51 57


78

ii. Products and Processes Average Manufacturing facility Operating Hours

020

040

060

080

0H

ours

per

Mon

th

11 17 18 21 29 38 42 48 49 50 54

Average Manufacturing Facility Operating Hours0

200

400

600

800

Hou

rs p

er M

onth

6 14 20 22 27 28 32 36 47 51 57

Average Manufacturing Facility Operating Hours

79

Number of Compounds Manufactured

0 10 20 30Count

54

50

49

48

42

38

29

18


0 20 40 60 80 100Count

57

51

47

32

28

27

22

20

14

6


80

Number of Manufacturing Processes

0 1 2 3 4 5Count

54

50

49

48

42

38

29

21

18

17

11


0 1 2 3 4 5Count

57

51

47

36

32

28

27

22

20

14

6


81

iii. Regulatory FDA Actions

0 1Count

5450494842382921181711

1999-2003Oral & Topical - FDA Actions

Warning Letters Consent Decrees

0 1Count

575147363228272220146

1999-2003Oral & Topical - FDA Actions


82

Manufacturing Facility Inspections

0 .25 .5 .75 1Average Number per Year

5450494842382921181711

By Inspection AgencyOral & Topical - Facility Inspections

FDA EMEAOther


575147363228272220146

By Inspection AgencyOral & Topical - Facility Inspections

FDA EMEAOther

83

iv. Other Services Provide Contract Manufacturing Services

0 10==No; 1==Yes

54

50

49

48

42

38

29

21

18

17

11

Provide Contract Manufacturing Services

0 10==No; 1==Yes

57

51

47

36

32

28

27

22

20

14

6


84

II. Human Resource Management

i. Facility Employment Total Facility Employment

0

500

1000

1500

Num

ber

1999 2000 2001 2002 2003Years

11 1718 2129 3842 4849 5054

1999-2003Oral & Topical - Total Facility Employees

85

0 20 40 60 80 100Percentage

575147363228272220146

(EOY 2002)Employees by Category

Operator Craft WorkerTechnician ProfessionalManagers Clerical

86

Average Total Facility Employees

050

01,

000

1,50

0Av

erag

e Em

ploy

ees

per Y

ear

11 17 18 29 38 42 48 49 50 54

1999-2003Oral & Topical - Avg. Total Facility Employees

050

01,

000

1,50

0Av

erag

e Em

ploy

ees

per Y

ear

6 14 20 22 27 28 32 36 47 51 57

1999-2003Oral & Topical - Avg. Total Facility Employees

87

Total Operators

0 200 400 600Count

5450494842382921181711

By CategoryTotal Operators (EOY 2002)

Mfg/Oper/Prod Process DevQA/QC RA/RCIT Plant Mgmt

0 100 200 300 400Count

575147363228272220146



88

Total Craft Workers

0 100 200 300Count

5450494842382921181711

By CategoryTotal Craft Workers (EOY 2002)


0 100 200 300Count

575147363228272220146



89

Total Technicians

0 50 100 150Count

5450494842382921181711

By CategoryTotal Technicians (EOY 2002)


0 20 40 60 80 100Count

575147363228272220146



90

Total Managers

0 50 100 150Count

5450494842382921181711

By CategoryTotal Managers (EOY 2002)


0 50 100 150 200Count

575147363228272220146



91

Total Professionals

0 50 100 150 200 250Count

5450494842382921181711

By CategoryTotal Professionals (EOY 2002)


0 100 200 300Count

575147363228272220146



92

Total Clerical Workers

0 20 40 60 80Count

5450494842382921181711

By CategoryTotal Clerical W orkers (EOY 2002)


0 20 40 60 80 100Count

575147363228272220146



93

Total Quits

0 20 40 60 80 100Count

5450494842382921181711

By Job ClassificationTotal Quits (EOY 2002)

Operators Craft WorkersTechnicians ProfessionalsManagers Clerical

0 20 40 60 80 100Count

575147363228272220146



94

Total Retires

0 5 10 15Count

5450494842382921181711

By Job ClassificationTotal Retires (EOY 2002)


0 5 10Count

575147363228272220146



95

Total Involuntary Layoffs

0 50 100 150Count

5450494842382921181711

By Job ClassificationTotal Involuntary Layoffs (EOY 2002)


0 50 100 150 200 250Count

575147363228272220146



96

Total Hires

0 50 100 150 200Count

5450494842382921181711

By Job ClassificationTotal Hires (EOY 2002)


0 50 100 150Count

575147363228272220146



97

ii. Employee Demographics Operator Demographics

0 10 20 30 40Years

5450494842382921181711

(EOY 2002)Operator Demographics

Avg. Years Employed Avg. Age

0 10 20 30 40 50Years

575147363228272220146



98

Craft Worker Demographics

0 10 20 30 40 50Years

5450494842382921181711

(EOY 2002)Craft Worker Demographics


0 10 20 30 40 50Years

575147363228272220146



99

Technician Demographics

0 10 20 30 40 50Years

5450494842382921181711

(EOY 2002)Technician Demographics


0 10 20 30 40 50Years

575147363228272220146



100

Manager Demographics

0 10 20 30 40 50Years

5450494842382921181711

(EOY 2002)Manager Demographics


0 10 20 30 40 50Years

575147363228272220146



101

Professional Demographics

0 10 20 30 40 50Years

5450494842382921181711

(EOY 2002)Professional Demographics


0 10 20 30 40 50Years

575147363228272220146



102

Clerical Demographics

0 10 20 30 40Years

5450494842382921181711

(EOY 2002)Clerical Demographics


0 10 20 30 40 50Years

575147363228272220146



103

Operator Educational Level

0 .2 .4 .6 .8 1Percentage

5450494842382921181711

(EOY 2002)Operator Education Level

High School Degree Bachelors DegreeMasters Degree Ph.D.


575147363228272220146



104

Craft Worker Educational Level


5450494842382921181711

(EOY 2002)Craft Worker Education Level



575147363228272220146



105

Technician Educational Level


5450494842382921181711

(EOY 2002)Technician Education Level



575147363228272220146



106

Manager Educational Level


5450494842382921181711

(EOY 2002)Manager Education Level



575147363228272220146



107

Professional Educational Level


5450494842382921181711

(EOY 2002)Professional Education Level



575147363228272220146



108

Clerical Educational Level


5450494842382921181711

(EOY 2002)Clerical Education Level



575147363228272220146



109

iii. Employee Training Basic Skills – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingBasic Skills

Operators Craft WorkersTechnicians Engineers

01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



110

Basic Skills – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingBasic Skills


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



111

Basic Science – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingBasic Science


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



112

Basic Science – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingBasic Science


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



113

Statistical Process Control – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingStatistical Process Control


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



114

Statistical Process Control – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingStatistical Process Control


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



115

Machine Operation – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingMachine Operation


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



116

Machine Operation – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingMachine Operation


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



117

Machine Maintenance – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingMachine Maintenance


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



118

Machine Maintenance – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingMachine Maintenance


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



119

Teamwork – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingTeamwork


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



120

Teamwork – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingTeamwork


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



121

Problem Solving – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingProblem Solving


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



122

Problem Solving – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingProblem Solving


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



123

Design of Experiments – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingDesign of Experiments


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



124

Design of Experiments – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingDesign of Experiements


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



125

Safety Procedures – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingSafety Procedures


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



126

Safety Procedures – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingSafety Procedures


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



127

Clean Room Procedures – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingClean Room Procedures


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



128

Clean Room Procedures – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingClean Room Procedures


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



129

cGMP – On-the-job Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

On-the-job TrainingcGMP


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



130

cGMP – Classroom Training

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Classroom TrainingcGMP


01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



131

iv. Teams Team Participation Percentage

020

4060

8010

0P

erce

ntag

e (%

)

11 17 18 21 29 38 42 48 49 50 54

By Job ClassificationTeam Participation Percentage


020

4060

8010

0P

erce

ntag

e (%

)

6 14 20 22 27 28 32 36 47 51 57



132

Approximate Teams per Participation

02

46

Cou

nt

11 17 18 21 29 38 42 48 49 50 54

By Job ClassificationApproximate Teams per Participant


02

46

810

Cou

nt

6 14 20 22 27 28 32 36 47 51 57



133

Team Number

0 5 10 15 20 25Number of Teams

5450494842382921181711

By Team TypeTeam Number

Quality Improvement Continuous ImprovementSelf-Directed Total Preventative MaintenanceOther


575147363228272220146



134

Average Team Size

0 20 40 60 80 100Average Number of Employees

5450494842382921181711

By Team TypeAverage Team Size


0 5 10 15Average Number of Employees

575147363228272220146



135

Quality Improvement Team Participation

0 20 40 60 80 100Average Percentage

5450494842382921181711

By Job ClassificationQuality Improvement Team Participation



575147363228272220146



136

Continuous Improvement Team Participation


5450494842382921181711

By Job ClassificationContinuous Improvement Team Participation



575147363228272220146



137

Self Directed Work Team Participation


5450494842382921181711

By Job ClassificationSelf-Directed W ork Team Participation



575147363228272220146



138

Total Preventative Maintenance Team Participation


5450494842382921181711

By Job ClassificationTotal Preventative Maintenance Team Participation



575147363228272220146



139

Other Team Participation


5450494842382921181711

By Job ClassificationOther Team Participation


0 20 40 60 80Average Percentage

575147363228272220146



140

III. Product/Process Development i. General Product Overview

Process Classification Count

0 5 10 15 20Number

54

50

49

48

42

38

29

21

18

17

11

By Process CategoryOral & Topical - Process Count

0 50 100 150Number

57

51

47

36

32

28

27

22

20

14

6

By Process CategoryOral & Topical - Process Count

141

Average Primary NDC Numbers

05

1015

Num

ber

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Avg. Primary NDC Numbers

02

46

8N

umbe

r

6 14 20 22 27 32 36 47 51 57

For Reported CompoundsOral & Topical - Avg. Primary NDC Numbers

142

Percentage First Manufactured at Another Facility

0.2

.4.6

.81

Per

cent

age

(%)

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Per. First Manufactured at Another Facility

0.2

.4.6

.81

Per

cent

age

(%)

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Per. First Manufactured at Another Facility

143

Average Duration Manufactured at Another Facility

01.

0e+0

62.

0e+0

63.

0e+0

6M

onth

s

11 17 18 21 29 42 48 49 50

For Reported CompoundsOral & Topical - Avg. Duration Manufactured at Another Facility

050

100

150

200

Mon

ths

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Avg. Duration Manufactured at Another Facility

144

ii. New Product Development Discovery Location

0.2

.4.6

.81

Perc

enta

ge

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Discovery Location

At Current Facility Other Location Within SBUOther Location Within Firm Another Corporation

0.2

.4.6

.81

Perc

enta

ge

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Discovery Location


145

Process Research Location

0.2

.4.6

.81

Perc

enta

ge

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Process Research Location


0.2

.4.6

.81

Perc

enta

ge

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Process Research Location


146

Pilot Development Location

0.2

.4.6

.81

Perc

enta

ge

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Pilot Development Location


0.2

.4.6

.81

Perc

enta

ge

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Pilot Development Location


147

Commercial Plant Transfer Location

0.2

.4.6

.81

Perc

enta

ge

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Commercial Plant Transfer Location


0.2

.4.6

.81

Perc

enta

ge

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Commercial Plant Transfer Location


148

Assay Development Location

0.2

.4.6

.81

Perc

enta

ge

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Assay Development Location


0.2

.4.6

.81

Perc

enta

ge

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Assay Development Location


149

Average Vertical Reporting Relationships – From Discovery to:

02

46

8N

umbe

r

18 21 29 38 42 48 49 50 54

From Discovery To:Oral & Topical - Avg. Vertical Reporting Relationships

Process Development Pilot DevelopmentTransfer Assay Development

02

46

8N

umbe

r

20 22 27 28 32 36 47 51 57

From Discovery To:Oral & Topical - Avg. Vertical Reporting Relationships


150

Average Vertical Reporting Relationships – From Process Research To:

02

46

8N

umbe

r

18 21 29 38 42 48 49 50 54

From Process Research To:Oral & Topical - Avg. Vertical Reporting Relationships

Discovery Pilot DevelopmentTransfer Assay Development

02

46

8N

umbe

r

20 22 27 28 32 36 47 51 57

From Process Research To:Oral & Topical - Avg. Vertical Reporting Relationships


151

Average Vertical Reporting Relationships – From Pilot Development To:

02

46

8N

umbe

r

18 21 29 38 42 48 49 50 54

From Pilot Development To:Oral & Topical - Avg. Vertical Reporting Relationships

Discovery Process DevelopmentTransfer Assay Development

02

46

8N

umbe

r

20 22 27 28 32 36 47 51 57

From Pilot Development To:Oral & Topical - Avg. Vertical Reporting Relationships


152

Average Vertical Reporting Relationships – From Transfer To:

02

46

8N

umbe

r

18 21 29 38 42 48 49 50 54

From Transfer To:Oral & Topical - Avg. Vertical Reporting Relationships

Discovery Process DevelopmentPilot Development Assay Development

02

46

8N

umbe

r

20 22 27 28 32 36 47 51 57

From Transfer To:Oral & Topical - Avg. Vertical Reporting Relationships


153

Average Vertical Reporting Relationships – From Assay Development To:

02

46

8N

umbe

r

18 21 29 38 42 48 49 50 54

From Assay Development To:Oral & Topical - Avg. Vertical Reporting Relationships

Discovery Process DevelopmentPilot Development Transfer

02

46

8N

umbe

r

20 22 27 28 32 36 47 51 57

From Assay Development To:Oral & Topical - Avg. Vertical Reporting Relationships


154

Discovery Personnel

020

4060

8010

0Pe

rcen

tage

(%)

11 17 18 21 29 38 48 49 50 54

Average Time Spent In:Oral & Topical - Discovery Personnel


020

4060

8010

0Pe

rcen

tage

(%)

6 14 22 27 28 32 36 47 51 57

Average Time Spent In:Oral & Topical - Discovery Personnel


155

Process Development Personnel

020

4060

8010

0Pe

rcen

tage

11 17 18 21 29 38 48 49 50 54

Average Time Spent In:Oral & Topical - Process Development Personnel


020

4060

8010

0Pe

rcen

tage

6 14 20 22 27 28 32 36 47 51 57

Average Time Spent In:Oral & Topical - Process Development Personnel


156

Pilot Development Personnel

020

4060

8010

0Pe

rcen

tage

11 17 18 21 29 38 48 49 50 54

Average Time Spent In:Oral & Topical - Commercial Plant Transfer Personnel


020

4060

8010

0Pe

rcen

tage

6 14 22 27 28 32 36 47 51 57

Average Time Spent In:Oral & Topical - Commercial Plant Transfer Personnel


157

Commercial Plant Transfer Personnel

020

4060

8010

0Pe

rcen

tage

11 17 18 21 29 38 48 49 50 54

Average Time Spent In:Oral & Topical - Pilot Development Personnel


020

4060

8010

0Pe

rcen

tage

6 14 22 27 28 32 36 47 51 57

Average Time Spent In:Oral & Topical - Pilot Development Personnel


158

Assay Development Personnel

020

4060

8010

0Pe

rcen

tage

11 17 18 21 29 38 48 49 50 54

Average Time Spent In:Oral & Topical - Assay Development Personnel


020

4060

8010

0Pe

rcen

tage

6 14 20 22 27 28 32 36 47 51 57

Average Time Spent In:Oral & Topical - Assay Development Personnel


159

Organization of Process Validation

0.2

.4.6

.81

Perc

enta

ge

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Organization of Process Validation

Standalone Group Part of Proc DevPart of Ops/Prod Part of EngineeringPart of QA/QC Part of RC/RA

0.2

.4.6

.81

Perc

enta

ge

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Organization of Process Validation


160

Average Total Development Hours

05.

0e+0

61.

0e+0

7N

umbe

r of H

ours

11 17 18 21 29 38 42 48 49 50 54

For Reported CompoundsOral & Topical - Avg. Total Development Hours

Process Development Scale upTransfer/Relocation Assay Development

02,

000

4,00

06,

000

8,00

010

,000

Num

ber o

f Hou

rs

6 14 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Avg. Total Development Hours


161

Total Individuals Involved

05

1015

2025

Num

ber o

f Ind

ivid

uals

11 17 18 21 29 38 48 49 54

For Reported CompoundsOral & Topical - Total Individuals Involved


02

46

8N

umbe

r of I

ndiv

idua

ls

6 20 22 27 28 32 36 47 51 57

For Reported CompoundsOral & Topical - Total Individuals Involved


162

IV. Performance i. Manufacturing

Batches Started

0

50

100

150

Num

ber

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Monthly Average For Reported CompoundsOral & Topical - Batches Started

163

0

5

10

15

20

25N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Batches Started

164

Batches Failed

0

1

2

3N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Monthly Average For Reported CompoundsOral & Topical - Batches Failed

165

0

.2

.4

.6N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Batches Failed

166

Batches Reworked

0

.01

.02

.03N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Monthly Average For Reported CompoundsOral & Topical - Batches Reworked

167

0

.01

.02

.03

.04N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Batches Reworked

168

Theoretical Yield

0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Monthly Average For Reported CompoundsOral & Topical - Theoretical Yield

169

0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Theoretical Yield

170

Actual Yield

0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Monthly Average For Reported CompoundsOral & Topical - Actual Yield

171

0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Actual Yield

172

Cycle Time

0

50

100

150D

ays

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Monthly Average For Reported CompoundsOral & Topical - Cycle Time

173

0

20

40

60

80

100D

ays

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Cycle Time

174

ii. Deviation Management

Product Unavailability

0

.05

.1

.15

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Average For Reported CompoundsOral & Topical - Product Unavailability

175

0

.02

.04

.06

.08Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Product Unavailability

176

Field Alerts

0

1

2

3N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Field Alerts

177

0

.5

1

1.5

2N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Field Alerts

178

Finished Product Recalls

-1

1Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Average For Reported CompoundsOral & Topical - Finished Product Recalls

179

-1

1Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Finished Product Recalls

180

Raw Material Deviations

0

20

40

60

80

100N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Raw Material Deviations

181

0

2

4

6N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Raw Material Deviations

182


0

50

100

150N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Prod. Comp. Deviations

183

0

10

20

30

40N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Prod. Comp. Deviations

184


0

50

100

Num

ber

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Prod./Proc. Deviations

185

0

100

200

300N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Prod./Proc. Deviations

186


0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Average For Reported CompoundsOral & Topical - Repeat Raw Material Deviations

187

-1

1Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Repeat Raw Material Deviations

188

Repeat Product Component Deviations

0

20

40

60

80Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Average For Reported CompoundsOral & Topical - Repeat Prod. Comp. Deviations

189

0

10

20

30Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Repeat Prod. Comp. Deviations

190


0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Average For Reported CompoundsOral & Topical - Repeat Prod./Proc. Deviations

191

0

10

20

30

40

50Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Average For Reported CompoundsOral & Topical - Repeat Prod./Proc. Deviations

192

iii. FDA Inspection

FDA Pre-Approval Inspections

0

1

2

3

4

Num

ber

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - FDA PA Inspections

193

0

1

2

3

4N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - FDA P-A Inspections

194

FDA General cGMP Inspections

0

5

10

15N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - FDA Gen. cGMP Inspections

195

0

2

4

6

8

10N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - FDA Gen. cGMP Inspections

196

FDA For Cause cGMP Inspections

0

2

4

6N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - FDA F-C cGMP Inspections

197

0

5

10

15N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - FDA F-C cGMP Inspections

198

Brazil Inspections

0

1

2

3

4N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Brazil Inspections

199

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Brazil Inspections

200

Canada Inspections

0

.2

.4

.6

.8

1N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Canada Inspections

201

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Canada Inspections

202

EMEA Inspections

0

1

2

3N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - EMEA Inspections

203

0

5

10

15

20

25N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - EMEA Inspections

204

Japan Inspections

0

.5

1

1.5

2N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Japan Inspections

205

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Japan Inspections

206

Other Country Inspections

0

1

2

3

4

5N

umbe

r

1999 2000 2001 2002 2003Year

11 1718 2129 3842 4849 5054

Yearly Total For Reported CompoundsOral & Topical - Other Country Inspections

207

0

1

2

3

4

5N

umbe

r

1999 2000 2001 2002 2003Year

06 1420 2227 2832 3647 5157

Yearly Total For Reported CompoundsOral & Topical - Other Country Inspections

208

Deviation Management iv. Electronic Tracking

Extent of Electronic Tracking

01

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 5412345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003

Extent of Electronic TrackingDeviation Management

01

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 5712345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003


209

IT System Tracking

01

23

Num

ber a

nd T

ype

11 17 18 21 29 38 42 48 49 50 5412345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003

IT System TrackingDeviation Management

By Lot By Deviation TypeBy People Assigned

01

23

Num

ber a

nd T

ype

6 14 20 22 27 28 32 36 47 51 5712345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



210

v. Process Analytical Technology Multivariate Data Analysis Tools

01

23

4N

umbe

r and

Typ

e

11 17 18 21 29 38 42 48 49 50 5412345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003

Multivariate Data Analysis ToolsProcess Analytic Technology

Stat. Design of Exper. Resp. Surface Method.Process Simulation Pattern Recog. Tools

01

23

Num

ber a

nd T

ype

6 14 20 22 27 28 32 36 47 51 5712345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



211

Process Analyzers/Process Analytic Chemistry Tools

01

23

Num

ber a

nd T

ype

11 17 18 21 29 38 42 48 49 50 5412345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003

Process Analyzers/Process Analytic Chemistry ToolsProcess Analytic Technology

Process Measurement Chem. Comp. MeasurementPhys. Attr. Measurement

01

23

Num

ber a

nd T

ype

6 14 20 22 27 28 32 36 47 51 5712345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



212

Process Monitoring, Control and Endpoint Tools

01

23

45

Num

ber a

nd T

ype

11 17 18 21 29 38 42 48 49 50 5412345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003

Process Monitoring, Control and Endpoint ToolsProcess Analytic Technology

Proc. Attr. Related to Qual. R/T Endpoint MonitoringCritical Element Adjustment Mathematical RelationshipsProcess Endpoint Monitoring

01

23

45

Num

ber a

nd T

ype

6 14 20 22 27 28 32 36 47 51 5712345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



213

Effectiveness of PAT Tools

01

23

4B

elie

f and

Typ

e

11 17 18 21 29 38 42 48 49 50 5412345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003

Effectiveness of PAT Tools to:Process Analytic Technology

Acquire Information Develop Risk-Mitig. Strat.Achieve Cont. Improvement Share Information

01

23

4B

elie

f and

Typ

e

6 14 20 22 27 28 32 36 47 51 5712345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



214

vi. Organization Final Responsibility for Lot Failure

0.2

.4.6

.81

0==N

o, 1

==Y

es

11 17 18 21 29 38 42 48 49 50 54

Final Responsibility for Lot FailureDeviation Management

QA Member Assigned QA ManagerHead of QA at Plant Plant ManagerCorporate QA

0.2

.4.6

.81

0==N

o, 1

==Y

es

6 14 20 22 27 28 32 36 47 51 57



215

Takes Lead in Responding to Deviations

0.2

.4.6

.81

0==N

o, 1

==Y

es

11 17 18 21 29 38 42 48 49 50 54

Takes Lead in Responding to DeviationsDeviation Management

Operations EngineeringQuality Regulatory AffairsGroup From Which Deviation Originated

0.2

.4.6

.81

0==N

o, 1

==Y

es

6 14 20 22 27 28 32 36 47 51 57



216

Must Review and Approve Deviation

0.2

.4.6

.81

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Must Review and Approve DeviationDeviation Management

Operations EngineeringQuality Regulatory Affairs

0.2

.4.6

.81

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



217

Typically participate in Review/Approval of Deviation

0.2

.4.6

.81

0==N

o, 1

==Ye

s

11 17 18 21 29 38 42 48 49 50 54

Typically Participate in Review/Approval of DeviationDeviation Management


0.2

.4.6

.81

0==N

o, 1

==Ye

s

6 14 20 22 27 28 32 36 47 51 57



218

Final Responsibility for Lot Release with Major Deviation

0.2

.4.6

.81

0==N

o, 1

==Y

es

11 17 18 21 29 38 42 48 49 50 54

Final Responsibility for Lot Release with Major DeviationDeviation Management


0.2

.4.6

.81

0==N

o, 1

==Y

es

6 14 20 22 27 28 32 36 47 51 57



219

Appendix B: Pharmaceutical – API V. Manufacturing Facility


89

10111213

LN(S

quar

e M

eter

s)

1999 2000 2001 2002 2003Years

03 0526 3133 3537 3940 4446 4852 5356

1999-2003API - Facility Size

220


05

1015

LN(S

quar

e M

eter

s)

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

1999-2003API - Average Facility Size

221


0

50000

100000

150000

200000Li

ters

1999 2000 2001 2002 2003Years

03 0526 3133 3537 3940 4446 4852 5356

1999-2003API - Max Reactor/Vat/Fermentor

222


050

,000

1000

0015

0000

2000

00Li

ters

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

1999-2003API - Max Reactor/Vat/Fermentor

223

ii. Products and Processes

Average Manufacturing facility Operating Hours

020

040

060

080

0H

ours

per

Mon

th

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



0 20 40 60 80 100Count

5653524846444039373533312653


224


0 1 2 3 4 5Count

5653524846444039373533312653


iii. Regulatory

FDA Actions

0 1Count

5653524846444039373533312653

1999-2003API - FDA Actions


225



5653524846444039373533312653

By Inspection AgencyAPI - Facility Inspections

FDA EMEAOther

iv. Other Services


0 10==No; 1==Yes

5653524846444039373533312653


226

VI. Human Resource Management

i. Facility Employment Total Facility Employment

01000200030004000

Num

ber

1999 2000 2001 2002 2003Years

03 0526 3133 3537 3940 4446 4852 5356

1999-2003API - Total Facility Employees

227


01,

000

2,00

03,

000

Aver

age

Empl

oyee

s pe

r Yea

r

3 5 26 33 37 39 40 44 46 48 52 53 56

1999-2003API - Avg. Total Facility Employees

Total Operators

0 100 200 300 400 500Count

5653524846444039373533312653



228

Total Craft Workers

0 100 200 300 400Count

5653524846444039373533312653



Total Technicians

0 100 200 300 400Count

5653524846444039373533312653



229

Total Managers

0 50 100 150 200Count

5653524846444039373533312653



Total Professionals

0 100 200 300 400Count

5653524846444039373533312653



230


0 20 40 60 80Count

5653524846444039373533312653



Total Quits

0 5 10 15 20 25Count

5653524846444039373533312653



231

Total Retires

0 50 100 150Count

5653524846444039373533312653




0 20 40 60 80Count

5653524846444039373533312653



232

Total Hires

0 50 100 150Count

5653524846444039373533312653



233

ii. Employee Demographics

Operator Demographics

0 10 20 30 40 50Years

5653524846444039373533312653




0 10 20 30 40 50Years

5653524846444039373533312653



234


0 10 20 30 40 50Years

5653524846444039373533312653




0 20 40 60Years

5653524846444039373533312653



235


0 10 20 30 40Years

5653524846444039373533312653




0 10 20 30 40 50Years

5653524846444039373533312653



236



5653524846444039373533312653





5653524846444039373533312653



237



5653524846444039373533312653





5653524846444039373533312653



238



5653524846444039373533312653





5653524846444039373533312653



239

iii. Employee Training

Basic Skills – On-the-job Training

01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



240


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



241


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



242


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



243


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



244


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



245


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



246


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



247


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



248


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



249


01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56




01

0==N

o, 1

==Ye

s

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56



250

iv. Teams Team Participation Percentage

020

4060

8010

0Pe

rcen

tage

(%)

3 26 31 33 35 37 39 40 44 46 48 52 53 56




02

46

810

Cou

nt

3 26 31 33 35 37 39 40 44 46 48 52 53 56



251

Team Number


565352484644403937353331263



Average Team Size

0 5 10 15 20Average Number of Employees

565352484644403937353331263



252



565352484644403937353331263





565352484644403937353331263



253



565352484644403937353331263





565352484644403937353331263



254



565352484644403937353331263



255

VII. Product/Process Development

i. General Product Overview Process Classification Count

0 20 40 60 80 100Number

5653524846444039373533312653

By Process CategoryAPI - Process Count

256

API Manufacturing Facilities Information

05

1015

2025

Num

ber

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

API - Manufacturing Facilities Information

Avg. Chemical Reactions Avg. Buyers


020

4060

80N

umbe

r

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Average Primary NDC Numbers

257


0.2

.4.6

.81

Per

cent

age

(%)

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Per. First Manufactured at Another Facility


01.

0e+0

62.

0e+0

63.

0e+0

6M

onth

s

3 26 31 33 35 37 44 46 48 53 56

For Reported CompoundsAPI - Avg. Duration Manufactured at Another Facility

258


0.2

.4.6

.81

Perc

enta

ge

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Discovery Location



0.2

.4.6

.81

Perc

enta

ge

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Process Research Location


259


0.2

.4.6

.81

Perc

enta

ge

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Pilot Development Location



0.2

.4.6

.81

Perc

enta

ge

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Commercial Plant Transfer Location


260


0.2

.4.6

.81

Perc

enta

ge

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Assay Development Location



02

46

810

Num

ber

3 5 26 35 37 39 40 46 48 52 53 56

From Discovery To:API - Avg. Vertical Reporting Relationships


261


02

46

810

Num

ber

3 5 26 35 37 39 40 46 48 52 53 56

From Process Research To:API - Avg. Vertical Reporting Relationships



02

46

810

Num

ber

3 5 26 35 37 39 40 46 48 52 53 56

From Pilot Development To:API - Avg. Vertical Reporting Relationships


262


02

46

810

Num

ber

3 5 26 35 37 39 40 46 48 52 53 56

From Transfer To:API - Avg. Vertical Reporting Relationships



02

46

810

Num

ber

5 26 35 37 39 40 46 48 52 53 56

From Assay Development To:API - Avg. Vertical Reporting Relationships


263

Discovery Personnel

020

4060

8010

0Pe

rcen

tage

(%)

3 5 26 33 35 37 39 40 46 48 52 53 56

API - Avg. Time Spent In:API - Discovery Personnel



050

100

150

Perc

enta

ge

3 26 33 35 37 39 40 46 48 52 53 56

API - Avg. Time Spent In:Process Development Personnel


264


020

4060

8010

0Pe

rcen

tage

3 26 33 35 37 39 40 46 48 52 53 56

Average Time Spent In:API - Commercial Plant Transfer Personnel



020

4060

8010

0Pe

rcen

tage

3 26 33 35 37 39 40 46 48 52 53 56

Average Time Spent In:API - Pilot Development Personnel


265


020

4060

8010

0Pe

rcen

tage

3 26 33 35 37 39 40 46 48 52 53 56

Average Time Spent In:API - Assay Development Personnel



0.2

.4.6

.81

Perc

enta

ge

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Organization of Process Validation


266


02.

0e+0

64.

0e+0

66.

0e+0

6N

umbe

r of H

ours

3 5 26 31 33 35 37 39 40 44 46 48 52 53 56

For Reported CompoundsAPI - Avg. Total Development Hours



05

1015

20N

umbe

r of I

ndiv

idua

ls

3 5 26 33 35 37 39 40 46 48 52 53 56

For Reported CompoundsAPI - Total Individuals Involved


267

VIII. Performance

i. Manufacturing

Batches Started

0

20

40

60

Num

ber

1999 2000 2001 2002 2003Year

03 0526 3133 3507 3940 4446 4852 5356

Monthly Average For Reported CompoundsAPI - Batches Started

268

Batches Failed

012345

Num

ber

1999 2000 2001 2002 2003Year

03 0526 3133 3507 3940 4446 4852 5356

Monthly Average For Reported CompoundsAPI - Batches Failed

269

Batches Reworked

0.2.4.6.81

Num

ber

1999 2000 2001 2002 2003Year

03 0526 3133 3507 3940 4446 4852 5356

Monthly Average For Reported CompoundsAPI - Batches Reworked

270

Theoretical Yield

020406080

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

03 0526 3133 3507 3940 4446 4852 5356

Monthly Average For Reported CompoundsAPI - Theoretical Yield

271

Actual Yield

020406080

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

03 0526 3133 3507 3940 4446 4852 5356

Monthly Average For Reported CompoundsAPI - Actual Yield

272

Cycle Time

0

20

40

60

80

Day

s

1999 2000 2001 2002 2003Year

03 0526 3133 3507 3940 4446 4852 5356

Monthly Average For Reported CompoundsAPI - Cycle Time

273



0.1.2.3.4.5

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Average For Reported CompoundsAPI - Product Unavailability

274

Field Alerts

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Field Alerts

275


-1

1Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Average For Reported CompoundsAPI - Finished Product Recalls

276


0

20

40

60N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Raw Material Deviations

277


020406080

100N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Production Component Deviations

278


0

50

100

150N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Product/Process Deviations

279


-100

-50

0

50

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Average For Reported CompoundsAPI - Repeat Raw Material Deviations

280


-100

-50

0

50

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Average For Reported CompoundsAPI - Repeat Prod. Comp. Deviations

281


-100

-50

0

50

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Average For Reported CompoundsAPI - Repeat Product/Process Deviations

282

iii. FDA Inspection


0

1

2

3

Num

ber

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - FDA Pre-Approval Inspections

283


0

5

10

15N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - FDA General cGMP Inspections

284


0.2.4.6.81

Num

ber

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - FDA For Cause cGMP Inspections

285

Brazil Inspections

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Brazil Inspections

286

Canada Inspections

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Canada Inspections

287

EMEA Inspections

0

.5

1

1.5

2N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - EMEA Inspections

288

Japan Inspections

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Japan Inspections

289


012345

Num

ber

1999 2000 2001 2002 2003Year

03 0526 3133 3537 3940 4446 4852 5356

Yearly Total For Reported CompoundsAPI - Other Country Inspections

290

IX. Deviation Management

i. Electronic Tracking Extent of Electronic Tracking

01

0==N

o, 1

==Ye

s

3 5 26 33 35 37 39 40 44 46 48 52 53 5612345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003


291

IT System Tracking

01

23

Num

ber a

nd T

ype

3 5 26 33 35 37 39 40 44 46 48 52 53 5612345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



ii. Process Analytical Technology Multivariate Data Analysis Tools

01

23

Num

ber a

nd T

ype

3 5 26 33 35 37 39 40 44 46 48 52 53 5612345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



292


01

23

Num

ber a

nd T

ype

3 5 26 33 35 37 39 40 44 46 48 52 53 5612345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003




01

23

45

Num

ber a

nd T

ype

3 5 26 33 35 37 39 40 44 46 48 52 53 5612345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



293


01

23

4B

elie

f and

Typ

e

3 5 26 33 35 37 39 40 44 46 48 52 53 5612345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345 12345

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



iii. Organization Final Responsibility for Lot Failure

0.2

.4.6

.81

0==N

o, 1

==Y

es

3 26 33 35 37 39 40 44 46 48 52 53 56



294


0.2

.4.6

.81

0==N

o, 1

==Y

es

3 26 33 35 37 39 40 44 46 48 52 53 56




0.2

.4.6

.81

0==N

o, 1

==Ye

s

3 26 33 35 37 39 40 44 46 48 52 53 56



295


0.2

.4.6

.81

0==N

o, 1

==Ye

s

3 26 33 35 37 39 40 44 46 48 52 53 56




0.2

.4.6

.81

0==N

o, 1

==Y

es

3 26 33 35 37 39 40 44 46 48 52 53 56



296

Appendix C: Pharmaceutical – Injectables X. Manufacturing Facilities


8

10

12

14

LN(S

quar

e M

eter

s)

1999 2000 2001 2002 2003Years

14 1819 2831 4345 55

1999-2003Injectable - Facility Size

297


05

1015

LN(S

quar

e M

eter

s)

14 18 19 28 31 43 45 51

1999-2003Injectable - Average Facility Size

298


0

10000

20000

30000

40000Li

ters

1999 2000 2001 2002 2003Years

14 1819 2831 4345 55

1999-2003Injectable - Max Reactor/Vat/Fermentor

299


010

,000

20,0

0030

,000

40,0

00Li

ters

14 18 19 28 31 43 45 51

1999-2003Injectable - Max Reactor/Vat/Fermentor

ii. Products and Processes Average Manufacturing facility Operating Hours

020

040

060

080

0H

ours

per

Mon

th

14 18 19 28 31 43 45 51


300


0 20 40 60 80 100Count

51

45

43

31

28

19

18

14



0 1 2 3 4 5Count

51

45

43

31

28

19

18

14


301

iii. Regulatory FDA Actions

0 .2 .4 .6 .8 1Count

51

45

43

31

28

19

18

14

1999-2003Injectable - FDA Actions



0 .5 1 1.5Average Number per Year

51

45

43

31

28

19

18

14

By Inspection AgencyInjectable - Facility Inspections

FDA EMEAOther

302

iv. Other Services Provide Contract Manufacturing Services

0 10==No; 1==Yes

51

45

43

31

28

19

18

14


303

XI. Human Resource Management

i. Facility Employment

Total Facility Employment

0

500

1000

1500

2000

Num

ber

1999 2000 2001 2002 2003Years

14 1819 2831 4345 51

1999-2003Injectable - Total Facility Employees

304


050

01,

000

1,50

0Av

erag

e Em

ploy

ees

per Y

ear

14 18 19 28 43 45 51

1999-2003Injectable - Avg. Total Facility Employees

Total Operators

0 500 1,000Count

51

4543

31

28

19

18

14



305

Total Craft Workers

0 200 400 600 800Count

51

4543

31

28

19

18

14



Total Technicians

0 100 200 300 400Count

51

4543

31

28

19

18

14



306

Total Managers

0 50 100 150 200Count

51

4543

31

28

19

18

14



Total Professionals

0 100 200 300 400Count

51

4543

31

28

19

18

14



307


0 20 40 60 80 100Count

51

4543

31

28

19

18

14



Total Quits

0 50 100Count

51

4543

31

28

19

18

14



308

Total Retires

0 10 20 30Count

51

4543

31

28

19

18

14




0 50 100 150 200 250Count

51

4543

31

28

19

18

14



309

Total Hires

0 100 200 300Count

51

4543

31

28

19

18

14



310

ii. Employee Demographics

Operator Demographics

0 10 20 30 40Years

51

45

43

31

28

19

18

14




0 10 20 30 40 50Years

51

45

43

31

28

19

18

14



311


0 10 20 30 40 50Years

51

45

43

31

28

19

18

14




0 10 20 30 40 50Years

51

45

43

31

28

19

18

14



312


0 10 20 30 40Years

51

45

43

31

28

19

18

14




0 10 20 30 40 50Years

51

45

43

31

28

19

18

14



313



51

45

43

31

28

19

18

14





51

45

43

31

28

19

18

14



314



51

45

43

31

28

19

18

14





51

45

43

31

28

19

18

14



315



51

45

43

31

28

19

18

14





51

45

43

31

28

19

18

14



316

iii. Employee Training Basic Skills – On-the-job Training

01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



317


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



318


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



319


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



320


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



321


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



322


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



323


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



324


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



325


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



326


01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51




01

0==N

o, 1

==Ye

s

14 18 19 28 31 43 45 51



327

iv. Teams

Team Participation Percentage

020

4060

8010

0Pe

rcen

tage

(%)

14 18 19 28 31 43 45 51




01

23

45

Cou

nt

14 18 19 28 31 43 45 51



328

Team Number

0 50 100 150Number of Teams

51

4543

31

28

19

18

14



Average Team Size

0 5 10 15 20Average Number of Employees

51

4543

3128

19

1814



329



51

45

43

31

28

19

18

14





51

45

43

31

28

19

18

14



330



51

45

43

31

28

19

18

14





51

45

43

31

28

19

18

14



331



51

45

43

31

28

19

18

14



XII. Product/Process Development i. General Product Overview

332

Process Classification Count

0 10 20 30 40Number

51

45

43

31

28

19

18

14

By Process CategoryInjectable - Process Count


01

23

45

Num

ber

14 18 19 31 43 45 51

For Reported CompoundsInjectable - Average Primary NDC Numbers

333


0.1

.2.3

.4.5

Per

cent

age

(%)

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Per. First Manufactured at Another Facility


-11

Mon

ths

14 18 28 31 45 51

For Reported CompoundsInjectable - Avg. Duration Manufactured at Another Facility

334


0.2

.4.6

.81

Perc

enta

ge

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Discovery Location



0.2

.4.6

.81

Perc

enta

ge

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Process Research Location


335


0.2

.4.6

.81

Perc

enta

ge

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Pilot Development Location



0.2

.4.6

.81

Perc

enta

ge

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Commercial Plant Transfer Location


336


0.2

.4.6

.81

Perc

enta

ge

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Assay Development Location



02

46

8N

umbe

r

18 19 28 43 45 51

From Discovery To:Injectable - Avg. Vertical Reporting Relationships


337


02

46

8N

umbe

r

18 19 28 43 45 51

From Process Research To:Injectable - Avg. Vertical Reporting Relationships



02

46

8N

umbe

r

18 19 28 43 45 51

From Pilot Development To:Injectable - Avg. Vertical Reporting Relationships


338


02

46

8N

umbe

r

18 19 28 43 45 51

From Transfer To:Injectable - Avg. Vertical Reporting Relationships



02

46

8N

umbe

r

18 19 28 43 45 51

From Assay Development To:Injectable - Avg. Vertical Reporting Relationships


339

Discovery Personnel

020

4060

8010

0Pe

rcen

tage

(%)

14 18 19 28 43 45 51

Average Time Spent In:Injectable - Discovery Personnel



020

4060

8010

0Pe

rcen

tage

14 18 19 28 43 45 51

Average Time Spent In:Injectable - Process Development Personnel


340


020

4060

8010

0Pe

rcen

tage

14 18 19 28 43 45 51

Average Time Spent In:Injectable - Commercial Plant Transfer Personnel



020

4060

8010

0Pe

rcen

tage

14 18 19 28 43 45 51

Average Time Spent In:Injectable - Pilot Development Personnel


341


020

4060

8010

0Pe

rcen

tage

14 18 19 28 43 45 51

Average Time Spent In:Injectable - Assay Development Personnel



0.2

.4.6

.81

Perc

enta

ge

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Organization of Process Validation


342


020

,000

40,0

0060

,000

80,0

0010

0000

Num

ber o

f Hou

rs

14 18 19 28 31 43 45 51

For Reported CompoundsInjectable - Avg. Total Development Hours



010

2030

4050

Num

ber o

f Ind

ivid

uals

18 19 28 43 45 51

For Reported CompoundsInjectable - Total Individuals Involved


343

XIII. Performance

i. Manufacturing

Batches Started

0

5

10

15

20

Num

ber

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Monthly Average For Reported CompoundsInjectable - Batches Started

344

Batches Failed

0

.5

1

1.5N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Monthly Average For Reported CompoundsInjectable - Batches Failed

345

Batches Reworked

0

.01

.02

.03N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Monthly Average For Reported CompoundsInjectable - Batches Reworked

346

Theoretical Yield

0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Monthly Average For Reported CompoundsInjectable - Theoretical Yield

347

Actual Yield

0

20

40

60

80

100Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Monthly Average For Reported CompoundsInjectable - Actual Yield

348

Cycle Time

0

20

40

60D

ays

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Monthly Average For Reported CompoundsInjectable - Cycle Time

349



0

.2

.4

.6

.8

1

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Average For Reported CompoundsInjectable - Product Unavailability

350

Field Alerts

0

2

4

6

8N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Field Alerts

351


0

.05

.1

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Average For Reported CompoundsInjectable - Finished Product Recalls

352


0

50

100

150

200N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Raw Material Deviations

353


0

20

40

60

80

100N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Production Component Deviations

354


0

500

1000

1500N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Product/Process Deviations

355


-100

-50

0

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Average For Reported CompoundsInjectable - Repeat Raw Material Deviations

356


-100

-50

0

50

Perc

enta

ge (%

)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Average For Reported CompoundsInjectable - Repeat Prod. Comp. Deviations

357


0

500

1000Pe

rcen

tage

(%)

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Average For Reported CompoundsInjectable - Repeat Product/Process Deviations

358

iii. FDA Inspection


0

1

2

3

Num

ber

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - FDA Pre-Approval Inspections

359


0

2

4

6

8

10N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - FDA General cGMP Inspections

360


0

.5

1

1.5

2N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - FDA For Cause cGMP Inspections

361

Brazil Inspections

0

.2

.4

.6

.8

1N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Brazil Inspections

362

Canada Inspections

0

.2

.4

.6

.8

1N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Canada Inspections

363

EMEA Inspections

0

.5

1

1.5

2N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - EMEA Inspections

364

Japan Inspections

-1

1N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Japan Inspections

365


0

.2

.4

.6

.8

1N

umbe

r

1999 2000 2001 2002 2003Year

14 1819 2831 4345 51

Yearly Total For Reported CompoundsInjectable - Other Country Inspections

366

XIV. Deviation Management

i. Electronic Tracking Extent of Electronic Tracking

01

0==N

o, 1

==Ye

s

14 18 19 28 43 45 511 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1=1999, 2=2000, 3=2001, 3=2002, 5=2003


367

IT System Tracking

01

23

Num

ber a

nd T

ype

14 18 19 28 43 45 511 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



ii. Process Analytical Technology

Multivariate Data Analysis Tools

01

23

4N

umbe

r and

Typ

e

14 18 19 28 43 45 511 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



368


01

23

Num

ber a

nd T

ype

14 18 19 28 43 45 511 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1=1999, 2=2000, 3=2001, 3=2002, 5=2003




01

23

45

Num

ber a

nd T

ype

14 18 19 28 43 45 511 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



369


01

23

4B

elie

f and

Typ

e

14 18 19 28 43 45 511 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1=1999, 2=2000, 3=2001, 3=2002, 5=2003



iii. Organization

Final Responsibility for Lot Failure

0.2

.4.6

.81

0==N

o, 1

==Y

es

14 18 19 28 43 45 51



370


0.2

.4.6

.81

0==N

o, 1

==Y

es

14 18 19 28 43 45 51




0.2

.4.6

.81

0==N

o, 1

==Ye

s

14 18 19 28 43 45 51



371


0.2

.4.6

.81

0==N

o, 1

==Ye

s

14 18 19 28 43 45 51




0.2

.4.6

.81

0==N

o, 1

==Y

es

14 18 19 28 43 45 51



372

Appendix D: PMRP Survey Instrument 1. Plant Liaison Information

373

374

2. Company and Business Unit Information

375

376

377

3A. Manufacturing Facility

378

379

380

3B. Manufacturing Facility

381

382

383

4. Human Resources

384

385

386

387

5. Product Development

388

389

390

391

392

393

394

6. Performance Metrics

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

7. Teams

429

430

431

432

433

434

435

436

437

438

439

440

441

8A. Deviation and Manufacturing Management

442

443

444

445

446

8B. Deviation Management

447

448

449

450

451

452

453

9. Supplement management

454

455

456

457

Documents

Pharmaceutical Manufacturing Research Project Final ...apps.olin.wustl.edu/faculty/nickerson/results/PMRPFinalReportSept2… · Research Project Final Benchmarking Report By Jeffrey