(Q)SAR Evaluation of Drug Impurities from the US FDA Scientific Perspective

Naomi L. Kruhlak, Ph.D.Scientific Lead, Computational Toxicology Consultation Service

Division of Applied Regulatory ScienceOffice of Clinical PharmacologyOffice of Translational SciencesFDA’s Center for Drug Evaluation and Research

2019 Pharmaceutical Industry and Regulators Symposium, Brasilia, Brazil

May 29, 2019

2

Disclaimer

The findings and conclusions in this presentation reflect the views of the authors and should not be construed to represent FDA’s views or policies.

The mention of commercial products, their sources, or their use in connection with material reported herein is not to be construed as either an actual or implied endorsement of such products by the Department of Health and Human Services.

www.fda.gov

3

Outline

▪ Chemical informatics at FDA/CDER

▪ ICH M7(R1) Guideline

• Application of expert knowledge

• Structural analog searching

• Applicability domain

▪ FDA/CDER Computational Toxicology Consultation Service

• Chemical registration

• (Q)SAR prediction workflow

▪ Evaluation of Applicant (Q)SAR data

• Software acceptability

• Out-of-domain results

• Reporting

www.fda.gov

Chemical Informatics at FDA/CDER

5

Chemical Informatics at FDA/CDER

▪ Applied regulatory research

• Create chemical structure-linked toxicological and clinical effect databases

• Develop and enhance toxicological and clinical effect prediction models through external collaborations

• Develop best practices for the application of computational models

▪ Computational Toxicology Consultation Service

• Provide (Q)SAR evaluations for drugs, metabolites, impurities, degradants, etc. to FDA/CDER safety and quality reviewers

• Perform structure-based searching for read-across purposes

• Provide expert interpretation of (Q)SAR data submitted to FDA/CDER

• Maintains a repository of chemical structures evaluated internally or used for CDER cheminformatics projects (currently ~30K)

www.fda.gov

6

In-House

(Q)SAR Consult

Database

Clinical Endpoints:• Liver toxicity

• Cardiotoxicity

• Renal toxicity

Non-Clinical Endpoints:• Genetic toxicity

• Rodent carcinogenicity

• Reproductive/developmental

toxicity

• Phospholipidosis

Reference Datasets:• Validation

• Read-across

• Public

• In-house

Training Sets:

Non-Clinical Toxicity

Clinical AEs

Benchmarking

Consultations

Pharmacological Endpoints:• Opioid receptor binding

• Blood-brain barrier permeability

Regulatory Research AreasChemical

Registration

System

Chemical Structures

Web-Based Searchable Databases

Data Sets

In Silico Models

7

▪ (Q)SAR modeling• Global models

• Binary endpoints

• Different modeling algorithms – combined predictions

• Apply expert knowledge to (Q)SAR analyses

▪ Commercial software used under RCAs • SAR and QSAR modeling and prediction platforms

• Data visualization tools

• Commercial databases

• Access to collaborative networks and consortia

(Q)SAR Modeling at FDA/CDER

www.fda.gov

8

(Q)SAR Software Used by FDA/CDER

▪ Statistical-Based Models

• CASE Ultra MultiCASE, Inc.

• Model Applier - Statistical Models Leadscope, Inc.

• Sarah Nexus Lhasa Limited

▪ Expert Rule-Based Models

• Derek Nexus Lhasa Limited

• Model Applier - Expert Alerts Leadscope, Inc.

• CASE Ultra - Expert Alerts MultiCASE, Inc.

All software above are used by FDA/CDER under Research Collaboration Agreements (RCAs)www.fda.gov

9

▪ Different methodologies yield different predictions

• Predictions are complementary

• Yields higher sensitivity and negative predictivity

• Second statistical system improves coverage

▪ Predictions are chemically meaningful and transparent

• Structural alerts and associated training set structures can be identified to explain why a prediction was made

• Application of expert knowledge is facilitated

▪ Software and models are publicly available

• Our results are reproducible by pharmaceutical applicants and others

(Q)SAR Software Selection Criteria

www.fda.gov

10

ICH M7(R1) Guideline

11

ASSESSMENT AND CONTROL OF DNA REACTIVE (MUTAGENIC) IMPURITIES IN PHARMACEUTICALS TO LIMIT POTENTIAL CARCINOGENIC RISK

▪ Goal of hazard assessment is to assign impurity class

www.fda.gov

(Q)SAR

experimental data

experimental data or (Q)SAR

The ICH M7(R1) Guideline

Mutagenic

Non-mutagenic

12

Section 6:

“A computational toxicology assessment should be performed using (Q)SAR methodologies that predict the outcome of a bacterial mutagenicity assay(Ref. 6). Two (Q)SAR prediction methodologies that complement each other should be applied. One methodology should be expert rule-based and the second methodology should be statistical-based. (Q)SAR models utilizing these prediction methodologies should follow the general validation principles set forth by the Organisation for Economic Co-operation and Development (OECD).

The absence of structural alerts from two complementary (Q)SAR methodologies (expert rule-based and statistical) is sufficient to conclude that the impurity is of no mutagenic concern, and no further testing is recommended (Class 5 in Table 1).”

How to Apply (Q)SAR Under ICH M7

www.fda.gov

13

OECD Validation Principles

▪ To facilitate the consideration of a (Q)SAR model for regulatory purposes, it should be associated with the following information:

1) a defined endpoint

2) an unambiguous algorithm

3) a defined domain of applicability

4) appropriate measures of goodness-of–fit, robustness and predictivity

5) a mechanistic interpretation, if possible

OECD (2007) Guidance document on the validation of (Quantitative) structure activity-relationship [(Q)SAR] models.www.fda.gov

14

Expert Knowledge

Model output “… can be reviewed with the use of expert knowledge in order to provide additional supportive evidence on relevance of any positive, negative, conflicting or inconclusive prediction and provide a rationale to support the final conclusion.”

For example:

▪ Identify and interpret alerting portion of the molecule

▪ Consider mechanism of reactivity, where possible

▪ Assess training set structures used to derive an alert and mitigating features

▪ Consider data from structurally similar compound (analog)

Expert knowledge is applied to all (Q)SAR analyses conducted in-house by FDA/CDERwww.fda.gov

15

(Q)SAR Model

NegativeMutagenicity

Prediction

Structural Alert

Mitigating Features

Application of Expert Knowledge

▪ Identify and interpret alerting portion of the molecule

▪ Consider mechanism of reactivity, where possible

▪ Assess training set structures used to derive alerts and mitigating features [review model output]

▪ Consider data from structurally similar compounds (analogs) not used by the model [search supplemental databases]

www.fda.gov

16

QueryMolecule

0.90Negative

0.77Negative

0.51Negative

0.49Positive

Identifying Structural Analogs

www.fda.gov

Global Similarity Searching Similarity Index

Analogs with relevant alert environment

0.47Negative

No shared alert

17

Sub-Structure Searching for Analogs

▪ Returns multiple hits containing a particular sub-structure, e.g. primary aromatic amine

▪ Can refine the query to identify the most relevant analogs

• For bacterial mutagenicity, local similarity is important.

• The most globally similar analog may not be the most relevant.

282 hits

7 hits

856 hits

Negative Negative Negative Positive

18

▪ Applicability Domain: Region of chemical space within which a model makes predictions with a given reliability

▪ Chemical space defined by structural attributes/properties of training set molecules

Applicability Domain Measurement

www.fda.gov

Descriptor 1

Training

Test

PhysicochemicalDescriptors

MolecularFragments

E.g.,

GlobalSimilarity

Test chemical fragments must be “known” to model

67% 53%

49% 48%

19

▪ Overall, different models have different coverage (applicability domain measurement)

• Even models using the same general method (e.g., fragment-based statistical models) can differ in coverage

• Can be used to our advantage to obtain a valid prediction

• However, when multiple models yield OODs, then extra attention needed

Out-of-Domain (OOD) Definitions

www.fda.gov

20

▪ For 3 models in combination (n = 519 chemicals):• Predictions for 50% of chemicals in agreement• Predictions for 13% of chemicals changed based on expert knowledge

+ = positive − = negativeEqv = equivocalOOD = out-of-domain

Impact of Expert Knowledge

www.fda.gov

▪ Particularly useful for resolving ambiguous (Q)SAR outcomes, such as equivocal predictions or out-of-domain results

Chem. No. Chemical Name

Bacterial Mutagenicity OverallSoftware

Prediction

OverallExpert

PredictionModel

1Model

2Model

3

1 Chemical 1 - - - - -

2 Chemical 2 - - Eqv Eqv +

3 Chemical 3 + - OOD + +

4 Chemical 4 - OOD - - -

5 Chemical 5 + + - + +

6 Chemical 6 - OOD OOD OOD OOD

7 Chemical 7 Eqv Eqv - Eqv -

FDA/CDER Computational Toxicology Consultation Service (CTCS)

22

▪ Have we seen this compound before?▪ Are experimental data available?▪ Have we previously performed a (Q)SAR analysis for this

compound?▪ Are there data for related compounds?

NoYes

Common ICH M7 Review Questions

Chemical registration enables us to answer these questions

www.fda.gov

23

Send data to requestor

Is chemical

in DB?

No

Yes

Standardize structure; create molfile; add to

DB

Adequate expt’ldata?

No

Yes

Prior CDER (Q)SAR

analysis?

No

Yes

Send report to requestor

Applicant (Q)SAR

data?

Yes

Acceptable?

Yes

Send report to requestor

No

No Run (Q)SAR analysis

Send report to requestor

CDER Chemical Dictionary▪ 30K structures▪ Experimental data▪ Prior (Q)SAR consult reports

CTCS Chemical Registration Process

24

Evaluation of Applicant (Q)SAR Data

25

(Q)SAR Software Acceptability

▪ Under the ICH M7 guideline, Applicants may submit (Q)SAR analyses performed using models that are fit-for-purpose• Commercially available• Freely available• Constructed in-house

▪ CDER has prior knowledge of several commercial and freely available (Q)SAR software

▪ For software that CDER has no prior knowledge of, supporting documentation demonstrating that a model is fit-for-purpose is recommended (e.g., QMRF)• Predict bacterial (Ames) mutagenicity• 2 models: expert rule-based and statistical-based• Consistent with OECD Validation Principles

www.fda.gov

26

▪ Typically, only problematic (Q)SAR submissions are sent to us for evaluation

• Well-documented submissions are handled by review divisions

• If a reviewer is concerned about the quality of a submission, or it uses an unfamiliar software, it is sent to us.

• Quality Issues: single methodology, read-across only, overall conclusions conflict with predictions with no explanation

▪ General rule is: Trust, but verify. Predictions are re-run only if there is a concern.

▪ Predictions with the most recent software version are preferred. Old predictions are acceptable unless there are known model changes that could impact conclusions.

▪ Expert analysis serves as a buffer to prediction changes with different software versions

Applicant (Q)SAR submissions

www.fda.gov

27

Out-of-Domain Results

▪ Common problem for new drug impurities

▪ 18% of impurities in new drugs approved in 2016 and 2017 had an out-of-domain (Q)SAR result, based on an internal study

▪ An out-of-domain result is not a prediction and does not contribute to a Class 5 assignment

▪ Application of expert knowledge can be used to address these gaps but higher bar to acceptance

▪ FDA/CDER uses a 2nd statistical system to resolve most out-of-domains in internal analyses

▪ These are areas with the greatest need for improved databases and models

www.fda.gov

28

Example – Out-of-Domain (OOD)

ModelBacterial

MutagenicityExpert Rule-Based NegativeStatistical-Based OOD*

Default Overall Prediction OODExperimental Data Negative

ModelBacterial

MutagenicityExpert Rule-Based NegativeStatistical-Based OOD*

Default Overall Prediction OODOverall Expert Prediction Negative

*Contains unknown fragment and/or has no nearest neighbors

Ames data from API is acceptable to qualify impurity since only difference is an additional non-reactive group

Late-stage Impurity

Parent Drug

Class 5*Contains same unknown fragment and/or has no nearest neighbors

29

Relevant Information for Reporting

▪ Materials and methods• Name and version of software and (Q)SAR models used• Prediction classification criteria, such as the cutoff or threshold values to

define a positive/negative/equivocal result

▪ Results and Conclusions• Individual predictions, as well as the overall conclusion (impurity class)• Confirmation that the impurity is within the model’s domain of applicability• Description of any confirmatory application of expert knowledge, including

analogs (where appropriate)• Rationale for superseding any prediction

▪ Appendix• Raw (Q)SAR outputs• Ames data for structurally related compounds used to confirm or refute a

prediction

30

Commonly Used Report Format

+ = positive; - = negative; Eqv = equivocal; OOD = out-of-domainwww.fda.gov

31

0

100

200

300

400

500

600

700

800

900

1000

2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

# o

f co

nsu

lts/

chem

ical

s

# (Q)SAR Consults # Chemicals Analyzed

CTCS (Q)SAR Consult Statistics

▪ 32% of consults include Applicant-submitted data

▪ 75% of consults for generic drugs

▪ Average of 17 chemicals evaluated per week

▪ Turnaround time of 10 business days

880 chemicals

297 consults

ICH M7 Finalized

(Q)SAR Consults, 2009-2018

32

Concluding Remarks

▪ Application of expert knowledge is an important component of (Q)SAR assessment under ICH M7

• Model transparency and interpretability facilitate application of expert knowledge

• Effective structural analog searching is critical

• Expert review of predictions is standard practice at FDA/CDER

▪ Regulatory (Q)SAR submissions still vary significantly in quality.Areas for improvement:

• Use of appropriate models (expert rule-based and statistical-based) that are consistent with OECD validation principles— May need to provide supporting documentation

• Appropriate handling of out-of-domain results• Adequate documentation of assessments, particularly if model

predictions are overruled based on expert knowledgewww.fda.gov

33

Concluding Remarks

▪ Internal process improvements enable CTCS to handle a large volume of (Q)SAR consultation requests

• Dedicated team of (Q)SAR experts

• Close communication and collaboration with review staff to ensure needs are met

• Robust chemical registration system

• Integration of (Q)SAR consults into review management platform

▪ External collaboration and outreach ensure access to state-of-the-art models and databases• Conduit for interacting with pharmaceutical stakeholders to share

knowledge and experiences

• Participation in industry consortia advancing the science of (Q)SAR modeling

• Identifies opportunities for future projectswww.fda.gov

34

Support:

• Critical Path Initiative• ORISE

• RCA partners

Acknowledgements

Govindaraj KumaranChris EllisLidiya StavitskayaBenon MugabeBecca RaczCurran LandryNeil HartmanMarlene KimLauren WoodardSuresh JayasekaraJian Yang[Andy Zych][Keith Burkhart]

www.fda.gov