ICH M7 - Lhasa Limited

ICH M7

CEO

[email protected]

Chris Barber

Brazilia, May 2018

Agenda

• The ICH M7 Guidelines

• The Process

• Negative Predictions

• Managing out of domains

• Using Strain Profiles

• Some Examples

The ICH M7 Guidelines

ICH

M7

Supports the use of in silico models

in decision-making

Adopted worldwide

Enables fast, safe decision-making

http://www.ich.org/products/guidelines/multidisciplinary/article/multidisciplinary-guidelines.html

1 Feb2018

14 March 2018



• Control measures can be determined by considering• mutagenic potential




• Control measures can be determined by considering• mutagenic potential

• concentrations of impurities


Option Control Process

1 [impurity] < specification in final drug product

2 [impurity] < specification in raw material, starting material or intermediate

3

[impurity] > specification in raw material, starting material or intermediate

+ fate and purge during subsequent steps ⇒ [impurity] in the final drug substance is

below the acceptable limit

4Sufficient knowledge of fate and purge to give confidence that final [impurity] <

acceptable limit without analytical testing


Decision-making under ICH M7

Database

searching

ICH

M7

Expert in silico

prediction

Statistical in

silico prediction

Expert

assessment

Classification

Reporting

Purge mitigation

Control

Impurity

identification

Test

Storing

Undertaking a Database Search

Public data

Pre-competitive

shared data

In-house dataClass 1

Mutagen

Carcinogen

Mutagen

Carcinogen ?

Class 2

Non-Mutagen

Class 5

Exact

match

Databases

Substructure /

similarity search Further supporting information for

expert review

Undertaking a Database Search

Public data

Pre-competitive

data

Sources include:

• FDA

• NTP

• ISSSTY

• Kirkland

• Hansen, Bursi

• MPDB

• Literature

In vitro genetox

• 185,018 | 10,602

In vivo genetox

• 12,488 | 2,995

Overall-call genetox

• 35,092 | 10,932

Carcinogenicity

• 16,398 | 3,866

Aromatic amines

• 9,402 | 666

Intermediates

• 21,507 | 1,338

Excipients

• 2,985 | 1,071

Consortia of Lhasa

members

In-house data

Freely available on-line resourcehttps://www.lhasalimited.org/products/lhasa-carcinogenicity-

database.htm

Long term carc studies

• 6,529 | 1529

https://www.lhasalimited.org/products/lhasa-carcinogenicity-database.htm

Expert Review Using in silico Models

Database

searching

ICH

M7

Classification

Reporting

Purge mitigation

Control

Impurity

identification

Test

Storing

Expert in silico

prediction

Statistical in

silico prediction

Expert

assessment


• Guidance documentation is readily available

• Establishing best practise in the application of expert review of

mutagenicity under ICH M7.

• Barber… Regul. Toxicol. Pharmacol. 2015, 73, 367

• Principles and procedures for implementation of ICH M7

recommended (Q)SAR analyses.

• Amberg… Regul. Toxicol. Pharmacol. 2016, 77, 13

• Lhasa’s website contains publications, presentations, examples...

• https://www.lhasalimited.org/ich-m7.htm

• Regulator’s website

• https://www20.anvisa.gov.br/coifaeng/calculos.html

https://www.lhasalimited.org/ich-m7.htm

https://www20.anvisa.gov.br/coifaeng/calculos.html


• “The absence of structural alerts from two complementary

(Q)SAR methodologies (expert rule-based and statistical) is

sufficient to conclude that the impurity is of no mutagenic

concern”

• “If warranted, the outcome of any computer system-based

analysis can be reviewed with the use of expert knowledge in

order to provide additional supportive evidence on relevance of

any positive, negative, conflicting or inconclusive prediction

and provide a rationale to support the final conclusion”



• A model should meet the OECD principles• a defined endpoint;

• an unambiguous algorithm;

• a defined domain of applicability;

• appropriate measures of goodness-of-fit, robustness and predictivity;

• a mechanistic interpretation, if possible.”

• A model must also support expert review• Make predictions and have good accuracy for your chemical space

• Provide some meaningful measure of confidence

• Be regularly updated with new knowledge

• Ideally be well understood by regulatory authorities

• Be transparent and highlight areas of uncertainty

• Make predictions that you understand, support or can overturn

Expert Review – Model Selection

Distinguishing between expert and statistical systems for application under ICH M7.

Barber… Regul. Toxicol. Pharmacol. 2017, 84, 124

Derek – an Expert System

1 Defined endpoint

2 Unambiguous algorithm

3 Defined applicability domain

4 Performance measure

5 mechanistic interpretation

Derek – Positive Predictions

CFSANn=1535

100 98

80

49

0

25

50

75

100

Certain Probable Plausible Equivocal

Accuracy of predictions

Assessing confidence in predictions made by knowledge-based systems. Judson… Toxicol. Res., 2013, 2, 70

Distinguishing between expert and statistical systems for application under ICH M7. Barber…Regul. Toxicol. Pharmacol. 2017, 84, 124

• Highlights the alert and gives a level of confidence

Derek – Negative Predictions

• “I have no concerns, there is nothing new for me here..

..I’ve seen these all features before in negative compounds”

• “There is nothing new for me here..

..but there is a feature that was in a compound I falsely predicted

as negative”

• “There is a feature I haven’t seen before

..you might want to check that out!”

If you asked a human expert the absence of a positive prediction..

More confidentAmount of

expert review


?

Is it alerting?

Contains unknown

features?

Contains features in

known false negatives?

InactiveInactive with

misclassified features

Inactive with

unclassified features

N

N N YY

It’s difficult, but important, to make negative predictions. Williams… Reg. Tox. and Pharmacol. 2016, 76, 79

YPositive prediction

Derek – Negative (unclassified feature)

• Highlights fragments in contexts not seen in public data

• May be present in confidential data

• Often driven by unusual ring systems

• Expert review recommended

Derek – Negative (misclassified feature)

• Highlights fragments seen in false positive predictions

• This is still a negative prediction

• It can be triggered for many reasons

• Inconsistent data• Will flag if only seen once (and many negative tests are seen)

• An innocent spectator

• A missing alert

• Expert review recommended

• Search for additional supporting compounds containing fragment

• Review conflicting results• Higher hurdle to dismiss positive results but you can

• Non-standard assay, impure sample…

Performance against 3 proprietary data sets – frequency (negative predicitivity)


?

Is it alerting?

Contains unknown

features?

Contains features in

known false negatives?

InactiveInactive with

misclassified features

Inactive with

unclassified features

N

N N YY

It’s difficult, but important, to make negative predictions. Williams… Reg. Tox. and Pharmacol. 2016, 76, 79

YPositive prediction

86-90%

(87-94%)

7-9%

(86-93%)

3-5%

(86-95%)

Using Predictions from Expert Systems

• Positive and negative predictions are not sufficient

• OCED principles are not sufficient

• Expert review also requires:

• Levels of confidence

• Where to focus attention

• Supporting examples

• Peer-reviewed expert commentaries

• Supporting literature references

Positive

plausible

probable

certain

Equivocal

Negative

..no mis/unclassified

..with misclassified

..with unclassified


Likely to conclude positiveVery strong evidence would be needed to overturn both

predictions

UncertainLikely to conclude positive without strong evidence to

overturn a positive prediction

Likely to conclude positiveLack of a second prediction

suggests insufficient evidence to draw any other

conclusion

In silicoprediction 1

In silicoprediction 2

Positive

Positive

Positive

O.O.D. or equivocal

Positive

Negative

Negative

O.O.D. or equivocal

Negative

Negative

UncertainConservatively could assign as positive.

May conclude negative with strong evidence showing feature driving a ‘no prediction’ is

present in the same context in known negative examples (without deactivating features)

Likely to conclude negativeExpert review should support this conclusion – e.g. by assessing any

concerning features (misclassified, unclassified, potentially reactive...)

O.O.D. = out of domain

Establishing best practise in the application of expert review of mutagenicity under ICH M7 Reg. Tox. and Pharmacol. 2015, 73, 367

Expert Review

Harder

Easiest

Expert

Review

Low confidence

Mis- or unclassified

Poor coverage of important

fragments

Own knowledge /

proprietary data disagrees

Close relevant examples

agree

Good coverage of

fragments

Both predictions agree

Fits own knowledge /

proprietary data

Conflicting predictions

Equivocal or out of domain

Expert Review – Conflicting Results

O.O.D. 2 0 3

+ 10 0 17

Equiv 8 0 6

- 38 2 12

- Equiv +

• Lhasa data sharing consortium group (n=777; 32% positive)

Frequency of Outcomes (%)

O.O.D. 29 - 62

+ 30 - 70

Equiv 23 - 51

- 8 15 34

- Equiv +

Probability of being positive (%)

Managing out of domains

https://www.lhasalimited.org/publications/dealing-with-out-of-domain-qsar-predictions-for-ich-m7-a-

regulatory-and-industrial-perspective/4476 (Kruhlak , Sept 2017)

Slides reproduced with permission of the author

https://www.lhasalimited.org/publications/dealing-with-out-of-domain-qsar-predictions-for-ich-m7-a-regulatory-and-industrial-perspective/4476









Using Strain Profiles in Expert Review

• Ames test uses different strains with different sensitivities

• A negative result not tested in 5 strains may miss something

• Strain-specific models perform badly• Too many data gaps

• Too many different strains used

• You should know if a negative result for an analogue was

tested in the most appropriate strain• Particularly if this is a key piece of data for expert review

• Hypothesis-level strain information• Which strain is most sensitive for chemicals in this class?

• How comprehensive is the data for supporting negative ex’s?

• Compound-level strain information• Which strains was this compound tested in?

• Is it negative because it is missing a key strain?

Using Strain Profiles in Expert Review

Agenda

• The ICH M7 Guidelines

• The Process

• Negative Predictions

• Managing out of domains

• Using Strain Profiles

• Some Examples

Example 1

• Conflicting predictions from the expert and statistical

systems

Example 1

• Well supported alert• No reason to immediately dismiss a positive prediction

Example 1

• Positive hypothesis for the epoxide was overwhelmed by others• Close examples are also positive

Example 1

• Model’s closest compound was not tested in

5-strains or in the presence of S9

Example 1

Mutagenic

Equivocal

Pro-mutagenic

arguments

Non-mutagenic

arguments

Non-mutagenic

Key hypothesis (epoxide) is positive

and this is supported by many

relevant examples that give good

coverage of the structure Key negative compound was

only tested in 2 strains w/out

metabolic activation

Example 2

• One equivocal and one weakly positive prediction..• You could ‘conservatively’ treat as a positive prediction

but…

Example 2

• Derek specifically notes that positive results are not

driven by the acid chloride but by the solvent• Literature references are included…

Example 2

• Sarah makes a weakly positive prediction• Insufficient data for own hypothesis – not all examples relevant

• Links to the source data allows review of solvents

acetone DMSOTHF

DMSO N/A

Example 2

Mutagenic

Equivocal

Pro-mutagenic

arguments

Non-mutagenic

arguments

Non-mutagenic

Half the relevant

examples are inactive

supporting DX

comments & publ’n

Alert tells us that

activity is driven by

reaction with solvent

and not by ROCl

Example 3


systems

Example 3

• Clear negative prediction

..with no mis- or unclassified features

Example 3

Positive 1/6 times in TA100

Positive 1/1 in D3052

(1984)

1 positive reference

(no experimental details

or strain information..)

• Positive prediction overturned by expert• Other reasons for activity or weak positive evidence

Example 3

Mutagenic

Equivocal

Pro-mutagenic

arguments

Non-mutagenic

arguments

Non-mutagenic

Positive examples have

more plausible reasons

for activity or have weak

evidence

Example 4


systems

Example 4

• Specifically includes sulphinates and includes references

• Describes mechanism

Example 4

• No specific hypothesis for this functional group

• Insufficient relevant examples to accept the prediction

Example 4

Mutagenic

Equivocal

Pro-mutagenic

arguments

Non-mutagenic

arguments

Non-mutagenic

Supporting

examples not

that relevantNo hypothesis

of alkyl

sulphinate

Summary of expert review

• Good software will help you reach a conclusion• You need to be able to convince yourself!

Conflicted predictions

• use of strain information

Two weak positives were overturned

• expert system commentary + key ref’s

• links to original underlying data


• ability to look for other causes of

activity in statistical system

• review of underlying data (of

remaining positives)


• expert system commentary

• assessment of supporting

compounds (statistical system)

Tools to support ICH M7 assessments

Database

searching

ICH

M7

Expert in silico

prediction

Statistical in

silico prediction

Expert

assessment

Classification

Reporting

Purge mitigation

Control

Impurity

identification

Test

Storing

Head office in Leeds, UKNot-for-profit

Educational charity

A membership organisation

Data & knowledge sharing

Honest broker

Sponsor

PhDs

Teaching

lecturesWork

experienceAcademic

Agrochem

Biotech

Chemical

CROCosmetic

Generics

Personal Products

Pharma

Government

Tobacco

MIP-DILI

Regulators

(members & collaborators)

Proprietary data mining

Predictive software

(expert & machine learnt)

Purge

Toxicity Metabolism

Degradation

Undergrad

projects

Publications &

presentations

Thank you

Documents

ICH M7 - Lhasa Limited