Regulatory Toxicology and Pharmacology · approach for a chemical risk assessment, in a manner analogous to the use of MoA in the ILSI KEDRF and HESI Q-KEDRF (Julien et al., 2009;

Regulatory Toxicology and Pharmacology 71 (2015) 463–477

Contents lists available at ScienceDirect

Regulatory Toxicology and Pharmacology

journal homepage: www.elsevier .com/locate /yr tph

Proposing a scientific confidence framework to help support theapplication of adverse outcome pathways for regulatory purposes

http://dx.doi.org/10.1016/j.yrtph.2015.02.0110273-2300/� 2015 The Authors. Published by Elsevier Inc.This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

⇑ Corresponding author at: EPA, Office of Research and Development, NationalCenter for Computational Toxicology, 109 T W Alexander Dr, RTP, NC 27711, USA.

E-mail address: [email protected] (G. Patlewicz).

Grace Patlewicz a,⇑, Ted W. Simon b, J. Craig Rowlands c, Robert A. Budinsky c, Richard A. Becker d

a DuPont Haskell Global Centers for Health and Environmental Sciences, 1090 Elkton Road, Newark, DE 19711, USAb Ted Simon LLC, 4184 Johnston Road, Winston, GA 30187, USAc The Dow Chemical Company, Toxicology & Environmental Research & Consulting, 1803 Building Washington Street, Midland, MI 48674, USAd Regulatory and Technical Affairs Department, American Chemistry Council (ACC), Washington, DC 20002, USA

a r t i c l e i n f o a b s t r a c t

Article history:Received 29 August 2014Available online 20 February 2015

Keywords:Scientific confidence framework (SCF)Adverse outcome pathway (AOP)Mode of action (MoA)Integrated approaches to testing andassessment (IATA)(Q)SARRead-acrossExposure:activity ratio (EAR)

An adverse outcome pathway (AOP) describes the causal linkage between initial molecular events and anadverse outcome at individual or population levels. Whilst there has been considerable momentum inAOP development, far less attention has been paid to how AOPs might be practically applied for differentregulatory purposes. This paper proposes a scientific confidence framework (SCF) for evaluating andapplying a given AOP for different regulatory purposes ranging from prioritizing chemicals for furtherevaluation, to hazard prediction, and ultimately, risk assessment. The framework is illustrated using threedifferent AOPs for several typical regulatory applications. The AOPs chosen are ones that have beenrecently developed and/or published, namely those for estrogenic effects, skin sensitisation, and rodentliver tumor promotion. The examples confirm how critical the data-richness of an AOP is for drivingits practical application. In terms of performing risk assessment, human dosimetry methods are neces-sary to inform meaningful comparisons with human exposures; dosimetry is applied to effect levelsbased on non-testing approaches and in vitro data. Such a comparison is presented in the form of an expo-sure:activity ratio (EAR) to interpret biological activity in the context of exposure and to provide a basisfor product stewardship and regulatory decision making.� 2015 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Societal demands for safer and more sustainable chemicalproducts are stimulating changes in toxicity testing and assess-ment frameworks. Chemical safety assessments are expected tobe conducted faster and with fewer animals, and at the same time,the number of chemicals that require assessment is also rising withthe number of different regulatory programmes increasing world-wide. These considerations have stimulated a shift in thinkingabout how toxicity testing and their evaluations need be conduct-ed in the future-moving away from extensive toxicity testing basedon phenotypic responses in animals towards pathway approachesbased on (quantitative) structure–activity relationships ((Q)SAR),toxicokinetics, physiological mechanisms and dose-dependent bio-logical changes underlying toxicity in exposed organisms. Since

‘‘safety,’’ by definition, includes both the inherent hazards of thesubstances that make up a product and exposures that occur as aresult of use of the product, improvements are needed in bothapproaches for evaluating intrinsic hazards and approaches fordetermining exposures. These visions were articulated to a largeextent in the 2007 NRC report ‘‘Toxicity Testing in the 21st Centu-ry: A Vision and a Strategy’’ (NRC, 2007) and the 2012 NRC report‘‘Exposure Science in the 21st Century: A Vision and A Strategy’’(NRC, 2012; Cohen Hubal et al., 2010).

A move towards more mechanistically based risk assessmentsimplies with it the use of in vitro tests, including high throughputand high content (HT/HC) screening methods, coupled with theapplication of a range of computational methods for data analysisand predictive modeling. Thus achieving the visions of Tox21 andEXPO21 relies on 4 key components:

� The generation of in vitro data.� The derivation of models from these biological activity assays

that predict downstream biological responses of toxicologicalrelevance.

http://crossmark.crossref.org/dialog/?doi=10.1016/j.yrtph.2015.02.011&domain=pdf

http://creativecommons.org/licenses/by-nc-nd/4.0/

http://dx.doi.org/10.1016/j.yrtph.2015.02.011

http://creativecommons.org/licenses/by-nc-nd/4.0/

mailto:[email protected]


http://www.sciencedirect.com/science/journal/02732300

http://www.elsevier.com/locate/yrtph

464 G. Patlewicz et al. / Regulatory Toxicology and Pharmacology 71 (2015) 463–477

� Exposure modeling to relate predicted downstream biologicalresponses of toxicological relevance to exposures from uses ofchemical products.� A tiered framework for proceeding to more complex assessment

procedures when greater precision is warranted to support aspecific regulatory or product stewardship decision.

A key, overarching component is a biological construct forappropriate interpretation of these data so that prediction modelscan guide regulatory uses and decision making. An adverse out-come pathway (AOP) could serve as such a construct.

2. Adverse outcome pathway

An AOP is a sequence of events from the first critical molecularevent (known as the molecular initiating event or MIE) to an in vivoadverse outcome (AO) (Ankley et al., 2010). Although the molecu-lar initiating event has been defined as the first key event (KE) inthe AOP causally linked to an adverse outcome, in practice theMIE is being used to characterize the first molecular interactionwhich itself might not be causal. The term ‘‘initial molecular event’’(IME) was coined by Patlewicz et al. (2013a) to replace the molecu-lar initiating event in an effort to represent this important distinc-tion1. Subsequent to the molecular initiating event, additional keyevents will contribute to and culminate in the occurrence of theadverse outcome. The Organization for Economic Co-operation andDevelopment (OECD) has developed guidance on developing andassessing AOPs that is in alignment with guidance from the WorldHealth Organization (WHO), International Program on ChemicalSafety (WHO-IPCS) and ILSI Health and Environmental Science Insti-tute (HESI) on mode of action (MoA), Human Relevance (HRF) andKey Event Dose Response (KEDRF) (Julien et al., 2009; Meek et al.,2003; Meek, 2008; Meek et al., 2014a,b; OECD, 2013).

OECD’s work programme for developing AOPs stemmed fromthe desire to enhance read-across within chemical categories. AOPsshould thus facilitate the transition from categories that have beenlargely structurally based to categories that are informed by theinclusion of additional biological information (van Leeuwen et al.,2009). In 2010, the OECD held a workshop entitled ‘‘UsingMechanistic Information in Forming Chemical Categories.’’ Thisworkshop discussed the types of activities that could form thebasis of an OECD AOP work programme including the developmentof a library of AOPs and MIEs which could subsequently be includ-ed in the OECD QSAR Toolbox (OECD, 2011). A complementary dri-ver was the 7th Amendment to the Cosmetics Directive whichestablished a ban on animal testing for repeated dose toxicity end-points for cosmetics by 2013 (EC, 2009; Hartung et al., 2011).Indeed an ongoing joint research effort between Cosmetics Europeand the European Commission known as SEURAT-1 is investigatingapproaches to replace the types of repeated dose toxicity testingthat would be necessary to assure the safety of cosmetic sub-stances by exploiting an AOP framework (http://www.seurat-1.eu/).

Whilst there is a wealth of activity on the development of AOPsin particular within the OECD programme, far less attention hasbeen placed on their evaluation and practical application in aregulatory context. The purpose of this paper is to propose a scien-tific confidence framework (SCF) to outline the types of consid-erations pertinent when applying and evaluating AOPs fordifferent regulatory purposes and to highlight its utility with afew illustrative examples. The SCF incorporates established think-ing regarding Mode of Action (MoA), the notion of ‘‘fit-for-purpose’’

1 Drewe et al. (2014) coined the term pre-MIE as an alternative to IME to make thesame distinction. This was in an effort to ensure that MIEs that were not truly causallylinked were not being used as direct predictors of the adverse outcome.

as a necessary aspect of problem formulation, and the need to con-sider human dosimetry (Becker et al., 2012, 2014b,c).

3. Challenges in applying AOPs in regulatory decision making: aframework to document scientific confidence

The OECD AOP work programme foresees AOPs as addressingseveral different regulatory purposes. These include (1) develop-ment of chemical categories based on biological responses (2)informing test method refinement/development and (3) develop-ing integrated approaches to testing and assessment (IATA) forhazard and risk assessment. Although not explicitly stated in theOECD work programme, AOPs can also be used for prioritizationpurposes, which may be viewed as a distinct application stemmingfrom chemical categorization based on biological responses. Inaddition, an AOP can be used as the central organizing conceptualapproach for a chemical risk assessment, in a manner analogous tothe use of MoA in the ILSI KEDRF and HESI Q-KEDRF (Julien et al.,2009; Simon et al., 2014).

The current OECD AOP work programme falls under the direc-tion of the Extended Advisory Group for Molecular Screening andToxicogenomics (EAGMST), and is focused on the development ofAOPs, associated guidance and knowledge management tools suchas the AOP Wiki. Although there is a workflow described to out-line the steps of AOP development, the endorsement and regula-tory application, as noted by Vinken (2013), has not yet beenconsidered in any great detail by the OECD. In an idealized case,an AOP would include a description of all key events, delineationof methods which can be used to measure each key event,descriptions of each key event relationship (KER), and quantitativemodels for each KER to permit statistical prediction of a down-stream key event from an upstream key event. If all of this infor-mation were available, quantitative predictions of the adverseoutcome (AO) could be made from an upstream key event. How-ever, for almost all AOPs, our current state of understanding doesnot allow for a quantitative prediction of a downstream key eventor the ultimate adverse outcome from an upstream key event.Typically, quantitative prediction models are lacking, and thuspredicting quantitative hazards falls short of achieving the desireddegree of scientific confidence. Therefore, the use of AOPs toquantitatively predict human toxicity or risks may not becomeroutine for some time to come. Nonetheless, depending uponthe degree of understanding, AOPs can still be practically usedin a number of ways for regulatory purposes. The extent to whichan AOP can be used in any of the applications delineated abovewill depend on the maturity or completeness of the AOP itself.The application of a given AOP to a specific regulatory challengewill depend in a large part on how the scientific basis of theAOP has been justified and documented.

Cox et al. (2014) put forward a scientific confidence frameworkdesigned to aid in the development, evaluation and communica-tion of the scientific confidence in Tox21 assays and their predic-tion models. Specifically the framework was designed as a meansof documenting the performance and robustness of assays andtheir prediction models within the context of a biological pathwaythat culminated in an adverse effect (i.e., an AOP) and was aimed ata given regulatory purpose, whether it be for priority setting, read-across, screening level hazard identification, etc. The frameworkwas derived using the OECD QSAR validation principles (OECD,2004) and the Institute of Medicine (IOM) biomarkers guidance(IOM, 2010). It is composed of three inter-related core elements,(1) analytical validation, (2) qualification and (3) utilization whichcan be readily adapted for AOPs. These three core elements havebeen integrated in an extended scientific confidence frameworkfor AOPs in a stepwise manner (Table 1).

http://www.seurat-1.eu/

http://www.seurat-1.eu/

Table 1Scientific confidence framework adapted for AOPs.

Step Scientificconfidence element

Description

1 AOP development Develop the AOP2 Mapping assays Develop new (or map existing) specific assays

to demonstrate key events within the AOP3 Analytical

validation of assaysConduct (or document) analytical validation ofeach assay. (Assessment of the biological basisand analytical performance of assays. Eachassay (including HT/HC screening assays)should map to a defined mechanistic endpoint(e.g., the intermediate or key event in the modeof action or AOP). A defined chemical domain ofapplicability and documentation of assayperformance characteristics (reliability,sensitivity, and specificity) and transparent datasets (to enable independent verification) shouldbe readily available)

4 Prediction modeldevelopment

Develop new (or map existing) models thatpredict a specific key event from one or moreprecursor key events. (The input data for theprediction models comes from the assaysdescribed in Steps 2 and 3 above)

5 Qualification ofprediction models

Conduct (or document) qualification of theprediction models. (Assessment of theprediction model derived from the (HT/HC)screening assays. A defined algorithm for eachprediction model is needed to ensuretransparency. Appropriate measures ofgoodness-of-fit, robustness and predictivity ofthe prediction model need to be presented.Some prediction models may be quantitative,others may be qualitative. Known limitations ofeach prediction model should also besummarized. Prediction models should becharacterized in sufficient detail to facilitatereview, reconstruction and independentverification of results)

6 Utilization forspecific purposes

Utilization: defining and documenting wherethere is sufficient scientific confidence to useone or more AOP-based prediction models for aspecific purpose (e.g., priority-setting, chemicalcategory formation, integrated testingstrategies (ITS), predicting in vivo responses,etc.)

7 Regulatoryacceptance

For regulatory acceptance and use, processesneed to be agreed upon and utilized to ensurerobust and transparent review anddetermination of fit-for-purpose uses of AOPs.This should include dissemination of allnecessary datasets, model parameters,algorithms, etc., to enable stakeholder reviewand comment, fully independent verificationand independent scientific peer review. Whilstthese processes have yet to be defined globally,in time, these should evolve to enable credibleand transparent use of AOPs with sufficientscientific confidence by all stakeholders

G. Patlewicz et al. / Regulatory Toxicology and Pharmacology 71 (2015) 463–477 465

Utilization as outlined in step 6 addresses the question of ‘‘whatthe degree of confidence is in predicting the adverse outcomewithin the AOP and whether this is sufficient to support the useof the AOP for a given purpose e.g., prioritization; read-across; haz-ard identification; etc.?’’ In order to arrive at such a confidencedetermination for an AOP, the OECD AOP Handbook guidance onconducting a Weight of Evidence (WoE) evaluation on an AOPcan be used (published in September 2014 on the AOP Wiki). Theprocedures in this guidance entail completing a template, usingevolved Bradford Hill (BH) considerations, in which each key eventand key event relationship within the AOP are evaluated andqualitatively scored as High (H). Moderate (M) or Low (L). TheWoE determinations help address the degree of scientific confi-dence for using the AOP for a defined purpose. In other words,

the confidence underpinning the use of an AOP for a specific appli-cation will be commensurate with the ability of the user to providesufficient scientific justification that the knowledge of key eventsand their relationships is such that a qualified and appropriatelyknowledgeable scientific reviewer would concur that the definedapplication proposed is scientifically supported. Thus, the users’justification should include a rationale explaining whether a quan-titative use of an AOP is justified, or if not, whether a qualitative useis supported.

If a specific key event or key event relationship is judged, in theWoE determination, to be weak, then one is likely to have low con-fidence proceeding along the AOP pathway beyond that key eventor key event relationship to predict subsequent key events or theadverse outcome. The weak level of confidence represents a break-point in the causal pathway. The most influential consideration inassessing the WoE in the overall AOP for regulatory application isthe biological plausibility of each of the key event relationshipsin the AOP. The essentiality of the key events and the extent ofempirical support for each key event relationship are secondaryconsiderations.

Per the WoE determinations in Table 2, one would have insuffi-cient scientific confidence in quantitatively predicting KE2 fromeither the IME or KE1. Thus, one would not be able to proceed pastthis breakpoint in the causal pathway to quantitatively predict KE3or the AO from the IME or KE1 although, qualitative predictions ofthe AO may be possible.

In terms of application of an AOP for priority setting purposes,where the decision is to identify substances that will undergo amore extensive evaluation using more complex and precise testmethods, perhaps all that is needed is solid scientific confidencein the biological plausibility of the AOP and high confidence inthe assay(s) that measure the IME. Results of such assays that mea-sure activities of substances indicative of the IME or KE1 could beused quantitatively for prioritization purposes, for read-across orfor IATA development. For IATA purposes (which are discussed inmore detail in Section 3.4), one may be able to use the AOP as adecision tree, in which assessment proceeds sequentially startingwith the IME. If a positive response is seen, the key event outcomeof the substance is then predicted using the KER. If there is a break-point in the AOP causal chain, then the IATA scope will be limitedand new data would need to be generated.

From a reviewer’s perspective, it is insufficient for an applica-tion of an AOP to simply document its use; one needs to indicatethe rationale for the specific use and the scientific support under-pinning such a use. Limitations of use should also be noted. Sinceexperience in developing, applying and reviewing AOPs is current-ly very limited, it is important to use the OECD AOP WoE templateprocedures to document one’s understanding of the WoE of theAOP and each key event and key event relationship. These WoEdeterminations and subsequent use of these determinations forcommunicating the scientific foundation supporting use of anAOP for a particular application will form the basis for expandingthe knowledgebase of AOPs. This will foster scientific discussionsand consensus will emerge as to which applications are scien-tifically sound and which are more limited or speculative.

Thus the scientific confidence framework (SCF) proposed herecomplements the OECD guidance on AOPs. Whilst the OECD guid-ance describes the approach to develop an AOP including the keyevents and key event relationships, it is largely silent on document-ing confidence in (1) the analytical methods used to measure keyevents, (2) the prediction models (which are in essence, the keyevent relationships) and (3) the rationale and justification for deci-sions to use (or not use) an AOP for a given regulatory application. Itis recognized that different regulatory decisions will requirediffering levels of scientific confidence; which will in turn impactthe amount and type of data required in the development and

Table 2WoE determinations for a hypothetical AOP.

Key Event /

Key Event

Relationship

IME KER1 KE1 KER2 KE2 KER3 KE3 KER4 AO

WoE High High High Low Moderate Moderate High High High

High

Low

HighConfidence in quantitatively predicting KE1 from IME1 & KER1

Confidence in quantitatively predicting KE2 from KE1 & KER2

Confidence in quantitatively predicting KE3 from KE2 & KER3 Moderate

Confidence in quantitatively predicting AO from KE3 & KER4

Notes: IME – initial molecular event, KE – key event, KER – key event relationship, AO – adverse outcome, WoE – weight of evidence.

Fig. 1. Tradeoff between the confidence in the use of an AOP to quantitativelypredict the adverse outcome (AO) and the associated model uncertainties depend-ing on the type of regulatory decision.


application of an AOP. The trade-off between the scientific confi-dence needed to make a quantitative AO prediction and the under-lying uncertainty is represented conceptually (Fig. 1).

In the following sections, we aim to highlight the utility of thescientific confidence framework for several regulatory purposesusing three AOPs, namely those for estrogen mediated activity,skin sensitisation, and sustained AHR activation leading to rodentliver tumors. These AOPs are referenced in the AOP Wiki whichwas made public in September (see https://aopkb.org/aopwiki/in-dex.php/Main_Page). The Wiki contains a reasonable though notexhaustive listing of AOPs currently under development. It isimportant to note that these illustrative examples are intendedto highlight some of the regulatory purposes that could be poten-tially considered for these specific AOPs. We have not performed adetailed evaluation of each AOP using the framework proposed norconsidered the extent to which these AOPs could address otherpurposes. Some of the AOPs herein may serve other purposesbeyond what we have described. We also have not addressed theissue of how substances could be practically assessed by exploitinginformation arising from multiple AOPs. That is beyond the scopeof this manuscript. We do however discuss aspects of dosimetryin how data generated from early events in AOPs need to be inter-preted in the appropriate context for risk assessment purposes.

3.1. Prioritization

In prioritization, substances are screened to identify those withmeasureable biological activity, and those judged in such screens

to be positive could then be ranked ordered from least potent tomost potent and weighted as to their margin of exposure (MOE),with the most potent and/or smallest MOE having the highest pri-ority to undergo more detailed evaluation or testing. Typically,applying an AOP for prioritization would entail using a specificassay, or a battery of assays, and a prediction model which relatesthe observed results to a biological response that is a given keyevent in an AOP. As a minimum, the biological basis of the assayor assays should be understood to facilitate interpretation in thecontext of a pathway. For priority setting, the most probable sce-nario is to use assays that measure the initial molecular event.The prediction model could be as simple and straightforward as‘‘positive responses in these assays correspond to a critical early(or initial) key event in the pathway leading to the adverse out-come.’’ For priority setting purposes, it is not necessary to have aprediction model that quantitatively relates responses in an earlykey event to the probability of development of the adverse out-come. Whilst the assay(s) should be anchored to a key event thatis relevant for the adverse outcome of concern, a full understand-ing of the quantitative dose–response of each key event or quanti-tative relationships between all of the key events in the AOP is notneeded for this type of application. Of course, it is recognized thathaving dose–response information would allow for rank orderingof chemicals from lowest to highest priority.

3.1.1. Case study 1: endocrine activity AOP leading to reproductivedysfunction

A prioritization application is discussed in Table 3 where thefocus is on using biological activity indicative of an early key event,in this case from estrogen receptor (ER) binding and transactiva-tion assays, as the basis for setting priorities. Specifically estrogenreceptor binding (ERBA) and transactivation assays (ERTA) are tak-en as early key events in the AOP of adverse human health effectsassociated with exposure to estrogenic agents. Fig. 2a depicts Step2 (mapping assays to the AOP KEs) and Step 3 (analytical validationof assays). Fig. 2b illustrates prediction model development (Step4) and qualification (Step 5). The cross validation AOP predictionmodel uses measurements of molecular interactions (ER binding)and cellular responses (ER-mediated transcription) to predictin vivo uterotrophic responses (Rotroff et al., 2013; Cox et al.,2014). In this case example, the positive and negative predictivity(balanced accuracy) of the model was shown to be 85%. As noted inCox et al., 2014, Step 6 (utilization) requires addressing such ques-tions as: what is the acceptable range of predictivity of the model

https://aopkb.org/aopwiki/index.php/Main_Page

https://aopkb.org/aopwiki/index.php/Main_Page

Table 3Use of the scientific confidence framework for an endocrine activity AOP within thecontext of prioritization.

Step Scientific confidenceelement

Action required

1 AOP development For the estrogen-mediated adverse outcomepathway for human health evaluation, link theKEs commencing with molecular interactions,proceeding through cellular and organresponses and culminating with adverseeffects

2 Mapping assays Identify assays that are used in the EndocrineDisruption Screening Programme (EDSP) (seehttp://www.epa.gov/endo/) to evaluate eachof the KEs in the AOP

3 Analytical validation For the specific ERBA and ERTA assays,document performance. Such as the studiespublished by EPA as part of their validationeffort for the EDSP (see http://www.epa.gov/endo/)


For example, use of ToxCast™ ERBA and ERTAassays to predict the in vivo EDSP Tier 1uterotrophic response per Rotroff et al. (2013)and Cox et al. (2014)

5 Qualification of theprediction models

Document the performance of the model(s)that predict KEdownstream from KEupstream. Forexample, per Cox et al. (2014), evaluation ofthe positive and negative predictivity of themodel developed using ToxCast™ ERBA andERTA assays to predict the in vivo EDSP Tier 1uterotrophic response


Describe the WoE of the specific KEs ofestrogen receptor binding and geneexpression in the AOP and the rationalesupporting use for setting priorities for EDSPscreening. Use the WoE guidance in the OECDAOP Handbook (OECD, 2014a), to documentevaluation of estrogen receptor binding andgene expression KEs and KERs for biologicalplausibility, essentiality, empirical evidenceand uncertainties. Describe the level of confi-dence arrived at in the context of specificapplications. Address questions such as: whatis the acceptable range of predictivity of themodel for prioritizing substances for EDSP?For bypassing certain receptor-mediated EDSPER-based Tier 1 assays? For bypassing thein vivo EDSP uterotrophic assay?


Make available in publications, or insupplemental material (or by other means)the datasets, model parameters, algorithmsand utilization rationales, etc. (for discussion,see Cox et al., 2014 as an example)

EDSP = endocrine disruption screening programme, ERBA = estrogen receptorbinding assay, ERTA = estrogen receptor transactivation assay, KE = key event,KER = key event relationship.


for prioritizing substances for EDSP? For bypassing certain recep-tor-mediated EDSP ER-based Tier 1 assays? For bypassing thein vivo uterotrophic assay? It has been shown that the prioritysetting approach can be improved by integrating exposure infor-mation with biological activity (Wambaugh et al., 2013). Usingthe example outlined in Table 3, Becker et al. (2014b,c) have shownthat calculating exposure:activity ratios (EARs) for estrogenic che-micals and comparing these to the EAR for the phytoestrogengenistein can provide a refined context for priority setting forregulatory decision making.

3.2. Chemical categories and associated read-across

A chemical category is defined as a group of chemicals whosephysicochemical and human health and/or ecotoxicologicalproperties and/or environmental fate properties are likely to besimilar or follow a regular pattern, usually as a result of structural

similarity (OECD, 2007; ECHA, 2008). The similarities may be basedon the following:

� A common functional group (e.g., aldehyde, epoxide, ester,specific metal ion).� Common constituents or chemical classes, similar carbon range

numbers.� An incremental and constant change across the category (e.g., a

chain-length category).� The likelihood of common precursors and/or breakdown prod-

ucts, via physical or biological processes, which result in struc-turally similar chemicals (e.g., the metabolic pathway approachof examining related chemicals such as acid/ester/salt).

These ‘‘similarity rationales’’ may be considered as representingthe overarching hypothesis for grouping two or more chemicalstogether as part of an analogue or category approach. The next steprequires the grouping to be justified on the basis of general consid-erations such as bioavailability, reactivity, and metabolism, whichthemselves are informed through non-testing approaches such as(Quantitative) Structure Activity Relationships ((Q)SARs) as wellas experimental test data. Once the category has been nominallyestablished, aspects pertinent to different apical endpoints arethen considered to determine the scope of the category for fillingdata gaps for specific endpoints. For example, a category or ana-logue approach derived on the basis of a metabolic pathwaybetween a parent and its primary metabolite may be applicableto all mammalian systemic endpoints but will not necessarily beapplicable to local effects such as irritation or indeed for environ-mental fate properties (OECD, 2007; Patlewicz et al., 2013b).Populating the data matrix (a matrix of category members vs.available information per endpoint) assists in this evaluation toidentify what consistency exists across the category members fordifferent hazard endpoints. Data gap filling approaches such asread-across or trend analysis can then be used to predict the effectsof untested chemicals (OECD, 2007; ECETOC, 2012). With theincrease of chemical regulatory programmes such as REACH, thedesire to exploit read-across approaches as a means of addressingdata gaps has increased but acceptance has been somewhat lackingin practice, especially for complex endpoints such as repeated dosetoxicity (Ball et al., 2014). This may be in part due to the absence ofa systematic framework to identify and address read-across uncer-tainties (Patlewicz et al., 2014a). Indeed this might also be due tothe fact that read-across has been traditionally anchored with che-mical structural similarity without a meaningful way of integratingbiological similarity. Exploiting information from AOPs to enhanceread-across could help resolve some of these uncertainties asapproaches to characterize the different key events would providea means of demonstrating biological similarity (Patlewicz et al.,2013b, 2014a; Ball et al., 2014). If an AOP were available for sucha complex endpoint, read-across predictions for an in vivo endpointcould be potentially substantiated by generating in chemico orin vitro data to characterize the molecular initiating event and/orother downstream key events as necessary. Thus in terms of theinformation considerations, analytical validation of the assayscharacterizing specific key events and qualification of their predic-tion models would ideally be required to justify the similarityrationale as well as the subsequent read-across prediction withina chemical category, whereas a full quantitative understanding ofthe key event relationships to predict the adverse outcome wouldnot be necessary. A discussion on how AOPs can be used in the for-mation of categories and associated read-across is also described inthe recently revised OECD grouping guidance (OECD, 2014a). Aspart of the OECD AOP work programme, AOPs are envisaged tobe implemented into the OECD QSAR Toolbox to facilitate forma-tion of chemical categories and their associated read-across.

http://www.epa.gov/endo/



Fig. 2a. Mapping relevant assays that anchor to key events in the AOP for endocrine activity, specifically the estrogen receptor binding (ERBA) and transactivation assays(ERTA) as early key events, the in vivo uterotrophic assay as an intermediate event and the 2 generational reproductive toxicity to characterize the adverse outcome ofreproductive dysfunction.

Fig. 2b. Characterizing the performance of a prediction model that relies on ER binding and ER mediated transcription to predict in vivo uterotrophic responses. Notes: EDSP –EPA’s endocrine disruptor screening program.


3.2.1. Case study 2; skin sensitisation AOP as applied to chemicalcategorization and read-across

This case study was chosen for a number of reasons. Skin sensi-tisation is an endpoint that has been well studied over many yearsand is sufficiently simple to illustrate the AOP construct. There is awealth of data that has been already generated in alternative testmethods that lend themselves to the development of structuralprofilers that would be required for the AOP to be incorporatedinto the OECD Toolbox. The OECD Toolbox profilers encode rele-vant structural characteristics and/or potential modes of actioninto rules. The structure–activity information may be based aroundchemistry principles and relevant to many endpoints or relatedtowards particular toxicity endpoints. Given that all the key eventsleading up to the adverse outcome are characterized and well sup-ported experimentally, the AOP for skin sensitisation could be con-sidered mature, though semi-quantitative AOP in nature. In terms

of the SCF, the richness of this AOP renders it potentially applicablefor different regulatory purposes including read-across (Table 4).

As noted in Table 4, the MIE is defined as the covalent interac-tion of the electrophilic test substance and/or its metabolite(s)with nucleophilic skin proteins and the AO as allergic contact der-matitis (OECD, 2012a,b; Landsteiner and Jacobs, 1936; Godfrey andBaer, 1971; Dupuis and Benezra, 1982). Numerous efforts havefocused on identifying the electrophilic features in chemicals andrepresenting these as structural alerts. Other strategies haveinvolved developing QSAR approaches, for example the hybridexpert system TIMES-SS which incorporates structure–toxicityand structure-metabolism rules, some of which are underpinnedwith 3D QSARs, to provide a semi quantitative measure of skin sen-sitisation potency (Patlewicz et al., 2014b). Reaction mechanisticdomains based on organic chemistry principles have also been for-mulated to facilitate the grouping of chemicals for the purposes of

Table 4Testing/non-testing approaches for key events within the skin sensitization AOP.

Key event number Key event Level of biologicalorganization

Testing/non-testing approaches

Key event (KE) 1(MIE)

Covalent binding between electrophile andskin protein

Molecular level DRPA (Gerberick et al., 2004, 2007), GSH (Schultz et al., 2005),QSARs/read-across

KE2 Activation of inflammatory cytokines Cellular response KeratinoSens™ (Emter et al., 2010, 2013), read-acrossKE3 Maturation and mobilization of dendritic cells Cellular response MUSST (Python et al., 2007), h-CLAT, read-acrossKE4 T-cell proliferation Organ response LLNA (OECD TG 429), QMM, read-acrossAdverse outcome

(AO)Allergic contact dermatitis Organism response GPMT (OECD TG 406), HRIPT

DRPA = direct reactivity peptide assay, GSH = glutathione depletion assay; MUSST = myeloid U937 skin sensitization test; h-CLAT = human cell line activation test;LLNA = local lymph node assay; QMM = quantitative mechanistic model; GPMT = guinea pig maximization test; HRIPT = human repeat insult patch test.

Table 5Use of the scientific confidence framework within the context of employing the AOPfor skin sensitization for chemical categorization and read-across.

Step Scientific confidenceelement

Action required

1 AOP development Semi quantitative AOP has been published byOECD in 2012

2 Mapping assays Non-testing approaches and assays for severalof the KEs have been mapped to the AOP, e.g.,KeratinoSens™, DPRA, h-CLAT, LLNA, GPMT(see Table 4)

3 Analytical validation KeratinoSens™, DRPA and h-CLAT have beenformally validated by ECVAM. DRPA andKeratinoSens™ and h-CLAT exist as draftOECD test guidelines. The LLNA (OECD TG429) has been extensively validated withrespect to guinea pig and human data. Forchemical categorization/read-across purposes,extracting the structural features of chemicalsexperimentally measured in these assaysprovides a measure of the structural domainof applicability


Prediction models to interpret the outcomesof each of the assays and how these relate tothe AOP can be found in the primaryreferences of the specific test protocols

5 Qualification of theprediction models

Assessment of some of the prediction modelsinclude Reisinger et al. (2014), Urbisch et al.(2014), van der Veen et al. (2014)


The similarity rationale to justify an initialcategory can be based on the reactionchemical domains (Aptula et al., 2005) and/orstructural alerts. To substantiate the rationaleand to perform a read-across prediction,reactivity and hydrophobicity informationwould be needed. The former can be derivedfrom the DRPA or other kinetic studies e.g.,GSH. QSAR approaches may also be useful aspart of the WoE approach. If the read-across isbeing used for hazard characterization or riskassessment, and/or if the reactivity informa-tion is not sufficient, other KE data may begenerated from corresponding assays to in-crease confidence in the outcome


Notable datasets of the available assaysinclude Natsch et al. (2013). Other datasetshave been made available within the OECDToolbox


read-across or to derive quantitative mechanistic models (QMMs)(Aptula et al., 2005). Appropriate measures of reactivity and/orhydrophobicity can be determined to estimate a measure ofsensitisation potency either through using a derived QMM or byread-across for ‘‘similar’’ chemicals as determined by their reactiondomain (Roberts et al., 2008). In terms of the experimental meth-ods that exist to measure reactivity, the direct peptide reactivityassay (DPRA) (Gerberick et al., 2004) or the glutathione depletionassay (GSH), (Schultz et al., 2005) are examples. The DRPA providesa measure of inherent reactivity but not in a form that can belinked back to a rate constant, which itself has been shown to bepredictive of T-cell proliferation as measured in the LLNA and rep-resented as KE4 in the AOP (Roberts et al., 2008; Roberts andPatlewicz, 2014). This is perhaps why the percentage of peptidedepletion over a 24 h period as measured in the DRPA gives riseto a weak quantitative relationship with the EC3 from the LLNA,i.e., the DPRA results do not accurately represent the MIE quantita-tively (Roberts and Patlewicz, 2014). Several of the experimentaltest systems cited (DRPA, KeratinoSens™, h-CLAT) have beenvalidated by ECVAM and of these two, namely the KeratinoSens™and the DRPA, are now OECD test guidelines (OECD, 2015a,b). Theapplication of the scientific confidence framework to evaluate thesuitability of the skin sensitisation AOP and associated assays forapplication in read-across is examined (Table 5).

In brief, the similarity rationale to group chemicals to evaluatetheir sensitising ability is ideally based on mechanistic reactiondomains or structural alerts encoding electrophilic features. Tosubstantiate the hypothesis, information characterizing the MIEwould need to be generated. This could take the form of the DRPAor preferably other reactivity experimental studies which providekinetic information. Information from QSARs can be helpful asadditional evidence as part of a WoE approach. Reactivity informa-tion should be sufficient to substantiate the similarity on accountof the strong evidence supporting the essentiality of this MIE anddue to the many QMM/QSAR studies that have related the MIE tothe EC3 value (i.e., the effective concentration at a simulation indexof 3) as measured in the LLNA. For a read-across prediction, thelevel of confidence will be both context- and chemical-specific.Chemical-specific since, depending on the reaction domain, infor-mation on hydrophobicity as modeled by LogKow, in conjunctionwith reactivity data, will be needed to justify the prediction of sen-sitisation potential and/or potency. Context-dependent since thetypical assays currently available to characterize the KEs are onlyable to provide information to discriminate sensitisers from non-sensitisers. Depending on these 2 factors, information from otherdownstream KEs such as data from the h-CLAT or KeratinoSens™may be needed to increase the confidence of the read-across pre-diction for sensitisation potential. Alternatively existing in vivodata for one or more of the category members could provide suffi-cient information to support a read-across prediction of potency.The integration of these different sources of information starts to

transition into the area of IATA which is discussed in more detailin Section 3.4.

3.2.2. Case study 3: AHR rodent liver tumor AOPThe MoA for rodent liver tumor promotion by the aryl hydrocar-

bon receptor (AHR) presents challenges for chemical categorizationand associated read-across even though it is one of the moreexperimentally rich MoAs (Budinsky et al., 2014) (see Table 6).

Table 6AOP for the sustained AHR activation leading to rodent liver tumors.

Key event number Key event Level of biologicalorganization

Testing/non-testing approaches

Initial molecularevent (IME)

Short-term AHR activation Molecular level Endogenous gene expression, reporter gene expression,receptor binding assay, (Q)SAR/read-across

Key event (KE) 1(MIE)

Sustained AHR activation Molecular level Endogenous gene expression from sub-chronic in vivostudiesa

KE2 Changes in apoptosis, proliferation and cellularhomeostasis

Cellular response In vitro primary parenchymal and non-parenchymal cellstudies; in vivo histopathology and immunohistochemicalstaining from in vivo repeated dose studies and initiation-promotion studies

In vivo (initiation-promotion) and in vitro primaryhepatocyte evidence

KE3 Hepatopathy constellation of histopathological changes,hyperplasia

Organ response In vitro primary liver cell studies; in vivo histopathology andimmunohistochemical staining from in vivo repeated dosestudies"bBrdU-labeling, oval cells and bile duct hyperplasia

Adverse outcome(AO)

Liver tumors Organism response Rodent cancer bioassay"Hepatocellular adenomas, cholangiomas andcholangiolar carcinomas

a Phenotypic anchoring of genomic changes, pathways and networks to KE2 and KE3 are current data gaps requiring a better understanding of subchronic and chronicchanges in transcription, that are direct or indirect outcomes of sustained AHR activation, in both parenchymal and non-parenchymal cells.

b BrdU = 5-bromo-2-deoxyuridine.


AHR ligands display diverse chemical structures. However notall AHR ligands are likely to be rodent liver tumor promoters, espe-cially the naturally occurring short-acting, readily metabolizedligands of lower potency (NTP, 2014). In contrast, the potent andpersistent co-planar AHR ligands represented by specific polychlo-rinated dibenzo-p-dioxins, furans and PCBs are readily recognizedfor their rodent liver tumor promotion hazard based on their abil-ity of inducing sustained AHR activation and the MIE (Zhao et al.,2008; Petkov et al., 2010; Cao et al., 2013). The challenge forcategorization stems from realizing that the initial molecular inter-action, e.g., AHR binding and activation may not be sufficient tolink early AHR activation-induced changes to the AO of livertumors. Indeed a diverse range of naturally occurring and syntheticligands bind to and activate the AHR to alter the expression of hun-dreds of genes, some of which presumably promote the clonalexpansion of altered hepatic foci (reviewed in Denison et al.,2011; Nguyen and Bradfield, 2008; Forgac et al., 2012; Yao et al.,2012; Boverhof et al., 2006; Rowlands et al., 2011; Fletcher et al.,2005). However, it is unlikely that the large number of naturallyoccurring or short-acting AHR ligands act as tumor promoters. Fur-thermore, AHR activation has been reported to inhibit apoptosiswithin altered hepatic foci providing the initiated cells an opportu-nity to replicate and acquire additional mutations (Chopra andSchrenk, 2011). AHR activation affects both parenchymal andnon-parenchymal cells, in a zonal-dependent manner, withinvolvement in AHR-mediated liver tumor promotion. The KEinformation points to an important role for abnormal hepatic stemcell growth and differentiation, e.g., biliary fibrosis, oval cell hyper-plasia (Hailey et al., 2005; Sheikh-Bahaei et al., 2010; Andersenet al., 1997; Chang et al., 2005). In addition to the indirect stimula-tion of clonal expansion produced by intrafocal apoptosis inhibi-tion, sustained AHR activation and its accompanyinghistopathological changes produce a direct proliferative environ-ment to stimulate the clonal expansion of altered focal growth.These complex KE aspects of this AOP illustrate the challenges withprioritization and categorization of a substance based on limiteddata such as AHR binding and/or activation. The applicability ofread-across for prioritizing and/or categorizing AHR ligands islargely limited to the straightforward examples of obvious, copla-nar halogenated polyaromatic hydrocarbons. For example, a recentNTP cancer bioassay showed that indole-3-carbinol, a naturallyoccurring AHR ligand, failed to produce liver tumors as well asthe classic hepatopathy findings closely linked to TCDD-induced

liver tumors in female Sprague Dawley rats (NTP, 2014; Haileyet al., 2005).

3.3. Informing test method refinement and/or development

AOPs as noted by the OECD, can provide information relevant totest method development or refinement, e.g., the need for develop-ment of a new method, or refinement of an existing method, tomeasure a specific key event. The need for a new test method orimprovement of an existing test method may arise when an AOPdeveloper attempts to map existing specific assays to each keyevent within the AOP. Alternatively the need may arise when adeveloper or user evaluates the performance of an assay used tomeasure a key event. The need for new methods or refinementsmay arise when the performance of a key event relationship, orprediction model, based on an existing assay falls short becauseof issues with the key event measurement method e.g., technicallimitations such as lack of metabolic competence or water solubi-lity limits. In any of these cases, since the confidence in the AOPhinges upon the scientific robustness in the methods used to mea-sure key events, application of an AOP may be limited until suchtime as one or more test methods are improved. Drawing againfrom the EDSP, as an example, consider the in vitro ToxCast™ testmethods used to evaluate thyroid effects. Rotroff et al. (2013)demonstrated that the ToxCast™ assays could not be used to pre-dict the in vivo thyroid-mediated effects in EDSP assays. This wasconfirmed by Cox et al. (2014). Whilst the ToxCast™ thyroid assayscomprised thyroid hormone receptor binding and transcriptionalactivation assays, it was determined that binding to the thyroidhormone receptor was not the MoA arising from altered clearanceof thyroid hormones. Although ToxCast™ includes assays thatmeasure enzymatic activity relevant to thyroid hormone clearance,Rotroff et al. (2013), concluded there was a lack of specificity forthyroid-active chemicals in these ToxCast™ assays. A number oftargets exist where chemicals may interact to interfere with thy-roid hormone signaling. This can be at the level of central regula-tion in the hypothalamus, at the pituitary gland, the thyroidgland itself or in peripheral tissues such as the liver that affectsthyroid hormone clearance (Murk et al., 2013). Thus, an AOP forthyroid-mediated effects would benefit from the development ofnew in vitro methods to replace the current animal test systemsin use which measure the effects of chemicals upon thyroidhormone clearance.

Fig. 3. Illustration of a possible progression along an integrated approach to testingand assessment (IATA).


Another example is that of the AOP for skin sensitisation andthe fact that the typical assay presently advocated to characterizethe MIE is the DRPA. This assay has several shortcomings whichlimit its scope for potency prediction (Roberts and Patlewicz,2014). Whilst the DRPA detects inherent reactivity, it does notprovide any information on the kinetics such as a relative rate con-stant (Roberts et al., 2008). A modification could be made to theprotocol as demonstrated in Roberts and Natsch (2009) whichwould address this specific limitation. Peptide depletion valuescannot be derived with sufficient accuracy for substances with lim-ited solubility in the solvents as prescribed by the DPRA protocol,hence substances that are too hydrophobic cannot be adequatelymeasured. The DRPA also has no metabolic competence. It needsto be coupled with other test or non-testing approaches to identifythe potential biological transformations as a result of oxidation orother enzymatic processes. Using the SCF to evaluate the assay andidentify its scope and limitations is critical in ensuring its valid useand in proposing refinements, all of which can ultimately be usedto update the corresponding AOP.

The Sustained AHR activation liver tumor promotion AOP alsoreveals test method refinement and development needs. Forexample, how can in vitro test methods represent the Area-Un-der-the-Curve phenomenon for differentiating between an initialmolecular event and the MIE for AHR-induced rodent liver tumorpromotion? How can test methods be designed to represent thecomplicated interactions of multiple cells types locked into theprecise zonal geometry of the hepatic lobule (Andersen et al.,1997; Chang et al., 2005; Sheikh-Bahaei et al., 2010)? Since2,3,7,8-tetrachlorordibenzo-p-dioxin-induced (TCDD-induced)AHR activation gives rise to both hepatocellular and cholangiolartumors, stem cell differentiation and proliferation responsesappear to play a central role in this AOP. Therefore, one could pro-pose that HT/HC hepatocellular stem cell methods are a neededtool to further examine this AOP. How can test methods be devel-oped to predict the complex histopathology that is necessary forAHR-induced liver tumor promotion (Goodman and Sauer, 1992;Hailey et al., 2005)? For instance, are there hepatic cells lines thatmirror the glutathione S-transferase (GSTP+) focal cells that appearto be the target for AHR-induced promotion and are therepathways involving apoptosis that explain how these cells escapeprogrammed cell death? Recently, evidence involving inflamma-tion obtained with (Tumor necrosis factor alpha) TNFa knockoutmouse model has shown the importance of inflammatory cytoki-nes in driving the mouse liver tumor response to TCDD (Pandeet al., 2005; Kennedy et al., 2014). How could HT methods bedesigned to examine the important role of inflammatory cytokinesin liver tumor promotion? Will genomic pathways and networksbe identified with the AHR liver tumor promotion key events thatcan enable HT in vitro results to be used to predict an AHR ligand’sliver tumor potential? Currently, a number of research efforts toidentify genomic signatures predictive of liver tumor potentialhave been published but the recommended genomic patterns donot align with gene changes that have been reported for the AHR(Heise et al., 2012; Fielden et al., 2008; Fletcher et al., 2005;Boutros et al., 2011; Vezin et al., 2004). Until clear genomic signalswith pathways clearly linked to the key events required for AHR-mediated liver tumor promotion are established, for bothparenchymal and non-parachymal cells, the current collection ofgenomic responses are not suitable for prediction (hazard) butcould be used to establish dose–response relationships for biologi-cal activity and possible risk assessment.

3.4. Integrated approaches to testing and assessment

Integrated approaches to testing and assessment (IATA) arepragmatic approaches which exploit existing information (includ-

ing human data and exposure information), alternative approaches(such as in chemico, in vitro including HT/HC screening assays), andtailored testing strategies. Ideally future IATA will rely on resultsfrom appropriate combinations of non-testing approaches, inchemico, in vitro tests that target key events, along with an AOPthat could provide sufficient information for hazard and riskassessment purposes (Fig. 3).

Fig. 3 highlights the interplay between gathering all existinginformation and structuring what additional information mightneed to be generated to address a specific regulatory purpose fora given chemical or group of chemicals. Indeed, this idea to consid-er exposure information along with measures of toxicity in the ini-tial stages of a risk assessment was articulated in the HESI RISK 21roadmap, which can be easily illustrated through a Matrix tooldesigned to identify exposure and toxicity data gaps and determinethe most appropriate studies to address them (Pastoor et al., 2014).For IATA, the type and extent of data generated would focus on oneor more key events. The number of key events and their associatedtest systems will drive the development of testing strategies tofacilitate the collection and interpretation of test outcomes. IATAhave been in widespread use for some time; indeed REACH articu-lates endpoint-specific integrated testing strategies (ITS) in itstechnical guidance for its information requirements (ECHA,2008). Aligning IATA with AOPs represents a step change in termsof IATA development in that it ensures specific testing is directedto what is most relevant for the decision in mind, for the chemi-cal(s) of interest, and which would be readily interpretable fromthe biological context.

The complexity of an IATA will depend on the completeness ofan AOP, the decision context, and the chemical(s) under consid-eration. Again the SCF guides the utility of an AOP for this purposeby evaluating the assays underpinning the KEs and their predictionmodels, and determining what combination of assays or non-test-ing approaches lend themselves to IATA development and for whattype of regulatory decision. The role AOPs play in informing thedevelopment of IATA for different regulatory purposes was dis-cussed in detail at the Workshop entitled ‘‘Advancing AOPs forIntegrated Toxicology and Regulatory Applications’’ in March2014 and the insights were summarized in Tollefsen et al. (2014).

3.4.1. Case study 2; skin sensitisation AOP as applied to IATAThere have been several efforts to consider how to combine in

silico, in chemico, in vitro, and in vivo data together as part of anIATA to make an assessment of skin sensitisation potential based


on the available AOP. The maturity of this AOP lends itself to thedevelopment of IATA for hazard identification and to a lesserextent for potency. The results of 2 in chemico assays and 1 in vitrotest characterizing the MIE, KE2, and KE3 (as described in Table 4)were collected for 145 chemicals and compared with the corre-sponding results from the LLNA (Natsch et al., 2013). A Bayesiannetwork analysis was derived to predict skin sensitisation potency(in terms of strong, moderate, weak categorizations) for variousclasses of sensitisers (Jaworska et al., 2011, 2013). Other testingstrategies have also been proposed including that of Bauch et al.(2012), Buist et al. (2013), Rorije et al. (2013), Tluczkiewicz et al.(2013) and Nukada et al. (2013) though not all of these have beenframed them in the context of AOPs. An IATA could in principlesolely focus on non-testing approaches such as QSARs beforedeciding on what additional data would be warranted to generateto address a specific purpose.

The AOP for skin sensitisation has been implemented in theOECD Toolbox to facilitate the formation of chemical categoriesand associated read-across using the knowledge and data for thedifferent key events (OECD, 2014b). This implementation couldbe likened to a type of IATA that focuses solely on existing dataand other non-testing approaches and marks a step change interms of how read-across and other QSAR information can beapplied and will be developed in the future. In the Toolbox, a setof profilers have been developed which encode available knowl-edge derived for substances that had been tested in assays thatcharacterize each of the key events. These profilers are not directpredictors of the key events or the adverse outcome but providea mechanistic basis for grouping chemicals together. The Toolboxcontains traditional profilers which encode known SAR informa-tion for chemicals tested in in vivo methods such as the LLNAand GPMT as well as hypothetical SARs based on organic chemistryreaction principles (Aptula and Roberts, 2006). These specific pro-filers are called the Protein Binding alerts by OASIS and OECD.There are also profilers for lysine and cysteine depletion basedon data generated on chemicals tested in the DRPA as well as pro-filers for chemicals tested in the KeratinoSens™ assay (Emter et al.,2010), the human cell line activation test (h-CLAT) (Sakaguchiet al., 2007), and myeloid U937 skin sensitisation test (MUSST)(Python et al., 2007). A user starts by profiling their chemicals onthe basis of protein binding alerts. If alerts are identified, then astepwise approach of gathering data for each of the differentevents and investigating the feasibility of deriving a read-acrossprediction is evaluated. Specific thresholds to pass from one keyevent to the next have been encoded into the AOP implementa-tion to help in the interpretation of the novel data. Dependingon the decision being made and the availability of experimentaldata for the related analogues at each step of the IATA and thechemical of interest, sufficient scientific confidence could bepotentially be reached at the molecular initiating event, or alter-natively other key events may need to be considered in theevaluation.

A complementary implementation of the skin sensitisation AOPwas proposed by Patlewicz et al. (2014c) through the developmentof a non-testing IATA which relies on a combination of existinginformation on the physical form of the substance itself (i.e.,whether it is a gas, liquid or solid), available experimental datafor skin sensitisation, skin corrosion, and mutagenicity data asextracted from the OECD Toolbox in addition to profilers forreactivity, as well as components for the expert system TissueMetabolism Simulator (TIMES) (Patlewicz et al., 2014b) to predictthe same endpoints. The underlying basis of this IATA is to considerwhat inherent data might exist for the substance of interest thatwould inform its skin sensitisation potential and to establishwhether any sensitisation testing would be justified e.g., the sub-stance is a gas or is corrosive.

3.4.2. Case study 3; AHR AOP as applied to IATAThe AHR case study (as detailed in Table 6) provides an exten-

sive dataset for the purposes of outlining how one can frame anIATA. A decision tree approach, summarized in brief in Fig. 4, isproposed to outline what additional, more complex, testing couldbe considered when a substance has the potential to be an AHRligand that could result in rat liver tumors. The IATA decision tree(Fig. 4) integrates consideration of exposure into the decision mak-ing. Previously (Becker et al., 2015), we discussed application of theexposure:activity ratio concept at the latter stages of this decisiontree. In this evolved IATA, we propose, that in addition to evaluat-ing activities in assays, exposure activity profiling can help to placesuch results of early molecular events and cellular responses intothe context of existing human exposures. Since sustained AHR acti-vation is the MIE, care will need to be exercised in interpretingexposure activity profiling based on assays that are not measuresof sustained activation.

This proposed AHR decision tree (Fig. 4) does not include testingin human parenchymal and non-parenchymal cells although thispossibility could be considered to see if differences in responsebetween rats and humans, e.g., transcriptional changes, responseto apoptotic stimuli, and differentiation and proliferation, are sodifferent as to call into question the human relevance for a ligandinducing AHR binding or activation.

Again the completeness of a given AOP facilitates the extent towhich an IATA can be developed, how encompassing it will be interms of whether it can be used for hazard identification or onlyfor prioritization or chemical categorization purposes. The twoAOPs cited here have been extensively delineated to permitdetailed IATA to be developed that can be applied for hazard iden-tification decisions and beyond.

3.5. Hazard and risk assessment

Identification of an adverse outcome is a necessary element forboth hazard identification and risk assessment. The use of AOPs forhazard identification and risk assessment, therefore will require afocus on the performance of analytical methods and predictionmodels at each key event and key event relationship within theAOP. In the utilization step, one will need to document the scien-tific support, based on the WoE, for using one or more of the keyevents and key event relationships to predict the adverse outcome.As a general rule, the more ‘downstream’ a key event is towardsthe adverse outcome, the greater the confidence that the relevantkey event relationship could be used to predict the adverse out-come. Conversely, the more ‘upstream’ a key event is from theadverse outcome, the less confidence in the use of its associatedkey event relationship to predict the adverse outcome.

Considerable research over the last 15 years has focused ondeveloping and applying methods to use knowledge of MoA toinform human health hazard and risk assessments (Meek et al.,2014a,b). Typically, when applying a MoA to characterize humanhazard or risk, one needs to understand not only the molecularinitiating event and adverse outcome, but also critical key eventsfurther along the pathway. One also needs to understand dose–response and temporal concordance of key events. When dataare available from a molecular initiating event, and yet knowledgeof the intermediate key events is not available, confidence wouldgenerally be lacking, and results measuring the molecular initiat-ing event (or other early key events) would be judged insufficientto allow a conclusion to be made regarding the likelihood that anadverse outcome would occur. This arises because we do not gen-erally have sufficient knowledge of the biology and complex biolo-gical interactions and dose–response and time courses to link all ofthe key events in an AOP together quantitatively. For example, oneof the main challenges in using any AOP stems from the fact that in

Fig. 4. Decision tree approach to guide testing for a substance with the potential to be an AHR ligand which results in rat liver tumor promotion. (See above-mentionedreferences for further information.)

2 Risk factors are derived on the basis of odds ratios analysis and represent a flag fora concern whereas predictors would imply an actual estimate of the AO.


many cases the molecular initiating event involves a combinationof events, some of which may not be known and some may notbe reflected in acute and sub-acute biological responses conduciveto HT/HC approaches. Thus, for many AOPs at the present time,when data are limited to measurements of the molecular initiatingevent or early key events, current information will not allowconclusions to be drawn regarding potential hazard or risk. How-ever, for some AOPs, where extensive downstream key events areavailable, coupled to robust pharmacokinetic and exposureinformation, hazard and risk can be more readily defined. TheAHR AOP is a good example. Quantitative modeling of the livertumor promotion response is made possible by extensivedosimetry data. These quantitative data include AHR-binding,AHR-induced transcriptional responses, rat liver initiation-promotion results, and interim and terminal NTP cancer bioassayresults (Budinsky et al., 2014; Hailey et al., 2005; Conolly andAndersen, 1997; Moolgavkar et al., 1996; Simon et al., 2009).Whilst the relevance of the rodent liver tumor response to humansremains in doubt, the quantitative aspects of this AOP provide anexample of how key event relationships can be understood on aquantitative basis (see Lorber et al., 2009). It is perhaps worthnothing that persistent AHR ligands produce rodent liver tumorsprovided that high levels of AHR activation are sustained. Incontrast, humans develop chloracne with clinical onset occurringin short order following fairly high levels of AHR activation.Consequently, chloracne may offer a non-controversial humanendpoint for risk assessment that can benefit from closer temporallinkage between gene changes in keratinocytes and sebocytes thatculminate in the apical outcome.

It may be possible to develop AOP hazard prediction modelsfrom a combination of assays characterizing early key events in

the AOP. These may include QSAR and/or predictions based on che-mical knowledge, in chemico, in vitro assays, possibly using hetero-logous expression systems and, in some cases, short term in vivoassays. An example is provided by recent work using the ToxCast™dataset from US EPA and almost 100 statistical tests of associationto predict 60 in vivo endpoints—the relationships between assayresults and adverse outcomes could be interpreted such that theassay results were risk factors2 but not predictors (Thomas et al.,2012). As more AOPs are developed, prediction models will likelyimprove. Before any AOP is used for either screening purposes orhazard prediction, additional work will be needed to develop andevaluate the associated prediction model, an example where suchan exercise was undertaken can be found in Cox et al. (2014).

Lessons learned from the use of well-established MoAs in haz-ard identification and risk assessment can be applied to AOPs.Recently the HESI RISK21 project investigated enhancements tothe existing mode of action/human relevance framework and keyevents (KE)/dose–response Framework to make the best use ofquantitative dose–response and timing information for key eventsas part of an effort to advance the next-generation of chemical riskassessments. The resulting quantitative KE/dose–response frame-work (QKEDRF) provides a structured quantitative approach forsystematic examination of the dose–response and timing of keyevents resulting from a dose of a bioactive agent that causes apotential adverse outcome (Simon et al., 2014). With its focus ofquantitative dose–response, the QKEDRF approach provides ameans for quantification of key event relationships. At this time,

3 The choice of the target value of an EAR is a regulatory decision. If the exposureconcentration of a chemical is 10,000 fold lower than the activity level of thatchemical, very little concern is warranted. If the exposure and activity are equalresulting in an EAR of 1, perhaps greater concern is appropriate. To contextualize EARvalues, Becker et al. (2014b) compared EARs for chemicals with estrogenic activity tothe EAR for genistein, a common phytoestrogen that is part of a healthy diet. The ratiobetween the EAR for a chemical and the EAR for genistein is called the RelativeEstrogenic Exposure Activity Quotient (REEAQ). The use of REEAQs contextualizes theEAR value in terms of a ubiquitous estrogenic exposure, that of genistein.


however, the quantitative aspects of AOPs will require additionaldevelopment and examples using these approaches and theQKEDRF is mentioned for completeness.

3.6. Considerations for dosimetry

Scholz et al. (2013) indicate that ‘‘most if not all types of toxicitycan be explained by an initial molecular interaction.’’ However,until more experience is gained with developing and applying AOPs,this claim remains a testable hypothesis. Thomas et al. (2013)recently proposed a tiered evaluation framework that comprisesthree tiers. Tier 1 utilizes an in vitro HTS battery, such as ToxCast™and Tox21, to evaluate activity of substances in biologically relevantassays, and then links these to exposure to enable a margin ofexposure (MOE) determination. If the MOE is adequate, no furtherevaluation would be needed. For substances with lower MOEs, Tier2 would entail in vivo evaluation of transcriptomics coupled to anMOE determination. Again, after Tier 2, substances with anadequate MOE would proceed no further, but substances withlower MOEs would proceed to Tier 3. Tier 3 consists of conventionalanimal toxicity tests, but which may be tailored based onknowledge gained from the preceding mechanistic in vitro andin vivo studies. Tier 3 is also coupled with a MOE evaluation.

This framework did not specifically incorporate AOPs, at thetime of its development though this could be a considerationnow. For example, in Tier 1, the Thomas et al. (2013) frameworkincorporates evaluation streams for chemicals that act via ‘‘specificMoAs’’ and those that act via ‘‘non-specific MoAs.’’ For substancesidentified in Tier 1 as acting by ‘‘specific MoAs’’, the biological pro-file determined could be compared to initial molecular events orkey events for a suite of AOPs. Following such a matching exercise,one will then have a set of AOPs that could be further evaluated.This may obviate the need for Tier 2 or Tier 3 evaluations, providedthere is sufficient scientific confidence for the use of the AOP. Alter-natively the AOP can be used along the lines of an IATA to tailorTier 2 or Tier 3 evaluations.

In general, for non-selective chemicals, consideration ofdosimetry is vital to development of an AOP whereas for selectivechemicals, the issues of potency and efficacy will likely be moreimportant. However, the dosimetry for some chemicals that actvia a selective mechanism in vitro remains necessary for under-standing in vivo results and for AOP development (Hengstleret al., 2011; Teeguarden et al., 2011, 2013; Teeguarden andHanson-Drury, 2013).

The prediction of concentrations measured in human body flu-ids from known administered doses of a chemical is a necessarypart of the application of AOPs. Results from HT/HC or other in vitroassays need to be contextualized in terms of human exposures(Becker et al., 2014b,c; Wetmore et al., 2011). In fact, as will beseen, the incorporation of exposure is often critical for increasingthe predictive power of models (Morgan et al., 2013).

In vitro assays produce an estimate of a concentration needed toproduce an effect in the test system. The exposure:activity ratio(EAR) has been developed for this purpose (Becker et al.,2014a,b,c). Values for exposure and activity may be expressed asoral doses, urinary concentrations or steady state blood or plasmaconcentrations. For comparison, both values must be expressed inthe same way. For example, if one wishes to compare the results ofestrogenic screening assays with urinary excretion data fromNHANES, then the assay result needs to be expressed as a urinaryconcentration using in vitro-to-in vivo extrapolation (IVIVE)(Aylward et al., 2011; Aylward and Hays, 2011; Becker et al.,2014b,c; Wetmore et al., 2011).

In its first use, the EAR compared human urinary concentrationsmeasured in NHANES to human urinary biomonitoring equivalents(Becker et al., 2014a). The EAR is also used to compare effect

concentrations from in vitro assays to steady state blood concen-trations estimated from human urinary data with IVIVE (Beckeret al., 2014a, Becker et al., 2014b; Wetmore et al., 2011).

Hence, one can think of the EAR as the environmentally relevanthuman exposure level divided by combined result from the rele-vant in vitro assays. If the EAR is well above the target value, indi-cating that exposure is low relative to the activity of the chemicaland there is sufficient confidence in the dosimetry model, then theEAR can be documented and published. Doing so will establish adegree of confidence that current human exposures will not leadto the adverse outcome within the population3.

The next step would be to refine the dosimetry model. Forexample, Wetmore et al. (2011) used a generic model that account-ed for plasma protein binding, phase I and phase II metabolism,bioavailability, and red blood cell partitioning. This model, albeitgeneric, was more sophisticated than those used in earlierattempts at IVIVE (Rotroff et al., 2010). The use of the initial EARfor screening depends on the level of confidence. Refinement ofthe dosimetry model may require laboratory experimentationand possibly the use of animals. Consequently, the experimentsneed to be well designed and narrowly targeted toward obtainingspecific results. Finally, the results of and confidence in the refinedEAR will determine whether additional testing, possibly in vivotesting in animals, should be contemplated.

In summary, the most useful piece of information for applyingan AOP in a risk assessment context would be a qualitativerelationship between the dose–response of the molecular initiatingevent and the dose–response of the adverse outcome. However, ifsufficiently robust dose–response data exist for a molecular initiat-ing event and/or other key events, this can be used to determinethe most appropriate dose–response model for an adverse out-come in a risk assessment (Simon et al., 2014). A relationshipexpressed using quantitative dose–response enables the consid-eration of exposure information with relative ease. The next mostpowerful piece could be provided by a strong correlation betweena measure of the molecular initiating event and the adverse out-come. Patlewicz et al. (2013a) provide a general approach forevaluation of prediction models based on their expected use.

4. Summary

The AOP framework provides an overarching basis for contextu-alizing in vitro methods and their prediction models in a mannerthat can be interpreted for regulatory decision making. Currentlythere is a strong motivation at least by the OECD to develop AOPsthat could be applied in one of three main avenues – informing testmethods, enhancing read-across, or helping to develop IATA. Ulti-mately these AOPs and the approaches they anchor will be usedto address regulatory questions such as prioritization, hazard iden-tification, and risk assessment. Whilst there has been significantmomentum to develop AOPs and start to populate an AOP Wikirepository, there has been little focus on the ultimate applicationof AOPs in these different regulatory applications. A harmonizedframework approach to establish scientific confidence is criticalto bolster scientific rigor, improve communication and under-standing of the scientific justification for specific applications,and enhance uptake and use of Tox21 approaches by regulatory


programmes globally. In addition to a framework, processes stillneed to be agreed upon and utilized to ensure robust and transpar-ent review and determination of fit-for-purpose uses of AOPs forregulatory acceptance and use.

Here a scientific confidence framework previously described inCox et al. (2014) has been adapted and extended for AOPs. TheSCF comprises 3 core elements – analytical validation of the assayscharacterizing key events within the AOP, qualification of predictionmodels derived from these assays and utilization of AOP based pre-diction models for specific purposes; the latter being anchoredclosely with the evolved Bradford Hill considerations, in which eachkey events and key event relationships in an AOP are evaluated andscored as part of a WoE determination. The SCF has been illustratedwith respect to a handful of example AOPs to demonstrate how thepractical application of an AOP strongly depends both on its com-pleteness and the decision context in mind. The example AOPs aredifferent in character and in their level of maturity. The AOP forestrogen activity lends itself to prioritization purposes throughthe use of assays that measure early key events specifically estrogenbinding and transactivation assays. A semi quantitative but com-plete AOP for skin sensitisation can be and is used in many typesof applications such as chemical categorization and read-across, aswell as IATA for hazard assessment purposes. A number of IATAhave been developed and the AOP itself has also been incorporatedinto the OECD Toolbox to support read-across. Applications of theAHR AOP are complicated by the lack of concrete assays to measurethe molecular initiating event. The only means to currently predictan AHR ligands ability to act as a rodent liver tumor promoter is totest the ligand in a sub-chronic or chronic assay that pays carefulattention to the classic histological changes (e.g., hepatopathy) thatare required for tumor promotion to ensue (Hailey et al., 2005).As such it’s predominant value is to develop an IATA for hazardand risk assessment purposes. The role of AOPs in hazard andrisk assessment have also been discussed in light of exposureconsiderations.

Conflict of interest

The authors had complete control over the design, conduct,interpretation, and reporting of the analyses included in thismanuscript. The contents of this manuscript are solely the respon-sibility of the authors and do not necessarily reflect the views orpolicies of their employers. Ted Simon received funding to supportthis research from the American Chemistry Council (ACC). RobertBudinsky, Craig Rowlands, and Grace Patlewicz are or wereemployed by companies engaged in chemical product manufactur-ing. Richard Becker is employed by ACC, a trade association of U. S.chemical manufacturers.

Acknowledgments

The perspectives and discussion in this manuscript were a pro-duct of the ACC Computational Profiling and Risk AssessmentWorkgroup chaired by Richard A Becker of ACC, Grace Patlewiczof DuPont and J Craig Rowlands of The Dow Chemical Company.We thank all the members for their valuable insights and contribu-tions. We would like to thank in particular Katy Goyak of ExxonMobil and Lynn Pottenger of The Dow Chemical Company for theircontributions.

References

Andersen, M.E., Birnbaum, L.S., Barton, H.A., Eklund, C.R., 1997. Regional hepaticCYP1A1 And CYP1A2 induction with 2,3,7,8-tetrachlorodibenzo-p-dioxinevaluated with a multicompartment genometric model of hepatic zonation.Toxicol. Appl. Pharmacol. 144, 145–155.

Ankley, G.T., Bennett, R.S., Erickson, R.J., Hoff, D.J., Hornung, M.W., Johnson, R.D.,Mount, D.R., Nichols, J.W., Russom, C.L., Schmieder, P.K., Serrrano, J.A., Tietge,J.E., Villeneuve, D.L., 2010. Adverse outcome pathways: a conceptual frameworkto support ecotoxicology research and risk assessment. Environ. Toxicol. Chem.29, 730–741. http://dx.doi.org/10.1002/etc.34.

Aptula, A.O., Patlewicz, G.Y., Roberts, D.W., 2005. Skin sensitisation: reactionmechanistic applicability domains for structure–activity relationships. Chem.Res. Toxicol. 18 (9), 1420–1426.

Aptula, A.O., Roberts, D.W., 2006. Mechanistic applicability domains for non animal-based prediction of toxicological end points: general principles and applicationto reactive toxicity. Chem. Res. Toxicol. 19, 1097–1105.

Aylward, L.L., Becker, R.A., Kirman, C.R., Hays, S.M., 2011. Assessment of margin ofexposure based on biomarkers in blood: an exploratory analysis. Regul. Toxicol.Pharmacol. 61, 44–52. http://dx.doi.org/10.1016/j.yrtph.2011.06.001.

Aylward, L.L., Hays, S.M., 2011. Consideration of dosimetry in evaluation of ToxCast™data. J. Appl. Toxicol. 31, 741–751. http://dx.doi.org/10.1002/jat.1626.

Ball, N., Bartels, M.J., Budinsky, R.A., Klapacz, J., Hayes, S.M., Kirman, C.R., Patlewicz,G.Y., 2014. The challenge of using read-across within the EU REACH regulatoryframework; how much uncertainty is too much? Dipropylene glycol methylether acetate, an exemplary case study. Regul. Toxicol. Pharmacol. 68, 212–221.

Bauch, C., Kolle, S.N., Ramirez, T., Eltze, T., Fabian, E., Mehling, A., Teubner, W., vanRavenzwaay, B., Landsiedel, R., 2012. Putting the parts together: combiningin vitro methods to test for skin sensitizing potentials. Regul. Toxicol.Pharmacol. 63, 489–504. http://dx.doi.org/10.1016/j.yrtph.2012.05.013.

Becker, R.A., Hays, S.M., Robison, S., Aylward, L.L., 2012. Development of screeningtools for the interpretation of chemical biomonitoring data. J. Toxicol. http://dx.doi.org/10.1155/2012/941082.

Becker, R.A., Rowlands, J.C., Patlewicz, G., Simon, T., 2014. Enhancing the Utility ofT21 Assessment Methods by Employing a Scientific Confidence Framework andExposure: Activity Profiling. Presented at ICCA-JRC Workshop, June 17–18, 2014Lugano, Switzerland.

Becker, R.A., Hays, S.M., Kirman, C.R., Aylward, L.L., Wise, K., 2014b. Interpretingestrogen screening assays in the context of potency and human exposurerelative to natural exposures to phytoestrogens. Birth Defects Res. B Dev.Reprod. Toxicol. 101, 114–124. http://dx.doi.org/10.1002/bdrb.21085.

Becker, R.A., Paul, K., Simon, T.W., Marty, S., Patlewicz, G., Rowlands, J.C., 2014c. Anexposure: activity profiling method for interpreting high-throughput screeningdata for estrogenic activity—proof of concept. Regul. Toxicol. Pharmacol.(submitted)

Becker, R.A., Budinsky, R.A., Patlewicz, G., Simon, T., Rowlands, C. 2015. The AdverseOutcome Pathway for Rodent Liver Tumor Promotion by Sustained Activation ofthe Aryl Hydrocarbon Receptor. in preparation.

Boutros, P.C., Yao, C.Q., Watson, J.D., Wu, A.H., Moffat, I.D., Prokopec, S.D., Smith,A.B., Okey, A.B., Pohjanvirta, R., 2011. Hepatic transcriptomic response to TCDDin dioxin-sensitive and dioxin-resistant rats during the onset of toxicity.Toxicol. Appl. Pharmacol. 251, 119–129.

Boverhof, D.R., Burgoon, L.D., Tashior, C., Sharratt, B., Chittim, B., Harkema, J.R.,Mendrick, D.L., Zacharewski, T.R., 2006. Comparative toxicogenomic analysis ofthe hepatotoxic effects of TCDD in Sprague Dawley rats and C57BL/6 mice.Toxicol. Sci. 94, 398–416.

Budinsky, R.A., Schrenk, D., Simon, T., Van den Berg, M., Maier, A., Silkworth, J.B.,Lesa, L.L., Brix, A., Gasiewicz, T., Kaminski, N., Perdew, G., Starr, T.B., Walker, N.J.,Rowlands, J.C., 2014. Mode of action and dose–response framework analysis forreceptor-mediated toxicity: the aryl hydrocarbon receptor as a case study. Crit.Rev. Toxicol. 44, 83–119. http://dx.doi.org/10.3109/10408444.2013.835787.

Buist, H., Aldenberg, T., Batke, M., Escher, S., Klein Entink, R., Kühne, R., Marquart, H.,Pauné, E., Rorije, E., Schüürmann, G., Kroese, D., 2013. The OSIRIS weight ofevidence approach: ITS mutagenicity and ITS carcinogenicity. Regul. Toxicol.Pharmacol. 67, 170–181. http://dx.doi.org/10.1016/j.yrtph.2013.01.002.

Cao, F., Li, X., Ye, L., Xie, Y., Wang, X., Shi, W., Zian, X., Zhu, Y., Yu, H., 2013. Moleculardocking, molecular dynamics simulation, and structure-based 3D_QSAR studieson the aryl hydrocarbon receptor agonistic activity of hydroxylatedpolychlorinated biphenyls. Environ. Toxicol. Pharmacol. 36, 626–635.

Chang, H., Wang, Y.J., Chang, L.W., Lin, P., 2005. A histochemical and pathologicalstudy on the interrelationship between TCDD-induced AR expression, AhRactivation, and hepatotoxicity in mice. J. Toxicol. Environ. Health Part A 68,1567–1579.

Chopra, M., Schrenk, D., 2011. Dioxin toxicity, aryl hydrocarbon receptor signaling,and apoptosis-persistent pollutants affect programmed cell death. Crit. Rev.Toxicol. http://dx.doi.org/10.3109/10408444.2010.524635.

Cohen Hubal, E.A., Richard, A., Aylward, L., Edwards, S., Gallagher, J., Goldsmith,M.R., Isukapalli, S., Tornero-Velez, R., Weber, E., Kavlock, R., 2010. Advancingexposure characterization for chemical evaluation and risk assessment. J.Toxicol. Environ. Health B Crit. Rev. 13, 299–313.

Conolly, R.B., Andersen, M.E., 1997. Hepatic foci in rats after diethylnitrosamineinitiation and 2,3,7,8-tetrachlorodibenzo-p-dioxin promotion: evaluation of aquantitative two-cell model and of CYP 1A1/1A2 as a dosimeter. Toxicol. Appl.Pharmacol. 146, 281–293.

Cox, L.A., Douglas, D., Marty, S., Rowlands, J.C., Patlewicz, G., Goyak, K.O., Becker,R.A., 2014. Applying a scientific confidence framework to a HTS-derivedprediction model for endocrine endpoints: lessons learned from a case study.Regul. Toxicol. Pharmacol. 69, 443–450.

Denison, M.S., Soshilov, A.A., He, G., DeGroot, D.E., Zhao, B., 2011. Exactly the samebut different: promiscuity and diversity in the molecular mechanisms of actionof the aryl hydrocarbon (dioxin) receptor. Toxicol. Sci. 124, 1–22. http://dx.doi.org/10.1093/toxsci/kfr218.

http://refhub.elsevier.com/S0273-2300(15)00039-2/h0010




http://dx.doi.org/10.1002/etc.34








http://dx.doi.org/10.1002/jat.1626






http://dx.doi.org/10.1155/2012/941082

http://dx.doi.org/10.1155/2012/941082

http://dx.doi.org/10.1002/bdrb.21085













http://dx.doi.org/10.3109/10408444.2013.835787










http://dx.doi.org/10.3109/10408444.2010.524635













http://dx.doi.org/10.1093/toxsci/kfr218



Drewe, W.C., Cayley, A., Benz, R., Kruhlak, N.L., Surfraz, B. Identification of AdverseOutcome Pathways for the Nephrotoxicity of Nucleoside and NucleotideAntiviral Drugs. Presented at the Society of Toxicology, March 24–27,Phoenix, AZ, USA.

Dupuis, G., Benezra, C., 1982. Allergic Contact Dermatitis to Simple Chemicals: aMolecular Approach. Dekker, New York and Basel.

ECHA. 2008. Guidance on information requirements and chemical safetyassessment. Chapter R.7a: Endpoint Specific Guidance. Available at <http://guidance.echa.europa.eu/docs/guidance_document/information_requirements_r7a_en.pdf?vers=20_08_08>.

ECETOC. 2012. Technical Report 116 Category approaches, read-across, (Q)SARavailable at <http://www.ecetoc.org/technical-reports>.

Emter, R., Ellis, G., Natsch, A., 2010. Performance of a novel keratinocyte-basedreporter cell line to screen skin sensitizers in vitro. Toxicol. Appl. Pharmacol.245, 281–290. http://dx.doi.org/10.1016/j.taap.2010.03.009.

Emter, R., van der Veen, J.W., Adamson, G., Ezendam, J., van Loveren, H., Natsch, A.,2013. Gene expression changes induced by skin sensitizers in theKeratinoSens™ cell line: discriminating Nrf2-dependent and Nrf2-independent events. Toxicol. In Vitro 27, 2225–2232. http://dx.doi.org/10.1016/j.tiv.2013.09.009.

European Commission, (EC) November 11, 2009. Regulation (EC) No. 1223/2009 ofthe European Parliament and of the Council of 30 November 2009 on cosmeticproducts.

Fielden, M.R., Nie, A., McMillian, M., Elangbam, C.S., Trela, B.A., Yang, Y., Dunn II, R.T.,Dragan, Y., Fransson-Stehen, R., Bogdanffy, M., Adams, S.P., Foster, W.R., Chen,S.J., Rossi, P., Kasper, P., Jacobson-Kram, D., Tatsuoka, K.S., Wier, P.J., Gollub, J.,Halbert, D.N., Roter, A., Young, J.K., Sina, J.F., Marlowe, J., Martus, H.J., Aubrecht,J., Olaharski, A.J., Roome, N., Nioi, P., Pardo, I., Snyder, R., Perry, R., Lord, P.,Mattes, W., Car, B.D., Predictive Safety Testing Consortium, CarcinogenicityWorking Group, 2008. Interlaboratory evaluation of genomic signatures forpredicting carcinogenicity in the rat. Toxicol. Sci. 103, 28–34.

Fletcher, N., Wahlstrom, D., Lundberg, R., Nilsson, C.B., Nilsson, K.C., Stockling, K.,Hellmold, H., Hakansson, H., 2005. 2,3,7,8-Tetrachlorodibenzo-p-dioxin (TCDD)alters mRNA expression of critical genes associated with cholesterolmetabolism, bile acid biosynthesis, and bile transport in rat liver: amicroarray study. Toxicol. Appl. Pharmacol. 207, 1–24.

Forgac, A.L., Kent, M.N., Makley, M.K., Mets, B., DelRaso, N., Jahns, G.L., Burgoon, L.D.,Reo, N.V., Zacharewski, T.R., 2012. Comparative metabolomic and genomicanalyses of TCDD-elicited metabolic disruption in mouse and rat liver. Toxicol.Sci. 125, 41–55.

Gerberick, G.F., Vassallo, J.D., Bailey, R.E., Chaney, J.G., Morrall, S.W., Lepoittevin, J.-P., 2004. Development of a peptide reactivity assay for screening contactallergens. Toxicol. Sci. 81, 332–343.

Gerberick, G.F., Vassallo, J.D., Foertsch, L.M., Price, B.B., Chaney, J.G., Lepoittevin, J.-P., 2007. Quantification of chemical peptide reactivity for screening contactallergens: a classification tree model approach. Toxicol. Sci. 97, 417–427. http://dx.doi.org/10.1093/toxsci/kfm064.

Godfrey, H.P., Baer, K., 1971. The effect of physical and chemical properties of thesensitizing substance on the induction and elicitation of delayed contacthypersensitivity. J. Immunol. 106, 431–441.

Goodman, D.G., Sauer, R.M., 1992. Hepatotoxicity and carcinogenicity in femaleSprague-Dawley rats treated with 2,3,7,8-tetrachlorordibenzo-p-dioxin(TCDD): a Pathology Working Group reevaluation. Regul. Toxicol. Pharmacol.15, 245–252.

Hailey, J.R., Walker, N.J., Sells, D.M., Brix, A.E., Jokinen, M.P., Nyska, A., 2005.Classification of proliferative hepatocellular lesions in harlan sprague-dawleyrats chronically exposed to dioxin-like compounds. Toxicol. Pathol. 33, 165–174.

Hartung, T., Blaauboer, B.J., Bosgra, S., Carney, E., Coenen, J., Conolly, R.B., Corsini, E.,Green, S., Faustman, E.M., Gaspari, A., Hayashi, M., Wallace Hayes, A., Hengstler,J.G., Knudsen, L.E., Knudsen, T.B., McKim, J.M., Pfaller, W., Roggen, E.L., 2011. Anexpert consortium review of the EC-commissioned report ‘‘alternative (non-animal) methods for cosmetics testing: current status and future prospects –2010’’. ALTEX 28, 183–209.

Heise, T., Schug, M., Storm, D., Ellinger-Ziegelbauer, H.J., Ahr, H., Hellwig, B.,Rahnenfuhrer, J., Ghallab, A., Guenther, G., Sisnaiske, J., Reif, R., Godoy, P.,Mielke, H., Gundert-Remy, U., Lampen, A., Oberemm, A.G., Hengstler, J., 2012. Invitro – in vivo correlation of gene expression alterations induced by livercarcinogens. Curr. Med. Chem. 19, 1721–1730.

Hengstler, J.G., Foth, H., Gebel, T., Kramer, P.-J., Lilienblum, W., Schweinfurth, H.,Völkel, W., Wollin, K.-M., Gundert-Remy, U., 2011. Critical evaluation of keyevidence on the human health hazards of exposure to bisphenol A. Crit. Rev.Toxicol. 41, 263–291. http://dx.doi.org/10.3109/10408444.2011.558487.

Institute of Medicine (IOM). 2010. Evaluation of biomarkers and surrogateendpoints in chronic disease. ISBN: 978-0-309-15129-0.

Jaworska, J., Harol, A., Kern, P.S., Gerberick, G.F., 2011. Integrating non-animal testinformation into an adaptive testing strategy – skin sensitization proof ofconcept case. ALTEX 28, 211–225.

Jaworska, J., Dancik, Y., Kern, P., Gerberick, F., Natsch, A., 2013. Bayesian integratedtesting strategy to assess skin sensitization potency: from theory to practice. J.Appl. Toxicol. http://dx.doi.org/10.1002/jat.2869.

Julien, E., Boobis, A.R., Olin, S.S., Ilsi Research Foundation Threshold Working Group,2009. The key events dose–response framework: a cross-disciplinary mode-of-action based approach to examining dose–response and thresholds. Crit. Rev.Food Sci. Nutr. 49, 682–689. http://dx.doi.org/10.1080/10408390903110692.

Kennedy, G.D., Nukaya, M., Moran, S.M., Glover, E., Weinberg, S., Balbo, S., Hecht,S.S., Pitot, H.C., Drinkwater, N.R., Bradfield, C.A., 2014. Liver tumor promotion by2,3,7,8-tetrachlorodibenzo-p-dioxin is dependent on the aryl hydrocarbonreceptor and TNF/IL-1 receptors. Toxicol. Sci. 140, 135–143.

Landsteiner, K., Jacobs, J.L., 1936. Studies on the sensitisation of animals with simplechemicals III. J. Exp. Med. 64, 625–639.

Lorber, M., Patterson, D., Huwe, J., Kahn, H., 2009. Evaluation of backgroundexposures of Americans to dioxin-like compounds in the 1990s and the 2000s.Chemosphere 77, 640–651.

Meek, M.E., 2008. Recent developments in frameworks to consider humanrelevance of hypothesized modes of action for tumours in animals. Environ.Mol. Mutagen. 49, 110–116. http://dx.doi.org/10.1002/em.20369.

Meek, M.E., Bucher, J.R., Cohen, S.M., Dellarco, V., Hill, R.N., Lehman-McKeeman,L.D., Longfellow, D.G., Pastoor, T., Seed, J., Patton, D.E., 2003. A framework forhuman relevance analysis of information on carcinogenic modes of action. Crit.Rev. Toxicol. 33, 591–653.

Meek, M.E., Boobis, A., Cote, I., Dellarco, V., Fotakis, G., Munn, S., Seed, J., Vickers, C.,2014a. New developments in the evolution and application of the WHO/IPCSframework on mode of action/species concordance analysis. J. Appl. Toxicol. 34,1–18.

Meek, M.E., Palermo, C.M., Bachman, A.N., North, C.M., Lewis, R.J., 2014b. Mode ofaction human relevance (species concordance) framework: evolution of theBradford Hill considerations and comparative analysis of weight of evidence. J.Appl. Toxicol. 34, 595–606.

Moolgavkar, S.H., Luebeck, E.G., Buchmann, A., Bock, K.W., 1996. Quantitativeanalysis of enzyme-altered liver foci in rats initiated with diethylnitrosamineand promoted with 2,3,7,8-tetrachlorodibenzo-p-dioxin or 1,2,3,4,6,7,8-heptachlorodibenzo-p-dioxin. Toxicol. Appl. Pharmacol. 138, 31–42.

Morgan, R.E., van Staden, C.J., Chen, Y., Kalyanaraman, N., Kalanzi, J., Dunn, R.T.,Afshari, C.A., Hamadeh, H.K., 2013. A multifactorial approach to hepatobiliarytransporter assessment enables improved therapeutic compound development.Toxicol. Sci. 136, 216–241. http://dx.doi.org/10.1093/toxsci/kft176.

Murk, A.J., Rijntjes, E., Blaauboer, B.J., Clewell, R., Crofton, K.M., Dingemans, M.M.,Furlow, J.D., Kavlock, R., Kohrle, J., Opitz, R., Traas, T., Visser, T.J., Xia, M., Gutleb,A.C., 2013. Mechanism-based testing strategy using in vitro approaches foridentification of thyroid hormone disrupting chemicals. Toxicol. In Vitro 27,1320–1346.

National Research Council (NRC), 2007. Toxicity Testing in the 21st Century: AVision and a Strategy. National Academies Press, Washington, DC.

National Research Council, 2012. Exposure Science in the 21st Century: a Vision anda Strategy. The National Academies Press, Washington, DC.

National Toxicology Programme. 2014. Toxicology studies of indole-3-carbinol (CASNo. 700-06-1_in F33/N rats and B6C3F1/N mice and toxicology andcarcinogenensis studies of indole-3-carbinol in Harlan Sprague Dawley ratsand B6C3F1 mice. NTP Technical Report 584.

Natsch, A., Ryan, C.A., Foertsch, L., Emter, R., Jaworska, J., Gerberick, F., Kern, P., 2013.A dataset on 145 chemicals tested in alternative assays for skin sensitizationundergoing prevalidation. J. Appl. Toxicol. http://dx.doi.org/10.1002/jat.2868.

Nguyen, L.P., Bradfield, C.A., 2008. The search for endogenous activators of the arylhydrocarbon receptor. Chem. Res. Toxicol. 21, 102–116.

Nukada, Y., Miyazawa, M., Kazutoshi, S., Sakaguchi, H., Nishiyama, N., 2013. Dataintegration of non-animal tests for the development of a test battery to predictthe skin sensitizing potential and potency of chemicals. Toxicol. In Vitro 27,609–618.

Organisation for Economic Cooperation and Development (OECD), 2012a. TheAdverse Outcome Pathway for Skin Sensitisation by Covalent Binding toProteins: Part 1: Scientific Evidence. OECD, Paris.

OECD. 2004. ENV/JM/MONO/(2004)24 <http://appli1.oecd.org/olis/2004doc.nsf/linkto/env-jm-mono(2004)24>.

OECD. 2007. Guidance on grouping of chemicals. OECD Series on Testing andAssessment No. 80. Organisation for Economic Co-operation and Development,Paris, France.

OECD. 2011. OECD Series on Testing and Assessment Number 138. Report of theWorkshop on Using Mechanistic Information in Forming Chemical Categories.ENV/JM/MONO(2011) 8. Organisation for Economic Co-operation andDevelopment. Paris, France.

Organisation for Economic Cooperation and Development, (OECD), 2012b. TheAdverse Outcome Pathway for Skin Sensitisation by Covalent Binding toProteins. Part 2: Use of the AOP to Develop Chemical Categories andIntegrated Testing and Assessment Approaches. OECD, Paris.

Organisation for Economic Cooperation and Development (OECD), 2013. GuidanceDocument on Developing and Assessing Adverse Outcome Pathways. OECD,Paris.

OECD. 2014. Guidance on grouping of chemicals. OECD Series on Testing andAssessment No. 194. Organisation for Economic Co-operation andDevelopment, Paris, France.

OECD. 2014b. How to use the Toolbox AOP workflow for Skin Sensitization.Accessible at <http://www.oecd.org/env/ehs/risk-assessment/Tutorial_1_How%20to%20use%20AOP%20for%20Skin%20sensitization_F_28012014.pdf>.

OECD. 2015a. Test Guideline 442c. in chemico skin sensitisation. Direct PeptideReactivity Assay (DPRA).

OECD. 2015b. Test Guideline 442d in vitro skin sensitisation. ARE-Nrf2 Luciferasetest method.

Pande, K., Moran, S.M., Bradfield, C.A., 2005. Aspects of dioxin toxicity are mediatedby interleukin 1-like cytokines. Mol. Pharmacol. 67, 1393–1398.



http://guidance.echa.europa.eu/docs/guidance_document/information_requirements_r7a_en.pdf?vers=20_08_08



http://www.ecetoc.org/technical-reports

http://dx.doi.org/10.1016/j.taap.2010.03.009

http://dx.doi.org/10.1016/j.tiv.2013.09.009

http://dx.doi.org/10.1016/j.tiv.2013.09.009





















http://dx.doi.org/10.1093/toxsci/kfm064

http://dx.doi.org/10.1093/toxsci/kfm064























http://dx.doi.org/10.3109/10408444.2011.558487





http://dx.doi.org/10.1080/10408390903110692










http://dx.doi.org/10.1002/em.20369

















http://dx.doi.org/10.1093/toxsci/kft176




















http://appli1.oecd.org/olis/2004doc.nsf/linkto/env-jm-mono(2004)24

http://appli1.oecd.org/olis/2004doc.nsf/linkto/env-jm-mono(2004)24








http://www.oecd.org/env/ehs/risk-assessment/Tutorial_1_How%20to%20use%20AOP%20for%20Skin%20sensitization_F_28012014.pdf

http://www.oecd.org/env/ehs/risk-assessment/Tutorial_1_How%20to%20use%20AOP%20for%20Skin%20sensitization_F_28012014.pdf




Pastoor, T.P., Bachman, A.N., Bell, D.R., Cohen, S.M., Dellarco, M., Dewhurst, I.C., Doe,J.E., Doerrer, N.G., Embry, M.R., Hines, R.N., Moretto, A., Phillips, R.D., Rowlands,J.C., Tanir, J.Y., Wolf, D.C., Boobis, A.R., 2014. A 21st century roadmap for humanhealth risk assessment. Crit. Rev. Toxicol. 44 (Suppl. 3), 1–5. http://dx.doi.org/10.3109/10408444.2014.931923.

Patlewicz, G., Simon, T., Goyak, K., Phillips, R.D., Rowlands, J.C., Seidel, S., Becker,R.A., 2013a. Use and validation of HT/HC assays to support 21st century toxicityevaluations. Regul. Toxicol. Pharmacol. 65, 259–268.

Patlewicz, G., Ball, N., Booth, E.D., Hulzebos, E., Zvinavashee, E., Hennes, C., 2013b.Use of category approaches, read-across and (Q)SAR: general considerations.Regul. Toxicol. Pharmacol. 67, 1–12.

Patlewicz, G., Ball, N., Becker, R.A., Booth, E.D., Cronin, M.T.D., Kroese, D., Steup, D.,Van Ravenzwaay, B., Hartung, T., 2014a. Food for thought. . .. Read-acrossapproaches – misconceptions, promises and challenges ahead. ALTEX 31, 387–396.

Patlewicz, G., Kuseva, C., Mehmed, A., Popova, Y., Dimitrova, G., Ellis, G., Hunziker,R., Kern, P., Low, L., Ringeissen, S., Robert, D.W., Mekenyan, O., 2014b. TIMES-SS– recent refinements as a result of an Industrial skin sensitisation consortium.SAR QSAR Environ. Res. 25, 367–391.

Patlewicz, G., Kuseva, C., Kesova, A., Popova, I., Zhechev, T., Pavlov, T., Roberts, D.W.,Mekenyan, O.M., 2014c. Towards AOP application – implementation of anintegrated approach to testing and assessment (IATA) into a pipeline tool forskin sensitization. Regul. Toxicol. Pharmacol. 69, 529–545.

Pelkonen, O., Tolonen, A., Rousu, T., Tursas, L., Turpeinen, M., Hokkanen, J., Uusitalo,J., Bouvier d’Yvoire, M., Coecke, S., 2009. Comparison of metabolic stability andmetabolite identification of 55 ECVAM/ICCVAM validation compounds betweenhuman and rat liver homogenates and microsomes – a preliminary analysis.ALTEX 26, 214–222.

Petkov, P.I., Rowlands, J.C., Budinsky, R., Zhao, B., Denison, M.S., Mekenyan, O., 2010.Mechanism-based common reactivity pattern (COREPA) modelling of arylhydrocarbon receptor binding affinity. SAR QSAR Environ. Res. 21, 187–214.http://dx.doi.org/10.1080/10629360903570933, 920009073 [pii].

Python, F., Goebel, C., Aeby, P., 2007. Assessment of the U937 cell line for thedetection of contact allergens. Toxicol. Appl. Pharmacol. 220, 113–124.

Reisinger, K., Hoffmann, S., Alépée, N., Ashikaga, T., Barroso, J., Elcombe, C., Gellatly,N., Galbiati, V., Gibbs, S., Groux, H., Hibatallah, J., Keller, D., Kern, P., Klaric, M.,Kolle, S., Kuehnl, J., Lambrechts, N., Lindstedt, M., Millet, M., Martinozzi-Teissier,S., Natsch, A., Petersohn, D., Pike, I., Sakaguchi, H., Schepky, A., Tailhardat, M.,Templier, M., van Vliet, E., Maxwell, G., 2014. Systematic evaluation of non-animal test methods for skin sensitisation safety assessment. Toxicol. In Vitro29, 259–270.

Roberts, D.W., Aptula, A.O., Patlewicz, G., Pease, C., 2008. Chemical reactivity indicesand mechanism-based read across for non-animal based assessment of skinsensitization potential. J. Appl. Toxicol. 28, 443–454.

Roberts, D.W., Natsch, A., 2009. High throughput kinetic profiling approach forcovalent binding to peptides. Application to skin sensitising potency of Michaelacceptor electrophiles. Chem. Res. Toxicol. 22, 592–603.

Roberts, D.W., Patlewicz, G., 2014. Integrated testing and assessment approaches forskin sensitization: a commentary. J. Appl. Toxicol. 34, 436–440.

Rorije, E., Aldenberg, T., Buist, H., Kroese, D., Schüürmann, G., 2013. The OSIRISweight of evidence approach: ITS for skin sensitisation. Regul. Toxicol.Pharmacol. 67, 146–156. http://dx.doi.org/10.1016/j.yrtph.2013.06.003.

Rotroff, D.M., Wetmore, B.A., Dix, D.J., Ferguson, S.S., Clewell, H.J., Houck, K.A.,Lecluyse, E.L., Andersen, M.E., Judson, R.S., Smith, C.M., Sochaski, M.A., Kavlock,R.J., Boellmann, F., Martin, M.T., Reif, D.M., Wambaugh, J.F., Thomas, R.S., 2010.Incorporating human dosimetry and exposure into high-throughput in vitrotoxicity screening. Toxicol. Sci. 117, 348–358.

Rotroff, D.M., Dix, D.J., Houck, K.A., Knudsen, T.B., Martin, M.T., McLaurin,K.W., Reif, D.M., Crofton, K.M., Singh, A.V., Xia, M., Huang, R., Judson,R.S., 2013. Using in vitro high throughput screening assays to identifypotential endocrine disrupting chemicals. Environ. Health Perspect. 121,7–14.

Rowlands, J.C., Budinsky, R., Gollapudi, B., Novak, R., Abdelmegeed, M., Cukovic, D.,Dombokowski, A., 2011. Transcriptional profiles induced by the arylhydrocarbon receptor agonists 2,3,7,8-tetrachlorodibenz-p-dioxin, 2,3,7,8-tetrachlorodibenzofuran and 2,3,4,7,8-pentachlorodibenzofuran in primary rathepatocytes. Chemosphere 85, 232–238.

Sakaguchi, H., Ashikaga, T., Kosaka, N., Sono, S., Nishiyama, N., Itagaki, H., 2007. Thein vitro skin sensitization test; human cell line activation test (h-CLAT) usingTHP-1 cells. Toxicol. Lett. 172, S93.

Scholz, S., Sela, E., Blaha, L., Braunbeck, T., Galay-Burgos, M., García-Franco, M.,Guinea, J., Klüver, N., Schirmer, K., Tanneberger, K., Tobor-Kapłon, M., Witters,H., Belanger, S., Benfenati, E., Creton, S., Cronin, M.T.D., Eggen, R.I.L., Embry, M.,Ekman, D., Gourmelon, A., Halder, M., Hardy, B., Hartung, T., Hubesch, B.,Jungmann, D., Lampi, M.A., Lee, L., Léonard, M., Küster, E., Lillicrap, A.,Luckenbach, T., Murk, A.J., Navas, J.M., Peijnenburg, W., Repetto, G., Salinas, E.,Schüürmann, G., Spielmann, H., Tollefsen, K.E., Walter-Rohde, S., Whale, G.,Wheeler, J.R., Winter, M.J., 2013. A European perspective on alternatives toanimal testing for environmental hazard identification and risk assessment.Regul. Toxicol. Pharmacol. 67, 506–530.

Schultz, T.W., Yarbrough, J.W., Johnson, E.L., 2005. Structure–activity relationshipsfor reactivity of carbonyl compounds with glutathione. SAR QSAR Environ. Res.16, 313–322.

Shah, I., Houck, K., Judson, R.S., Kavlock, R.J., Martin, M.T., Reif, D.M., Wambaugh, J.,Dix, D.J., 2011. Using nuclear receptor activity to stratify hepatocarcinogens.PLoS ONE 6, e14584. http://dx.doi.org/10.1371/journal.pone.0014584.

Sheikh-Bahaei, S., Maher, J.J., Hunt, C.A., 2010. Computational experiments revealplausible mechanisms for changing patterns of hepatic zonation of xenobioticclearance and hepatotoxicity. J. Theor. Biol. 265, 718–733.

Simon, T., Aylward, L.L., Kirman, C.R., Rowlands, J.C., Budinsky, R.A., 2009. Estimates ofcancer potency of 2,3,7,8-tetrachlorodibenzo(p)dioxin using linear and nonlineardose–response modeling and toxicokinetics. Toxicol. Sci. 112, 490–506.

Simon, T., Stoney Simons Jr., S., Preston, R.J., Boobis, A.R., Cohen, S.M., Doerrer, N.G.,Fenner-Crisp, P.A., McMullin, T., McQueen, C.A., Rowlands, J.C., 2014. The use ofmode of action information in risk assessment: quantitative key events/dose–response framework for modeling the dose–response for key events. Crit. Rev.Toxicol., in press

Teeguarden, J., Hanson-Drury, S., Fisher, J.W., Doerge, D.R., 2013. Are typical humanserum BPA concentrations measurable and sufficient to be estrogenic in thegeneral population? Food Chem. Toxicol. 62, 949–963. http://dx.doi.org/10.1016/j.fct.2013.08.001.

Teeguarden, J.G., Calafat, A.M., Ye, X., Doerge, D.R., Churchwell, M.I., Gunawan, R.,Graham, M.K., 2011. Twenty-four hour human urine and serum profiles ofbisphenol a during high-dietary exposure. Toxicol. Sci. 123, 48–57. http://dx.doi.org/10.1093/toxsci/kfr160.

Teeguarden, J.G., Hanson-Drury, S., 2013. A systematic review of Bisphenol A ‘‘lowdose’’ studies in the context of human exposure: a case for establishingstandards for reporting ‘‘low-dose’’ effects of chemicals. Food Chem. Toxicol. 62,935–948. http://dx.doi.org/10.1016/j.fct.2013.07.007.

Thomas, R.S., Black, M., Li, L., Healy, E., Chu, T.-M., Bao, W., Andersen, M., Wolfinger,R., 2012. A comprehensive statistical analysis of predicting in vivo hazard usinghigh-throughput in vitro screening. Toxicol. Sci. http://dx.doi.org/10.1093/toxsci/kfs159.

Thomas, R.S., Philbert, M.A., Auerbach, S.S., Wetmore, B.A., Devito, M.J., Cote, I.,Rowlands, J.C., Whelan, M.P., Hays, S.M., Andersen, M.E., Meek, M.E.B., Reiter,L.W., Lambert, J.C., Clewell, H.J., Stephens, M.L., Zhao, Q.J., Wesselkamper, S.C.,Flowers, L., Carney, E.W., Pastoor, T.P., Petersen, D.D., Yauk, C.L., Nong, A., 2013.Incorporating new technologies into toxicity testing and risk assessment:moving from 21st century vision to a data-driven framework. Toxicol. Sci. 136,4–18. http://dx.doi.org/10.1093/toxsci/kft178.

Tluczkiewicz, I., Batke, M., Kroese, D., Buist, H., Aldenberg, T., Pauné, E., Grimm, H.,Kühne, R., Schüürmann, G., Mangelsdorf, I., Escher, S.E., 2013. The OSIRIS weightof evidence approach: ITS for the endpoints repeated-dose toxicity (RepDoseITS). Regul. Toxicol. Pharmacol. 67, 157–169. http://dx.doi.org/10.1016/j.yrtph.2013.02.004.

Tollefsen, K.E., Scholz, S., Cronin, M.T., Edwards, S.W., de Knecht, J., Crofton, K.,Garcia-Reyero, N., Hartung, T., Worth, A., Patlewicz, G., 2014. Applying adverseoutcome pathways (AOPs) to support integrated approaches to testing andassessment (IATA). Regul. Toxicol. Pharmacol. 70, 629–640.

Tonnelier, A., Coecke, S., Zaldívar, J.-M., 2011. Screening of chemicals for humanbioaccumulative potential with a physiologically based toxicokinetic model.Arch. Toxicol. http://dx.doi.org/10.1007/s00204-011-0768-0.

Urbisch, D., Mehling, A., Guth, K., Ramirez, T., Honarvar, N., Kolle, S., Landsiedel, R.,Jaworska, J., Kern, P.S., Gerberick, F., Natsch, A., Emter, R., Ashikaga, T.,Miyazawa, M., Sakaguchi, H., 2014. Assessing skin sensitization hazard inmice and men using non-animal test methods. Regul. Toxicol. Pharmacol. 71,337–351.

van der Veen, J.W., Rorije, E., Emter, R., Natsch, A., van Loveren, H., Ezendam, J.,2014. Evaluating the performance of integrated approaches for hazardidentification of skin sensitizing chemicals. Regul. Toxicol. Pharmacol. 2014(69), 371–379.

van Leeuwen, K., Schultz, T.W., Henry, T., Diderich, B., Veith, G.D., 2009. Usingchemical categories to fill data gaps in hazard assessment. SAR QSAR Environ.Res. 20, 207–220.

Vezin, C.M., Walker, N.J., Olson, J.R., 2004. Subchronic exposure to TCDD, PeCDF, PCB126 and PCB 153: effect on hepatic gene expression. Environ. Health Perspect.112, 1636–1644.

Vinken, M., 2013. The adverse outcome pathway concept: a pragmatic tool intoxicology. Toxicology 312, 158–165. http://dx.doi.org/10.1016/j.tox.2013.08.011.

Wambaugh, J.F., Setzer, R.W., Reif, D.M., Gangwal, S., Mitchell-Blackwood, J., Arnot,J.A., Joliet, O., Frame, A., Rabinowitz, J., Knudsen, T.B., Judson, R.S., Egeghy, P.,Vallero, D., Cohen Hubal, E.A., 2013. High-throughput models for exposure-based chemical prioritization in the ExpoCast project. Environ. Sci. Technol. 47,8479–8488.

Yao, C.Q., Prokopec, S.D., Watson, J.D., Pang, R., P’ng, C., Chong, L.C., Harding, N.J.,Pohjanvirta, R., Okey, A.B., Boutros, P.C., 2012. Inter-strain heterogeneity in rathepatic transcriptomic responses to 2,3,7,8-tetrachlorodibenzo-p-dioxin(TCDD). Toxicol. Appl. Pharmacol. 260, 135–145.

Wetmore, B.A., Wambaugh, J.F., Ferguson, S.S., Sochaski, M.A., Rotroff, D.M.,Freeman, K., Clewell, H.J., Dix, D.J., Andersen, M.E., Houck, K.A., Allen, B.,Judson, R.S., Singh, R., Kavlock, R.J., Richard, A.M., Thomas, R.S., 2011. Integrationof dosimetry, exposure and high-throughput screening data in chemical toxicityassessment. Toxicol. Sci. 125, 157–174. http://dx.doi.org/10.1093/toxsci/kfr254.

Zhao, Y.Y.I., Tao, F.M., Zeng, E.Y., 2008. Theoretical study of the quantitativestructure–activity relationships for the toxicity of dibenzo-p-dioxins.Chemosphere 73, 86–91.

http://dx.doi.org/10.3109/10408444.2014.931923

http://dx.doi.org/10.3109/10408444.2014.931923

























http://dx.doi.org/10.1080/10629360903570933


















































http://dx.doi.org/10.1371/journal.pone.0014584












http://dx.doi.org/10.1016/j.fct.2013.08.001





http://dx.doi.org/10.1093/toxsci/kfs159

http://dx.doi.org/10.1093/toxsci/kfs159

http://dx.doi.org/10.1093/toxsci/kft178







http://dx.doi.org/10.1007/s00204-011-0768-0
















http://dx.doi.org/10.1016/j.tox.2013.08.011














Documents

Regulatory Toxicology and Pharmacology · approach for a chemical risk assessment, in a manner analogous to the use of MoA in the ILSI KEDRF and HESI Q-KEDRF (Julien et al., 2009;