MI-210: Essentials of Population PKPD Modeling and Simulation · Chapter 0 Getting Started 0.1 Course Introduction MI-210: Essentials of Population PKPD Modeling and Simulation provides

MI-210: Essentials of Population PKPDModeling and Simulation

Marc R. Gastonguay, Ph.D.Chairman and Scientific Director

and

Metrum Institute Faculty

Contents

Contents 2

0 Getting Started 30.1 Course Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30.2 Student Expectations and Requirements for Certificate . . . . . . . . . . . . . . . . 30.3 Course Content Management Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40.4 Metrum Institute R Web Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40.5 About the Course Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50.6 Course Presentation and Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60.7 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1 Introduction to Population PKPD Modeling 81.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2 Introduction to Population PKPD Modeling and Simulation . . . . . . . . . . . . 9

1.2.1 Population PKPD Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.2.2 History and Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.2.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.2.4 Modeling and Simulation in Drug Development . . . . . . . . . . . . . . . 12

1.3 Introduction to Nonlinear Regression Concepts . . . . . . . . . . . . . . . . . . . . . 181.3.1 A Phase 1 Exposure-Response Example . . . . . . . . . . . . . . . . . . . . . . 181.3.2 The Method of Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . . . 191.3.3 Maximum Likelihood Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 191.3.4 Least-Squares Objective Functions . . . . . . . . . . . . . . . . . . . . . . . . . 261.3.5 Diagnostic Plots for Regression Models . . . . . . . . . . . . . . . . . . . . . . 26

1.4 Examples (Excel Workbook) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

1.5.1 Problem 1.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411.5.2 Problem 1.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

1.6 Maximum Likelihood Models for Population Repeated Measures Data . . . . 431.6.1 Data for Population Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431.6.2 Hierarchical Population Mixed-Effects Models . . . . . . . . . . . . . . . . . 441.6.3 Population NLMEM Objective Functions . . . . . . . . . . . . . . . . . . . . 46

2

MI-210: Essentials of Pop PKPD M&S CONTENTS

1.6.4 Diagnostics for Population NLME Models . . . . . . . . . . . . . . . . . . . . 611.7 Study Guide Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2 Population PK-PD Data Requirements and Formatting 752.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752.2 Data Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 762.3 Data Specification for the NONMEM® System . . . . . . . . . . . . . . . . . . . . . . 77

2.3.1 General Data Requirements for NMTRAN . . . . . . . . . . . . . . . . . . . . 782.3.2 Data Requirements for NMTRAN with PREDPP . . . . . . . . . . . . . . . . 802.3.3 Data Item Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 822.3.4 NMTRAN Control Records: $DATA and $INPUT . . . . . . . . . . . . . . . 86

2.4 Data Assembly Points to Consider and Best Practices . . . . . . . . . . . . . . . . . 872.5 Practice Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 902.6 Study Guide Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3 Coding Population Nonlinear Mixed Effects Models 923.1 Coding Pharmaco-Statistical Models through NMTRAN . . . . . . . . . . . . . . . 933.2 Interpreting Modeling Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 993.3 The PREDPP Model Library for Pop PK Models . . . . . . . . . . . . . . . . . . . . . . 1043.4 Practice Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1073.5 Study Guide Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4 Covariate Model Building 1094.1 Objectives of Covariate Model Development . . . . . . . . . . . . . . . . . . . . . . . 1104.2 Common Covariate Model Parameterizations . . . . . . . . . . . . . . . . . . . . . . 111

4.2.1 Covariate-Parameter Models for Continuous Covariates . . . . . . . . . . 1114.2.2 Covariate-Parameter Models for Categorical Covariates . . . . . . . . . . 1124.2.3 Combining Continuous & Categorical Covariates . . . . . . . . . . . . . . . 1124.2.4 Desirable Properties of Covariate Model Parameterizations . . . . . . . 113

4.3 Data Reduction: Before You Start Covariate Model Building . . . . . . . . . . . . 1134.4 Covariate Modeling Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

4.4.1 Traditional Covariate Screening Methods in Population PKPD . . . . . 1154.4.2 Other Covariate Modeling Methods . . . . . . . . . . . . . . . . . . . . . . . . . 118

4.5 Inferences about Covariate Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1234.6 Other Statistical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1254.7 Aligning Methods with Modeling Purpose . . . . . . . . . . . . . . . . . . . . . . . . . 1284.8 Practice Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1294.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1294.10 Study Guide Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

5 Simulation 1315.1 Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1315.2 Calling Random Number Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

3


5.3 Simulation Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

6 Model Qualification 1526.1 Can We Agree on a Name? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1536.2 A Risk-Based Approach to Model Qualification . . . . . . . . . . . . . . . . . . . . . . 1536.3 What to Evaluate or Qualify? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1546.4 Model Qualification Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

6.4.1 Assumption Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1566.4.2 Test Data Sets for Model Qualification . . . . . . . . . . . . . . . . . . . . . . . 1586.4.3 Log-Likelihood Profile: Qualification of Parameter Estimates . . . . . . 1586.4.4 Bootstrap: Qualification of Parameter Estimates . . . . . . . . . . . . . . . 1606.4.5 Leverage Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1626.4.6 Qualification Based on Predictive Performance . . . . . . . . . . . . . . . . 1636.4.7 Posterior Predictive Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1666.4.8 When is Evaluation Less Important? . . . . . . . . . . . . . . . . . . . . . . . . 169

6.5 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

7 Direct and Indirect Continuous Pop PK-PD Models 1747.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1757.2 Basic PD Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1767.3 $PRED Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1767.4 Linking PK and PD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1777.5 PK-PD Modeling Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1787.6 Direct PK-PD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1797.7 Effect Compartment PK-PD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1817.8 Non-Parametric Effect Compartment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1827.9 PK-PD Model with Tolerance* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1827.10 Indirect PD Response Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1837.11 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

8 Regulatory Guidance and Best Practices 1878.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1878.2 Regulatory Support for M&S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

8.2.1 Regulatory Meetings/Interactions: End of Phase IIa Meeting . . . . . . 1888.2.2 Regulatory Review and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 1898.2.3 Regulatory Guidance Documents . . . . . . . . . . . . . . . . . . . . . . . . . . 1898.2.4 FDA Population Pharmacokinetics Guidance . . . . . . . . . . . . . . . . . 1908.2.5 EMEA Guidance on Pop PK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

8.3 Review of Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1938.3.1 Systems and Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1938.3.2 Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1948.3.3 Modeling and Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

4


8.3.4 Reporting and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . 1968.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

A Course Project 198A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198A.2 Project Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

B Complete Courseware License 200

Bibliography 205

5

Chapter 0

Getting Started

0.1 Course Introduction

MI-210: Essentials of Population PKPD Modeling and Simulation provides an extensiveoverview of topics in population pharmacokinetics and pharmacodynamics. The courseduration and content is equivalent to a single semester 3 credit course at a typical institu-tion of higher learning. Each week’s topic will consist of a lecture (two hours) followed bya hands-on lab (one hour). The general plan will be as follows:

• Lectures will be on Tuesdays at 2 PM EDT.

• Hands-on labs will be on Fridays at 2 PM EDT (in some cases, the lecture may finishduring the first part of the lab on Friday).

0.2 Student Expectations and Requirements for Certificate

• All students are expected to attempt the hands-on exercises prior to the Friday lab.Instructors will not grade homework assignments, but will review solutions with theentire class on a weekly basis.

• A midterm take-home exam will be assigned at the midpoint of the course.

• A final take-home exam will be assigned at the end of the course (due one week afterit is posted).

6

MI-210: Essentials of Pop PKPD M&S 0. Getting Started

• Students will be required to complete and submit a modeling project before the endof the course. This project will be based on a real-world (or similar) problem, andwill include components of data assembly, model development and evaluation, anda brief report. More details to follow...

• Course grade will be based on the midterm (25%), final (25%) and modeling project(50%)

0.3 Course Content Management Site

All students should already have an account to access the main course website. Here’s thelink: (http://training.metruminstitute.org). Postings to this site will automatically gener-ate an email message to your own e-mail accounts. This site is intended to be the primaryrepository for all course resources including:

• News Forum: Here you’ll find updates about class schedule, assignments, etc.

• Discussion Forum: Direct your questions about course content to instructors orother students here. You can contribute to ongoing discussions or start a new thread.

• Technical Support: Use this forum to submit technical support tickets for problemswith any of the Metrum Institute web resources.

• Links to the Metrum Institute Web Server (SIMI), and GoToWebinar webcast regis-tration form.

• Course Materials: You’ll find course notes, examples, and links to recorded lecturesunder each weekly class heading.

0.4 Metrum Institute R Web Server

All modeling and simulation examples will be run on the Metrum Institute R Web Server(SIMI). Metrum Institute has taken care of all software installation and licensing for you.This includes NONMEM® 7, Intel® Fortran, NMQual (and Perl), R, and MIfuns. User-name, password, and instructions for login to the server were supplied via email. Thelogin instructions are also listed below:

7

http://training.metruminstitute.org


1. Use a web browser (all mainstream browsers supported) and go to the SIMI webaddress:http://serverapp.org/sge

2. Enter your username and password.

3. Click the log in button.

4. Click on the Guide item for additional information on the use of SIMI.

0.5 About the Course Materials

Each week, you will find a new chapter of this book along with any other course materials(examples and problems) on the main course website (http://training.metruminstitute.org).The package will be a zip file. Normally you should:

1. Download the package to your computer.

2. Login to SIMI.

3. Upload the package to your home directory on SIMI.

4. Find the package in the file list.

5. Click the unpack link in the far right column next to the package. The first time youdo this, a new folder called MI210W with the contents of the package will be created.Subsequent packages will unpack into that same folder.

6. You might also find it useful to extract from the zip file the chapter in PDF format foruse on your local machine.

Some book conventions to be aware of: Code that appears within a sentence will use afixed-width font like this. Blocks of code will appear as black text on a light bluebackground

like so.

The output of that code may follow and will appear as blue text

just like this.

8

http://serverapp.org/sge

http://training.metruminstitute.org


0.6 Course Presentation and Topics

The current offering of MI210 represents a significant re-write of the original MI210 course,with the goal of teaching the theory and application of population PKPD modeling withinthe context of real-world problems, rather solely focusing on technical details about howto operate a particular piece of software.

Given this goal, all of the hands-on examples will be related to a realistic modeling andsimulation progression throughout the course of a hypothetical drug development pro-gram. Our hypothetical drug is MI2005A, a novel mechanism n.c.e. under investigation asan analgesic for acute and chronic pain. We’ll tackle the following modeling and simula-tion tasks as part of the development of MI2005A:

• Phase 1: Single ascending dose study

– Evaluate concentration-response relationship for QT prolongation

– Assemble and format population PK data

– Develop a base population PK model

– Simulate expected multiple-dose PK profiles

• Phase 1: Multiple ascending dose study

– Develop multiple-dose PK model

– Develop PKPD models for biomarker and toxicity endpoints

– Perform a predictive check model qualification

– Simulate to explore doses for Phase 2A

• Phase 2a: Phase 2a proof of concept study

– Refine multiple-dose PK model

– Develop exposure-response models for clinical efficacy and toxicity endpoints

– Simulate expected Phase 2b study design

9


• Phase 3: Phase 2b completed and Phase 3 ongoing

– Write formal modeling and simulation plan

– Assemble and format data from Phase 2 and 3 studies

– Develop population PK model with covariates

– Revise exposure-response models

– Perform predictive check and bootstrap model qualification

– Prepare regulatory-submission report

Interspersed in the hypothetical development program problems will be a series of didac-tic lectures on the following topics:

• Introductory concepts, theory and estimation methods for population nonlinearmixed-effects modeling

• Data requirements and formatting

• Development of the basic pharmaco-statistical model structure for PK and PKPDsystems

• Modeling effects of covariates

• Monte-Carlo simulation

• Model qualification methods

• Regulatory guidance and best practices

0.7 Licensing

Your use of the materials provided in this course is subject to the terms of use described inthe courseware license agreement in Appendix B.

10

Chapter 1

Introduction to Nonlinear Regression andNonlinear Mixed-Effects Models forRepeated Measures Population Data

1.1 Overview

• Introduction to Population PKPD Modeling and Simulation

– Definition

– History and Rationale

– Software

– Modeling and Simulation in Drug Development

• Fitting a Model to Data: Nonlinear Regression

– Phase 1 Exposure-Response Examples

– The Method of Maximum Likelihood

– Least-Squares Objective Functions

– Diagnostics for Regression Models (Individual or Naive-Pooled Data)

• Maximum Likelihood for Population Repeated Measures Data

– Hierarchical Mixed-Effects Models

– Some Population NLMEM Objective Functions

– Diagnostics for Population Models

11

MI-210: Essentials of Pop PKPD M&S 1. Introduction to Population PKPD Modeling

1.2 Introduction to Population PKPD Modeling and Simu-lation

1.2.1 Population PKPD Defined

What is Population PK-PD?

• Determine PK (PD) model structure for the population

• Estimate typical (mean) population PK (PD) parameters and inter-individual vari-ability

• Estimate individual PK (PD) parameters

• Estimate residual (and inter-occasion) variability

• Identify measurable sources of variability in PK (PD) and describe their relationshipto PK (PD) parameters

• Study all of these things in the intended patient population

Why Population PK-PD?

• Goal: To understand factors leading to variability in pharmacokinetic and pharma-codynamic response for appropriate drug use?

– Efficient (and cost effective) way to screen large number of diverse individualsfrom the target population

– Allows for investigation of multiple factors at once:

multiple drug interactions, food effects, pathophysiology, other demographicfactors, and combinations of these

12


Population vs. Traditional Approaches for PK-PD Data

Population PK-PD

• Sparse sampling

• Single large study or pooled data

• Diverse patient population

• Efficient means of studying several fac-tors

• Relevant results

• Complex data analysis resulting in usefulmodel

• Exploratory (?)

Traditional PK-PD

• Extensive sampling

• Single small study

• Homogeneous population

• Single factor per study

• Often extrapolate results

• Non-compartmental data analysis

• Confirmatory

Modeling to Explain Variability in Pharmacologic Response

Goal: To understand factors leading to variability in pharmacokinetic and pharmacody-namic response for appropriate drug use and trial design

How: Develop models to describe and quantify measurable and unexplained sources ofvariability

Population Data Analysis Methods

• Standard two-stage

• Naive pooled or averaged data

• Nonlinear mixed-effects modeling (various estimation methods)

These terms will be defined later.

13


1.2.2 History and Rationale

• Original Goal: To develop model-based method for analyzing (sparse) populationdata from routine clinical observations. Traditional data analysis methods were notadequate for this purpose.

• 1977: Prototype population analysis software developed at UCSF

• 1979: First version of NONMEM (Nonlinear Mixed-Effects Model) for IBM main-frame

• 1980: Sheiner and Beal first publish details of NONMEM system.

• 1988: Discrete non-parametric maximum likelihood method (Mallet et al.)

• 1990: Lindstrom and Bates NLME method

• 1991 (and ‘98, ‘99): FDA/Academia/Industry Conferences on PK-PD in drug devel-opment

• 1992: Smooth non-parametric maximum likelihood method (Davidian and Gallant)

• 1992: Mixed-effects model, linear in random effects (Vonesh and Carter)

• 1992: NONMEM Users Net established

• 1993: Bayesian hierarchical approach using Markov-Chain Monte Carlo methods(Bennet and Wakefield)

• 1994: AAPS forms Population PK-PD Focus Group

• 1995-1996: 23% of FDA drug submissions contained population PK (PD) analysisreports

• 1999: Final FDA Guidance—Population Pharmacokinetics

• 1999: Joint FDA/PhRMA Population PK-PD Workshop

Current Applications

• Academic Research: applied, methodological, and software

• Clinical Therapeutics: Population priors for (Bayesian) individualization of drugtherapy

• Drug Development: Population PK-PD modeling and simulation to impact both in-ternal and regulatory decisions (e.g. strategic M&S)

14


1.2.3 Software

• NONMEM

• MONOLIX

• NLME (S-PLUS and R)

• S-ADAPT

• Phoenix NLME

• WinBUGS, OpenBUGS (with BUGSModelLibrary)

• SAS PROC NLMIXED (SAS)

list not meant to be inclusive

1.2.4 Modeling and Simulation in Drug Development

"Drug Development=Model-Building”

• Quantitative support for decision making

• Population parameter estimation

• Transition from Phase I to Phase II/III

• Dose selection for Phase III

• Dose adjustment in special populations

• Confirm drug interaction studies

• Support for confirmatory efficacy/safety

Sources of PK-PD Data in Drug Development

• Pre-clinical toxicokinetics/PK development

• Extensive sampling Phase I studies in small numbers of healthy volunteers or pa-tients

• Sparse to moderate sampling Phase II studies in a focused group of patients

• Sparse sampling in large-scale Phase III efficacy and safety trials in patients

• Published literature, competitor data

15


• Post-marketing studies

16

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling

©2007 metrum institute 4

Introduction to Nonlinear Regression and Nonlinear Mixed-Effects Models

Pharmacometrics

!! Study PK/PD/TD response in context of disease progression

!! Determine typical population response

!! Understand and quantify variability in PK and response

!! Quantify unexplained variability & uncertainty

DOSE CONCENTRATION RESPONSE

…the science of interpreting and describing

pharmacology in a quantitative fashion (e.g. through

modeling and simulation)

DIS

EA

SE

PR

OG

RE

SS

IO

N

17


Goal of M&S in Drug Development

Increase efficiency of drug development and minimize cost of missed opportunities:

• Focus on lead candidates sooner

• Decrease time to market

• Decrease probability of failed or useless trials

• Develop the right dose(s)

• Avoid post-marketing labeling/dose changes or withdrawals

Why Pharmacometrics in Industry?

• Maximize value of prior information

– All available data within company, literature, previous trials, related compound-s/analogues, competitors

• Models allow exploration and quantitative evaluation of

– Current knowledge of product performance

– Competing strategies/ downstream options

– Novel (adaptive) trial strategies

– Sensitivities to key assumptions and uncertainties

• Integrates knowledge and provides a common platform for communication and col-laboration

– Across drug development disciplines (Scientific, Clinical, Commercial, Finan-cial)

– Between development team and decision-making bodies

– Across time

18

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



M&S Throughout Drug Development

Competitor Data

& Market Expectations:

Simulation & Optimization

of Phase III Trial Outcome

Probabilities

Bayesian Data Analysis &

Adaptive Designs: Early

Probability of Success &

Design for Next Phase

Pop PKPD M&S

for Labeling

Support

Preclinical &

Early Development:

PK-PD, Systems

Biology M&S

Therapeutic Area

Knowledge: Disease

Progression & Meta-

Analysis M&S

Toxicology

PK

Biomarker

E-R

Human

MTD, PK

Biomarker,

Tolerability, E-R

Biomarker, Efficacy,

Tolerability E-R

Dose-Response

Covariates, Pop PK-PD

Efficacy,

Safety & Dose

Special

Populations

New

Formulations

Bridge to New

Indications

Figure 1.1: M&S Throughout Drug Development

19


Regulatory Support for M&S: ICH Guidance Documents

• Dose-Response Information to Support Drug Registration (E4)

• Studies in Support of Special Populations: Geriatrics (E7)

• Ethnic Factors in the Acceptability of Foreign Clinical Data (E5)

• Clinical Investigation of Medicinal Products in the Pediatric Population (E11)

Regulatory Support for M&S: FDA Guidance Documents

• Population Pharmacokinetics (FDA and EMEA)

• Exposure-Response Relationships

• General Considerations for the Clinical Evaluation of Drugs (FDA 77-3040)

• General Considerations for Pediatric Pharmacokinetic Studies

• Pharmacokinetics in Patients with Impaired Renal Function

• Pharmacokinetics in Patients With Impaired Hepatic Function

Regulatory Support for M&S: End of Phase IIa Meeting

• Proposed by FDA at Nov. 2003 Clinical Pharmacology Advisory Committee Meeting

• Goal: To improve design and outcome of Phase 2b studies to improve success ratein Phase 3

• Quantitative analysis using exposure-response data

• Important addition to traditional FDA-sponsor meetings during development

• Pilot program: voluntary sponsor participation

• Several meetings conducted to date

Regulatory Support for M&S: Current Practice

• Review of Submitted Sponsor-Conducted Analyses

• Regulatory Decisions Based on FDA-Conducted M&S

• Advocating Model-Based Drug Development and Drug Review Processes (Part ofCritical Path Initiative)

20


1.3 Introduction to Nonlinear Regression Concepts

1.3.1 A Phase 1 Exposure-Response Example

A multiple cohort ascending single dose study has been conducted and is the first in hu-man exposure to MI2005A. Plasma drug concentrations, as well as various safety end-points have been measured. One of the endpoints of interest is the effect of MI2005A oncardiac repolarization, as measured by QT-interval prolongation. In this particular study,serial ECGs were collected in a time-matched baseline and on active treatment for eachsubject. QT-interval was measured and corrected for heart-rate and the resulting changefrom baseline QTc values are available for analysis.

concentration (ng/mL)

delta

QT

c (m

sec)

0

10

20

30

0 500 1000 1500 2000

Figure 1.2: Delta QTc vs. Plasma MI2005a Concentration

21


1.3.2 The Method of Maximum Likelihood

Fitting a Model to Data

• Observe data

• Specify a model

• Choose initial estimates for parameters

• Obtain best estimates of parameters, given data and model

• Evaluate model fit

Terminology

Dependent Variable (DV): observed data to be described by the model (e.g. plasma drugconcentration)

Independent Variable: observed or measured quantities used in model prediction - theseare typically the primary predictors of the system response (e.g. dose or time in PKmodels)

Covariate Factors: other observed or measured quantities used in model prediction—typically used to explain inter-individual differences in parameters (e.g. weight,CLcr)

Parameters: variables to be estimated, given the data and proposed model (e.g. CL, V)

1.3.3 Maximum Likelihood Approach

Let observed data Y be described with a single parameter (θ ) model.

The probability of the data P(Y ) can be modeled as a function of the parameter θ .

When viewed as a function of its parameters the probability of the data P(Y ) is known asthe likelihood of the data, given the model parameters: L(Y | θ )

The value of θ which mazimizes L(Y | θ ) is known as the maximum likelihood estimate(MLE) of θ and is denoted θ

22

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Value of !"

L(Y|!)

Maximum Likelihood

Estimate of !"

depicted as a probability density function

Maximum Likelihood

Figure 1.3: Maximum Likelihood depicted as a probability density function

23


Likelihood Function

The likelihood function of θ is related to P(Y | θ );L(Y | θ ) = c ·P(Y | θ )L(Y | θ )∝ P(Y | θ )

For practical reasons, it is more convenient to work with logarithms. The log-likelihoood(L) function is the natural log of L;l (Y | θ ) = log L(Y | θ ) = log c + log P(Y | θ )l (Y | θ )∝ log P(Y | θ )

The MLE of θ is the value of θ that maximizes l (Y | θ ). To find the MLE of θ , search for θsuch that:

l ′(Y | θ ) =d l (Y | θ )

dθ= 0

In some simple situations, and algebraic solution exists and θ can be calculated with aformula. Usually, the roots of the equation must be determined numerically.

θ should be evaluated for the possibility that it is only a relative or local maximum.

Linear Regression

• Simple condition with algebraic solution of first-derivative equation (see Draper &Smith)

• Obtain MLE and standard errors without a computer.

• Useful for linear relationships

• Can be applied to linear transformations of nonlinear data

• Caution: Transformations can induce improper weighting (e.g. Scatchard plot).

24




Linear Regression (continued)

Nonlinear Regression

• No closed-form solution of first-derivative equation

• Better way to deal with nonlinear data

• Maximize log-likelihood function (or minimize –2*log-likelihood)

• Use iterative numerical techniques to determine MLE of model parameters

• Gradient methods also provide standard errors of parameter estimates

MLE for Normally-Distributed Measurements

Given:Y is vector of data(y1, y2...yn )

y j ∼N ( f (x j ,θ ),σ2)

with independent variables x and model parameters θ

εj ≡ y j − f (x j ,θ )∼N (0,σ2) and are statistically independent

Then,P(y j ) = (2πσ2)−1/2 exp

h

(y j− f (x j ,θ ))2

−2σ2

i

ML Objective Function

25


P(y j ) = (2πσ2)−1/2 exp

(y j − f (x j ,θ ))2

−2σ2

L(Y | x j ,θ ,σ2) =n∏

j=1

(2πσ2)−1/2 exp

(y j − f (x j ,θ ))2

−2σ2

−2 ln L(Y | x j ,θ ,σ2) =n∑

j=1

ln 2π+n∑

j=1

lnσ2+n∑

j=1

(y j − f (x j ,θ ))2

σ2

OFML(x j ,θ ,σ2) =n∑

j=1

(y j − f (x j ,θ ))2

σ2+ lnσ2

Maximum Likelihood Assumptions

• Independence of residuals (obsj−predj)

• Normally-distributed residuals with varianceσ2

• Normal distribution assumption not as important: symmetric distributions workwell

26




Parameter Space

Figure 1.4: Parameter Space

Parameter Search Algorithms

• Steepest descent

– Iterative gradient (derivative) method

– Useful when far away from minimum

• Gauss Newton

– Approximates surface and solves for minimum

– Useful when close to minimum

• Marquardt

– Iterative method that employs steepest descent initially and Gauss Newtonwhen gradient is shallow (near minimum)

• Simplex

– Fast, iterative, derivative free vertex method

– No standard errors of the estimate

27




Parameter Space

Figure 1.5: Parameter Space

Local Minima

• Can be problematic with gradient methods (more than one place on surface withpartial derivative=0)

• Simplex method is less susceptible

• Re-run from different initial estimates

• Check goodness-of-fit diagnostic plots (more later)

Heteroscedastic Variance

• The residual variance, σ2, does not necessarily have to be constant across observa-tions.

• In PK,σ2 is often proportional to the observed concentration.

• Both σ2 and its functional relationship to concentration (or prediction) can be esti-mated with MLE approaches.

28


1.3.4 Least-Squares Objective Functions

Ordinary Least Squares

OFOLS =n∑

j=1

(obsj−predj)2

Weighted Least Squares (where W is typically 1/obs)

OFW LS =n∑

j=1

(obsj−predj)2 ·Wj

Extended Least Squares (Maximum Likelihood)

OFE LS =n∑

j=1

(obsj−predj)2

σ2j

+ lnσ2j

1.3.5 Diagnostic Plots for Regression Models

• Predicted vs. Observed

• Residuals vs. Predicted

• Weighted Residuals vs. Predicted

• Residuals vs. Time*

• Weighted Residuals vs. Time*

• Observed and Predicted vs. Time*

*assuming Time is independent variable

29

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Predicted vs. Observed

Figure 1.6: Predicted vs. Observed

30

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Residuals vs. Prediction

Figure 1.7: Residuals vs. Prediction

31

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Residuals vs. Time

Figure 1.8: Residuals vs. Time

32

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Weighted Residuals vs. Prediction

Figure 1.9: Weighted Residuals vs. Prediction

33

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Weighted Residuals vs. Time

Figure 1.10: Weighted Residuals vs. Time

34

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling

1.4 Examples (Excel Workbook)


35

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling


36


Results of Heteroscedastic Data

Parameter EstimatesParameter LR OLS WLS ELS True

CL (L/hr) 3.18 3.11 3.28 3.09 3.00V (L) 25.52 24.82 25.14 25.30 20.00

% error CL 5.65 3.69 8.42 2.77% error V 21.63 19.41 20.45 20.95

37

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Diagnostics

38

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Diagnostics

39

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Diagnostics

40

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Diagnostics

41

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Diagnostics

42

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



Diagnostics

43


1.5 Problems

1.5.1 Problem 1.1.

Using NLRworkbook.xls as a template, develop a model and estimate maximum likelihoodparameter estimates for the pooled Phase 1 MI2005A delta QTc concentration-responsedata. Compare estimates under different least-squares objective functions. Data can befound in a separate file (dqtc.csv). This is a population data set, but for this exerciseyou may treat the entire data set as a single individual (also known as a naive-pooledapproach). Assume a linear structural model, but explore different assumptions aboutresidual error patterns (e.g. homoscedastic vs. heteroscedastic).

concentration (ng/mL)

delta

QT

c (m

sec)

0

10

20

30

0 500 1000 1500 2000

Figure 1.11: Delta QTc vs. Plasma MI2005a Concentration

1.5.2 Problem 1.2.

Using NLRworkbook.xls as a template, develop a model and estimate maximum likeli-hood parameter estimates for the pooled Phase 1 MI2005A AST concentration-responsedata. Compare estimates under different least-squares objective functions. Data can befound in a separate file (ast.csv). This is a population data set, but for this exercise you maytreat the entire data set as a single individual (also known as a naive-pooled approach).

44


Consider linear and non-linear (e.g. Emax) structural models, and explore different as-sumptions about the residual error variance.

DV

AS

T

0

20

40

60

80

0 500 1000 1500

Figure 1.12: Observed AST vs. Plasma MI2005a Concentration

45


1.6 Maximum Likelihood Models for Population RepeatedMeasures Data

1.6.1 Data for Population Analyses

Dependent Variable Types

• Continuous

– concentrations, blood pressure, weight

• Categorical (this is a subject for another course)

– Dichotomous (yes/no)

– Ordered Categorical (pain rating scale)

– Counts (number of seizures/month)

– Non-Ordered Categorical (EEG sleep stages)

– Time-to-Event (survival)

Later, we will discuss the use of categorical covariates as predictors of continuous dependentvariables.

Data Terminology

• Repeated Measures

– More than one observation per individual

• Longitudinal

– Repeated measures over time

• Sparse

– Too few samples to characterize the model accurately with data from only oneindividual

• Extensive

– Enough samples to characterize the model accurately with data from only oneindividual

46


1.6.2 Hierarchical Population Mixed-Effects Models

Population PK Analysis Methods

• Naive Pooled Analysis

• Naive Averaged Analysis

• Standard Two-Stage

• MAP Bayesian

• Nonlinear Mixed-Effects

– Parametric

– Non-Parametric

Naive Pool

• Combine all the data as if they came from a single individual.

• Fit the model to all the data using least-squares regression (e.g., WLS) and estimatemodel parameters (θ ).

Weighted Least Squares for Naive Pool (where W is typically 1/obs):

OFW LS,N P =n∑

j=1

(obsj − f (θ )j )2 ·Wj

Naive Average

• Obtain the average concentration across individuals at each time point.

• Fit the model to average data with least-squares regression (e.g., WLS) and estimatemodel parameters (θ ).

• Requires PK sampling at same time for each individual

Weighted Least Squares for Naive Average (where obsj =mean(obsj ) across individuals):

OFW LS,N A =n∑

j=1


47


Standard Two-Stage

Step 1: Estimate each individual subject’s PK and/or PD parameters from rich data usingleast-squares, then explore resulting covariate-parameter relationships with regres-sion techniques (requires extensive sampling).

for each individual:

OFW LS,STS =n∑

j=1


across individuals:θC Li = A + B ·C RC L i

Step 2: Summarize parameters to obtain population mean, variance, and covariance.

MAP Bayesian Estimation

Maximum a posteriori probability estimates of individual random effects (ηi ), which isequal to population mean parameter − individual parameter), conditional on estimatesof population mean (θ ), interindividual (co)variance (Ω), and residual variance (σ2) areobtained by minimizing the following objective function for each individual’s data:

OFPOSTHOCi =Σj

logσ2+(yi j − f i j (θ ,ηi ))2/σ2

+η′

iΩ−1ηi

(NONMEM POSTHOC objective function for additive variance models—more later)

...where yi j and f i j are the observed and model predicted values, respectively, for individ-ual i and sampling point j .

48


1.6.3 Population NLMEM Objective Functions

Variability in Pharmacologic Response

Goal: To understand factors leading to variability in pharmacokinetic and pharmacody-namic response for appropriate drug use and trial design

How: Develop models to describe and quantify measurable and unexplained sources ofvariability

Nonlinear Mixed Effects Models

• One-stage (simultaneous) estimation of fixed effects and random effects parameters

• Utilize nested random error hierarchy (typically, at least inter-individual and resid-ual levels).

• Employ maximum likelihood (or similar) estimation methods.

Fixed Effects: estimated parameters relating observed or measured variables (e.g. dose,time, weight, GFR) to model prediction.

Random Effects: unexplained random variability (e.g. inter-individual variability, resid-ual error)

Nested Random Effects: Population PK-PD models typically describe at least two levelsof random effects with a nested (hierarchical) model.

• Level 1: Inter-Individual Variability

• Level 2: Residual Variability

49

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling©2007 metrum institute 102


50

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling©2007 metrum institute 103


51

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



52

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



53

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



! = 0

54

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



! = 0

! = !i 55

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling



! = 0

! = !i 56


Random Effects Hierarchy of Typical Population PK Model

• 2 levels of random effects

– Inter-individual Variability

C L i = T V C L · e ηC L,i

– Residual Variability

yi j =k0

C L i

1− e−C Li

Vi·Ti j

· e−C Li

Vi·t ′i j + εi j

Nested Random Effects

yi j =k0

T V C L · e ηC L,i

1− e−T V C L·eηC L,i

T V V ·eηV,i Ti j

· e−T V C L·eηC L,i

T V V ·eηV,i t′i j + εi j

where:

η∼N (0,ω2)

ε ∼N (0,σ2)

Inter-Individual Variance Models

Just like residual variance models in maximum likelihood methods, inter-individual ran-dom effects may be described according to different patterns or distributions. Here aresome common models:

Additive (constant variance)

C L i = θ1+θ2CrCli +η1i

Proportional (constant coefficient of variation)

C L i = (θ1+θ2CrCli ) · (1+η1i )

Exponential (constant CV, log normal distribution)

C L i = (θ1+θ2CrCli ) ·exp(η1i )

57


or

ln(C L i ) = ln(θ1+θ2CrCli )+η1i

Residual Variance Models

Assumptions about residual error variance structure are similar to those addressed inMaximum Likelihood estimation with individual data.

Homoscedastic (additive, or constant variance)

Yi j = Fi j + ε1i j

Heteroscedastic (proportional, or constant CV)

Yi j = Fi j · (1+ ε1i j )

Exponential (approximates constant CV)

Yi j = Fi j ·exp(ε1i j )

or

ln Yi j = ln Fi j + εi j

Combination Additive and Proportional

Yi j = Fi j · (1+ ε1i j )+ ε2i j

58


Nonlinear Mixed-Effects Methods (Parametric vs. Non-Parametric Interindividual Ran-dom Effects)

• Parametric (assumed distribution for inter-individual random effects)

– First-Order (FO), First-Order Conditional Estimation (FOCE), Laplacian, NLME,Bayesian Hierarchical

• Non-Parametric (no assumed distribution of inter-individual random effects, butresidual variance model must be known)

– NPEM, NBML, NPAG (discrete density), NLMIX (continuous smooth density)

Each method makes assumptions. It is the modeler’s responsibility to check thoseassumptions and apply the method that is most useful for the intended purpose.

ML Objective Function for Single-Subject Data

OFM L(x ,θ ,σ2) =n∑

j=1

(y j − f (x j ,θ ))2

σ2+ logσ2

ML Objective Function for Population Model

For an individual

−2 log L i (Yi | x i ,θ ,C i ) =OFi (x i ,θ ,C i ) =n i∑

j=1

(y j − f (x j ,θ ))T C−1i (y j − f (x j ,θ ))

+ log(detC i )

For the population

−2 log L(Y | x ,θ ,C ) =OF (x ,θ ,C ) =N ,n i∑

i=1,j=1

(yi j − f (x i j ,θ ))T C−1i (yi j − f (x i j ,θ ))

+N∑

i=1

log(detC i )

59


Individual Variance-Covariance Matrix

C i =G iΩG Ti +HiΣH T

i

G i is a matrix of partial derivatives of f i j (θ ,x i j )with respect to η

Ω is population inter-individual covariance matrix

Hi is a matrix of partial derivatives of f i j (θ ,x i j )with respect to ε∑

is a covariance matrix for residual error, ε (usually diagonal)

Covariance Matrix for Residual Error

∑

=

σ1,1 σ1,2

σ2,1 σ2,2

∑

is a covariance matrix for residual error, ε

Dimension is related to number of residual variance terms

If no covariance between residual errors off-diagonal elements are 0

Inter-individual Covariance Matrix

Ω=

ω1,1 ω1,2 ω1,3

ω2,1 ω2,2 ω2,3

ω3,1 ω3,2 ω3,3

Ω is population inter-individual covariance matrix (covariance of η)

Dimension is determined by the number of inter-individual variance parameters

If no covariance between parameters, off-diagonal elements = 0

60


Correlation of ETAs



Correlation of ETAs

Figure 1.13: Correlation of ETAs

When BLOCK OMEGA is estimated for variances A and B and covariance AB:

corrA B =covA Bp

varA ·p

varB

61


First-Order Approximation

yi j = f i j (θ +ηi )+ εi j

yi j = f i j (θ )+∂ f i j (θ )∂ ηi

(ηi = 0) ·ηi + εi j +ξi j

ψi j = E (yi j ) = f i j (θ )+∂ f i j (θ )∂ ηi

(ηi = 0) ·0+0+E (ξi j )

ψi j = f i j (θ )+E (ξi j )

ψi j∼= f i j (θ )

ξi j the error associated with approximation

ψi j the expected value of yi j ; the prediction

Likelihood Approximations

Classic Estimation Methods implemented in NONMEM

(Newer estimation methods will be discussed later in this course.)

• First-Order Method (FO): η= 0use POSTHOC step for conditional ηi

• First-Order Conditional Estimation (FOCE): η=ηi

• Laplacian Approximation: a 2nd order expansion and η=ηi

• Hybrid: η= 0 or ηi

• eta-epsilon interaction: η = ηi and ηi included in residual variance when non-additive

62


POSTHOC Step: Caution with MAP Bayes

• Individual parameter estimate may dominate even when data are very sparse. Thebalance is affected by relative magnitude of inter-individual covariance (Ω) and resid-ual variance (σ2).

• Data Dominated Estimate (Ωσ2)

• Pop. Prior Dominated Estimate (σ2Ω)

OFPOSTHOCi =Σj

logσ2+(yi j −M i j (θ ,ηi ))2/σ2

+η′

iΩ−1ηi

FOCE-Interaction Example

Only relevant when residual variance model is a function of model prediction (e.g. propor-tional, exponential, etc.):

Yi j = Fi j a + Fi j b ·E PS(1)i j

Fi j a Fi j b

FO η= 0 η= 0FOCE η=ηi η= 0FOCE/INT η=ηi η=ηi

where Fi j is model prediction for individual i and sampling time j

63


1.6.4 Diagnostics for Population NLME Models

• Minimum Objective Function Value

• Fixed and Random Effect Parameter Estimates

• Standard Errors

• ETABAR (with conditional estimation)

• Diagnostic Plots

• AIC

• Likelihood Ratio Test

WRES and CWRES

Formerly (NONMEM VI and earlier), WRES was used for all NONMEM estimation meth-ods:

EFO,i j ( f ) = f (x i j ,θ , 0)

C i FO(yi j ) =d f

dηi

ηi=0·Ω ·

d f T

dηi

ηi=0+diag

d f

d εi j

εi j=0·Σ ·

d f T

d εi j

εi j=0

WRESi j =(yi j −EFO,i j ( f ))p

C i FO(yi j )

CWRES is more appropriate for conditional estimation:

EFOC E ,i j ( f ) = f (x i j ,θ ,ηi )−d f

dηi

ηi=ηi

·ηi

C i FOC E (yi j ) =d f

dηi

ηi=ηi

·Ω ·d f T

dηi

ηi=ηi

+diag

d f

d εi j

εi j=0·Σ ·

d f T

d εi j

εi j=0

C W RESi j =(yi j −EFOC E ,i j ( f ))p

C i FOC E (yi j )

64


WRES and CWRES in NONMEM

• WRES is proper diagnostic for FO only

• CWRES is more accurate diagnostic for all conditional estimation methods

• CWRES may also be useful under FO to investigate possible impact of conditionalestimation

• Multiple variations of the CWRES are now available as standard output in NONMEMVII

• See NONMEM help topic "PRED, RES, WRES" and PAGE 2006 presentation by A.Hooker et al.

65

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling

NONMEM Output: Minimum Value of the Objective Function

********************

**************************************** MINIMUM VALUE OF OBJECTIVE FUNCTION *************************************

*********************************************************************************************************************************************** 2433.919 **************************************1*****************************************************************************************

66

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling

NONMEM Output: Final Parameter Estimates



67

MI-210:E

ssentials

ofP

op

PK

PD

M&

S1.In

trod

uctio

nto

Pop

ulatio

nP

KP

DM

od

eling

NONMEM Output: Standard Errors



68


NONMEM Output: ETABAR

ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, AND THE P-VALUE ISGIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0.

ETABAR: -0.16E-01 -0.28E-01 -0.11E-01 0.49E-02

P VAL.: 0.76E+00 0.69E+00 0.75E+00 0.61E+00

NOT a test of NORMAL distribution!

69


Diagnostic Plots for Population Models



Observed and Predicted vs. Time

Figure 1.14: Observed and Predicted vs. Time

70




Observed vs. Predicted

Figure 1.15: Observed vs. Predicted



RES & WRES vs. Predicted

Figure 1.16: RES and WRES vs. Predicted

71




RES & WRES vs. Time

Figure 1.17: RES and WRES vs. Time



CWRES vs. Prediction and Time

Figure 1.18: CWRES vs. Prediction and Time

72




Distribution of CWRES

Figure 1.19: Distribution of CWRES



Distribution of ETAs

Figure 1.20: Distribution of ETAs

73




Correlation of ETAs

Figure 1.21: Correlation of ETAs

74


Akaike Information Criterion

• AIC is a useful statistic to compare goodness-of-fit between competing models (alongwith other diagnostics) for the same data set.

• Lower AIC= better fit to the data

• AIC = 2 (no. parameters) + (−2 log L)

• Model A

– Min OFV = −2 log L = 2469

– No. parameters = 12

– AICA = 2(12)+2469= 2493

• Model B

– Min OFV =−2 log L = 2454

– No. parameters = 14

– AIC B = 2(14)+2454= 2482

2482< 2493 Model B fits the data better than Model A.

Model-Based Inference

• Hypothesis testing by model-building

• Point and interval estimates (or posterior distributions) of model parameters andderived quantities

Hypothesis Testing

• Comparisons should be pre-specified in the data analysis plan.

• Specify alternative models to be tested.

• Likelihood Ratio Test

• Randomization Test

• Should adjust for multiple comparisons

• Not as useful when objective is learning

75


Likelihood Ratio Test

• Compare goodness-of-fit for nested models.

• Determine the statistical significance of model parameters.

• −2x (difference of log-likelihoods) from nested models: approximately asymptoti-cally χ2 distributed, a useful statistic

(−2 log L full)− (−2 log L reduced)∼χ2(df =∆p ) (1.1)

∆p is difference in number of parameters

• Compare the difference in −2 log L between models (∆OF V ) with χ2 distributiontable for specific df.

• Full model:

– CL=THETA(1) + THETA(2)*SMK + ETA(1)

– −2 log L full = 483

• Reduced model:

– CL=THETA(1) + ETA(1)

– −2 log L reduced = 490

∆OF V = 7p-value critical value0.05 3.84 (1 df)0.005 7.88(1 df)

BUT: What is magnitude of SMK effect and how well do we know it?

• χ2 approximation can be sensitive to assumptions.

– Normality of Residuals Assumption

– Likelihood Approximations (e.g. estimation methods)

• Actual significance level may be very different.

• FOCE/Interaction or Laplacian/Interaction result in ∆OFVs that are closest to χ2

distribution when model is correct

• FO is not reliable for statistical significance.

• Empirical methods are more accurate than theoretical assumptions (e.g. random-ization test).

76


Inference by Estimation

• When objective is learning (not confirming), estimation and description are moreuseful than hypothesis testing.

• Point and interval estimates of model parameters and derived quantities can beused to determine clinical relevance of model components.

• Confidence intervals can be obtained from asymptotic standard errors in NONMEM,the log-likelihood profile, or more robustly by bootstrap methods (more later).

1.7 Study Guide Questions

• Which diagnostic plot is most informative about the pattern of residual error vari-abilty?

• Which diagnostic plot is most informative about the performance of the modelingstrategy with respect to residual error variance?

• What are some strategies that can useful in detecting and avoiding local minima?

• What are the distributional assumptions associated with the maximum likelihoodobjective function?

• Can Maximum Likelihood methods be accurately applied to continuous data thatare non-normally distributed?

• Describe the key assumptions made in the FO approximation.

• What is the eta-epsilon interaction?

• If ETABAR is not significantly different from zero, does that indicate that underlyingassumptions about inter-individual random effects have been met?

77

Chapter 2

Population PK-PD Data Requirementsand Formatting

2.1 Overview

• Data Terminology

• Introduction to NONMEM® Software

• NMTRAN Requirements and Formats

• Data Assembly Points to Consider and Best Practices

– Data Specification Document

– Data Quality and Audit Trail

– CRF and Database Considerations

• Practice Problems

• Study Guide Questions

78

MI-210: Essentials of Pop PKPD M&S 2. Population PK-PD Data Requirements and Formatting

2.2 Data Terminology

• Dependent Variable (DV)

– Quantity to be modeled

– May be multivariate (e.g. concentration and effect, parent and metabolite)

– Sometimes useful to use a transformation of observed scale, such as log, orlogit transformations.

• Independent Variable(s)

– Known, observed or measurable quantities which serve as primary predictorsof the dependent variable (e.g. Dose, Time).

– In regression models, independent variables are assumed to be known withouterror.

• Covariates

– Known, observed or measured variables which may be predictive of model pa-rameters or response.

• Repeated Measures

– More than one observation per individual

• Longitudinal

– Repeated measures over time

• Sparse

– Too few samples to characterize the model accurately with data from only oneindividual

• Extensive

– Enough samples to characterize the model accurately with data from only oneindividual

79


2.3 Data Specification for the NONMEM® System

Although this course is aimed at teaching population modeling methods in general, thedata specification format developed as part of the NONMEM® system is powerful andflexible and has been adopted by other population modeling software tools.

• Nonlinear Mixed-Effects Modeling Software

• Comprised of multiple components:

– NONMEM Estimation Core

– PRED or (user-supplied prediction routine)

– NMTRAN (NONMEM® Translator): converts user-friendly model code (con-trol stream) and data set to NONMEM® fixed format FORTRAN (FCON andFDATA)

NMTRAN Model Specification: NMTRAN Translates ASCII text model codefile (control stream) to NONMEM® FORTRAN subroutine format

· $PRED (simple algebraic model specification)

· PREDPP (Prediction for Population Pharmacokinetics = library of PKmodels)

NMTRAN Data Set: NMTRAN translates ASCII text data set to fixed formatFORTRAN data set. Data file name and contents are specified to NON-MEM® via $DATA and $INPUT records of control stream (more on theselater).

NONMEM/NMTRAN Data Terminology

• Data Item (columns)

– Value corresponding to a particular heading or column in the data set

– NONMEM® 7 data set may include up to 50 data items at one time, but ASCIIdata set can include more items that may be ignored for a particular run.

• Data Record (rows)

– Each row in the data set is a data record.

– With PREDPP, each data event (e.g. dose, PK observation) is represented by adata record.

80


2.3.1 General Data Requirements for NMTRAN

• Data records for an individual must be contiguous.

• Data file must be space- or comma-delimited ASCII text (no tabs!).

• Each data record must have a value or placeholder ( . or 0) for every data item.

• A comment indicator allowed for an entire record.

• Only numeric characters, comment indicators, and placeholders allowed (except fordate and time separators in PREDPP data sets).

• Numerical values in the data file may now be up to 24 characters long (ID values inthe data file may be up to 14 digits long)

• Users specify different NMTRAN data formats depending on type of model subrou-tine:

– $PRED or user-supplied PRED subroutine: no data pre-processor (difficult tospecify observational-type data)

– PREDPP: useful data pre-processor (can specify any type of recursive dosing/ob-servation scenario)

Required Data Items for NONMEM® Estimation Core

• NONMEM estimation core requires DV, MDV data items for individual data

• In addition, ID data item is required for population models

• L2 data item is optional (more in MI212 course)

• NMTRAN will impute default values for MDV

Some Data Items Defined

• DV: Dependent Variable

• MDV: Missing Dependent Variable (1=missing, 0=not missing)

• ID: Individual Identifier (also known as L1 data item in population models)

• L2: Level-Two Indicator (used to identify records for correlated residual errors)

81


Data for $PRED or User-Written PRED

• Only NONMEM data items (DV, MDV, ID, and L2) are recognized.

• ID and DV must be present for population models.

• All other data items are user-defined (avoid NONMEM/NMTRAN and PREDPP re-served names).

• Model is not recursive, so each data record must contain all necessary data items forprediction calculation.

• General NONMEM/NMTRAN data requirements also apply.

Example 1

• NMTRAN data set for $PRED model: (C t =Dose/V ·exp−CL/V ·t )

• Single-dose IV bolus (100mg) with concentration observations (mg/L) at 2 and 6hours post dose.

C ID TIME DOSE DV. 1 2 100 24. 1 6 100 18. 2 2 100 29. 2 6 100 21

• Text file saved as example_1.csv (comma-separated values)

• View in spreadsheet (above) or text editor (here):

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

!"#"$%&'()*&+&,#-$./*$012%30

<

=#%%<';*,>4;'-+),-,4,* $?

!"#"$%&'()*&+&,#-$./*$012%30

45"+67&$8

6 @*A,'1-/*').B*8'.)'*A.;3/*C$DE)B'FE0;;.6)*3.>.,*8

B./4*)G

6 H-*I'-+'(AE*/'F3>*B-04)'3.9*G'0>',*A,'*8-,0>'FJ*>*G&

=#%%<';*,>4;'-+),-,4,* $K

!"#"$%&'()*&+&,#-$./*$012%30

9(&-#)/,-:

82


2.3.2 Data Requirements for NMTRAN with PREDPP

PREDPP Data Possibilities (Library of PK Models)

• Single or multiple IV or PO doses

• Single or multiple zero-order infusions

• Steady-state dosing/infusions

• Irregular dosing intervals

• Irregular observation times

• Multivariate observations (e.g. concentration and effect)

• Time-dependent covariates

• Combinations of any/all of the above

Data for PREDPP

• Part of the model code is included in data set.

• Each data record represents a single event.

• Events can be DV observations, doses or other (e.g. covariate measurement).

• Data records are also used to reset amounts to zero and to turn compartments on/off.

• Recursive subroutine will back-fill current value of covariates to time of precedingdata record.

• General NONMEM/NMTRAN data requirements also apply.

83


Required PREDPP Data Items

• Required for pop. models: ID, DV, AMT, TIME (default values for MDV and EVID areimputed automatically by NMTRAN)

• EVID

– Specifies type of event for each data record with PREDPP model.

0 = observation of DV1 = dose2 = other (on/off, covariate change, etc.)3 = reset system to zero amount4 = reset and dose

• AMT

– Amount of drug mass entering system (e.g. dose)

– User to keep track of units

• TIME

– Time of event

– Can be elapsed time or observed clock time

– Can be used in conjunction with DATE

Example 2

• NMTRAN data set (example_2.csv) for PREDPP model

• Single-dose IV bolus (100mg) with concentration observations (mg/L) at 2 and 6hours post-dose

C ID TIME AMT DV. 1 0 100 .. 1 2 . 24. 1 6 . 18. 2 0 100 .. 2 2 . 29. 2 6 . 21

84


2.3.3 Data Item Names

• Reserved: ID, L1, L2, DV, MDV, RAW_, MRG_, TIME, DATE, DAT1, DAT2, DAT3, DROP,SKIP, EVID, AMT, RATE, SS, II, ADDL, CMT, PCMT, CALL, CONT

• Do not use reserved parameter names: (e.g., ETAn, CL, V, K, KA, S1, F0, R1, F1, etc.)

• User-defined data item labels may be up to 20 characters long, but must begin witha letter.

Example 3

• NMTRAN data set (example_3.csv) for PREDPP model

• Multiple oral dosing (30 mg q8 hours) for 2 days

• Observational-type PK sampling

New Data Items

• ADDL: number of additional doses

• II: inter-dose interval (same scale as TIME)

Implementation for Example 3

• One dosing record specifies all 6 doses for an individual (1 + 5 additional).

• Observation times are elapsed times from start of study.

C ID TIME AMT ADDL II DV WT CLCR. 1 0 30 5 8 . 64 80. 1 18 . . . 126 64 80. 1 23.8 . . . 33 64 70. 2 0 30 5 8 . 81 110. 2 10 . . . 150 81 110. 2 42 . . . 45 81 125

85


Example 4

• Steady-state constant rate infusion (10 mg/hour)

• PK observation at end of infusion

• Second constant rate infusion (20 mg/hour) starts at 4 hours after termination offirst infusion and lasts for 2 hours.

• PK sample upon termination of 2nd infusion and at 6 hours after start of 2nd infu-sion

New Data Items

• SS: steady-state data item

1 = SS dose (reset initial amounts to zero)2 = superposition of SS dosenull = non-SS dose or non-dosing event

• RATE: zero-order rate

> 0 = rate in mass/time−1 = estimate rate−2 = estimate durationnull = bolus dose or non-dosing event


• NMTRAN/PREDPP data set (example_4.csv)

• SS infusion: AMT=0, TIME is end of infusion

• Other infusions: AMT=total amount, TIME is start of infusion

• Two events can have same TIME.

C ID TIME AMT RATE SS DV0 1 0 0 10 1 00 1 0 0 0 0 124.90 1 4 40 20 0 00 1 6 0 0 0 135.60 1 10 0 0 0 110.6

86


Example 5

• Crossover from oral to IV dosing at steady-state

• 1 compartment disposition model

• Initialize steady-state oral dosing of 50 mg q 12 hours.

• PK sample taken at next trough

• Observe next oral dose time and use actual time.

• PK sample at approximately 3 hours post-dose

• IV bolus dose (50 mg) at 12 hours after last dose

• PK sample at 1 hour post-IV bolus

• Date and time collected as calendar date and clock time

New Data Items

• CMT: compartment data item

0 = output compartment (e.g. urine data)>0 = model compartment number (for dose, observation or any otherevent)<0= turn off compartment number (e.g. end of urine collection interval)

NMTRAN will impute default CMT value for PREDPP PK model library.

87


Compartmental Model

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

!"#"$%&'()*&+&,#-$./*$012%30

$<

=#%%>';*,?4;'-+),-,4,* #@

!"#"$%&'()*&+&,#-$./*$012%30

45"+67&$8

A*B'7.,.'",*;)

C!D&'E0;3.?,;*+,'8.,.'-,*;

''%' 04,34,'E0;3.?,;*+,'F*G9G'4?-+*'8.,.H

I% ;08*/'E0;3.?,;*+,'+4;J*?'F10?'80)*K'

0J)*?L.,-0+'0?'.+M'0,N*?'*L*+,H

O% ,4?+'011'E0;3.?,;*+,'+4;J*?

F*G9G'*+8'01'4?-+*'E0//*E,-0+'-+,*?L./H

6A!DPQA'B-//'-;34,*'8*1.4/,'C!D'L./4*'10?

2P(722'25';08*/'/-J?.?MG

=#%%>';*,?4;'-+),-,4,* R%

!"#"$%&'()*&+&,#-$./*$012%30

9/+6"*#+&,#"7$1/:&7

!

"#$%$&'%()*

+

",$-$&'%()&.&%/($*

01

2345


• NMTRAN/PREDPP data set (example_5.csv)

• CMT 1=depot, 2=central (see PREDPP GUIDE)

• DATE and TIME will be transformed to elapsed time by NMTRAN/PREDPP.

C ID DATE TIME AMT SS II DV CMT AGE0 1 10/20/2001 8:00 50 1 12 0 1 420 1 10/20/2001 19:52 0 0 0 124.97 2 420 1 10/20/2001 19:53 50 0 0 0 1 420 1 10/20/2001 22:45 0 0 0 135.6 2 420 1 10/21/2001 8:06 50 0 0 0 2 420 1 10/21/2001 9:12 0 0 0 150.9 2 42

88


2.3.4 NMTRAN Control Records: $DATA and $INPUT

Instructions for NMTRAN related to the data set processing are provided via the NMTRANcontrol stream in two specific blocks of code.

$DATA

[filename|*] [(format)] [IGNORE=c1] [NULL=c2] [NOWIDE|WIDE][CHECKOUT] [RECORDS=n1] [LRECL=n2] [NOREWIND|REWIND][NOOPEN] [LAST20=n3] [TRANSLATE=(list)]

Example:

$DATA example_5.csv IGNORE=C WIDE

$INPUT

• List data items corresponding to data set columns.

• Drop unwanted items.

• Assign synonyms to NMTRAN data items.

Example:

$INPUT C ID DATE=DROP TIME AMT SS II DV=CONC CMT AGE

89


2.4 Data Assembly Points to Consider and Best Practices

NMTRAN Data Assembly Issues

• Imputing time-varying covariates

• Imputing missing covariates

• Correct order of records for events occurring at same time

• Missing samples

• Missing dose time

• Missing observation time

Data Specification Document

• Identify ahead of time how data problems will be handled.

• Link NMTRAN data items to source databases.

• Specify format of NMTRAN data set.

• Specify plausible range of values for data items (useful for edit checks).

• Identify special instructions for creation of NMTRAN data set.

90

MI-210:E

ssentials

ofP

op

PK

PD

M&

S2.Po

pu

lation

PK

-PD

Data

Req

uirem

ents

and

Form

atting

Example Data Specification Document

(data_specs.xls)

Data Item Description Units Original Name Source Format Range Continuous/Categorical

No.Levels

Comments

ID NONMEM ID Number N/A SID PKdata.xls integer 1 to 450 categorical 450 create NONMEM ID from SID -start numbering at 1

TIME 24 hr clock time N/A ClockTime PKdata.xls hh:mm 00:00 to 23:59 N/A N/A make sure 00:00 is associated withnew day

DATE calendar date N/A StudyDate PKdata.xls mm/dd/yyyy 1/1/1999 to10/30/2001

N/A N/A use 4 digit year to avoid confusion

AMT dose amount mg DOSE Dosing.xls decimal 0 or "." to 1000 continuous N/A Insert placeholder (0 or "." whennon-dosing record)

DV drug concentration mg/L PK PKdata.xls decimal 0 to 2000 continuous N/A Insert placeholder (0 or "." whennon-observation record)

EVID event ID N/A N/A N/A integer 0 to 4 categorical 5 0=observation, 1=dose, 2=other,3=reset, 4=reset & dose (use 4 forsecond period of crossover)

TYPE observation type N/A N/A N/A integer 0 to 2 categorical 3 0=PK obs, 1=PD obs, 2=doserecords

CMT model compartment N/A N/A N/A integer 1 to 2 categorical 2 1=depot (dosing), 2=central (ob-servations)

SS steady-state indicator N/A N/A N/A integer 0 to 1 categorical 2 0= non steady-state dose,1=steady-state dose

II interdose interval N/A N/A N/A integer 0 to 12 categorical 2 0=observations and non-multipledose record; 12=multiple doserecord

ADDL additional doses N/A N/A N/A integer 0 to 20 continuous N/A shortcut code for multiple dosingrecords, use 0 for non-multipledosing records

WT weight kg WGHT demog.sd2 decimal 0 to 200 continuous N/A For times without weight observa-tion, back-fill from next observedweight.

AGE age years AGE demog.sd2 decimal 18 to 95 continuous N/A noneSEX sex N/A GEND demog.sd2 integer 0 to 1 categorical 2 0=male, 1=femaleSMK smoking status N/A SMOKE demog.sd2 integer 0 to 1 categorical 2 0=nonsmoker, 1=smokerCLCR creatinine clearance mg/dL N/A derived from de-

mog.sd2decimal 20 to 150 continuous N/A calculate using Cockroft-Gault

and truncate at 150 mL/min

Table 2.1: Data Specs

NOTES:1 Back-fill covariate values when changing over time2 Assign EVID=2 where AMT=0 or "." and DV=0 or "."3 Use separate records for each event, even if they occur simultaneously4 Different covariate values measured at the same time should be on same record5 NMTRAN data file should be sorted by ID, DATE, and TIME6 Allow first column to be placeholder (".") for comment indicator7 Header row should have ÔCÕ in the first field8 Do not allow blank fields - convert to "."9 Impute missing continuous covariates with population median by SEX and identify that value was imputed10 Impute missing categorical covariates with most common value in population and identify that value was imputed

91


Data Quality Diagnostics

• Index plots: Graph all data items vs. ID.

• Plot data: DV vs. TIME (or time post-dose) with ID number as label.

• Plot histograms of continuous data items.

• Box plots of continuous data items by categorical variables

• Summary statistics of NMTRAN data items should be compared to source data andprotocol.

Data Assembly Audit Trail

• Goals

– Trace data set to source

– Reproduce data set assembly

• Methods

– Good records (e.g. notebook, data specification document, electronic journal,etc.)

– Use programmable data manipulation tools. (e.g. SPLUS, SAS, AWK, ExcelMacros, etc.)

Data Management Considerations

• Source database files should link all data to associated study, subject ID, calendardate, and actual clock time. (Study day is not always reliable.)

• Source database files should use the same name and units for the same variable inmultiple data sets.

• A unique identifier should link each PK (or PD) sample with study, subject ID, cal-endar date, and actual clock time.

• Dose amounts, dates, and times should be recorded in patient diary or CRF and endup in database.

• Each CRF entry should be linked to study, subject ID, calendar date, and actual clocktime when applicable.

• CRF and database should not contain duplicate information in different areas.

92


Help on Data Assembly

• See NONMEM® help items on $DATA, $INPUT, and specific reserved data itemnames

• MIfuns R package includes useful data formatting functions (more detail providedin MI205 course: R Programming for Pharmacometrics).

2.5 Practice Problems

Problem 1

Create NMTRAN data set for 2 subjects:

• Multiple dosing (100 mg controlled-release oral tablet, q 12 hours for 2 days)

• PK sampling after first dose and last dose at 2, 4, 6, 12 hours post-dose

• Insert arbitrary values for concentrations.

• Estimate zero-order release/absorption rate.

• $PRED model (no PREDPP library)

Problem 2

Create NMTRAN data set for 2 subjects:

• Multiple dosing (100 mg controlled-release oral tablet, q 12 hours for 2 days)

• PK sampling after first dose and last dose at 2, 4, 6, 12 hours post-dose

• Insert arbitrary values for concentrations.

• Estimate zero-order release/absorption rate.

• To be analyzed with PREDPP library

93


Problem 3

Create an NMTRAN data set for this hypothetical study:

• Multiple dosing (200 mg q 12 hours) Phase III study of Drug-X

• First dose is IV infusion (over 1 hour), but remaining doses are oral.

• At 3 weeks, a dose increase is allowed (increase to 400 mg bid), depending on re-sponse.

• Sparse PK sampling after first dose and at 1, 3, and 6 weeks (1, 6, 10 hours post AMdose)

• 1-compartment disposition with t1/2 = 8-10 hours

• Covariates (AGE, WT, CLCR, SEX)

• Assume PREDPP libraries will be used for modeling.

• Create hypothetical NMTRAN data set for 2 subjects (1 with/1 without week 3 doseincrease).


• What are the core required data items for population models in NONMEM® ? ForNMTRAN with PREDPP?

• When is it useful to use EVID = 3? EVID = 2 ?

• What are acceptable field delimiters for NMTRAN data sets?

• Why is good practice to use a data specification document?

94

Chapter 3

Coding Population Nonlinear MixedEffects Models

Overview

• Coding Pharmaco-Statistical Models through NMTRAN

• Interpreting Modeling Output

• The PREDPP Model Library for Pop PK Models

• Practice Problems

• Study Guide Questions

95

MI-210: Essentials of Pop PKPD M&S 3. Coding Population Nonlinear Mixed Effects Models

3.1 Coding Pharmaco-Statistical Models through NMTRAN

The NONMEM® System

• Nonlinear Mixed-Effects Modeling Software

• Comprised of multiple components:

– NONMEM Estimation Core

– PRED (user-supplied prediction routine)

– REDPP (Prediction for Population Pharmacokinetics = library of Pop PK mod-els)

– NMTRAN (NONMEM Translator): converts user-friendly data set and modelcode to NONMEM fixed format FORTRAN

Coding Models for NONMEM/NMTRAN

• User-written PRED routine (FORTRAN)

– No data pre-processor or model library

– User supplies partial derivatives

• NMTRAN control stream with $PRED

– No data pre-processor or model library

– Automatic partial derivatives

• NMTRAN control stream for PREDPP

– Data pre-processor and model library

– Automatic partial derivatives

NMTRAN Control Streams

• ASCII text file

• Create in text editor (e.g. Notepad)

• No tabs

• All uppercase letters (except file names)

96


• 80-character line length limit

• ; indicates comment for all characters to the right

$PRED Model

• Simple NMTRAN control stream

• Code (user) must define link between independent variables and DV.

• Data items required for population models: ID, DV

• Required control stream records for estimation:

– $PROBLEM

– $INPUT

– $DATA

– $PRED

– $THETA

– $OMEGA

– $SIGMA

– $ESTIMATION

NMTRAN Control Records

(also see NONMEM VII help files)

• $PROBLEM

– Problem description or title, any useful text

Example:$PROB Pop PK Analysis of Drug X Base Model

• $INPUT

– Defines data set contents

97


Example:$INPUT C ID TIME AMT DV AGE WT

• $DATA

– Defines data set name and file reading options

Example:$DATA 123.CSV IGNORE=C

• $PRED

– Model code section defined by user

– Defines fixed and random effect parameters

• $THETA

– Initial estimates for fixed-effects parameters

– (LOWER, INITIAL, UPPER) -or-

– (LOWER, INITIAL) -or-

– (INITIAL)

Examples:

Three fixed effects parameters, one with a lower bound, one with no bounds andone with both upper and lower bounds.$THETA (0, 20)(0.045)(0, 0.5, 1)

A fixed element of the $THETA vector.$THETA (0 FIX)

• $OMEGA

– Initial estimates for variance-covariance of inter-individual random effects

– Can be DIAGONAL (default) or BLOCK (includes off-diagonal elements)

– No bounds on estimation

98


– FIX will fix entire BLOCK

Examples:

Diagonal covariance matrix for random effects on CL and V:$ OMEGA 0.04 0.09

Full block covariance matrix for random effects on CL and V:$ OMEGA BLOCK (2)0.040.01 0.09

Diagonal and partial block covariance matrix for random effects on ka, ALAG, CLand V (with covariance between the latter two):$ OMEGA 0.04 0.09$ OMEGA BLOCK (2)0.040.01 0.09

• $SIGMA

– Initial estimates for variance-covariance of residual random effects

– Can be DIAGONAL (default) or BLOCK

– No bounds on estimation

– FIX will fix entire BLOCK

Examples:

A single residual variance term.$ SIGMA 0.04

Modeling covariance in residual error between multivariate endpoints (e.g. parentand metabolite data).$ SIGMA BLOCK (2)0.040.01 0.09

99


• $ESTIMATION

– Specifies parameter estimation options

– Defines model specification file name

Examples:

FO method with MAP Bayes (POSTHOC) individual estimation.$EST MAXEVAL=0 POSTHOC

FOCE with interaction method.$EST MAX=9999 METHOD=1 INTERACTION SIGDIG=2 PRINT=10 MSF=001.MSF

• $TABLE

– Specifies options and file name for NONMEM output table

– May have more than one $TABLE per problem

$TABLE NOPRINT ONEHEADER FILE=001.TAB ID TIME IPRE CL V ETA(1)

$TABLE NOPRINT NOHEADER FIRSTONLY NOAPPEND FILE=123.PRN ID TRTCL V1 V2 KA ETA(1)ETA(2)ETA(3)AGE WT CLCR SEX

100


$PRED Example

$PROB EMAX MODEL

$INPUT ID DV CONC

$DATA PDDATA.CSV IGNORE=C

$PRED

EMAX=THETA(1)*EXP(ETA(1))

EC50=THETA(2)*EXP(ETA(2))

E=EMAX*CONC/(EC50+CONC)

Y=E + EPS(1)

$THETA (0, 100) (0, 20) ; EMAX and EC50

$OMEGA 0.4 0.4 ; Diagonal random effects matrix

$SIGMA 9 ; Additive (constant) residual variance

$ESTIMATION MAXEVAL=9999 PRINT=10

$TABLE ID CONC ; DV PRED RES WRES are automaticallyadded

101


3.2 Interpreting Modeling Output

NONMEM® output file

• Review model and estimation options.

• Monitoring of search

– Gradients should be non-zero and becoming more shallow near minimum

– Check for successful minimization or significant digits if terminates in round-ing errors

– Did any parameters hit boundary?

• ETA-BAR

• Minimum value of the objective function

• Parameter estimates

• Standard errors of the estimates

Useful Variance Terms

• For proportional (and exponential) variance models, an approximate coefficient ofvariation (%CV) can be calculated from the variance estimate; e.g. for proportionalvariance parameter A:

%CV A =p

varA ·100

• For additive variance models, a standard deviation (SD) is calculated; e.g. for addi-tive variance parameter B:

SDB =p

varB

102


Correlation of ETAs

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9.+8':-;4/.,-0+

!"#$%$#&'()'"#)*+,&('-$+")*.)/+#0(1

<

=#%%>';*,?4;'-+),-,4,* @

!"#$%$#&'()'"#)*+,&('-$+")*.)/+#0(1

6 AB!(CD

! "+-,-./'*),-;.,*)'10?'E.?-.+F*6F0E.?-.+F*'01'-+,*?6-+8-E-84./'?.+80;*11*F,)

! G.+'H*'7"DCBIDJ'K8*1.4/,L'0?'MJBG5

! I0'H04+8)'0+'*),-;.,-0+

! N"O'P-//'1-Q'*+,-?*'MJBG5

A'B!(CD''%R%S''%R%@

A'B!(CD'MJBG5'K#L

%R%S

%R%$'%R%@

2/3452)6+"-7+()408+7#1

=#%%>';*,?4;'-+),-,4,* $%

!"#$%$#&'()'"#)*+,&('-$+")*.)/+#0(1

6%RS 6%RT 6%R# 6%R$ %R% %R$ %R#

UD#

(UDT

6$R$(6%%V6WR%(6%%>6$R%(6%%>SR%(6%%>

6$R$(6%%V

6WR%(6%%>

6$R%(6%%>

SR%(6%%>

6%RT 6%R# 6%R$ 6%R% %R$ %R#

6$R$(6%%V

6WR%(6%%>

6$R%(6%%>

SR%(6%%>

(UD$

6%RS

6%RT

6%R#

6%R$

%R%

%R$

%R#

6%RT

6%R#

6%R$

6%R%

%R$

%R#

6$R$(6%%V6WR%(6%%>6$R%(6%%>SR%(6%%>

(

(UDS

6+770('-$+")+9):351

When BLOCK OMEGA is estimated for variances A and B and covariance AB, a correlationcoefficient is useful to calculate:

corrA B =covA Bp

varA ·p

varB

103


Distribution of ETAs

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

!"#$%$#&'()'"#)*+,&('-$+")*.)/+#0(1

<

=#%%>';*,?4;'-+),-,4,* $$

!"#$%$#&'()'"#)*+,&('-$+")*.)/+#0(1

6 @A*+'BCDE5'D!(FG'-)'*),-;.,*8'10?'H.?-.+I*)'G'J'B

.+8'I0H.?-.+I*'GBK

!!!!

"#$$%& !"#$ %&

$%&% ! $%&&

2+330('-$+")+4)5671

=#%%>';*,?4;'-+),-,4,* $#

!"#$%$#&'()'"#)*+,&('-$+")*.)/+#0(1

8$1-3$9&-$+")+4)5671

Recall that parametric methods assume that random effects are centered and zero and areNormally-distributed.

NONMEM table file

• Import to graphics program and create diagnostic plots: DV vs. PRED, RES vs. PRED,WRES vs. PRED, etc.

• List population and individual predictions and individual parameter estimates (ifconditional estimation was implemented).

• Plot distributions of ETAs.

Individual Estimates

• Individual estimates are obtained for all estimation methods, except FO (and zero

104


elements of HYBRID).

• With FO or HYBRID, run POSTHOC step to obtain conditional estimates of individ-ual parameters.

• Code in NMTRAN Control Stream:

For $PRED Model:

$PRED...Y = EFFECT + EPS(1)IPRED=EFFECT; or whatever variable name is used for prediction

For PREDPP Model:

$PRED ...$ERROR ...IPRED=F; Prediction is saved in a variable called F.; Could also use a user-defined variable.

– $EST POSTHOC [or use conditional est. method]

– Include parameters and IPRED in $TABLE

Model Specification File

• Capture conditions of parameter space at the end of a run for future use (parame-ters, gradients).

• Useful in case of crash, early termination, or to implement additional NONMEMsteps

• On $ESTIMATION, include model specification output file:

$EST PRINT=5 MAX=9999 MSFO=001.MSF

• To use in a future run, add $MSFI block and remove all $THETA, $OMEGA, $SIGMA:

$MSFI 001.MSF$EST PRINT=5 MAX=0 MSFO=002.MSF POSTHOC

105


$COVARIANCE

• Obtain asymptotic standard errors of the estimates, given a successful minimiza-tion.

• Include in control stream with original model run or use $MSFI to run as a secondstep

• $COVARIANCE (some options available but not typically necessary)

• With complex models, completion of default covariance step is often problematic.

• In NONMEM output, standard errors of estimates, correlation matrix of estimates,and covariance matrix of estimates are produced.

Standard Errors of the Estimates

• It is useful to express standard errors (SE) relative to the parameter estimates (PE);e.g. for parameter C, a percent relative standard error (%RSE) can be calculated (thisis also referred to as a %CV):

%RSEC =SEC

PEC·100

• Approximate, symmetric 95% confidence intervals can also be calculated from theSE and PE. These are often inaccurate for variance parameters and nonlinear mod-els.

95%CIC = PEC ±1.96 ·SEC

Estimated ETA-Shrinkage

• When using ETA-based diagnostics after conditional estimation, it is useful to havean estimate of how much the individual ETA estimates have been influenced by thepopulation prior information (recall words of caution about MAP Bayes estimation).One estimate of this is the estimated ETA-Shrinkage. This calculation provides anestimate of how much the individual random effects have been shrunken towardthe mean value of zero.

106


3.3 The PREDPP Model Library for Pop PK Models

• Library of PK subroutines

• User-friendly code for most continuous PK-PD problems

• 13 different ADVANs (advance the solution from record to record in a recursive cal-culation)

• TRANS routine allows translation to physiologic parameters (e.g. CL, V) or parame-terization as micro-constants (e.g. K10, K12, etc.)

PREDPP Subroutines

ADVAN 1 one-compartment intravenous input

ADVAN 2 one-compartment with 1st order depot

ADVAN 3 two-compartment intravenous input

ADVAN 4 two-compartment with 1st order depot

ADVAN 11 three-compartment intravenous input

ADVAN 12 three-compartment with 1st order depot

ADVAN 5 and 7 general linear models

ADVAN 6, 8, 9, 13 differential equation models

ADVAN 10 one-compartment intravenous input with saturable elimination

See NMHELP for TRANS routines and requirements for each model.

107


NMTRAN Control Records (for PREDPP)

• $SUBROUTINE

– Specifies PREDPP subroutine (ADVAN and TRANS)

$ SUB ADVAN2 TRANS2 ; one-compartment model with 1st orderabsorption

• $PK

– Defines required PK model parameters and inter-individual variance models.

• $ERROR

– Defines residual variance model(s)

$PK Additional Parameters

In addition to micro-constants or basic PK parameters (e.g. CL, V):

• Scale parameter: Sn

• Bioavailability: Fn

• Rate: Rn

• Duration: Dn

• Absorption lag: ALAGn

• Compartment initialization: A_0(n)

where n is the applicable compartment number

108


$PROB Pop PK model

$INPUT C ID DV AMT II ADDL TIME

$DATA ../data.CSV IGNORE=C

$SUB ADVAN2 TRANS2

$PKCL=THETA(1)*EXP(ETA(1))V=THETA(2)*EXP(ETA(2))KA=THETA(3)*EXP(ETA(3))S2=V

$ERRORY=F *(1+ERR(1))IPRED=F

$THETA(0, 8) ;CL(0, 50); V(0, 0.45) ;KA

$OMEGA0.040.040.04

$SIGMA0.04

;$MSFI=./.msf$ESTIMATION MAXEVAL=9999 PRINT=10 METHOD=0 POSTHOC MSFO

=../001.msf;$COVARIANCE$TABLE NOPRINT ONEHEADER FILE=../001.TABID TIME IPRED EVID

$TABLE NOPRINT ONEHEADER FILE=../001par.TABID TIME ETA1 ETA2 ETA3 EVID

109



Problem 1

• Population analysis of in-vitro enzyme activity data in 20 different liver slice samples

• Estimate Vmax and Km for population (naive pool and nlmem)

• Study design includes evaluation of activity at fixed concentrations: 0.1, 0.3, 1, 3, 10,30, 100, 300 and 1000 µM

• Data.../indpopmodels/example/enz/enz.csv

Getting Started

• Load example to SIMI server

• Import and view data in R (enz.R)

• Use text editor to create control stream

• Run NONMEM from MIfuns R package with NONR()

Problem 2

• Population analysis of QTc exposure response data from Phase 1

• Develop structural model

• Estimate population parameters

• Estimate individual parameters

• What is expected value of QTc prolongation at concentrations of 150 ng/mL and1500 ng/mL?

• Data.../indpopmodels/example/QT/dqtc.csv

Problem 3

• Population analysis of AST exposure response data from Phase 1


110




• What is expected value of AST at concentrations of 150 ng/mL and 1500 ng/mL?

• Data.../indpopmodels/example/AST/ast.csv

Problem 4

• Population PK analysis of a typical Phase 1 data set




• What is the population typical CL/F and its inter-individual variability?

• Data.../indpopmodels/example/PopPK/prob_2.csv


• Explain how you might interpret the standard error of the estimates.

• List three things that you should look for in the Monitoring of Search output.

• List three diagnostic plots that you should create for a population nlme model.

• How can you make sense of the off-diagonal elements of a covariance matrix?

• Provide an initial estimate for the first element of OMEGA, assuming a proportionalinter-individual %CV equal to 30%.

• Provide an initial estimate for the first element of SIGMA, assuming a constant resid-ual error with standard deviation equal to 5.

111

Chapter 4

Covariate Model Building

Overview

• Objectives of Covariate Model Development

• Common Covariate Model Parameterizations

• Data Reduction

• Covariate Modeling Methods

– Screening Methods Outside of the Population Model Context

– Stepwise Methods

– Full Covariate Models

• Making Inferences about Covariate Effects

• Aligning Covariate Modeling Methods with Objectives

112

MI-210: Essentials of Pop PKPD M&S 4. Covariate Model Building

4.1 Objectives of Covariate Model Development

• Explain “random” variability in parameters and response

• Understand causes of variability and apply the knowledge

– For better clinical therapeutic use (dosing, adjustment, labeling)

– To allow for better control in clinical trials

– In other words, make inferences about covariate effects from modeling results

• Improve predictive performance of the model

– For subjects in the current data set

– For simulations of future studies

– For future patient populations

113


4.2 Common Covariate Model Parameterizations

4.2.1 Covariate-Parameter Models for Continuous Covariates

Linear Additive:TVCL= θ1+θ2 ·WT i

CL= TVCL ·exp(ηi )

Linear Centered:TVCL= θ1+θ2 · (WT i −WT r e f )


Power Model:TVCL= θ1 · (WT i )θ2


Power Model with Normalized Covariate:

TVCL= θ1 · (WT i/WTref )θ2


Log-Transformed Power Model with Normalized Covariate:

LNCL= θ1+θ2 · log(WT i/WTref )+ηi

CL= exp(LNCL)

where:

WT i = individual weight

WT r e f = reference weight

114


4.2.2 Covariate-Parameter Models for Categorical Covariates

Linear Additive:TVCL= θ1+θ2 ·SMK i


Linear Proportional:TVCL= θ1 · (1+θ2 ·SMK i )


Power ModelTVCL= θ1 ·θ SMK i

2


Log-Transformed Power Model:

LNCL= θ1+SM K i · log(θ2)+ηi

CL= exp(LNCL)

where:

SMK i = individual smoking status; 0=no, 1=yes

4.2.3 Combining Continuous & Categorical Covariates

TVCL= θ1 · (WTi/WTref )θ2 ·θ SMK i3


where:

WT i = individual weight

WT r e f = reference weight

SMK i = individual smoking status; 0=no, 1=yes

115


4.2.4 Desirable Properties of Covariate Model Parameterizations

• Accurate description of observed range of data

• Model constrained to be plausible value (e.g. positive CL)

• Physiologically relevant

• Clinically relevant and useful

e.g. allometric model

TVCL=C L ref · (WTi/WTref )0.75

TVV =Vref · (WTi/WTref )1.0

4.3 Data Reduction: Before You Start Covariate Model Build-ing

• Examine covariate data.

– Identify range and distribution of continuous covariates.

– Count number in each category for categorical covariates.

– Identify strong correlations or collinearity between covariates.

Select covariates with unique information.

May require composite/interaction to convert to single variable.

• Was the study designed to estimate covariate effects?

• How do inclusion criteria impact choice of covariates to include in modeling?

• Does covariate make sense given prior knowledge?

• Which covariates are of interest from clinical perspective?

116


Correlation/Collinearity

Covariate effects to be included in model should be independent, i.e. they carry unique information.

Rule of thumb: Be cautious when |corr. coef.| > 0.3.

Check for Independence between Continuous and Categorical Covariates

• Explore graphically.

• Apply ANOVA and/or Kruskal-Wallis test.

117


Solutions to Correlation/Collinearity

• Avoid simultaneous inclusion of suspect covariates.

• Remove correlation.

– MDRD calculation for renal function is normalized by BSA, and can be in-cluded simultaneously with measures of body size.

• Seek additional data where the same variables are independent.

– Include data from renal impairment study where CRCL WT are not likely to becorrelated.

• Create a single summary variable to represent correlated predictors.

– BMI reflects both weight and height.

• Fix one of the covariate-parameter relationships.

– Age and weight are highly correlated in pediatrics, but fixing weight relation-ship to an allometric expression allows estimation of age effects.

• Reserve correlated covariates for secondary exploratory modeling goals.

4.4 Covariate Modeling Methods

4.4.1 Traditional Covariate Screening Methods in Population PKPD

Outside of population model context:

• Exploratory Graphics of Individual Random Effects (η) vs. Covariates

• Generalized Additive Modeling

– Multiple regression technique (in R/S) that allows for quick screen of linear andnon-linear covariate-parameter relationships

– Stepwise method based on empirical Bayes parameter estimates

– Focuses on individual parameter out of PK-PD model context (does not ac-count for correlation)

– Does not account for time-varying covariates or imbalance in data

– Results are dependent upon precision and accuracy of individual parameterestimates.

– Implemented in XPOSE package

118


Both Exploratory Graphics, GAM can be misleading in the presence of: η-shrinkage.

Exploratory Graphics

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

!"#$%&$'()*"+(,)-.&,+&/0

<

=#%%>';*,?4;'-+),-,4,* $$

!"#$%&$'()*"+(,)-.&,+&/0

!"#$%&$'()12%((/&/0)*('3"+4

6 7.,.'@*84A,-0+'.+8'?.,-0+./'A0B.?-.,*')*/*A,-0+'$)1%&"%&

6 (C3/0?.,0?D'9?.3E-A)

6 @*9?*))-0+'01'-+8-B-84./'3.?.;*,*?)'F0?'@*)-84./)G'B)H

A0B.?-.,*)

! !4/,-3/*'/-+*.?'?*9?*))-0+

! I*+*?./-J*8'K88-,-B*'!08*/-+9'FIK!G

=#%%>';*,?4;'-+),-,4,* $#

!"#$%&$'()*"+(,)-.&,+&/0

567,"%$'"%8)9%$73&24

2/0,'(LK-'0?'"+8-B-84./

@*)-84./)'F2-'M'LN2-G'B)H

O0B.?-.,*)',0'*C3/0?*

,?*+8'.+8')E.3*'01

?*/.,-0+)E-3)H

:0;*'?*/.,-0+)E-3)';.D

P*'+0+/-+*.?'0?

)*9;*+,*8H

Within population model context:

• Stepwise Forward Addition

• Stepwise Backward Elimination

• Stepwise Forward/Backward

We’ll discuss problems with these methods later

Stepwise Forward Addition

• Include covariates one step at a time using Likelihood Ratio Test.

• Do complete univariate screen of all covariate-parameter relationships.

• Add “best” covariate and repeat univariate screen with remaining covariates.

• Continue until no significant covariates are left.

• Suffers from lack of interaction between combinations of covariates

119


Stepwise Backward Elimination

• Start with “FULL” model and remove covariates one at a time in stepwise mannerusing Likelihood Ratio Test.

• Set each covariate fixed effect in FULL model to zero in turn.

• Remove covariate that has smallest impact on the model (and is not significant).

• Start new step with reduced model.

• Continue until all remaining covariates are significant.

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

!"#$%&$'()*"+(,)-.&,+&/0

$<

=#%%>';*,?4;'-+),-,4,* @$

!"#$%&$'()*"+(,)-.&,+&/0

1'$'&2'&3$,)!"/2&+(%$'&"/2

6 2.?)-;0+A'2?-+B-3/*

6 2?0C/*;)'D-,E':,*3D-)*'F*9?*))-0+

6 GE00)-+9'!08*/'H4-/8-+9'G?-,*?-.

6 7*,*?;-+.,-0+'01'IB,4./':-9+-1-B.+B*'J*K*/

6 2?-;.?A'LA30,E*)-)'M*),-+9'K)N'(O3/0?.,0?A'I+./A)-)

6 2.?.;*,*?'(),-;.,-0+'K)N'LA30,E*)-)'M*),-+9

=#%%>';*,?4;'-+),-,4,* @#

!"#$%&$'()*"+(,)-.&,+&/0

1'(45&2()-$365$%+)7,&8&/$'&"/

PE.,'E.33*+)'DE*+'.

B0K.?-.,*'*11*B,'-)'),.,-),-B.//A

Q)-9+-1-B.+,RS'C4,'+0,'B/-+-B.//A

-;30?,.+,T

"1'.'B0K.?-.,*'*11*B,'-)'+0,

),.,-),-B.//A'Q)-9+-1-B.+,RS'80*)

,E-)';*.+',E.,',E*?*'-)'+0

*11*B,T

!"#"$%&'%"#()'"&*+%),&$,,

-./$0)$%"'&$0"&')1+)2)-$+%

$%&#345446

!"2)+"&/$0)$+-"&7.8",

'%0*-%*0"&$+8&#$0$7"%"0'5

!*+&9:;<

=%"#&6

;><&?>*,,@

:A&B&!6&C&!DEFG&C&!HIJ

<&B&!K&C&!LIJ

=%"#&K="%&"$-M&!&%.&+*,,

/$,*"&$+8&0"-.08&;><+2.0&"$-M&0*+

N2&$+O&$0"&+.%&')1+)2)-$+%

#345446&?";><&3645PP@

0"7./"&-./$0)$%"&()%M

'7$,,"'%&";><

=%"#&+

Q!GRS:GR&T;RGAU

Q>NVEA&T;RGAU

=%$0%&()%M&2*,,&7.8",What happens when a covari-ate effect is statistically “sig-nificant”, but not clinically im-portant?

If a covariate effect is notstatistically “significant”, doesthis mean that there is no ef-fect?

Forward/Backward Stepwise

• Typically univariate forward selection (at p<0.05) is followed by stepwise backward(at p<0.01 or p<0.001), but this is problematic.

• Automated full forward/backward stepwise model building in NONMEM (with lin-earization) has been proposed by Jonsson & Karlsson (PSN software).

• All stepwise approaches require a large number of model runs.

120


Problems with stepwise regression

• Based on methods (e.g. F tests for nested models) that were intended to be used totest pre-specified hypotheses.

• P-values are difficult to interpret and difficult to adjust appropriately for multiplecomparisons.

• Regression coefficients are typically over-estimated (e.g. selection bias, false-positivefindings).

• Confidence intervals are falsely narrow. - Severe problems in the presence of corre-lated or collinear predictors (estimation bias, interpretation difficulties).

• Resulting models may be predictive of the current data set, but are often difficult tointerpret or generalize.

• NONMEM® likelihood approximations can result in incorrect p-values, even whenmodel is known.

• Reconciling statistically significant effects with clinically important effects is chal-lenging.

• Lack of statistical significance does not necessarily indicate lack of effect.

• Even with very large data sets, rigorous model building, testing and cross-validation(conditions not typically seen in population PKPD), stepwise selection often fails.

4.4.2 Other Covariate Modeling Methods

• WAM Algorithm

• Genetic Algorithm

• Full Covariate Models

WAM Algorithm

• Kowalski and Hutmacher proposed Wald Approximation to LRT for ranking of co-variate sub-models.

• Requires successful “FULL” model run and variance- covariance matrix of the esti-mates ($COVARIANCE step)

121


• Run 10 best models instead of all models in full stepwise method.

• Preliminary results indicate similar results to stepwise backward

• Performance is sensitive to correct choice of estimation method.

Genetic Algorithm

• Automated model building algorithm based on theory of natural selection (appliedto Population PK-PD by M. Sale).

• Model components are viewed as genetic traits that impact survival probability.

• Best models are defined by objective criteria, such as objective function value, suc-cessful minimization, successful COVARIANCE, etc.

• The choice of selection criteria is subjective, though.

• Experimental and computationally intensive, but favorable results so far.

Full Covariate Model Approach

• Define stable base model structure based on GOF criteria.

• Data Reduction: Avoid searching across all possible covariates.

• Requires rational selection of potential covariates for full model (similar to WAM orwell-executed Stepwise Backwards Method)

• Re-parameterize as necessary to develop a stable full model

• Estimate all parameters of full model and construct intervals or posterior distribu-tions (SEs, bootstrap, Bayes).

• Make inferences based on posterior distributions of estimated covariate effects.

• Explore remaining trends as secondary hypothesis-generating step.

122


Guiding Factors for Selection of Covariates for the Full Model

• First, perform data reduction step

• Create focused questions about specific covariate effects in the current data set basedon:

– Scientific or clinical interest

– Mechanistic plausibility

– Prior knowledge about covariate effects

(These should be defined a priori in the analysis plan)

• Exploratory graphics (view trend and shape of covariate- parameter relationships)

• Avoiding simultaneous inclusion of collinear/correlated predictors

• Adequate study design/range for covariates of interest

Pre-specified Covariate Plan

Covariate Model Parameters Rationale

Weight CL, V1, Q, V2 Clinical interest

Age CL Clinical interest

Race CL, V1 Clinical interestBridging goal

Disease State Type CL Clinical interest

Child-Pugh Score CL Clinical interestPrior knowledge of hepatic eliminationsmechanism; CYP3A4

Drug X Interaction CL Clinical interestKnown CYP3A4 inhibitor and commoncon-med

etc...

123


Stable Parameterization of Full Model

TVP= θn ·m∏

1

covmi

ref m

θm+n

·p∏

1

θcovpi

(p+m+n )

where: the typical value of a model parameter (TVP) is described as a func-tion of m individual continuous covariates (covmi) and p individual categori-cal (0–1) covariates (covpi) such that θn is an estimated parameter describingthe TVP for an individual with covariates equal to the reference covariate val-ues (covmi = refm, covpi = 0); (θm+n ) and (θp+m+n ) are estimated parametersdescribing the magnitude of the covariate-parameter relationships

Some Examples

124


125


Check Full Model for Remaining Trends

–  Plot CWRES or η from Full Model vs. any covariates in database.

- and/or -

–  Proceed with exploratory stepwise regression, starting at the full model. (This is purely an hypothesis-generating exercise).

4.5 Inferences about Covariate Effects

• Clinically Important: Posterior distribution of covariate effect results in pre-definedclinically important change in parameter or derived parameter (e.g. greater than+/-20% of null value).

• Not Clinically Important: Posterior distribution of covariate effect lies within a pre-defined, unimportant effect size (e.g. within+/- 20% of null value). Could be impor-tant in combination with other effects.

• Insufficient Information: Posterior distribution of covariate effect is broad and spansacross values of covariate effect that are both clinically important and not clinicallyimportant.

...

• Descriptive Approach: Describe probability of being clinically important using pos-terior distribution and reference range, or link to exposure-response. Visualize im-pact on derived parameters, such as AUC, Cmax, etc.

126


Quantify probability of covariate effect being clinically important on PK parameter or derived quantity (e.g. AUC).

Numbers indicate percent of posterior probability distribution relative to reference region.

Plot created using MIfuns R package (metruminstitute.org)

Reduction of Full Model for Predictive Purposes

• Drop covariate effects that meet both of these criteria:

– Not statistically significant (e.g. C.I. includes null value)

– Not clinically important (e.g. entire C.I. is contained within no effect range)

• Retain all other effects (any one or more of these criteria):

– Clinically important

– Statistically significant (e.g. C.I. excludes null value)

– Not statistically significant, but may be clinically important (e.g. characterizedby insufficient information with C.I. extending into ranges of potential clinicalimportance)

Removal of covariate effects from the full model should not impact coefficients for otherremaining effects.

127


Full Model: Results

• Point and interval estimates (or posterior distributions) can be used to assess clinicalrelevance of covariate effects, and how precisely they are estimated.

• Full model results in a more accurate reflection of estimation precision than step-wise regression.

• Covariate selection is not subject to problems of eta-shrinkage.

• Understand why some covariates have no impact:

– truly no effect

– not enough information to estimate effect

• Parameters with 95% CI near null value can be dropped and the reduced model ex-plored, if necessary.

4.6 Other Statistical Considerations

• Parsimony Principle

• Problems with Stepwise Regression

• Choosing Model Building Criteria

• Determination of Actual Significance Level

• Primary Hypothesis Testing vs. Exploratory Analysis

• Parameter Estimation vs. Hypothesis Testing

Parsimony Principle

“All things being equal, choose the simpler model.”

• Stepwise reduced models do not allow for inferences about “non-significant” covari-ate effects and are, therefore, not “equal” to the full model.

• For the purpose of making inferences about covariate effects, the full model is themost parsimonious model.

128


Parsimony Principle Restated

• Occam’s razor: “When competing hypotheses are equal in other respects, select thehypothesis that introduces the fewest assumptions and postulates the fewest enti-ties while still sufficiently answering the question.”

• Isaac Newton: “We are to admit no more causes of natural things than such as areboth true and sufficient to explain their appearances.”

• Albert Einstein: “Make everything as simple as possible, but not simpler.”

• Unknown: “The simplest explanation that covers all the facts is usually the best.”

Likelihood Ratio Test

• Compare goodness-of-fit for nested models.

• Determine theoretical statistical significance of model parameters.

• −2·(difference of log-likelihoods) from nested models: approximately asymptoti-cally 2 distributed, a useful statistic

(−2 log L full)− (−2 log L reduced)∼χ2(d f =∆p )∆p is difference in number of parameters

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

!"#$%&$'()*"+(,)-.&,+&/0

$<

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&$'()*"+(,)-.&,+&/0

6 B0;3.?*',C*'8-11*?*+D*'-+'E#/09F'G*,H**+';08*/)'I!

JKLM'H-,C'"#'8-),?-G4,-0+',.G/*'10?')3*D-1-D'81N

6 K4//';08*/&

! !1234536789):)345367;9<=*>):)536789

! 6#/09F14//'O'123

6 P*84D*8';08*/&

! !1234536789):)536789

! 6#/09F?*84D*8'O'145

!!"#$%$&

'()*+,- ./010.*+$)*+,-

2324 5367$89$:;<

23224 &366$89$:;<

6&7(,&8""+)9$'&"):(;'

-?3@)AB$')&C)D$0/&'.+()"E)=*>)(EE(F')G)B"H)H(,,)+")H()I/"H)&'J

=#%%>';*,?4;'-+),-,4,* @>

!"#$%&$'()*"+(,)-.&,+&/0

6 "#'.33?0Q-;.,-0+'D.+'G*')*+)-,-R*',0'.))4;3,-0+)N

! S0?;./-,T'01'?*)-84./)'.))4;3,-0+

! F-U*/-C008'.33?0Q-;.,-0+)'I*N9N'*),-;.,-0+';*,C08)M

6 VD,4./')-9+-1-D.+D*'/*R*/';.T'G*'R*?T'8-11*?*+,N

6 KJB(W"+,*?.D,-0+'0?'F.3/.D-.+W"+,*?.D,-0+'?*)4/,'-+'!JKL)

,C.,'.?*'D/0)*),',0'"#'8-),?-G4,-0+'HC*+';08*/'-)'D0??*D,N

6 KJ'-)'+0,'?*/-.G/*'10?'),.,-),-D./')-9+-1-D.+D*N

6 (;3-?-D./';*,C08)'.?*';0?*'.DD4?.,*',C.+',C*0?*,-D./

.))4;3,-0+)'I*N9N'?.+80;-X.,-0+',*),MN

6&7(,&8""+)9$'&"):(;'<)=%">,(?;129


Likelihood Ratio Test: Problems

• Approximation can be sensitive to assumptions.

– Normality of residuals assumption

– Likelihood approximations (e.g. estimation methods)

• Actual significance level may be very different.

• FOCE/Interaction or Laplacian/Interaction result in OFVs that are closest to χ2 dis-tribution when model is correct.

• FO is not reliable for statistical significance.

• Empirical methods are more accurate than theoretical assumptions (e.g. random-ization test).

Likelihood Ratio Test: Problems

• Suggestion: Use LRT as one of several model selection criteria, but not as a hypoth-esis testing tool.

e.g. Model selection requires critical change in OFV = 6.8 for 1 df (nostatement of statistical significance—just an improvement in goodness offit).

• LRT may be more useful in selecting structural model than for covariate effects.

• Also consider: AIC, diagnostic plots, minimization status, precision and plausibilityof parameter estimates, predictive check simulations

If You Really Need a P-Value: Randomization Test (RT)

• Create random permutations of data.

• Analyze with test and reference models.

• Compare observed statistic from final model (∆OFV) to permuted (null) distribu-tion.

• Obtain distribution of ∆OFVs and observe critical ∆OFV for desired significancelevel.

• Relies on empirical distribution rather than χ2 approximation

• When do you need a p-value?

130


4.7 Aligning Methods with Modeling Purpose

• Hypothesis Generation/Data Mining

– Stepwise methods can be useful.

• Inferences Based on Estimation of Covariates Effects

– Use full covariate model.

• Prediction

– Stepwise models can be useful, with large enough data set, and adequate testdata sets, when selection bias is likely to be low; still be careful to include datareduction step (see discussion of WAM).

– Full covariate models result in more appropriate (larger) parameter uncertainty,but can lead to larger prediction error of response.

• Hypothesis Testing

– Requires a priori adequately powered design and model specification.

– Stepwise is not appropriate.

– Limit full model to specific hypothesis to be tested.

131



Problem 1

• Population PK data set, 300 subjects, 4 samples each

• 1-cmt model with large inter-individual variability

• Significant drug-drug interaction identified in vitro with commonly used combination—confirm presence in current clinical trial

• Other covariates include weight, age, race, sex, liver function tests, creatinine clear-ance, conmed2, conmed3

• Describe the steps you might undertake to build a covariate model. Include details,such as plots you might create, estimation methods used, etc.

Problem 2

• Population PK model with covariates

• Multiple dosing: 10 doses x 900mg q12

• Drug administered by 3-hour constant-rate infusion

• Multiple samples after first and last dose, plus troughs along the way

• 1-cmt disposition: C L ∼ 13 L/hr; V∼ 75 L

• Specify model-building methods and GOF criteria.

• Build a covariate model

• Define impact of renal impairment on CL for drug label (be careful).

• Data: PopPKwCovs.csv

4.9 References

• Boeckmann, A.J., Sheiner, L.B. and Beal, S.L. NONMEM Users Guide – Part V: Intro-ductory Guide (1994).

• Burnham KP, Anderson DR. Model selection and multimodel inference: A practicalinformation- theoretic approach. 2nd ed. New York: Springer-Verlag, 2002.

132


• Gastonguay, MR. A Full Model Estimation Approach for Covariate Effects: InferenceBased on Clinical Importance and Estimation Precision. The AAPS Journal; 6(S1),Abstract W4354, 2004.

• Harrell, F.E. Regression Modeling Strategies. 2001; Springer-Verlag. NY.

• Jonsson EN, Karlsson MO. Automated covariate model building within NONMEM.Pharm Res 1998; 15(9):1463-1468.

• Jonsson EN, Karlsson MO. Xpose–an S-PLUS based population pharmacokinetic/phar-macodynamic model building aid for NONMEM. Comput Methods Programs Biomed1999; 58(1):51-64.

• Kowalski KG, Hutmacher MM. Efficient Screening of Covariates in Population Mod-els Using Wald’s Approximation to the Likelihood Ratio Test. J. Pharmacokinet Phar-macodyn 2001; 28(3):253-275.

• Mandema JW, Verotta D, Sheiner LB. Building population pharmacokinetic–pharmacodynamicmodels. I. Models for covariate effects. J Pharmacokinet Biopharm 1992; 20(5):511-528.

• Sale, M. Unsupervised machine learning based model building in NONMEM. (pre-sentation at 2000 ECPAG meeting).

• Wade JR, Beal SL, Sambol NC. Interaction between structural, statistical, and co-variate models in population pharmacokinetic analysis. J Pharmacokinet Biopharm1994; 22(2):165-177.

• Wahlby U, Jonsson EN, Karlsson MO. Assessment of actual significance levels forcovariate effects in NONMEM. J Pharmacokinet Pharmacodyn 2001; 28(3):231-252.http://www.pitt.edu/~wpilib/98sort.html (see topics 167, 168 and169 on stepwise methods).


• Name 2 different methods of covariate model building.

• List 3 problems with stepwise regression methods.

• Identify 3 items to consider as part of the data reduction step.

• What is the parsimony principal?

133

http://www.pitt.edu/~wpilib/98sort.html

Chapter 5

Simulation

5.1 Topics

• Review Simulation in NONMEM

• Uncertainty vs. Variability

• Typical Examples

134

MI-210: Essentials of Pop PKPD M&S 5. Simulation

Uses of Simulation

• Summarize data/concepts

• Interpolate between observed data

• Extrapolate to new conditions

• Express expected range of variability in response relationship

• Express expected level of uncertainty in response relationship

• Best model performance (e.g. posterior predictive check)

• Numerical integration of difficult problems (e.g. MCMC in hierarchical Bayes)

Fixed-Parameter vs. Monte Carlo Simulations

• Fixed-Parameter simulations reflect one individual‘s response or typical population(mean) response.

• Monte Carlo (stochastic) simulations reflect expected range of variability in responseunder current model assumptions.

Monte Carlo Simulations

• Include random variability

• Require pseudo-random number generator

– Quality is more important as number of samples increases

• Allow random variability to be included at multiple levels: (observation, individual,trial, etc.)

• Simulate from many types of distributions (Normal, Uniform, Log-Normal, etc.)

Practical Advantages of Using NONMEM®for Simulation

• Rapid and reliable conversion from estimation mode to simulation mode

• Both components in one tool

• Easily reproduced

• Robust random number generator

• Can automate simulation/estimation cycles and/or batches

Random Effect Distributions

• In NONMEM, pseudo-random number generation is available using Normal andUniform distributions.

135


• Combinations of these will produce other useful sampling distributions:

– exp(Normal) = log-Normal Distribution

– Normal2 =X 2 Distribution

– Normal +Normal = Bimodal

• Given known probability density function, any distribution can be simulated.

5.2 Calling Random Number Routines

CALL SIMETA (ETA)CALL SIMEPS (EPS)

• Default is one call per individual (SIMETA) or once per observation (SIMEPS)

• Multivariate Normal, mean = zero with variance covariance specified in $OMEGAor $SIGMA

• First seed is for SIMETA and SIMEPS

CALL RANDOM (2, R)

• Distribution is Normal (0, 1) or Uniform (0-1)

• Second or subsequent seeds

136


Simple Fixed-Value Simulations

• Useful to describe typical (mean) response for individual or population

• Does not reflect expected variability or uncertainty in response

• Method (see fixedsim.ctl):

– Create template data set with desired dosing regimen, sampling times and co-variates (DV = placeholder)

– Enter desired parameter estimates in $THETA

– $EST MAXEVAL=0

– $TABLE ID PRED TIME etc.

– PRED is typical-value simulation



Simulation with NONMEM®


137


Simulating Individual Data Given Estimated ETAs

• Simulate expected response for each individual in data set, given conditional esti-mates of η

• Method (see indest.ctl):

– Add template records to each individual in population data set for desired sim-ulation scenario and set MDV=1

– Enter initial parameter estimates in $THETA, $OMEGA, $SIGMA

– $ESTIMATION MAX=9999 POSTHOC

– IPRED=F

– $TABLE ID IPRED TIME MDV , etc.

– Simulated values are in IPRED when MDV=1

Simulating Individual Data Given Estimated ETAs



Simulating Individual Data

Given Estimated ETAs

138


$SIMULATION

e.g. $SIMULATION (5465) (5468 UNIFORM) SUB=10 ONLY

• Monte Carlo Simulation

• Seed: any number from 0 to 21474836447

– distributions: NORMAL, UNIFORM

– NEW (vector of random effects changes with each call to SIMETA or SIMEPS)

• ONLYSIM: PRED defined-items are calculated with simulated random effects, noestimation allowed

• SUBPROBLEMS: number of simulation replicates


• One level of random effects (residual), which reflects measurement/process noise:individual PK parameters are known

• Method (see indsim.ctl):

– Create template individual data set with desired dosing, sampling and covari-ates (DV=placeholder)

– Create individual model control stream

– Insert desired parameter estimates in $THETA, $OMEGA (no $SIGMA in indi-vidual model)

– remove $ESTIMATION

– $SIMULATION (1234) ONLYSIM

– $TABLE ID TIME DV (DV is simulated value)

139






140


Simulating Population Data

• Multi-level random effects model (inter-individual and residual variabilities)

• Simulate expected typical value, and expected variability

• Method (see popsim.ctl):

– Create template data set with desired dosing, sampling (DV=placeholder)

– Insert desired parameter estimates in $THETA, $OMEGA and $SIGMA

– $SIMULATION (1234) SUB=(n) ONLYSIM

– $TABLE (DV is simulated value)





141


Simulating Covariate Distributions

To simulate Normal distribution of WT:

W T = T H E TA(1)+E TA(1)

To apply bounds on simulated variable:

IF (ICALL.EQ.4) THEN

WT=THETA(1) + ETA(1)

DOWHILE (WT.LT.20.OR.WT.GT.100)

CALL SIMETA(ETA)

WT=THETA(1) + ETA(1)

ENDDO

ENDIF

$SIM (8799 NEW)

To simulate a uniform 3-category covariate:

IF(ICALL.EQ.4.AND.NEWIND.NE.2) THEN

CALL RANDOM (2, R)

IF(R.LE.0.33) TRT=0

IF(R.GT.0.33.AND.R.LE.0.66) TRT=1

IF(R.GT.0.66) TRT=2

ENDIF

...

$SIM (3546) (6466 UNIFORM)

142


Simulation with Uncertainty

• Considerable uncertainty exists for some simulation values for population PKPDmodel or covariate parameters.

• This uncertainty can be described by a prior probability density function.

• The uncertainty random effect is implemented as inter-trial variability (1 call persubproblem).

Hierarchy of Random Variability and Uncertainty in Simulation

• Intra-individual, residual error (ε)

– 1 draw from (0,σ2) per observation, constant fixed-effect parameters (θ )

• Inter-individual error (η) in parameter

– 1 draw from (0,ω2) per individual, constant fixed-effect parameters (θ )

• Uncertainty in models and parameters

• 1 draw from prior distribution for θ , Ω, Σ per trial

143


©2007 metrum institute ©2008 metrum institute 26

Uncertainty in ln(TVCL) Uncertainty in Var CL

144


Uncertainty in the Model (Competing Models)

• Through Uncertainty in Parameters:

– Combinations of some parameters approximate a different model structure.

– e.g. High EC50 approximates linear model

• Simulate from Expected Probability of Each Model: P(Model A) = 0.7 P(Model B) =0.3

– Draw random uniform variable (0-1), R

– Model A if R <= 0.7; Model B if R > 0.7

Obtaining Measures of Uncertainty

• Results from prior modeling exercise

– Variance-covariance matrix of estimates

– Bootstrap parameter distributions

– Bayesian posterior distributions

• Review of literature for ranges of plausible values

• Poll experts (everyoneÕs view can be part of the simulation)

145


A Simple Example of Simulation Including Parameter Uncertainty

IF(IREP.EQ.1.AND.NEWIND.EQ.0) COUNT = 0

IF(ICALL.EQ.4) THEN

IF(IREP.NE.COUNT) THEN

F1 = -1

DOWHILE(F1.LE.0)

CALL RANDOM(2,R) ;2nd seed is pseudo-Normal

F1 = THETA(2) + THETA(3)*R ;R is SD

ENDDO

ENDIF

COUNT = IREP

AUCR = THETA(1) + ETA(1)

AUCT = AUCR * F1 + ETA(2)

Y = AUCR*(1-FLG) + AUCT*FLG + EPS(1)

ENDIF

- See simuncert.ctl

146


Uncertainty Distribution of F1



Uncertainty Distribution of F1

147


Assembling Simulation Model Components


Assembling Simulation Model Components

Pop PK Model

(NONMEM; NPBootstrap)

PD Models Efficacy/AEs

(BUGS; Posterior Distributions)

Clinical Outcome Model

(Meta-Analysis of Literature Data; Var-Cov Matrix of Estimates)

Trial Simulation Model

Trial Conduct Model

(Dropout Rate; Range of Expert Opinions)

Simulation Tool: Requirements

1. Monte Carlo simulation hierarchy with multiple levels of nested random effects (atleast 3)

2. Ability to incorporate joint uncertainty distributions from other methods (e.g. boot-strap, Bayesian)

3. Simulation and estimation (ML) for typical population PK and PD systems in sametool

4. Programmable/extensible language with data manipulation and graphics capability

5. Platform neutral (Win, Unix, Linux, Mac OS X)

Current Simulation Tools

Some programs with Monte Carlo simulation capabilities at parameter uncertainty levelare available, but not all requirements are met:

• WinBugs

• NONMEM PRIOR subroutine

148


• Trial Simulator

• Others...

MIfuns R Package

1. Contains functions that facilitate draws from the uncertainty distributions at inter-trial level, maintaining joint distribution (covariance) of parameters

-OR-

1. Samples from previously determined uncertainty distributions (e.g. Bootstrap, BayesianPosteriors)

2. Generates NONMEM control streams for simulation (estimation)

3. Runs NONMEM or R for simulation (and possibly estimation) of each trial

4. Summarizes the results of each trial and across all trials

149

MI-210:E

ssentials

ofP

op

PK

PD

M&

S5.Sim

ulatio

n

Parameter type(NONMEM name)

Distribution Parameters of the distribution Implementation How to assign distribution parameters based onNONMEM run

Single uncorrelated pop-ulation parameter(THETA)

Normal Mean µ, varianceσ2 Standard R functionrnorm(., µ,σ)

µ: population parameter estimate;σ: standard error of the parameter estimate.

Set of correlated popula-tion parameters(THETA)

MultivariateNormal

Vector of mean values M, variance-covariance matrix Σ

Standard R functionmvrnorm(., M, Σ)

M: vector of population parameter estimate;Σ: variance-covariance matrix of the parameter es-timates.

Variance of the randomeffect(OMEGA)

ScaledInverse X 2

Number of degrees of freedom v, scale s 2. Standard R functionvs 2/rchisq(., v)

v: number of patients used to obtain the estimate;s 2: estimated variance of the random effect.

Variance-covariance ma-trix of the random effects(OMEGA)

InverseWishart

Number of degrees of freedom v, scale ma-trix S. Implicit parameter is the S matrix di-mension k.

Proprietary R functionmyriwish(k, v, vS) basedon the standard riwish()function

v: number of patients used to obtain the estimate;vS: estimated variance-covariance matrix of therandom effect.

Variance of the error term(SIGMA)

ScaledInverse X 2

Number of degrees of freedom v,scale s 2. Standard R functionvs 2/rchisq(., v)

v: number between the number of patients andthe number of observations used to obtain the es-timate;s2: estimated variance of the error.

150

MI-210:E

ssentials

ofP

op

PK

PD

M&

S5.Sim

ulatio

n

Parameters Generated from Uncertainty Distributions (or Bootstrap, Bayesian Posteriors, etc.)


Parameters Generated from Uncertainty Distributions

(or Bootstrap, Bayesian Posteriors, etc.)

1 full set of simulation

parameters per trial

(each row = 1 trial)

151

MI-210:E

ssentials

ofP

op

PK

PD

M&

S5.Sim

ulatio

n

Typical NONMEM $SIM Control Stream



$PROB RUN# 001!

$INPUT C ID AMT TIME EVID DV WT SS II!

$DATA ../Example2Data.csv IGNORE=@!

$SUBROUTINE ADVAN4 TRAN4 !

$PK!

TVCL = THETA(1)*(WT/70)**0.75!

TVV2 = THETA(2)*WT/70!

V3 = THETA(3)*WT/70 !

Q = THETA(4)*(WT/70)**0.75!

CL = TVCL*EXP(ETA(1))!

V2 = TVV2*EXP(ETA(2))!

F1 = 2!

S2 = V2/1000!

T1 = TVCL/TVV2!

T23 = Q/TVV2!

T32 = Q/V3!

TL1 = ((T1+T23+T32)+SQRT((T1+T23+T32)**2-4*T1*T32))/2!

TVKA = THETA(5)+TL1!

KA = TVKA*EXP(ETA(3))!

EMAX = THETA(6)*EXP(ETA(4))!

EC50 = THETA(7)*EXP(ETA(5))!

$ERROR!

CONC=A(2)/S2!

EFF = EMAX*CONC/(EC50+CONC)!

Y=EFF*EXP(EPS(1))!

IPRED=CONC*EXP(EPS(2))!

$THETA !

100 ; 1 TVCL !

1000 ; 2 TVV2 !

7500 ; 3 TVV3 !

500 ; 4 TVQ !

0.5 ; 5 TVKA!

1 ; 6 EMAX!

500 ; 7 EC50!

$OMEGA !

0.04 ; 1 CL!

0.09 ; 2 V2!

1.00 ; 3 KA!

0.09 ; 4 EMAX!

0.25 ; 5 EC50!

$SIGMA !

0.01 ; 1 EFF!

0.04 ; 2 PK!

$SIMULATION (12345) (6789 UNIFORM) !

$TABLE EVID TIME CONC IPRED EFF DV NOPRINT NOHEADER !

NOAPPEND FILE=../001.tab!

Typical NONMEM $SIM Control Stream

152


Constraining Simulated Parameters

• When simulating from a multi-variate Normal covariance matrix, use caution aboutplausible values for population- level parameters.

• Constrain model so that plausible values are simulated, e.g.:

LN C L = THETA

C L = exp(LN C L)

• Bootstrap distributions and Bayesian posteriors may already be constrained to plau-sible values

Sensitivity to Uncertainty in PD Parameters



Effect of uncertainty in

EMAX: % of patients

with trough effect

within the desired range

Simulated EMAX:

Black: median

Red: 95% CI

Conclusion: Precise

knowledge of EMAX is

very important

Uncertainty in PD Parameters

Effect of uncertainty in EMAX: % of patients with trough effect within the desired rangeSimulated EMAX: Black: median Red: 95% CI Conclusion: Precise knowledge of EMAX isvery important

153




Effect of uncertainty in

EC50: % of patients

with trough effect within

the desired range

Simulated EC50:

Black: median

Red: 95% CI

Conclusion: Uncertainty

in EC50 is less

important than

uncertainty in EMAX

Uncertainty in PD Parameters

Effect of uncertainty in EC50: % of patients with trough effect within the desired range

Simulated EC50: Black: median Red: 95% CI

Conclusion: Uncertainty in EC50 is less important than uncertainty in EMAX

5.3 Simulation Problem

• Already developed a model for a population PK data set for a multiple- dose study:200mg q8h

• New formulation has increased oral bioavailability by 50%

• Simulate expected population variability under new formulation and new regimen:200mg q12

• Compare with expected response under old formulation and regimen

• Old formulation model and data are included: simulation.ctl and data.csv, respec-tively

154

Chapter 6

Model Qualification

Overview

• Validation?

• A Risk-Based Approach

• Assumption Checking

• Evaluation Methods

– Parameters

– Predictive Performance

• Sensitivity Analysis

155

MI-210: Essentials of Pop PKPD M&S 6. Model Qualification

6.1 Can We Agree on a Name?

• Model Appropriateness

• Model Checking

• Model Evaluation

• Model Qualification

• Model Validation

• Model Verification

“Validation” Is Misleading

All models are wrong but some are useful.(Box, G. E. P. (1979). Robustness in the strategy of scientific model building. In R.L. Launer, and G. N. Wilkinson, (eds.) Robustness in Statistics. New York: AcademicPress)

Models are never completely valid, but the application of a model to a specific purpose(s)can be evaluated.

6.2 A Risk-Based Approach to Model Qualification

• Identify:

– What are the deficiencies of the model?

– What are the resulting risks for the modeling and decision-making process?

• Assess:

– Will these deficiencies/risks impact the intended use of the model?

• Manage:

– What are the strategies for managing these risks?

156


6.3 What to Evaluate or Qualify?

• The model itself?

– Structural PK-PD model

– Models for covariate-parameter relationships

– Random effect models

• The performance of model-based applications and inferences?

– Parameter estimates and confidence intervals

– Hypothesis tests

– Predictions/simulations with model

How?

• Before implementing a model evaluation method:

– Look at the data carefully

– Review modeling objective

– Develop model according to GOF criteria

– Check model assumptions

– Define model evaluation method and criteria for decision making

157


6.4 Model Qualification Methods

• Assumption Checking

– Randomization test

• Stability/precision of parameter estimates

– Validation through parameter prediction errors (Bruno et al. JPB 24:153- 172,1996)

– Log-Likelihood Profile

– Bootstrap (parametric and non-parametric)

– Cross-Validation/Leverage analysis

• Assessment of model performance

– Prediction errors (from internal or external test set)

– (Posterior) Predictive Check

• Sensitivity Analysis

– Part of the simulation effort

158


6.4.1 Assumption Checking

• Test assumptions of structural model

• Test assumptions in statistical model

• Test estimation method assumptions

• Primarily a model-building and data analysis concern (especially when analysis in-volves hypothesis testing)

Assumption Checking: Model

• Does the model fit the observed data?

• Has the search reached a global minimum?

• Are parameter estimates consistent with prior knowledge?

• Are random effect distributions consistent with modeling assumptions (e.g. Normaland distributions, centered on zero)?

• What is the impact of assumptions in data recording/assembly? (Try plausible sce-narios.)

Assumption Checking: Estimation Method

• Are assumptions about likelihood approximation valid? (Try more rigorous methodsand compare diagnostics.)

• Are assumptions about test statistics accurate?

Randomization Test

• Checks assumptions about χ2 distribution of delta −2 log L between nested models

• Determines actual significance level for a given model comparison (Full vs. Re-duced)

• Distribution of delta −2 log L for null hypothesis is generated by fitting model to re-peated random permutations of data set.

159


Randomization Test: Method

1. Fit Reduced (H0) and Full (H1) models to original data set and obtain delta −2 log L

2. Randomly permute (scramble) variable of interest (e.g. covariate) across entire dataset.

3. Fit Full (H1) model to permuted data set and compare to Reduced model to obtainnew delta −2 log L.

4. Repeat 2–3 several times (hundreds or thousands).

5. Generate distribution of delta −2 log L and pick quantile of interest (95% for p <0.05).

Distribution of delta−2 log L for Null

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%<';*,>4;'-+),-,4,* $?

!"#$%&'(")*+",-%*./&%$&'(")

0(1'2(3$'(")*"4*!567"87*4"2*9$%%

!"#$%#&'()&*$$+,-&..(/(01(

=#%%<';*,>4;'-+),-,4,* $@

!"#$%&'(")*+",-%*./&%$&'(")

6 "+,*>+./

! 7.,.':3/-,,-+9'A)3/-,'8.,.'-+,0';08*/'B4-/8-+9'C',*),-+9'8.,.')*,)D

! E>0))'F./-8.,-0+'A;4/,-3/*')3/-,)D

! G00,),>.3'A>*).;3/-+9'0>')-;4/.,-0+D

6 (H,*>+./'A)*3.>.,*'8.,.')*,D

+",-%*./&%$&'("):*;-1'*0&'&

–2LogLikelihood Difference

160


6.4.2 Test Data Sets for Model Qualification

• Internal

– Data Splitting (split data into model building and testing data sets)

– Cross Validation (multiple splits)

– Bootstrap (resampling or simulation)

• External (separate data set)

6.4.3 Log-Likelihood Profile: Qualification of Parameter Estimates

1. Obtain final model and parameter estimates.

2. Fix parameter of interest (e.g. THETA(1)) at range of values above and below themaximum likelihood estimate.

3. Perform estimation runs for each of the fixed values of THETA(1).

4. Record minimum objective function value (MOFV) at each fixed value of THETA(1).

5. Plot MOFV vs. THETA(1).

• Values of THETA(1) that increase the objective function by 3.84 units are defined as95% confidence interval.

• Accuracy of this method is highly dependent upon accuracy of likelihood approxi-mation in estimation method.

161


!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$%

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* $?

!"#$%&'(")*+",-%*./&%$&'(")

0"120(3-%(4"",*!5"6(%-

6 @./4*)'01'AB(ACD$E',F.,'-+G>*.)*',F*'0HI*G,-J*'14+G,-0+

HK'LM=N'4+-,)'.>*'8*1-+*8'.)'?OP'G0+1-8*+G*'-+,*>J./M

6 CGG4>.GK'01',F-)';*,F08'-)'F-9F/K'8*3*+8*+,'430+

.GG4>.GK'01'/-Q*/-F008'.33>0R-;.,-0+'-+'*),-;.,-0+

;*,F08M

<#%%=';*,>4;'-+),-,4,* #%

!"#$%&'(")*+",-%*./&%$&'(")

0"120(3-%(4"",*!5"6(%-

(zoom in at minimum)

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$$

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* #$

!"#$%&'(")*+",-%*./&%$&'(")

0"120(3-%(4"",*!5"6(%-

<#%%=';*,>4;'-+),-,4,* ##

!"#$%&'(")*+",-%*./&%$&'(")

7""'8'5&#

$? @A,.-+'1-+./';08*/'.+8'3.>.;*,*>'*),-;.,*)?

#? B>*.,*')*C*>./'DE4+8>*8)F'01'>*3/-G.,*'8.,.')*,)'AH'>*6).;3/-+9'D4+-,I-+8-C-84./F'J-,E'>*3/.G*;*+,'1>0;0>-9-+./'8.,.')*,?

K? 2*>10>;'*),-;.,-0+'>4+'J-,E'1-+./';08*/'0+'*.GE'01',E*>*3/-G.,*'8.,.')*,)?

L? M>0;'8-),>-A4,-0+'01'3034/.,-0+'3.>.;*,*>'*),-;.,*)N0A,.-+'O4.+,-/*)'01'-+,*>*),?

P? (;3->-G./'QPR'B"'-)'8*1-+*8'AH'#?P,E'.+8'QS?P,E

O4.+,-/*)'01'3.>.;*,*>'*),-;.,*)?

162


6.4.4 Bootstrap: Qualification of Parameter Estimates


2. Create several (hundreds) of replicate data sets by resampling (unit=individual)with replacement from original data set.

3. Perform estimation run with final model on each of the replicate data sets.

4. From distribution of population parameter estimates, obtain quantiles of interest.

5. Empirical 95% CI is defined by 2.5th and 97.5th quantiles of parameter estimates.

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$#

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* #?

!"#$%&'(")*+",-%*./&%$&'(")

0""'1'2&#*3")4(,-)5-*6)'-2/&%1

<#%%=';*,>4;'-+),-,4,* #@

!"#$%&'(")*+",-%*./&%$&'(")

6 A0+62.>.;*,>-B'C00,),>.3

! D*6).;3/*'>*3/-B.,*'8.,.')*,)'EF>*3/.B*;*+,'G4+-,'H'"7IJ

! :,>.,-1K'>*6).;3/-+9'.B>0))'B0L.>-.,*)F8.,.',K3*)',0'*+)4>*>*3>*)*+,.,-L*'>*3/-B.,*'8.,.')*,)J

! D*)4/,)'.>*'B0+8-,-0+./'0+',M*'8.,.J

6 2.>.;*,>-B'C00,),>.3

! :-;4/.,*'>*3/-B.,*'8.,.')*,)'1>0;'1-+./';08*/J

! 'N0L.>-.,*'8-),>-O4,-0+)'.>*'3>*)*>L*8J

! D*)4/,)'.>*'B0+8-,-0+./'0+';08*/J

6 PBB4>.BK'01'O0,M';*,M08)'8*3*+8)'430+'.BB4>.BK'01/-Q*/-M008'.33>0R-;.,-0+'-+'*),-;.,-0+';*,M08J

0""'1'2&#*+-'7",1

163


Bootstrap Methods

• Non-Parametric Bootstrap

– Re-sample replicate data sets w/replacement (unit = ID).

– Stratify re-sampling across covariates/data types to ensure representative repli-cate data sets.

– Results are conditional on the data.

• Parametric Bootstrap

– Simulate replicate data sets from final model. Covariate distributions are pre-served.

– Results are conditional on model.

• Accuracy of both methods depends upon accuracy of likelihood approximation inestimation method.

Bootstrap Considerations

• Number of replicates?

– 1000 is a good starting point.

– Check stability of CI’s as number of replicates increases (500, 1000, 1500, etc.).

• Which runs to include in bootstrap parameter distribution?

– Runs with successful MIN and $COV

– Runs with successful MIN regardless of $COV

– All runs with parameters reported (includes MIN terminations)

164


6.4.5 Leverage Analysis

1. Obtain final model and parameter estimates with entire data set.

2. Split data set into m approximately equal size (by individual) subsets.

3. Fit the model to data from m-1 subsets.

4. Record population parameter estimates.

5. Repeat for each of the unique subsets.

Leverage Analysis

• Useful to explore stability of parameter estimates

• Assists in identifying highly influential individual(s) or outliers

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* #@

!"#$%&'(")*+",-%*./&%$&'(")

0-/-1&2-*3)&%45(5

6 A)*14/',0'*B3/0?*'),.C-/-,D'01'3.?.;*,*?'*),-;.,*)

6 E))-),)'-+'-8*+,-1D-+9'F-9F/D'-+1/4*+,-./'-+8-G-84./H)I'0?

04,/-*?)

=#%%>';*,?4;'-+),-,4,* #>

!"#$%&'(")*+",-%*./&%$&'(")

0-/-1&2-*3)&%45(5

!"#"$%&%#

'(&)$"&%

*#+((,-".)/"&)+01234(%&

2+.)/1.)0%1#%5#%(%0&(15+)0&1%(&)$"&%167)0".1$+/%.8911:#+;%01.)0%(1#%5#%(%0&1<=>

?+07)/%0?%1)0&%#@".(167)0".1$+/%.89

165


6.4.6 Qualification Based on Predictive Performance

A Simple Qualitative Evaluation

1. Predict into validation data set with model and parameters estimated from indexdata

2. Create diagnostic plots

• Typical diagnostic scatter plots (PRED vs. DV, RES and WRES vs. TIME andPRED)

• RES vs. covariates

• PRED, DV vs. covariates

Validation Through Predictions

• Prediction error: PE i =C p r e d i −Cob s i

• Summary metrics

– Bias: MPE, MSPE

– Precision: MSE, MAE, RMSE

• Statistical Issues

– heteroscedastic variance SPE i = PE i/SDi , or use log(PE )

– more than 1 observation per individual leads to correlated prediction errorsand invalid statistical tests

Single-Split or External Prediction

1. Split data once by individuals (e.g. 70/30).

2. Estimate with index set.

3. Predict into validation set.

4. Calculate prediction errors.

5. Create diagnostic plots.

OR — Estimate with one data set and predict into an entirely new data set (external).

166


Cross-Validation

1. Procedure similar to Leverage Analysis

2. Split data into multiple subsets.

3. Each of the m-1 estimation subsets is used to predict into the remaining (unused)subset.



Cross-Validation

1. Procedure similar to Leverage Analysis

2. Split data into multiple subsets.

3. Each of the m-1 estimation subsets is used to predict into the remaining (unused)subset.



Cross-Validation Example 1

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @@

!"#$%&'(")*+",-%*./&%$&'(")

01"2234&%(,&'(")

$A 2?0B*84?*')-;-/.?',0'C*D*?.9*'E+./F)-)

#A :3/-,'8.,.'-+,0';4/,-3/*')4G)*,)A

@A (.BH'01',H*'06$'*),-;.,-0+')4G)*,)'-)'4)*8',0'3?*8-B,

-+,0',H*'?*;.-+-+9'I4+4)*8J')4G)*,A

KA L?*.,*'8-.9+0),-B'3/0,)A

MA L./B4/.,*'3?*8-B,-0+'*??0?)A

=#%%>';*,?4;'-+),-,4,* @K

!"#$%&'(")*+",-%*./&%$&'(")

01"2234&%(,&'(")*.5&6#%-*7

!"#$%&'#$

()*#"+#$

167


Cross-Validation Example 2

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%<';*,>4;'-+),-,4,* ?@

!"#$%&'(")*+",-%*./&%$&'(")

01"2234&%(,&'(")*.5&6#%-*7

!"#"$%&'()$*+$,-../+$!'0#'&"1$234/-/$!-//4&#"#-'56$78'556$9:::+

=#%%<';*,>4;'-+),-,4,* ?A

!"#$%&'(")*+",-%*./&%$&'(")

!"2'-1("1*!1-,(8'(/-*09-8:

6 2>030)*8',0'BC*BD'3*>10>;.+B*'01'C-*>.>BC-B./'E.F*)-.+

;08*/)

6 70')-;4/.,-0+)'G.)*8'0+',C*';08*/'H'3.>.;*,*>)'I;*.+

.+8'J.>-.+B*K'>*)4/,'-+'3.>.;*,*>'I0>'>*)30+)*K

8-),>-G4,-0+)',C.,'.>*')-;-/.>',0',C*'0G)*>J*8'8-),>-G4,-0+L

!"#$%&'"('%#)'+",-%*01-23()4*&),*5-)6('(/('7*8)&%76(69*:)*;&7-6(&)*<&'&

8)&%76(69*01&#=&)*&),*>&%%?*@-A*B"C3*DEFFGH9

Data from: M. Riggs. Doctoral Thesis Dissertation, UConn,2000.

168


6.4.7 Posterior Predictive Check

• Proposed to check performance of hierarchical Bayesian models

• Do simulations based on the model and parameters (mean and variance) result inparameter (or response) distributions that are similar to the observed distribution?

Gelman et al. Model Checking and Sensitivity Analysis. In Bayesian Data Analysis. Chap-man and Hall: New York (1995).

Posterior Predictive Check


2. Simulate several (100+) replicates of the original data set using final model fixed andrandom effect parameters.

3. Estimate population fixed and random effect parameters for each one of these repli-cates.

4. Simulate several (100+) new replicates with each replicate using simulation param-eters equal to one of the sets of estimated parameters from step 3.

5. From each of the simulations in step 4, calculate a characteristic of the data that isof interest (e.g. C max).

6. Summarize C max across all simulations (e.g. median, 1st quartile, etc.).

7. Calculate the same summary statistic from the original data set.

8. Plot distribution of simulated statistic with observed value.

9. Calculate PPC p-value (= fraction of simulated data > observed value).

169


Posterior Predictive Check

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#%

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* ?@

!"#$%&'(")*+",-%*./&%$&'(")

!"0'-1("1*!1-,(2'(/-*34-25

<#%%=';*,>4;'-+),-,4,* A%

!"#$%&'(")*+",-%*./&%$&'(")

!"0'-1("1*!1-,(2'(/-*34-25

6 BC*'>*)4/,-+9'8-),>-D4,-0+'>*1/*E,)'D0,C'4+E*>,.-+,F'-+

3.>.;*,*>'*),-;.,*)'.+8'>.+80;'-+,*>6-+8-G-84./'.+8

>*)-84./'G.>-.D-/-,FH

6 BC-)'3>0E*))'-)'G*>F'E0;34,.,-0+.//F'-+,*+)-G*H

• The resulting distribution reflects both uncertainty in parameter estimates and ran-dom inter-individual and residual variability.

• This process is very computationally intensive.

Predictive Check (A Simplification)

1. Assume uncertainty in parameters is small relative to other sources of variability.

2. Perform one set of simulations using final model parameters.

3. Calculate statistic of interest as in Posterior Predictive Check (steps 5–8).

OR — Compare observed data with simulated data.

Predictive Check Example 1

170


!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#$

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* ?$

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*56*7(8#%(9(1&'("):

$@ A))4;*'4+B*>,.-+,C'-+'3.>.;*,*>)'-)');.//'>*/.,-D*',0

0,E*>')04>B*)'01'D.>-.F-/-,C@

#@ 2*>10>;'0+*')*,'01')-;4/.,-0+)'4)-+9'1-+./';08*/

3.>.;*,*>)@

G@ H./B4/.,*'),.,-),-B'01'-+,*>*),'.)'-+'20),*>-0>'2>*8-B,-D*

HE*BI'J),*3)'K'L'=M@

NO'6'H0;3.>*'0F)*>D*8'8.,.'P-,E')-;4/.,*8'8.,.@

<#%%=';*,>4;'-+),-,4,* ?#

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.;&8#%-*<

!"#"$%&'()$*+$,-../+$!'0#'&"1$234/-/$!-//4&#"#-'56$78'556$9:::+Data from: M. Riggs. Doctoral Thesis Dissertation, UConn, 2000.


!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

##

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* ?@

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*7

!"!#$%"&'()*+&!"#$%&'%()%*+'%,-.-/$%,-./0%1112131$%-445'

<#%%=';*,>4;'-+),-,4,* ??

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*8*93(:'";0&6:<

adapted from: Shi, J. et al. JPKPD, (28): 555-575, 2001.

171


Predictive Check Example 3 (histograms)

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

##

!"#$%&'(")*+",-%*./&%$&'(")

<#%%=';*,>4;'-+),-,4,* ?@

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*7

!"!#$%"&'()*+&!"#$%&'%()%*+'%,-.-/$%,-./0%1112131$%-445'

<#%%=';*,>4;'-+),-,4,* ??

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*8*93(:'";0&6:<

Predictive Check Example 4 (Q-Q Plot)

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*7*89:9*!%"';

=#%%>';*,?4;'-+),-,4,* @B

!"#$%&'(")*+",-%*./&%$&'(")

6 CD*';08*/'-)'34?*/E'8*)F?-3,-G*H

6 CD*'-;3.F,'01';08*/68*?-G*8'-+1*?*+F*)'-)'-+)-9+-1-F.+,H

6 CD*';08*/'-)'.33/-*8',0'.'34?30)*',D.,'-)'+0,'*.)-/E'G./-8.,*8

I*J,?.30/.,-0+KH

6 CD*'90./'01';08*/'G./-8.,-0+'-)')-;3/E',0').,-)1E'.'?*L4-?*;*+,M'0?

FD*FN'.'O0JH

!"#$%&'(%)# *(+,!)&-"./)%&/0)&#"./&1&'(2/0.

<3-)*(=*./&%$&'(")*>-==*?6#"0'&)'@

6.4.8 When is Evaluation Less Important?

• The model is purely descriptive.

• The impact of model-derived inferences is insignificant.

172


• The model is applied to a purpose that is not easily validated (extrapolation).

• The goal of model validation is simply to satisfy a requirement, or check a box.

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*7*89:9*!%"';

=#%%>';*,?4;'-+),-,4,* @B

!"#$%&'(")*+",-%*./&%$&'(")

6 CD*';08*/'-)'34?*/E'8*)F?-3,-G*H

6 CD*'-;3.F,'01';08*/68*?-G*8'-+1*?*+F*)'-)'-+)-9+-1-F.+,H

6 CD*';08*/'-)'.33/-*8',0'.'34?30)*',D.,'-)'+0,'*.)-/E'G./-8.,*8

I*J,?.30/.,-0+KH

6 CD*'90./'01';08*/'G./-8.,-0+'-)')-;3/E',0').,-)1E'.'?*L4-?*;*+,M'0?

FD*FN'.'O0JH

!"#$%&'(%)# *(+,!)&-"./)%&/0)&#"./&1&'(2/0.

<3-)*(=*./&%$&'(")*>-==*?6#"0'&)'@

6.5 Sensitivity Analysis

• Sometimes, the model is applied to a purpose that is not easily evaluated (extrapo-lation).

• It may be more important to determine how the model inadequacies affect the con-clusions drawn from the modeling application.

• Example: clinical trial simulations

Types of Sensitivity Analysis

• Local Sensitivity Analysis (fixed-point perturbations)

• Global Sensitivity Analysis (based on uncertainty distributions across all parame-ters)

Local Sensitivity Analysis

• Can be performed by evaluation of gradients w.r.t. each parameter:

sensitivity= d(response)/d(parameter)

• Or by simulating with fixed-point perturbations in the parameter and subsequentcomparison of simulation response endpoints

Fixed Value of ZDVSL % Trials Successfula

0.25 30.6%0.5 70.4%0.735 93.05%1.0 99.0%

a Results reflect 500 Simulated trials of 2000 patients.

Limitations of Local Sensitivity Analysis

• Only reflects sensitivity to uncertainty in 1 parameter (assumption) at a time

• Inefficient: Must repeat the simulation exercise for each parameter of interest

173


• Conclusions about sensitivity to assumptions are only accurate if fixed values of allother parameters are correct.

Global Sensitivity Analysis

• Uncertainty is quantitatively defined for all parameters (models).

• Monte Carlo methods are required to simulate from uncertainty distributions (usu-ally requires one set of simulations with large number of replicates).

• Uncertainty distributions are implemented as inter-trial variability.

• Sensitivity of simulation outcome(s) to assumptions can be viewed over a continu-ous range of parameter uncertainty.

• Characterizes sensitivity to uncertainty in all model parameters (assumptions) si-multaneously

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&'(")*+",-%*./&%$&'(")

0%"1&%*2-)3('(/('4*5)&%43(3

6 :*+)-,-B-,C'01')-;4/.,-0+'04,D0;*E)F',0'.))4;3,-0+)D.+'G*'B-*H*8'0B*?'.'D0+,-+404)'?.+9*'01'3.?.;*,*?4+D*?,.-+,CI

6 JK.?.D,*?-L*)')*+)-,-B-,C

,0'4+D*?,.-+,C'-+'.//';08*/

3.?.;*,*?)'E.))4;3,-0+)F

)-;4/,.+*04)/C

=#%%>';*,?4;'-+),-,4,* @M

!"#$%&'(")*+",-%*./&%$&'(")

0%"1&%*2-)3('(/('4*5)&%43(3*()*!67!8

6 (N.;3/*)'-+'2O25'/-,*?.,4?*&')-;4/.,-0+'1?0;'.'?.+9*'01

3.?.;*,*?'B./4*)

! O0-)P'QI'*,'./I'0"1(2"%3*4##%3*!5&67&2"%3'$$%&'<R6RRP'$RR$I

6 S*H';*,K08)'3?030)*8'-+D/48*'4)*'01'14LLC'+4;G*?)

! S*),0?0BP'"I'*,'./I'86$9*+-'&:3*8(;#";3'A%&#<T6#>#P'#%%#I

6 2O25'*N.;3/*)',C3-D.//C'/4;3'3.?.;*,*?'4+D*?,.-+,C'.+8

-+,*?U-+,?.6-+8-B-84./'B.?-.G-/-,C',09*,K*?I

Global Sensitivity Analysis in PK-PD

• Examples in PBPK literature: simulation from a range of parameter values

– Bois, F. et al. Toxicol. Appl. Pharmacol. 110: 79-99, 1991.

• New methods proposed include use of fuzzy numbers

– Nestorov, I. et al. Drug Metab. Dispos. 30:276-282, 2002.

• PBPK examples typically lump parameter uncertainty and inter/intra-individual vari-ability together.

Global Sensitivity Analysis in CTS

• Incorporate parameter uncertainty in CTS using Bayesian prior probability distribu-tions for mean and variance parameters in the simulation model

– Gillespie, W.R., et al. Clin. Pharmacol. Ther. 65: PIII-21, 1999.

• A hierarchical model is employed where uncertainty is implemented as inter-trialvariability in the model and parameters.

174


Global Sensitivity Analysis Example 1

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%<';*,>4;'-+),-,4,* ??

!"#$%&'(")*+",-%*./&%$&'(")

0%"1&%*2-)3('(/('4*5)&%43(3*()*672

6 "+@0>30>.,*'3.>.;*,*>'4+@*>,.-+,A'-+'BC:'4)-+9'D.A*)-.+

3>-0>'3>0E.E-/-,A'8-),>-E4,-0+)'10>';*.+'.+8'F.>-.+@*

3.>.;*,*>)'-+',G*')-;4/.,-0+';08*/

! H-//*)3-*I'JKLKI'*,'./K'0%()1*!2&34&5"%1*62-31''M?&'2"""6#$I'$NNNK

! O'G-*>.>@G-@./';08*/'-)'*;3/0A*8'PG*>*'4+@*>,.-+,A'-)

-;3/*;*+,*8'.)'-+,*>6,>-./'F.>-.E-/-,A'-+',G*';08*/'.+8

3.>.;*,*>)K

=#%%<';*,>4;'-+),-,4,* ?M

!"#$%&'(")*+",-%*./&%$&'(")

0%"1&%*2-)3('(/('4*5)&%43(3*.8&9#%-*:


!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*7

!"#$%&#"'()*&+"',-%.'/*#"01&"

=#%%>';*,?4;'-+),-,4,* @>

!"#$%&'(")*+",-%*./&%$&'(")

B0;3*,-+9

,C*?.3D

?*)30+)*'.,'#

E**F)G

8(6$%&'(")*9('3*:)1-0'&()';<)-9*,"=-*&),*0->(6-)?

Simulation with Uncertainty (new dose and regimen)

175


!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

#<

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&'(")*+",-%*./&%$&'(")

!0-,(1'(/-*23-14*.5&6#%-*7

!"#$%&#"'()*&+"',-%.'/*#"01&"

=#%%>';*,?4;'-+),-,4,* @>

!"#$%&'(")*+",-%*./&%$&'(")

B0;3*,-+9

,C*?.3D

?*)30+)*'.,'#

E**F)G

8(6$%&'(")*9('3*:)1-0'&()';<)-9*,"=-*&),*0->(6-)?

Global Sensitivity Analysis Example 2

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

<%

!"#$%&'(")*+",-%*./&%$&'(")

=#%%>';*,?4;'-+),-,4,* @A

!"#$%&'(")*+",-%*./&%$&'(")

0%"1&%*2-)3('(/('4*5)&%43(3*.6&7#%-*8

=#%%>';*,?4;'-+),-,4,* B%

!"#$%&'(")*+",-%*./&%$&'(")

9"*9:();*51"$'<

6 CD.,'.?*',D*'4+8*?/E-+9'.))4;3,-0+)'.+8'-+.8*F4.G-*)'01

.'3.?,-G4/.?';08*/H

6 C-//',D*'.))4;3,-0+)'.+8'-+.8*F4.G-*)'01'.';08*/'D.I*'.

)-9+-1-G.+,'-;3.G,'0+',D*'-+1*?*+G*)'8?.J+'1?0;

.33/-G.,-0+)'01',D*';08*/H

To Think About:

• What are the underlying assumptions and inadequacies of a particular model?

• Will the assumptions and inadequacies of a model have a significant impact on theinferences drawn from applications of the model?

176

Chapter 7

Direct and Indirect Continuous PopPK-PD Models

Overview

• Utility of PK-PD Models

• esign and Data Issues

• Individual vs. Population Concentration-Response

• Basic Pharmacodynamic Models

• Simultaneous vs. Sequential Fit

• Direct PK-PD Models

• Effect-Compartment PK-PD Model

• Time-Dependent PD (tolerance)

• Indirect PD Response Models

177

MI-210: Essentials of Pop PKPD M&S 7. Direct and Indirect Continuous Pop PK-PD Models

7.1 Background

Utility of PK-PD Models

• Explain, understand, and control the Dose-Concentration-Response relationship

• Explain variability in response

• Provide framework for clinical trial simulations

• Provide support for clinical efficacy and demonstrate sensitivity of clinical trial

Population PK-PD Study Design

• Concentration-response within individual is key!

• With PK-PD model, it is not necessary to match concentrations and responses intime.

• Sparse sampling is possible but need multiple concentration-response observation-s/subject

• Aim for variety of exposure-response conditions.

– Single and multiple dosing

– Rising and falling concentrations

– Range of doses

178


7.2 Basic PD Models

Sigmoid Emax (or Emax when n=1)

E = E0+Emax ·C n/(EC50+ c n )

Truncated Sigmoid Emax

E = E0+SLPn ·C n

LinearE = E0+SLP ·C

Log-linearE = SLP · log(C )

7.3 $PRED Example

$PROB EMAX MODEL$INPUT ID DV CONC$DATA PD_DATA.CSV IGNORE=C$PREDEMAX=THETA(1)*EXP(ETA(1))EC50=THETA(2)*EXP(ETA(2))E=EMAX*CONC/(EC50+CONC)Y=E + EPS(1)$THETA (0, 100) (0, 20)$OMEGA 0.4 0.4$SIGMA 9$ESTIMATION MAXEVAL=9999 PRINT=10$TABLE ID CONC

179


7.4 Linking PK and PD

Some Continuous PK-PD Models

• Direct Link

• Effect Compartment

• Time-Dependent Models (e.g. tolerance)

• Indirect Pharmacodynamic Response Models

(All of these PK-PD model structures can also be applied to categorical outcome end-points.)

Model-Predicted or Observed PK?

• PK model allows prediction of concentrations at all PD observation times Ð no needto match pairs.

• PK model eliminates residual error (noise) in concentration data, which is importantassumption about predictor variables.

• PK model is easily extended to more complex model (e.g. effect compartment).

• PK & PD predictions are dependent upon accuracy of PK model.

IPRED Concentrations

• Sometimes in PK-PD modeling, it is not necessary to go through the entire popula-tion PK model building process.

• IPRED concentrations or individual PK parameters from a base model can be usedto drive PK-PD.

• Useful when:

– concentration prediction is for current data set only

– understanding PK is not a goal

– IPRED provides a good fit to PK data

180


7.5 PK-PD Modeling Steps

• Sequential Fit (1):

– Model PK first and fix parameters to final individual estimates

– Model PD conditional on fixed individual PK parameters (include PD data only)

• Sequential Fit (2):

– Model PK first and fix parameters to final population estimates

– Model PK-PD data given fixed population PK parameters (include PK & PDdata)

• Simultaneous Fit:

– Model PK & PD data with final estimates from sequential fit as initial estimates,allowing all parameters to be estimated

– “Gold Standard”

Sequential vs. Simultaneous

• Sequential Fit (2)

– Saves computational time and generally provides answers that are just as goodas simultaneous fit

• Sequential Fit (1)

– Is less computationally intense than Simultaneous or Sequential Fit (2) andproduces results that are almost as good if individual PK parameters are ac-curate. Preferred method when conditional estimation is not possible.

PK-PD Models in NONMEM®

• Direct link PD models coded through $ERROR of any ADVAN subroutine

• Effect-compartment (and tolerance) models require additional kinetic compartmentwith PD model coded in $ERROR

• Indirect PD response requires differential equation ADVAN (6, 8, 9)

181


7.6 Direct PK-PD

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9.+8':-;4/.,-0+

<

=#%%<';*,>4;'-+),-,4,* $?

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

0340!'5-*$2/'")'6765859

6 7->*@,'/-+A'27';08*/)'@08*8',B>049B'C(DDED'01'.+FG7HGI')4J>04,-+*

6 (11*@,6@0;3.>,;*+,'K.+8',0/*>.+@*L';08*/)'>*M4->*.88-,-0+./'A-+*,-@'@0;3.>,;*+,'N-,B'27';08*/'@08*8'-+C(DDED

6 "+8->*@,'27'>*)30+)*'>*M4->*)'8-11*>*+,-./'*M4.,-0+G7HGI'KOP'<P'QL

=#%%<';*,>4;'-+),-,4,* $O

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

!"#$%&'0340!

!"#$%&'()*+,

'"-%./0&'()*1,

2/

'345

!"#$#%&'(

K)**'*R.;3/*'@08*&'8->*@,S3A38T@,/L(see example code: direct_pkpd.ctl)

Simultaneous PK-PD Data for NMTRAN

ID TIME AMT DV TYPE (1=PK, 0=PD)1 0 100 . 11 2 . 15 11 2 . 95 01 4 . 22 11 6 . 14 11 7 . 79 02 0 100 . 1

Individualized PK/Sequential PK-PD Data for NMTRANID TIME AMT DV=PD CLI VI1 0 100 . 5 451 1 . 98 5 451 2 . 95 5 451 4 . 80 5 452 0 100 . 6.4 522 1 . 75 6.4 522 2 . 65 6.4 52

182


PD in $ERROR (Simultaneous)

$ERROREMAX=THETA(3)*EXP(ETA(3))EC50=THETA(4)*EXP(ETA(4))CONC=F*(1+ERR(1))EFF=EMAX*F/(EC50+F) + ERR(2)Y=CONC*TYPE + EFF*(1-TYPE);TYPE = 1 FOR PK, 0 FOR PD

183


7.7 Effect Compartment PK-PD

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9.+8':-;4/.,-0+

$%

<#%%=';*,>4;'-+),-,4,* $?

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

0!'")'678898':;"<.2&()$-./=

@(AABA

(!CDEFG(FCHIJK(D2H(FCHIJJ

(LM%EFG(FCHNJK(D2H(FCHNJJ

LBOLEPKH$Q(AAH$JJ

(PPE(!CDKPRH(LM%QPJ'Q'(AAH#J

SELBOLKFS2('Q'(PPKH$6FS2(J

TFS2('E'$'PBA'25U'%'PBA'27

<#%%=';*,>4;'-+),-,4,* #%

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

7>>$%&',-<1(#&<$)&'0340!

!"#$%&'()*+,

'"-%./0&'()*1,

2/

'345

!"#$#%&'(

677"8%&'()*9,

21"

2":;

H)**'*V.;3/*'W08*&'*W0;3$X3Y38ZW,/J(see example code: ecomp1_pkpd.ctl)

Effect Compartment Code in NONMEM

• Use multi-compartment analytical ADVANs: 3, 4, 11 and 12 (ecomp1_pkpd.ctl)

• General linear models in ADVANs 5 and 7 (ecomp2_pkpd.ctl)

• Differential equations in ADVANs 6, 8 and 9 (ecomp3_pkpd.ctl)

Simultaneous Effect Compartment PK-PD Data for NMTRAN

ID TIME AMT DV CMT TYPE1 0 100 . 1 11 2 . 15 2 11 2 . 95 3 01 4 . 22 2 11 6 . 14 2 11 7 . 79 3 02 0 100 . 1 1

(TYPE 1=PK, 0=PD ; CMT 1=GUT, 2=PK, 3=EFFECT)

184


7.8 Non-Parametric Effect Compartment

• Drive PK-PD link without compartmental model

• Linear interpolation of PK data (requires extensive sampling)

• See example code: (nonpar_pkpd.ctl)

• Can also be accomplished with constrained spline models

7.9 PK-PD Model with Tolerance*

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$#

<#%%=';*,>4;'-+),-,4,* #?

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

6-)40(#(7$&#"%'899$%&',-71(#&7$)&

6 7>-@*'25627'/-+A'B-,C04,'D0;3.>,;*+,./';08*/

6 E-+*.>'-+,*>30/.,-0+'01'25'8.,.'F>*G4->*)'*H,*+)-@*

).;3/-+9I

6 :**'*H.;3/*'D08*&'F+0+3.>J3A38KD,/I

6 L.+'./)0'M*'.DD0;3/-)C*8'B-,C'D0+),>.-+*8')3/-+*';08*/)

<#%%=';*,>4;'-+),-,4,* #N

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

0340!'5-*$2':"&;'<-2$#()%$=

!"#$%&'()*+(',-./01234567.28167.9

0:;<)=';%>"*?%+@A*=@+'%<A#='%=*B'"<+A'%?*;'BC%/*"AD''=%'=%<BE%F/86%GHHC349GI4J%4KLL

!"#$%&"'%()*

+,-./0

1)

,023

145%$*

145%$*

!""#"$%&'("#)*+",#-*(.'/'+0)-(

*Adapted from nicotine acute tolerance model: Porcheet et al. JPET 244:(1)231, 1988

See example code: tol_pkpd.ctl

185


7.10 Indirect PD Response Models

• Response is modeled as a continuous function of time with differential equation

• Drug effect is mediated as inhibition or stimulation of response formation or offset.

• PD model effect is on k i n or kou t and not directly on response itself

Indirect PD Response: Baseline Response

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

=#%%>';*,?4;'-+),-,4,* #@

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

+)*"#$%&'0!'6$/1-)/$'5-*$2/

6 A*)30+)*'-)';08*/*8'.)'.'B0+,-+404)'14+B,-0+'01',-;*

C-,D'8-11*?*+,-./'*E4.,-0+

6 7?49'*11*B,'-)';*8-.,*8'.)'-+D-F-,-0+'0?'),-;4/.,-0+'01

?*)30+)*'10?;.,-0+'0?'011)*,G

6 27';08*/'*11*B,'-)'0+'H-+'0?'H04,'.+8'+0,'8-?*B,/I'0+

?*)30+)*'-,)*/1

=#%%>';*,?4;'-+),-,4,* #J

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

+)*"#$%&'0!'6$/1-)/$7

8(/$2")$'6$/1-)/$

!"#$%&#"'(!)*+, *-./

!

0+12

!!!!

"#

"$!"!%&'!%()* "#

#"!%&' #%()*

Baseline Response in NONMEM

• R at time zero (R0) is equal to k i n/kou t

• Initialize amount in data set with unit dose (1) to response compartment (e.g. CMT=3)

• Estimate R0 using bioavailability (F3) parameter

k i n = θ (1) ∗exp(ETA(1))

kou t = θ (2) ∗exp(ETA(2))

F 3= k i n/kou t ;RESPONSE AT TIME ZERO

186


Indirect PD Response:1. Inhibition of Formation

(ind1_pkpd.ctl)

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

=#%%>';*,?4;'-+),-,4,* #@

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

6(/$2")$'7$/1-)/$'")'8985:5

6 A'.,',-;*'B*?0'CA%D'-)'*E4./',0'F-+GF04,

6 "+-,-./-B*'.;04+,'-+'8.,.')*,'H-,I'4+-,'80)*'C$D',0

?*)30+)*'J0;3.?,;*+,'C*K9K'L!MNOD

6 (),-;.,*'A%'4)-+9'P-0.Q.-/.P-/-,R'CSOD'3.?.;*,*?

5"TNMU(MVC$DW(X2C(MVC$DD

5YZMNMU(MVC#DW(X2C(MVC#DD

SON5"TG5YZM''''''''[A(:2YT:('VM'M"!('\(AY

=#%%>';*,?4;'-+),-,4,* #>

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

+)*"#$%&'0!'7$/1-)/$;

<='+)>"?"&"-)'-@'A-#B(&"-)C")*<D1E1*=%&2F

!"#$%&#"'(!)*+, *-./

0'1'2

3!45

!

6+78

!!!!

"#

"$!%&'!""

(#$% !)*)&'")

##

$#

%#%#%#%#

&#

'#

(#(#(#(#

"%+,- !#

'$(#$%)"

Indirect PD Response:2. Inhibition of Offset

(ind2_pkpd.ctl)

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

=#%%>';*,?4;'-+),-,4,* #@

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

!"#$%&#"'(!)*+, *-./

0'1'2

3!45

!

6+78

+)*"#$%&'0!'6$/1-)/$7

89'+):";"&"-)'-<'=<</$&>")*8?1@1*9%&2A

!!!!

"#

"$!%&'!%()* ""!

+#$% ",-,&'",

##

$#

%#%#%#%#

&#

'#

(#(#(#(#

"#

'$+#$%)"

=#%%>';*,?4;'-+),-,4,* A%

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

!"#$%&#"'(!)*+, *-./

0'9'2

3!45

!

6+78

+)*"#$%&'0!'6$/1-)/$7

B9'C&"D.2(&"-)'-<'E-#D(&"-)'>")*B?1@1*9%&2A

!"

!#!$%&!!"

'"#$ !('(%&"(

##

$#

%#%#%#%#

&#

'#

(#(#(#(#

"$)*+ !"

187


Indirect PD Response:3. Stimulation of Formation

(ind3_pkpd.ctl)

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

=#%%>';*,?4;'-+),-,4,* #@

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

!"#$%&#"'(!)*+, *-./

0'1'2

3!45

!

6+78

+)*"#$%&'0!'6$/1-)/$7

89'+):";"&"-)'-<'=<</$&>")*8?1@1*9%&2A

!!!!

"#

"$!%&'!%()* ""!

+#$% ",-,&'",

##

$#

%#%#%#%#

&#

'#

(#(#(#(#

"#

'$+#$%)"

=#%%>';*,?4;'-+),-,4,* A%

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

!"#$%&#"'(!)*+, *-./

0'9'2

3!45

!

6+78

+)*"#$%&'0!'6$/1-)/$7

B9'C&"D.2(&"-)'-<'E-#D(&"-)'>")*B?1@1*9%&2A

!"

!#!$%&!!"

'"#$ !('(%&"(

##

$#

%#%#%#%#

&#

'#

(#(#(#(#

"$)*+ !"

Indirect PD Response:4. Stimulation of Offset

(ind4_pkpd.ctl)

!"#$%&'(')*+"#$%&',-*&"*.-./'0-1.23&"-*'0450!

6-+$2/

!"#$%&'())*+,-./)'01'2034/.,-0+'25627'!08*/-+9

.+8':-;4/.,-0+

$<

=#%%>';*,?4;'-+),-,4,* @$

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

+)*"#$%&'0!'6$/1-)/$7

89':&";.2(&"-)'-<'=<</$&>")*8?1@1*9%&2A

!"#$%&#"'(!)*+, *-./

0'1'2

3!45

!

6+78

!!!!

"#

"$!%&'!%()* ""!

+#$% ",+,&'",

##

$#

%#%#%#%#

&#

'#

(#(#(#(#

"#

=#%%>';*,?4;'-+),-,4,* @#

!"#$%&'()*'+)*"#$%&',-)&").-./'0-1.2(&"-)'0340!'5-*$2/

=&B$#'+)*"#$%&'0!'6$/1-)/$'5-*$2/

6 A.?1.?-+&'3?0,B?0;C-+',-;*'-)'*+830-+,'10?'8?49'*11*D,'0+

D/0,,-+9'1.D,0?)'EF.?1GD,/H

6 I0;3*?,J';08*/'10?',4;0?')-J*'0?'C.D,*?-./'9?0F,B

E90;3*?,JGD,/H

188


7.11 References

General Reviews

1. Holford NH, Sheiner LB. Understanding the dose-effect relationship: clinical appli-cation of pharmacokinetic-pharmacodynamic models. Clin Pharmacokinet 1981;6(6):429-453.

2. Holford NH, Sheiner LB. Pharmacokinetic and pharmacodynamic modeling in vivo.CRC Crit Rev Bioeng 1981; 5(4):273-322.

3. Holford NH, Sheiner LB. Kinetics of pharmacologic response. Pharmacol Ther 1982;16(2):143-166.

4. Gibaldi, M. and Perrier, D. Kinetics of pharmacologic response in Pharmacokinetics,2nd Ed. Marcel Dekker, New York, 221-270, 1982.

5. Dayneka NL, Garg V, Jusko WJ, Comparison of Four Basic Models of Indirect Phar-macodynamic Responses. J. Pharmacokinet. Biopharm. 21:457-478, 1993.

Classic Examples

6. Sheiner LB, Stanski DR, Vozeh S, Miller RD, Ham J. Simultaneous modeling of phar-macokinetics and pharmacodynamics: application to d-tubocurarine. Clin Phar-macol Ther 1979; 25(3):358-371.

7. Porchet, H.C., Benowitz, N.L. and Sheiner, L.B. Pharmacodynamic model of toler-ance: Application to nicotine. Journal of Pharmacology and Experimental Thera-peutics. 244: 231-236, 1988.

8. Nagashima R., OÕReilly R.A., and Levy G. Kinetics of pharmacologic effects in man:The anticoagulant action of warfarin. Clin. Pharmacol. Ther. 10:22, 1969.

9. Jusko, WJ. A pharmacodynamic model for cell-cycle-specific chemotherapeutic agents.J. Pharmacokinet. Biopharm. 1: 175, 1973.

189

Chapter 8

Regulatory Guidance and BestPractices

8.1 Overview

• Support for Population PK-PD Modeling and Simulation by Regulatory Agencies

– General Regulatory Support of Population PK

– FDA Population PK Guidance

– EMEA Population PK Guidance

• Review of Best Practices

– Systems, Procedures

– Data

– Modeling and Simulation

– Reporting and Communication

190

MI-210: Essentials of Pop PKPD M&S 8. Regulatory Guidance and Best Practices

8.2 Regulatory Support for M&S

• Regulatory Meetings/Interactions

• Internal Reviews and Analyses

• Guidance Documents

8.2.1 Regulatory Meetings/Interactions: End of Phase IIa Meeting

http://www.fda.gov/downloads/DrugsGuidanceComplianceRegulatoryInformation/Guidances/ucm079690.pdf

• Proposed by FDA at Nov. 2003 Clinical Pharmacology Advisory Committee Meeting

• Goal: Improve design and outcome of Phase 2b studies to improve success rate inPhase 3

• "The overall purpose of an EOP2A meeting is to discuss options for trial designs,modeling strategies, and clinical trial simulation scenarios to improve the quantifi-cation of the exposure- response information from early drug development."

• "The goal of these meetings is to optimize dose selection for subsequent trials toimprove the efficiency of drug development. The exposure-response data discussedmight be pertinent to evaluation of efficacy outcomes or adverse outcomes."

• "In addition, the meetings would provide opportunities for discussions of complexissues pertaining to drug interactions, trials in special populations defined by ge-netic characteristics or other biomarkers, and other PK or PK/PD relationships."

• Important addition to traditional FDA-sponsor meetings during development, al-though sponsor participation is voluntary

191




8.2.2 Regulatory Review and Analysis

• Review of Submitted Sponsor-Conducted Analyses

• Regulatory Decisions Based on Agency-Conducted M&S

• Advocating Model-Based Drug Development and Drug Review Processes (Part ofCritical Path Initiative)

8.2.3 Regulatory Guidance Documents

• Population Pharmacokinetics (FDA & EMEA)

• Exposure-Response Relationships (FDA)

• General Considerations for the Clinical Evaluation of Drugs (FDA)

• General Considerations for Pediatric Pharmacokinetic Studies (FDA)

• Pharmacokinetics in Patients with Impaired Renal Function (FDA, EMEA)

• Pharmacokinetics in Patients With Impaired Hepatic Function (FDA)

• Dose-Response Information to Support Drug Registration (ICH E4)

• Studies in Support of Special Populations: Geriatrics (ICH E7)

• Ethnic Factors in the Acceptability of Foreign Clinical Data (ICH E5)

• Clinical Investigation of Medicinal Products in the Pediatric Population (ICH E11)

• and others...

192


8.2.4 FDA Population Pharmacokinetics Guidance

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm072137.pdf

(Points to Consider)

• Methods

• When to Use

• Study Design and Execution

• Assay

• Data Handling

• Data Analysis

• Population Analysis Report

• Labeling

• Examples

• References

• Glossary

Study Design and Execution

• Sampling Frequency

– Single or multiple troughs

– Full PK screen

• Sample on more than one occasion.

• Simulation to Evaluate Design

• Protocol (add-on or stand-alone)

– Clinical study

– Data analysis

• Execution: GCP, GLP, compliance

Data Handling

• Types of Data

• Real-Time Data Assembly

• Procedures for Data Assembly, QC

193




• Eliminating or Including Data

• Missing Data

• Dealing with Outliers

• Data Integrity and Computer Software

Data Analysis

• Exploratory Data Analysis

– Graphics and summary statistics

• Model Development

– Objectives, hypotheses and assumptions

– Pre-specify criteria and explain rationale

– Evaluate results (diagnostic plots, parameter estimates, standard errors, localminima, sensitivity to outliers, and estimation methods)

• Model Validation

– Internal Data (single split, cross-validation, bootstrap)

– External Data

– Prediction: prediction errors, diagnostic plots

– Validation through Parameters

– Posterior Predictive Check

Study Report

1. Summary

2. Introduction

• Objectives, hypotheses, and assumptions

3. Methods

• Assay

• Data: description of collection and assembly

• Data analysis: method, rationale, assumptions, algorithms used

194


4. Results

• Tables and plots (usual popPK diagnostics, but also frequently request pre-dicted and observed time-course for each individual)

• Thorough description

• Complete output for final model and key intermediates

• Analysis flowchart or log of model runs and decision process (not specificallymentioned in guidance, but recurring request)

5. Discussion

6. Application of results

• Support labeling on PK, efficacy and safety

• Individualize dosing

• Define additional studies

7. Appendix

• Hardcopy of portion of data set

• Final model code and output

• Include study protocol

8. Electronic Files

• Report

• Data files and code

Including Results in Labeling

• Total number of subjects

• Number of subjects in subpopulations, when relevant

• Precision of parameter estimates (standard error or confidence interval)

• Label should indicate that results were obtained from population analysis.

195


8.2.5 EMEA Guidance on Pop PK

http://www.emea.europa.eu/pdfs/human/ewp/18599006enfin.pdfGUIDELINE ON REPORTING THE RESULTS OF POPULATION PHARMACOKINETIC ANAL-YSES

• Final Adopted in 2007

• Similar to FDA Guidance, but focuses on reporting (indirectly implies expectationsfor analysis)

• More specific technical detail

– Expected content of Pop PK reports

– Analysis methods

– Diagnostics

– Somewhat NONMEM-centric (but states that general concepts apply to all tools)

8.3 Review of Best Practices

8.3.1 Systems and Procedures

Develop Consistent Systems & Procedures

• Software Installation/Qualification

• Protocol & Data Analysis Plan

• Data Handling & Analysis Tracking (audit trail)

• Data Analysis

• Scientific Review

• Report Writing & Presentation of Results

196

http://www.emea.europa.eu/pdfs/human/ewp/18599006enfin.pdf


Software Installation Qualification

• Most software packages are not truly “validated” (including NONMEM)

• Qualification of installation provides reasonable evidence that program is perform-ing as expected

• Install software and any patches/fixes

• Run test cases and compare with standard results

• Track future analyses and link to qualified installation

• Document this process

• Software qualification makes scientific sense and is advocated (required) by regula-tory agencies

8.3.2 Data Preparation

• Plan ahead

• Protocol, CRF, & Database Design

• Data Analysis Plan

Data Specification Document

• Identify ahead of time how data problems will be handled.

• Link NMTRAN data items to source databases.

• Specify format of NMTRAN data set.

• Specify plausible range of values for data items (useful for edit checks).

• Identify special instructions for creation of NMTRAN data set.

Data Quality Diagnostics

• Index plots: Graph all data items vs. ID.

• Plot data: DV vs. TIME (or time post-dose) with ID number as label.

• Plot histograms of continuous data items.

• Box plots of continuous data items by categorical variables

• Summary statistics of NMTRAN data items should be compared to source data andprotocol.

• Run simple prediction model as test case (MAXEVAL=0)

197


Data Assembly Audit Trail

• Goals

• Trace data set to source

• Reproduce data set assembly

• Methods

• Good records (e.g. notebook, data specification document, electronic journal, etc.)

• Use programmable data manipulation tools. (e.g. R, SPLUS, SAS, AWK, Excel Macros,etc.)

8.3.3 Modeling and Simulation

Modeling and Simulation Plan

(MandSPlanTemplate.doc)

• List analysis objectives.

• Define data sources.

• Define known assumptions for modeling process.

• Describe potential models and covariate-parameter relationships.

• Define data analysis methods.

– Specific hypothesis

– Estimation methods

– Model comparison criteria

• Specify model evaluation methods.

• Outline any simulations that may be planned as part of the analysis.

• Include literature references supporting proposed methods

Model-Based Analysis: Key Principles

• Reproducibilty

– Documentation

– Traceable process (e.g. version control of code)

• Consistency

198


– Modeling and simulation plan (describe deviations)

– Process standards

• Scientific Quality

– State of the art science

– Scientific review(s) ; Don’t wait until end of the analysis

– Quality control checks

• Focus

• What is intended purpose of modeling process?

• Which deliverables are necessary to make a drug development impact?

Modeling Methods/Process

• Define model development and application goals up-front

– Estimation

– Prediction

– Subsequent trial simulation

• Understand assumptions inherent in model specification and estimation methodsand check assumptions

– Random effect distributions

– Estimation approximations

• Assess model goodness-of-fit with appropriate diagnostics

– Conditional Weighted Residuals

• Covariate modeling should be focused on clinical interest

– Define covariate-parameter relationships up-front

– Exploratory screen has a role for hypothesis generation

• Model evaluation methods should focus on intended model application

– Precision of parameters (non-parametric bootstrap)

– Future use in simulation (predictive check)

• Subsequent simulations should appropriately incorporate random effects hierarchyand parameter uncertainty

– Not all models are equal - reflect range of possible outcomes

199


– Simulation endpoints as probability distributions

8.3.4 Reporting and Communication

• Communicate.

– Internal: within development team

– External: with regulatory agencies, CROs, clinical investigators

• DonÕt compromise on a good plan.

– More on this in Study Design lecture

Report Writing

(rpt_template.doc)

• Automate creation of figures and tables using programmable tools when possible.

• Include scientific review, QC and QA of report.

• Use version control tools, or carefully track versioning.

• Ensure that report conclusions are supported by results.

• Include sections as described in FDA / EMEA Guidance

8.4 References

• Fadiran EO, Jones CD, Ette EI. Designing population pharmacokinetic studies: per-formance of mixed designs. Eur J Drug Metab Pharmacokinet 2000; 25(3- 4):231-239.

• Williams PJ, Ette EI. The role of population pharmacokinetics in drug developmentin light of the Food and Drug Administration’s ’Guidance for Industry: populationpharmacokinetics’. Clin Pharmacokinet 2000; 39(6):385- 395.

• Sun H, Fadiran EO, Jones CD, Lesko L, Huang SM, Higgins K et al. Population phar-macokinetics. A regulatory perspective. Clin Pharmacokinet 1999; 37(1):41-58.

• Ette EI, Sun H, Ludden TM. Balanced designs in longitudinal population pharma-cokinetic studies. J Clin Pharmacol 1998; 38(5):417-423.

• Ette EI, Kelman AW, Howie CA, Whiting B. Interpretation of simulation studies forefficient estimation of population pharmacokinetic parameters. Ann Pharmacother1993; 27(9):1034-1039.

200

Appendix A

Course Project

A.1 Introduction

One of the requirements for course credit is completion of a small population modelingproject. This project represents 50% of the course grade. The project is due within twoweeks of the final course lecture. Students are welcome to use the Metrum Institute SIMIserver for analyses related to the course projects. Please direct any questions to courseinstructor.

A.2 Project Requirements

Students will define their own projects, according to the following required elements:

• Choose a real-world or simulated population data set (PK or PD), but in the interestof maintaining reasonable run times, please restrict the size of the data set and limitthe analysis to a single endpoint. For extensively sampled data, please limit to 30individuals or less. For sparsely sampled data, please limit to 100 individuals or less.Please also select an example that does not require numerical solution of differentialequations. In other words, avoid nonlinear PK systems with saturable processes.

• Include a data specification document and an NMTRAN formatted data set.

• Develop at least two alternative population models (could be a base model and finalmodel) for the endpoint in question and compare them on the basis of:

– minimization status

– diagnostic plots

201

MI-210: Essentials of Pop PKPD M&S A. Course Project

– goodness of fit statistics

– model output

– parameter point-estimates

– precision of parameter estimates

• Summarize methods, results and conclusions with supporting figures and tables ina brief report, including sections consistent with regulatory guidance documents.Report format should be PDF document, if possible.

• Discuss results including assumptions inherent in the analysis.

• Include supplemental files:

– data specification document

– NMTRAN data set

– control streams and output files for two models

202

Appendix B

Complete Courseware License

203

COMPLETE COURSEWARE LICENSE AGREEMENT RIGHTS RESERVED: THIS COMPLETE COURSEWARE LICENSE IS SEPARATE AND DISTINCT FROM THE OPEN COURSEWARE LICENSE, WHICH IS PROVIDED FOR PUBLIC USE OF A SPECIFIED SUBSET OF METRUM INSTITUTE TRAINING MATERIALS. SEE " Courseware Info" at http://metruminstitute.org. THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED. BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. TO THE EXTENT THIS LICENSE MAY BE CONSIDERED TO BE A CONTRACT, THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS. 1. Definitions "Adaptation" means a work based upon the Work, or upon the Work and other pre-existing works, such as a translation, adaptation, derivative work, or any other form in which the Work may be recast, transformed, or adapted including in any form recognizably derived from the original, except that a work that constitutes a Collection will not be considered an Adaptation for the purpose of this License. "Collection" means a collection of literary or artistic works, such as encyclopedias and anthologies, or performances, phonograms or broadcasts, or other works or subject matter other than works listed in Section 1(f) below, which, by reason of the selection and arrangement of their contents, constitute intellectual creations, in which the Work is included in its entirety in unmodified form along with one or more other contributions, each constituting separate and independent works in themselves, which together are assembled into a collective whole. A work that constitutes a Collection will not be considered an Adaptation (as defined above) for the purposes of this License. "Distribute" means to make available to anyone other than You, the original and copies of the Work or Adaptation, as appropriate, through sale or other transfer of ownership. "Licensor" means the individual, individuals, entity or entities that offer(s) the Work under the terms of this License. "Original Author" means the individual, individuals, entity or entities who created the Work or if no individual or entity can be identified, the publisher. "Work" means the scientific work offered under the terms of this License including without limitation any production in the literary, scientific and artistic domain, whatever may be the mode or form of its expression including digital form, such as a book, pamphlet and other writing; a lecture, address, sermon or other work of the same nature; a broadcast; a phonogram; a compilation of data to the extent it is protected as a copyrightable work. "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation. "Publicly Perform" means to perform public recitations of the Work and to communicate to the public those public recitations, by any means or process, including by wire or wireless means

or public digital performances; to make available to the public Works in such a way that members of the public may access these Works from a place and at a place individually chosen by them; to perform the Work to the public by any means or process and the communication to the public of the performances of the Work, including by public digital performance; to broadcast and rebroadcast the Work by any means including signs, sounds or images. "Reproduce" means to make copies of the Work by any means including without limitation by sound or visual recordings and the right of fixation and reproducing fixations of the Work, including storage of a protected performance or phonogram in digital form or other electronic medium. “Commercial” means any copy, application or Adaptation of the Work for the direct or indirect purpose of revenue generation “Non-Commercial” means any copy, application or Adaptation of the Work for purposes that are not directly or indirectly related to revenue generation. 2. Fair Dealing Rights. Nothing in this License is intended to reduce, limit, or restrict any uses free from copyright or rights arising from limitations or exceptions that are provided for in connection with the copyright protection under copyright law or other applicable laws. 3. License Grant. Subject to the terms and conditions of this License, Licensor hereby grants You fair use of this copyrighted Work for noncommercial educational purposes or for commercial and noncommercial research purposes. For example, You may adapt example code sets (the Work) and apply them to a commercial research project. 4. Restrictions. The license granted in Section 3 above is expressly made subject to and limited by the following restrictions: Under the terms of this License, You may not Distribute, Reproduce, or Publicly Perform the Work. For example, You may not use courseware (the Work or any adaptations) to create a new public training course, whether commercial or non-commercial. You may not implicitly or explicitly assert or imply any connection with, sponsorship or endorsement by the Original Author, Licensor and/or Attribution Parties, as appropriate, of You or Your use of the Work, without the separate, express prior written permission of the Original Author, Licensor and/or Attribution Parties. For the avoidance of doubt: Non-waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; Waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License if Your exercise of such rights is for a purpose or use which is otherwise than noncommercial as permitted under Section 4(b) and otherwise waives the right to collect royalties through any statutory or compulsory licensing scheme; and, Voluntary License Schemes. The Licensor reserves the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers

voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License that is for a purpose or use which is otherwise than noncommercial as permitted under Section 4(c). 5. Representations, Warranties and Disclaimer UNLESS OTHERWISE MUTUALLY AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE WORK, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU. 6. Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 7. Termination This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above. 8. Miscellaneous If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable. No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent. This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You. The rights granted under, and the subject matter referenced, in this License were drafted utilizing the terminology of the Berne Convention for the Protection of Literary and Artistic Works (as amended on September 28, 1979), the Rome Convention of 1961, the WIPO Copyright Treaty of 1996, the WIPO Performances and Phonograms Treaty of 1996 and the Universal Copyright Convention (as revised on July 24, 1971). These rights and subject matter take effect in the relevant jurisdiction in which the

License terms are sought to be enforced according to the corresponding provisions of the implementation of those treaty provisions in the applicable national law. If the standard suite of rights granted under applicable copyright law includes additional rights not granted under this License, such additional rights are deemed to be included in the License; this License is not intended to restrict the license of any rights under applicable law.

Bibliography

208