Actuarial Instruments in Risk Assessment

Yale University Law & Psychiatry DivisionHoward Zonana MD

Madelon Baranoski PhDMichael Norko MD

Alec Buchanan PhD MD

Governor’s Sentencing and Parole Review task Force

December 3, 2007

Antisocial Personality Disorder and Psychopathy

Howard Zonana MD

Connecticut Mental Health Center

Yale University School of Medicine

12/3/2007

Antisocial Personality DisorderDSM IV-TR

• Pervasive pattern of disregard for, and violation of, rights of others that begins in childhood or early adolescence and continues into adulthood.

• The person must be at least age 18 and must have a history of Conduct Disorder before age 15

Conduct Disorder

• Aggression towards people and animals

• Destruction of property,

• Deceitfulness or theft

• Serious violation of rules

Diagnostic Criteria for ASPDThree or more of the following:

• Failure to conform to social norms with respect to lawful behaviors as indicated by repeatedly performing acts that are grounds for arrest

• Deceitfulness, as indicated by repeated lying, use of aliases, or conning others for personal profit or pleasure

• Impulsivity or failure to plan ahead• Irritability and aggressiveness, as indicated by

repeated physical fights or assaults

Diagnostic Criteria for ASPD

• Reckless disregard for safety of self or others

• Consistent irresponsibility as indicated by repeated failure to sustain consistent work or honor financial obligations

• Lack of remorse, as indicated by being indifferent to or rationalizing having hurt, mistreated, or stolen from another

Epidemiology

• Prevalence rates of 2-3% for men and 1% for women in the general population

• Up to 60% in male prisoners• After age 30 the most flagrant antisocial

behaviors tend to decrease• Genetic and environmental factors contribute to

the risk• Both adopted and biological children of parents

with antisocial personality disorder are at increased risk for the disorder

Epidemiology

• The odds of developing antisocial personality disorder for those leaving formal education at 11 years was almost five times that of those remaining in education until 15 years,

Actuarial Measures and Risk

Madelon Baranoski, PhD

Associate Professor

Yale School of Medicine

Outline

• Actuarial measures and how they are developed

• Assessing criminality and antisocial personality

• Measures pertinent to re-offense– PCL-R (Psychopathy Checklist-Revised)– VRAG (Violence Risk Appraisal Guide) – LSI (Level of Service Indicator)

Actuarial Measures

• Actuarial refer to prediction by statistics• Analysis first used by insurance companies to

calculate financial risk • Measures developed through analysis of

outcomes that are associated with “predictor variables”– Variables weighted according to their ability to

differentiate between groups – Weighted variables combined to form a scale– Scale cross-validated on different populations to

derive estimates of probability that specific outcome will occur in a particular time

– Production of “life tables”

Development of life table life expectancies

at age 65 for American males

Paternal Death < 65

Smoking >10 Years

Obesity- BMI>30

Diabetes

DeadLiving

Actuarial Risk Assessment

• Identification of individuals at higher risk because of selected traits that correlate with criminal recidivism or violence

• Established through empirical association of traits with violence

Development of Actuarial Criminal Risk Measures

Personality Studies Criminality Studies

Predictor Variables of Criminal Behavior

• Offenders of Interest– Repeat offender– Violent offender– Sex offender

• Characteristics of offender– Personality – Attitude– Behavior– Substance use and addiction

• Situational characteristics– Poverty– Gang affiliation– Family business

Psychopathy Check List-Revised(PCL-R)

• Developed as research tool to study antisocial personality disorder

• Interview/collateral information provides data for assessing 20 areas of personality/behavior

• Results identify two domains– Behavioral domain– Personality domain

(Robert D. Hare, 1990)

PCL-R

• Personality Domain– Glibness/superficial

charm– Grandiose sense of

self– Pathological lying– Conning/manipulative– Lack of remorse/guilt– Shallow affect– Callous/lack of

empathy– Failure to accept

responsibility for actions

• Behavioral Domain– Boredom/need for

stimulation– Parasitic lifestyle– Poor behavioral

controls– Early behavioral

problems– Lack of long-term goals– Impulsivity– Irresponsibility– Juvenile delinquency– Revocation conditional

release – Criminal variety

PCL-R Considerations

• Strong correlation with criminal recidivism, violence, and sexual violence

• Inter-rater reliability• Scores indicate need for

monitoring vs. treatment

• Abbreviated version

• Ineffective for assessment of mental health risk

• Predicts life long risk, not imminent risk

• Insensitive to treatment effect or changes in situational factors

• Accuracy depends on extensive collateral data

• Requires extensive training

Strengths Limitations

Level of Service Inventory-Revised• Blend of actuarial and dynamic factors• Measures 54 risk/need factors over 10

domains– Criminal history, employment/education,

family/marital, accommodation, leisure/recreation, friends/assoc, emotional/mental health, attitudes/orientations (Andrews & Bonta, 1995)

• Structured Interview with collateral data • Total risk/need score correlated with re-

offense• Identified target areas for intervention

Comparison

• PCL-R– Extensive use in Canadian

system– Requires specialized training,

collaterals – Best prediction at high and

low scores – Strong reliability across

studies– Specifically excludes AXIS I

mental health disorders– Most data on men

• LSI– Extensive use in American

correctional systems (Ohio studies)

– Requires training in structured interview

– Collateral data recommended, not required

– Variable outcomes across study sites

– Includes persons with mental illness

– Best prediction at high and low scores

– Most data on men

Distribution of Risk Category

0

10

20

30

40

50

60

Low Low-Moderate

Moderate Moderate-High

High

%N=2006

Lowencamp & Latessa, 2006

Re-Incarceration Based on Risk Classification

0

10

20

30

40

50

60

Low Low-Moderate

Moderate Moderate-High

High

%

Risk Category Levels

• Low – 0-13

• Low/Moderate – 14-23

• Moderate – 24-33

• Moderate-High – 34-40

• High – 41-54

Violent and Sexual Offenses by PCL-R Scores

0

10

20

30

40

50

60

70

80

<11 11 to 20 21 to 30 Over 30

Re-ArrestViolent OffenseSexual Offense

%N=3478

10% 13% 42% 35%

Actuarial-Standard Measures on Inmates

• Advantages– Identifies groups

most likely to re-offend

– Assesses criminality as style

– Provides standard data base for program and time evaluation

– Provides bases for cost and program allocation

• Limitations– Requires training

and fidelity checks– Limited accuracy for

any individual assessment

– Cannot predict the unusual

– Accuracy related to time of follow-up

– Requires different tools for different types of criminal acts

What is the Goal?What is the Goal?• What are the outcomes of interest?

– Type of Crime: General criminal recidivism vs. violence – Over what period: Within probation/parole vs. lifetime– Under what circumstance: In prison, in community with

supervision, in community• Who are being assessed?

– Persons with diagnosed mental illness– Persons screened for absence of mental illness– All persons– Men and women

• What level of risk is acceptable?– Zero tolerance vs. violence reduction– Reduction of overall crime vs. specific crime (juvenile,

domestic, sex offenses)

• How certain is adequate certainty?– Would you rather incarcerate many more to avoid one bad

outcomes or risk one bad outcome to avoid over incarceration • What cost is tolerable and for how long?

• What are the outcomes of interest?– Type of Crime: General criminal recidivism vs. violence – Over what period: Within probation/parole vs. lifetime– Under what circumstance: In prison, in community with

supervision, in community• Who are being assessed?

– Persons with diagnosed mental illness– Persons screened for absence of mental illness– All persons– Men and women

• What level of risk is acceptable?– Zero tolerance vs. violence reduction– Reduction of overall crime vs. specific crime (juvenile,

domestic, sex offenses)

• How certain is adequate certainty?– Would you rather incarcerate many more to avoid one bad

outcomes or risk one bad outcome to avoid over incarceration • What cost is tolerable and for how long?

What Actuarial/Standard Measures Can Not Do

• Predict rare occurrence (“crime of the century”)• Address violence from mental health disorders• Predict first offenses• Prove prevention• Hold statistical accuracy for individual

assessments • Replace educated assessors

Requirements for All Actuarial Measurements

• Availability of data

• Standard use of measure

• Use on standardized population

• Adequate follow-up

• Customized to cultural, setting, and group

Meaning of Actuarial Test Outcome

Michael Norko MD

Associate Professor of Psychiatry

Yale University School of Medicine

Meaning of Actuarial Test Outcome

• Risk level

• Positive predictive power

ACME Risk Screening Tool

(ARST)

ARST Validation Data

• Separates into low risk and high risk

– Statistically significant separations

– Quite good AUC of 75%

• High risk has average risk of 37%

• Low risk has average risk of 9%

• Overall risk in population is 18.5%

What does 37% risk mean?

What does 37% risk mean?

What does 37% risk mean?

So what does it mean?

Using the ARST

The Results

What’s the Outcome?

The “Low Risk” Group

The “High Risk” Group

Meaning of Actuarial Test Outcomes

• % Risk level

–X% of people just like the subject will commit act w/in y period of time

• Positive predictive power–The % of the people predicted to

commit the act who actually do

Positive Predictive Power

• PPP almost never > .50

• In other words, the majority of nearly all identifiable high risk populations never commit the predicted act– For example, False Positive rates for PCL-R

in literature are between 50-75%• Freedman: J Am Acad Psych Law 2001

Accuracy of Predictions of Offending

Alec Buchanan PhD MD

Associate Professor of Psychiatry

Yale University School of Medicine

Indices of effectiveness of validated prediction studies 1970 – 2000 (from Buchanan and Leese, 2001)

0.5

1

1970 1980 1990 2000year

IoE Fitted values

Number needed to detain

• NND

• the number of individuals who would need to be detained in order to prevent one violent act

• the inverse of positive predictive value

Buchanan and Leese Lancet (2001) 358, 1955-59

Relationship between Number Needed to Detain (NND) and prevalence (p) when sensitivity = 0.73 and specificity = 0.63

0

5

10

15

20

25

30

0 0.1 0.2 0.3 0.4 0.5 0.6

p

NN

D

NND and base rates

20 % 10% 5%

Receiver operating characteristics of predictions of conviction at 10.5 years (all offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sensitivity

Chance

Receiver operating characteristics of predictions of conviction at 10.5 years (all offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sensitivity

ASChance

Receiver operating characteristics of predictions of conviction at 10.5 years (all offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sensitivity

ASChanceAS + C

Receiver operating characteristics of predictions of conviction at 10.5 years (all offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sensitivity

ASChanceAS + CAS + C + D

Receiver operating characteristics of predictions of conviction at 10.5 years (serious offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sens

itivi

ty

Chance

Receiver operating characteristics of predictions of conviction at 10.5 years (serious offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sens

itivi

ty

AS

Chance

Receiver operating characteristics of predictions of conviction at 10.5 years (serious offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sens

itivi

ty

AS

Chance

AS + C

Receiver operating characteristics of predictions of conviction at 10.5 years (serious offences)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1-specificity

Sens

itivi

ty

AS

Chance

AS + C

AS + C + D

How accurate are predictions of offending ?

• … not sufficiently better than chance to allow “prevention by detention” of unusual offences without detaining many people who would not have offended

• this may not improve much

• at 10% prevalence present psychiatric technology would detain 6 who would not offend for every 1 who would

• … at best

How accurate are predictions of offending (1)?

• Better than chance

• How much better?

Index of effectiveness

3/{log[Sn/(1-Sn)] +log[(Sp/(1-Sp)]}

Which information helps us predict?

Will accuracy improve?

What does this mean?