UK evaluation of evidence based

parenting programmes: Past and

future

Geoff Lindsay

Evidence based parenting programmes and social inclusion

Middlesex University, 20 September 2012

Geoff.lindsay@warwick.ac.uk

overview

• Developing EB parenting programmes – local

level

• National policy driven – Parenting Early

Intervention Programme 2006-11

• From targeted to universal parenting

programmes: CANparent trial 2012-14

• Future?

1. Initial developments

• Development of a programme

– Key role of the developer

– Theoretical rationale and parents aimed at

– Initial studies: try–out leading to randomized

controlled efficacy trial

– More trials

– Trials by independent researchers

– Effectiveness trial(s) in real life community settings

– Scaling up

• Opting in by individual organisations

• Local, regional, national policy

Developments in UK

• England has implemented national policies of

supporting parents through EB parenting

programmes

• 3-stage process to develop EB practice

– Literature review (Moran et al. 2004)

– Parenting Early Intervention Pathfinder (Lindsay et

al. 2008, 2011a)

– Parenting Early Intervention Programme (Lindsay et

al 2011b)

• Scaling up

• Evaluation of both outcomes and processes

2. The PEIP evaluation 2006-11

• Government initiative, LAs funded

• Pathfinder 2006-8 in 18 LAs,

– 3 EB programmes

– Focus children 8-13 years with or at risk of developing

behavioural, emotional and social difficulties

• Study of effectiveness across 18 LAs?

– Are there different effects for different programmes?

• What influence successful implementation on this scale?

• Implications for policy and practice?

Pathfinder evidence 2006-08

• Positive parent outcomes

– All 3 programmes were effective

– Parents rated them highly

• Process factors also important

– LA variations in efficiency and cost effectiveness

– Importance of the organisations support to implement

• On basis of positive evidence: government

funded a national roll out across all LAs in

England

The PEIP evaluation 2008-11

• Government initiative, all LAs funded

• Can evidence-based parenting programmes be

implemented effectively when rolled out across a

whole country?

– Are there different effects for different programmes?

• What factors influence successful

implementation on this scale?

• Implications for policy and practice?

Purpose of the PEIP evaluation

• All PEIP recognised programmes were evidence-

based (in controlled conditions- efficacy trials)

• Pathfinder showed the effectiveness of the three

programmes (across 18 LAs)

• PEIP explored national roll out across England

– Impact on parent and child outcomes

– LA and organisational factors in implementation

– Differential effects by parents/programme?

• One-year follow up

• Cost effectiveness

The programmesFive main programmes

Pathfinder 2006-08 (Wave 1):

• Triple P

• Incredible Years

• Strengthening Families Strengthening Communities

PEIP 2008-11 (Waves 2 & 3) – above plus:

• Strengthening Families 10-14

• Families and Schools Together (FAST)

– Insufficient data from FAST for the main quantitative analyses

– Similarly for Parent Power, STOP and Parents Plus (added later

in the PEIP)

Quantitative data• Pre-course questionnaires

– 6,143 parents

– Attending 860 groups

– In 43 LAs

• Post-course questionnaires

– returned by 3,325 (54%) of original sample.

• Change from pre-course to post-course in

scores on key parent and child measures

• Is there improvement?

• If so, how much?

Parent mental well-being: Warwick

Edinburgh Mental Well-being Scale

•Each item rated 1-5 (none to all of the time) score range

14–70, highly reliable (alpha=0.94), national norms

•Looking for increase in score

14 items, e.g.:

- I've been feeling optimistic about the future

- I've been feeling useful

- I've been feeling relaxed

Parenting Laxnesse.g. 1 7

1. When I give a fair threat or

warning ....

I always do

what I said

I often don't

carry it out

2. If my child gets upset when I

say 'No' ....

I stick to

what I said

I back down

and give in to

my child

•Each item scored 1-7, score range 6-42,

•Good reliability (alpha=.77).

•High scores represent less effective practice so looking for a

reduction in score.

Parenting Over-reactivity

Each item rated 1-7, score range 6-42, good reliability

(alpha=.72). Looking for reduction in score.

e.g.

When my child misbehaves ....

I raise my

voice or yell

I speak

calmly to my

child

When there's a problem with

my child ....

things build

up and I do

things I don't

mean to

Things don't

get out of

hand

Did PEIP target the ‘right’ parents?-

‘disadvantaged’ Yes

• 91% biological parents to the target child, 85% female

• Wide range of minority ethnic groups (19% vs 8% nationally)

• High proportion single parents (44% vs. 24% national

statistics) two-thirds living in rented property (63% vs. 27%)

• 54% no qualifications or only some GCSEs, but 20% with

HE qualifications and 11% with degrees

• High levels of support needs: 49% had seen GP in last 6

months and 21% had seen a social worker

• Low levels of mental well-being: 75% scored below the

national median as started their programme.

• So, skewed to disadvantaged but note also the range

Parents’ mental well-being at start

Did PEIP target the right children? Yes

• Mean age 8.6 years (but wide range: 37% age 0-7;

54% age 8-13; 9% age 14+)

• 62% boys

• 49% entitled to Free School Meal (16% nationally)

• 12% - statements of SEN (3% nationally)

• 31% - additional educational support in school

• High % had behavioural, emotional and social

difficulties with a negative impact on everyday life

Strengths & Difficulties Questionnaire (SDQ)

• 25 items, each rated on 3 point scale (not true,

somewhat true, certainly true)

• E.g. My child:

– Often has temper tantrums or hot tempers

– Often fights with other children or bullies them

• UK national norms from parents of a sample of

10,000+ children aged 5-15

• About 5-6 times higher prevalence of

behavioural problems than the norm

SDQ at pre-course

(% above clinical cut-off)

SDQ Scale National% PEIP%

Emotional symptoms 11.4 39.2

Conduct problems 12.7 60.7

Hyperactivity 14.7 48.3

Peer problems 11.7 44.7

SDQ Total difficulties 9.8 56.5

Prosocial scale 2.3 18.8

Impact score 8.8 60.4

p < .001 in all cases

Did the parent and child outcomes

improve after the groups?

• All effects are

statistically highly

significant

• Large effect sizes for

parent gains

• Effect size is a standardised

way of presenting the change

in outcomes:

<0.2 small,

0.5 medium;

0.8+ large.

• Parent outcomes show the

largest effects because

directly targeted

Were there different effects by

Programme?• Four programmes had sufficiently large samples to

support comparison:

• All programmes were effective in improving

outcomes, but some (relatively small) differences in

favour of Triple P compared with SFP 10-14

– Taking into account pre-group scores to look at

change/improvement

– Controlling for a range parent and child background factors

(gender, housing, education, ethnicity, child age etc.)

– Including fixed effects for LA

e.g. Comparison of programme effects

relative to Triple P

How did parents rate the parenting group?

• Generally very positive for all programmes

• Two factors

– Group leader style

• made me feel respected (98%);

• understood me and my situation (98%);

• I could be honest about my family (98%);

• interested in what I had to say (98%) etc.

– Helpfulness of the programme

• has been helpful (98%);

• helped me personally to cope (95%);

• helped me deal with child’s behaviour (96%);

• I have fewer problems than before (84%).

Group leaders’ styleHelpfulness of

programme

• SFP 10-14 significantly lower ratings - possibly because mean pre-

course scores for SFP 10-14 were particularly low

One year follow up

• Parent mental well-being fell back somewhat, but still better

than at pre-course

• Improvements in Parenting Scale scores and child

outcomes completely maintained

30

32

34

36

38

40

42

44

46

48

Pre-course Post-course Follow-up

Me

an

sco

re

with

95

% c

on

fid

en

ce

in

terva

l

Mental well being Parenting Child difficulties

Quantitative data: conclusions

• Clear evidence of success of the PEIP

– Parent and child improvements across a range of

important dimensions

– Parents very positive about the group experience

(those who completed)

– Maintained positive outcomes one year later

• Some variation between programmes in gains

– On these particular measures

– Programmes do have different foci

– No random allocation to treatment

Qualitative data

• 429 interviews with

– LA strategic leads and/or operational leads

– other professionals involved in parenting support,

e.g., parenting experts

– parenting group facilitators

– school representatives

– parents

LA factors & quality outcomes

• Strategic leadership & operational co-

ordination combined supported roll-out of the

PEIP in LAs.

• Where these were not in place, the PEIP was

less efficient in organising groups & reaching

parents

– Strategic leadership, including the existence of a

parenting strategy, helped establish the PEIP and

support it through the roll out

– Strategic leadership meshed the PEIP with existing

priorities & infrastructure.

LA factors & quality outcomes

• Operational co-ordination helped PEIP to be

delivered across an LA.

• Models differed – some LAs had one central co-

ordinator & others divided the co-ordination role

on a geographic basis.

– Each model could be effective

Qualitative data conclusions

• PEIP roll-out across LAs was not an easy task,

but

• Possible to achieve positive outcomes with a

diverse workforce & in a range of settings

• Facilitators needed to be able to engage

parents, be well trained, to deliver with fidelity &

to have access to supervision

• Strategic & operational management &

leadership necessary for successful LA roll-out.

So,…

• Overall PEIP was effective on all our measures

– And improvements were maintained one year later

• All 4 programmes were effective

– with some differences between them

• Large variation between LAs in numbers of parents

supported - cost effectiveness varied as a result

• Strategic and operational leadership and management

were crucial

• Use of a wide range of facilitators possible

• Very successful government initiative and clear

evidence supporting use of these programmes

3. CANparent trial

• Change of government – and policy focus

• From targeted to universal parenting classes

• 3 trial areas

– Camden, High Peak Derbyshire, Middlesbrough

• 14 providers of parenting classes

– All met criteria shown by research to be

characteristics of effective parenting programmes

• But lower level of evidence than PEIP

• Aim: to examine whether a market can be

developed so

– no need to subsidise (at all? partially?)

– Providers will want to develop their provision across

England/UK

• Parents receive £100 value vouchers to

‘purchase’ a parenting programme they choose

from those available in that trial area

• 4th non-voucher trial area - Bristol

The CANparent study• Strand 1: Supply side

– How the programme providers develop over the trial

• Strand 2: Survey research and impact evaluation

– Penetration surveys: in trial and comparison areas

• parents attitudes to parenting classes, awareness and

take up.

– Participating parents

• 2000 parent sample: improvements in mental well-

being, satisfaction and sense of effectiveness as a

parents, dealing with daily hassles

• All parents: satisfaction survey

• Strand 3: cost effectiveness

4. future?

• Importance of both targeted and universal

support?

• Parental choice?

• Funding?

• ‘Evidence based’ concept

• Will providers welcome opportunities o develop

large scale?

• Infrastructure?

• Support for implementing? For facilitators?

Maintenance of fidelity?

Conclusions

More information: http://www2.warwick.ac.uk/fac/soc/cedar/

Geoff.lindsay@warwick.ac.uk

For information about the CANparent evaluation

http://warwick.ac.uk/canparent