Monitoring Physical Activity Outcomes in Local Interventions

Andrew Power

Strategic Manager (Physical Activity), County Durham Sport

Background / Delivery and Impact

What we currently gather• ID

• Gender

• Age / DOB

• Address

• Postcode

• Email

• Tel

• Emergency contact

• Disability

• Ethnicity

• Sexual orientation

• Faith

• Permissions (surveys, news, etc)

• Registration date

• First attendance

• Last attendance

• Employment status

• Baseline PA

• Ongoing PA (survey data)

• How found out (referral, leaflets, etc)

• Participation group (CVD, T2D, family, other)

• Satisfaction data

• Attendance data

• Contact/session hours

• Sessions delivered

• Cost per participant

• Cost per attendance

• Geo location

• Correlates: population densities, IMD, Sport England market segments

• Contribution against set outcomes

• Media uploads

Tools used

Loughborough Single Item • Screening of inactivity at baseline

Stanford 7 day recall• Measure of total physical activity• Used to track change

Objective monitoring• Accurate, real time data /

comparison with self-report

Accelerometry

Objective monitoring

Real time measurement

Tracks change

Hourly, daily, weekly patterns

Added validation of subjective tools

Objective measures

Journey

• CPAL - £4.5m NHS funded outcome driven pathway• Demonstration of impact required PA monitoring• SEF for Obesity - 7 day recall adopted (total PA)• 28 projects: baseline and monthly follow up for 6

mths+• Paper based at sessions / postal survey / telephone,

etc• Data in paper files / Excel / Access / for every

individual

x 13,129!!

• Developed an ROI tool (for CVD risk) • Assumes valid PA data (increasing after 6 months = 62%)• Discounts those we have no influence over e.g. how many

would go on to develop CVD with/without our intervention• Derives the potential economic impact for the intervention • £2.10 - £3.20 return for every £1 invested

http://www.countydurhamsport.com/physical_activity/CPAL

Journey (cont.)

• 28 projects x 13,129 participants = months of chasing paper!

• Significant data set to model / draw comparisons against

• Auditing revealed cyclical variation in protocols and data returns across the 28 projects/23 organisations

• Recommended focus on consistency and efficiency without losing validity

• Resolve the difference between 'reported' and 'actual'

Key issues from CPAL

• Move4Life - 1/5th of the annual budget of CPAL

• Cost and capacity to monitor significantly reduced

• Move into Sport - opportunity to take learning to the next level - objective monitoring

Key issues moving forward

Re-engineering the process...

£40,000 baseline

data

£87,000 follow up

data

Re-engineering the process...

£8,000 gather & report

£18,000 audit and analysis

£153,000 monitoring

cost

M4L and MiS approach

£zero cost of follow up fully

automated

£16,000 for 2,500 participant data

gathering entry and analysis

The automated approach...

• 'Upshot' - developed by the Football Foundation• Demonstrating outcomes from community based projects• Performance dashboard shows progress towards targets• Reports on recruitment, attendance, sessions/hours delivered• Tells the 'stories' behind statistics by using photos, videos,

documents and audio files, uploaded as media• Can quickly identify high performing areas• Address underperforming areas while there's still time to effect

change

Participants, sessions delivered, attendance, etc

Qualitative feedback, images, programme informationLink indicators to

outcomes, reports generated on progress

Registers of attendance, individual, group and programme attendance reports

Attendance registers updated via smartphone

The automated approach...

• Define the outcomes and activities of a project and

show how they link together• Identify gaps in data entry as it happens• Basic analysis on who's participating in projects, their

profile, how often and provides the ability to track

change• Aggregated anonymised performance data at funder

level across projects• Automated follow up systems, email and survey

functions

Satisfaction surveys, 3,6,9 &12 month PA surveys

Responses are tagged to individual profiles and aggregated / downloadable

Response rates are now boosted initially by telephone surveys (max 20 per week) and overlaid by Upshot and mail merged postal surveys (again using Upshot)

View who responds, reminder system for those who haven't yet

View aggregated data on responses ('survey monkey' style)

Filter participant by age, gender, ethnicity and project

View participant distributions against population densities, deprivation indices and....

...against market segments overlays which help direct focus of delivery

• Lower returns (60% follow up data in CPAL)

now 15-30%

• More robust, consistent and efficient system

of data collection

• Modelling against increasing data set +

objective assessment = increase validity

THANK YOU

Andrew Power

Strategic Manager (Physical Activity)

County Durham Sport

Tel: 03000 264 617

E: andrew.power@durham.gov.uk

Twitter: @behaviourchange