Advice Advice ServicesServices

Measuring PerformanceMeasuring Performance

The Leicester ExperienceThe Leicester Experience

Best Value Review 2001Best Value Review 2001 ““Poor Service, probably going Poor Service, probably going

to improve”to improve” Not clear how services linked Not clear how services linked

to corporate priorities, and to corporate priorities, and why the services were why the services were delivered in the way that they delivered in the way that they were…were…

Advice Strategy 2002Advice Strategy 2002

Set out the intention to Set out the intention to introduce a common introduce a common performance monitoring performance monitoring framework for directly delivered framework for directly delivered & voluntary sector advice & voluntary sector advice servicesservices

Signed up to the Quality MarkSigned up to the Quality Mark Single commissioning point for Single commissioning point for

voluntary sector advice servicesvoluntary sector advice services

Work on performance Work on performance measuresmeasures Participation in national pilot Participation in national pilot

lead by Citizens Advicelead by Citizens Advice Need to progress quickly in light Need to progress quickly in light

of BVRof BVR Focus on outcomes rather than Focus on outcomes rather than

throughput measurementthroughput measurement What tangible results from What tangible results from

advice and how to compare advice and how to compare services?services?

The Monitoring SystemThe Monitoring System

Common definitions – levels of Common definitions – levels of advice, cases and enquiriesadvice, cases and enquiries

Monitoring requirements scaled to Monitoring requirements scaled to level of advicelevel of advice

Outcomes of advice at specialist Outcomes of advice at specialist levellevel

Customer feedback informationCustomer feedback information Mystery shoppingMystery shopping Financial informationFinancial information

New Monitoring SchemeNew Monitoring Scheme

Started in Leicester 2003/04Started in Leicester 2003/04 Nottingham 2004/05Nottingham 2004/05 Amended prior to introduction Amended prior to introduction

in Nottinghamin Nottingham Supported by set of guidelines Supported by set of guidelines

and Access Databaseand Access Database Further work on IT support Further work on IT support

requiredrequired

What does it Measure?What does it Measure?

Information RequestsInformation Requests The number of people requesting The number of people requesting

information (e.g. leaflets) but not the information (e.g. leaflets) but not the number of leafletsnumber of leaflets

EnquiriesEnquiries A headcount of enquirers, not the A headcount of enquirers, not the

number of issuesnumber of issues CasesCases

Specialist casework on which Specialist casework on which outcome monitoring takes placeoutcome monitoring takes place

Equal Opportunities Equal Opportunities InformationInformation For both enquiries and casesFor both enquiries and cases

GenderGender DisabilityDisability Ethnicity monitoringEthnicity monitoring

But…But… For cases we use the CRE full codes For cases we use the CRE full codes

(includes mixed race)(includes mixed race) For enquiries we use the CRE short For enquiries we use the CRE short

codes (Black, Asian, White, Chinese, codes (Black, Asian, White, Chinese, Other)Other)

Ward informationWard information

Again, for both enquiries and Again, for both enquiries and casescases Records the ward residency of Records the ward residency of

the service userthe service user Some “City” coding if of no fixed Some “City” coding if of no fixed

abodeabode High levels of “City” coding not High levels of “City” coding not

accepted unless the service is accepted unless the service is particularly geared to serving particularly geared to serving homeless users.homeless users.

Case RecordingCase Recording

Numbers of casesNumbers of cases Opened in the quarterOpened in the quarter Closed in the quarterClosed in the quarter Case outcomesCase outcomes Advice specific outcomes (e.g. Advice specific outcomes (e.g.

benefit raised, debt benefit raised, debt rescheduled)rescheduled)

General outcomes (won, lost, no General outcomes (won, lost, no further contacts)further contacts)

ClaimsClaims

At the present time At the present time completion of claims for completion of claims for DLA/AA are classed together DLA/AA are classed together with caseswith cases

Some work in Leicester to Some work in Leicester to separate these where a separate these where a General Help with Casework General Help with Casework service is provided (e.g. where service is provided (e.g. where claim outcomes are known to claim outcomes are known to the agency)the agency)

Case OutcomesCase Outcomes

Welfare BenefitsWelfare Benefits Amount raised (annualised & Amount raised (annualised &

lump sums)lump sums) DebtDebt

Amount of debt managedAmount of debt managed Housing outcomes (see next Housing outcomes (see next

slide)slide) Welfare benefit outcomesWelfare benefit outcomes

Housing OutcomesHousing Outcomes

HousingHousing Possession preventedPossession prevented Possession suspendedPossession suspended Eviction preventedEviction prevented Eviction suspendedEviction suspended Adverse homelessness decision Adverse homelessness decision

overturnedoverturned Money gained in damagesMoney gained in damages Welfare Benefit outcomesWelfare Benefit outcomes

Case Outcomes Case Outcomes (continued…)(continued…) EmploymentEmployment

Amount of award/ settlementAmount of award/ settlement Other employment outcomes Other employment outcomes

(e.g. reinstatement) – not (e.g. reinstatement) – not required to be broken downrequired to be broken down

Immigration & NationalityImmigration & Nationality Immigration status determined Immigration status determined

positivelypositively Welfare benefit outcomesWelfare benefit outcomes

Other Areas of AdviceOther Areas of Advice

Health & Community CareHealth & Community Care EducationEducation No outcomes set in the No outcomes set in the

current monitoring scheme.current monitoring scheme.

Financial MonitoringFinancial Monitoring

Information is specific to the Information is specific to the advice contract – not the advice contract – not the organisation as a wholeorganisation as a whole

Requires apportionment of Requires apportionment of organisational costs to the various organisational costs to the various contracts (e.g. how much is contracts (e.g. how much is Council, how much LSC?)Council, how much LSC?)

Requires cost centring where more Requires cost centring where more than one type of advice service is than one type of advice service is provided (e.g. housing casework, provided (e.g. housing casework, initial “generalist” advice services)initial “generalist” advice services)

Procedural MonitoringProcedural Monitoring

Replaced by one requirement Replaced by one requirement to hold the Community Legal to hold the Community Legal Service Quality MarkService Quality Mark

No Council auditing of No Council auditing of procedures governed by the procedures governed by the Quality MarkQuality Mark

Customer FeedbackCustomer Feedback

Three areas of assessmentThree areas of assessment Access to the serviceAccess to the service Quality of service receivedQuality of service received Impact of the serviceImpact of the service

Set questions in each of the areas Set questions in each of the areas which must be included in surveyswhich must be included in surveys

Set standard for numbers of returnsSet standard for numbers of returns Must be done for individual aspects Must be done for individual aspects

of the service (e.g. telephone help-of the service (e.g. telephone help-line, casework service = 2 customer line, casework service = 2 customer feedback exercises)feedback exercises)

One Year On – how is it One Year On – how is it used?used? Good information on the Good information on the

relative use of Information, relative use of Information, General Help and Specialist General Help and Specialist servicesservices 13,000 information 13,000 information 14,000 enquiries14,000 enquiries 1,400 cases1,400 cases

Information on Case OutcomesInformation on Case Outcomes 48% won, 19% no further contact48% won, 19% no further contact Available by agencyAvailable by agency

Information on Service Information on Service UsersUsers Difference in client groups Difference in client groups

across agenciesacross agencies Higher levels of BME access to Higher levels of BME access to

face to face advice servicesface to face advice services But proportionate access to But proportionate access to

phone (generally reflecting phone (generally reflecting make-up of population)make-up of population)

Total of 51% BME, 49% WhiteTotal of 51% BME, 49% White 23% Disabled, 57% Female23% Disabled, 57% Female

Ward Information on Ward Information on UsersUsers Two wards in the city account Two wards in the city account

for 31% of all advice service for 31% of all advice service usersusers Based on need or configuration Based on need or configuration

of the supply of services?of the supply of services? Overlay with the IMD – some Overlay with the IMD – some

areas clearly not provided for.areas clearly not provided for.

Outcome InformationOutcome Information

Specialist welfare rights services Specialist welfare rights services raised £1,100,000 in 571 cases raised £1,100,000 in 571 cases (average of £1,926 per user)(average of £1,926 per user)

General Help with Casework claims General Help with Casework claims raised over £2 millionraised over £2 million

Some contentious comparisons Some contentious comparisons made between these types of made between these types of service.service.

Issues relating to not including Issues relating to not including gains achieved under LSC gains achieved under LSC contracts by specialist agencies.contracts by specialist agencies.

Debt and Housing Debt and Housing OutcomesOutcomes Over £2.5 million of debts Over £2.5 million of debts

rescheduledrescheduled 38 possessions prevented38 possessions prevented 140 possessions suspended140 possessions suspended 111 evictions suspended111 evictions suspended 94 closed housing cases, 369 94 closed housing cases, 369

debt cases.debt cases.

Employment OutcomesEmployment Outcomes

£217,288 raised in awards £217,288 raised in awards and settlementsand settlements

40 other employment benefits40 other employment benefits Likely to be affected by Likely to be affected by

Dispute Resolution Rules in Dispute Resolution Rules in the future.the future.

212 specialist cases closed212 specialist cases closed

Assisting in Service Assisting in Service PlanningPlanning Leicester’s Advice Services ReviewLeicester’s Advice Services Review A “challenging context” - reduction of A “challenging context” - reduction of

35% in the Advice Services Budget35% in the Advice Services Budget Information from the monitoring Information from the monitoring

system used in the review to assess system used in the review to assess performance of internal and voluntary performance of internal and voluntary sector teamssector teams

No performance case for outsourcing No performance case for outsourcing advice services – broadly comparable advice services – broadly comparable productivity and outcomesproductivity and outcomes

Identifying GapsIdentifying Gaps

Problems with access limited to Problems with access limited to fixed points of the cityfixed points of the city

Those areas historically funded Those areas historically funded were well served, others notwere well served, others not

Outer estates in particular Outer estates in particular neglectedneglected

Range of different client criteria Range of different client criteria a potential barrier to access a potential barrier to access (mystery shopping)(mystery shopping)

ConclusionConclusion

This is far from perfect, but…This is far from perfect, but… It is helpful and gives solid It is helpful and gives solid

information on who is accessing information on who is accessing services and where;services and where;

Provides for comparison of specialist Provides for comparison of specialist casework services and teams;casework services and teams;

Gives solid information on gains to Gives solid information on gains to help build political support for advice;help build political support for advice;

Provides a starting point for informed Provides a starting point for informed debate about future priorities.debate about future priorities.

Next Steps…Next Steps…

Nottingham and Leicester Nottingham and Leicester benchmarking performance this benchmarking performance this yearyear

Group of LA’s and LSC Group of LA’s and LSC developing a ‘toolkit’ - including developing a ‘toolkit’ - including Manchester and SunderlandManchester and Sunderland

‘‘Toolkit’ available to other LA’s Toolkit’ available to other LA’s in early 2005 for discussionin early 2005 for discussion