Achieving Family Economic Success with the People We Serve Using

Results Based Accountability

United Way of Greater New HavenAnnie E. Casey Foundation

February 16, 2010

2

Results for Today

1. Can apply the fundamentals of Results Based Accountability (RBA)

2. Know the different types of and how to use performance measures in performance accountability

3. Are equipped to make applications to the United Way of Greater New Haven and the Casey Foundation

3

Various Outcome Models

Balanced ScorecardOur Own Hybrid

Scales and Ladders

Logic Model Results Mapping

Outcome Funding Framework

4

AECF and UWGNH use

Results

Based

Accountability

Results Matter“It matters if we succeed or fail. It

matters if things get better or worse.” Mark Friedman

5

a focused way of thinking and taking action that starts with ENDS

(defining the change you want to make) and works backwards to the means

(developing powerful strategies to bring about the change).

Results Based Accountability

6

Crosswalk RBA and Logic Frameworks

Feature Logic RBA

Program centric with depiction of expectations (processes) based on assumptions & “ifs & thens”

Uses Factor analysis based on

trend line data

Results centric with course of action to improve results

Population & program focus Program Both

Data to inform decision making – trend line

Shared accountability

Involvement of partners

Defined terms

7

1. Has consistent terms to use.

2. Uses a simple, step-by-step way to take action to make it bettertake action to make it better for populations of children, individuals or families.

3. Brings people together on what they they care aboutcare about – conditions of well-being.

4. Provides a common frameworkcommon framework for human services, government, funders and communities to work together.

5. Has a dual focus – population & program level change

Common Common LanguageLanguage

Common Common SenseSense

Common Common GroundGround

Why RBA Practice

8

What is the Added Value of RBA?

Draws people together on what they care about – better results for children & families

Makes the “silo” way of working obsolete because people align their actions as partners for a shared result

Uses data to inform decisions vs. the politics of the day

Uses what works vs. favorite programs to achieve results

Shares accountability across partners for better results

Tracks progress regularly by using data to course correct or to accelerate the work

9

RBA Practice - Language ConsistencyTo avoid - many terms, few definitions, & interchanging

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

10

Two Profoundly Different Accountabilities

Performance Populations For Customers Whole in a place or geographic

area

In Jobs Agency All Families in the City

11

Common Language

Population Accountability

Result (Outcome) – quality of life condition Families are economically successful.

Indicator – quantifies the result in the form of a percent, rate or ratio - % of residents employed

12

PopulationPopulation

AccountabilityAccountability

Whole PopulationWhole Population

++

Desired ResultDesired Result

++

PlacePlace

=

Population Accountability

13

PopulationPopulation

AccountabilityAccountability

Whole PopulationWhole Population

FamiliesFamilies

++

Desired ResultDesired Result

Have Economic StabilityHave Economic Stability

++

Place Place New HavenNew Haven

=

Population Accountability

14

Whole Populations in a Place

EX Children in Low Income Families in New Haven

ResultQuality of Life Change

EX: Children & families achieve financial stability

Indicator Trend LineQuantifies the Result

EX: 60% of families are employed in 2008 & 61% in 2009

Strategies What works to change the indicator Trend Line

Multi-sector jobs pipeline

Program or Service PopulationRe-entry Residents

Performance Measures

(How Much, How Well, Difference Made) 120 residents in training, 70% complete training, 60 (50%) residents obtain jobs

Actions to Take

(Partners commit to take action) Hospital agrees to hire ex-offenders, Library sponsors a job fair,

Union establishes a apprentice program

POPULATIONR

BA

PERFORMANCE

15

7 Q’s of Population Accountability

1. Which population(s) are we concerned about?

2. What quality of life conditions do we want for the population(s)? (results)

3. What measures can tells us that these conditions are improving? (indicators)

4. How are we doing on the most important indicators for all populations? (trend line) What factors are affecting the trend line? (story behind the data)

5. Which partners are needed to change the trend line?

6. Which strategies based on evidence, practice or experience are powerful enough to change the trend line?

7. What actions are going to take as a group?

16

Shifting from Population

Accountability to

Performance Accountability

17

Performance

Accountability

CustomersCustomersRecently released OffendersRecently released Offenders

++

ProgramProgram

Shea ServicesShea Services

++

Difference MadeDifference Made# retaining jobs for 6 months# retaining jobs for 6 months

==

Performance Accountability

18

An approach to help us better see the link between our work and our performance

and the results achieved for the people we serve in our programs.

Performance Accountability Is . . .

19

Performance Measures

A Closer Look

20

1. How much did we do? (QuantityQuantity)

2. How well did we do it? (QualityQuality)

3. Who is better off? (ImpactImpact)

Performance Measures AnswerThree Questions

21

1.1. How many did we serve? (Customers)

# of individuals served through jobs pipeline

# of agencies did we refer to?

2.2. How many activities did we provide?

# of training sessions held

# of events held in the neighborhood

How much did we do?

22

1. Are our efforts Timely? % of referrals completed in 24 hours

2. What are the completion or Attendance rates? % new workers completing training

3. Are our customers Satisfied? % of employers satisfied with job hires

4. What are the Costs per unit? TASCS

$400 per jobs skills training session held

5. What are professional or organizational Standards? Worker to Coaches ratio (10 to 1)

How well did we do?

23

What Difference Made – Better Off

1. Behaviors (#/% of participants who report to work on time)

2. Attitudes (Opinions)

(#/% of participants who value seeking a degree)

3. Circumstances (#/% of participants who have a job at program completion)

4. Knowledge/ Skills (#/% of individuals who have job increased skills) BACKS

24

Employment Readiness Class

How much did we do? (Quantity) # participating in the class

How well did we do it? (Quality) % satisfied with class

Who is better off? (Impact) #/% who get a job

25

How MuchHow Much How WellHow Well

Better OffBetter Off

How Performance Measures Fit Together

26

What difference did we make?

Performance Measures Start with

Effort

Effect

How hard did we try?

27

How much did we

do?

( # )

How well did we do

it?

( % )

Quantity Quality

Performance Measures Look at Both

28

Eff ort

HowMuch

We Did

HowWell

Here is the Fit of Performance Measures

Eff ect

29

How much service did we deliver?

#

How welldid we

deliver it?

%

How much change / effect

did we produce?

#

What quality of change / effect

did we produce?

%

Quantity Quality

Eff

ect

Eff

ort

Better OffBetter Off

The Fit In Detail

30

# of youth participating

in a jobs skills class

% of youth satisfied with the program

Quantity Quality

Eff

ect

E

ffo

rt

Is anyone better off?

# of youth who have a

job

% of youth who have a

job

Job Skills Class

31

Performance Measures Tips

32

Performance Measures Are NOT Equal

How well did we do it?How much did we do?

LeastImportant

2nd Most Important

3rd Most Important

MOSTImportant

Is Anyone Better Off?

33

100 job applicants in the program

Numbers Tell Only Part of the Story

40% of applicants complete course

11 22

33 44

%50 applicants employed

Numbers tell you the exact “number” of participants who were better off. (Q#3)

Percentages tell the degree of success of the program/organization. (Q#4)

#50% of applicants employed

# %

34

Measuring Improvement

Point in Time Improvement # / % of agencies showing increased

results in 2008

Point to Point Over Time # / % of agencies showing results for

two years

35

Performance Measures: How Many

Priority Measures – Most Important

Secondary Measures

Data Development Work to Do

2 or 3 or 4 “Headline” Measures

Others that you may want to track . . .

Need to gather data to measure what you don’t have

Data that needs repair (definition, quality, timely)

36

Setting Performance Targets

Helps us understand what we are aiming for and if we don’t reach targets then we can work to discover what didn’t work and what we can do better.

Helps to answer the key questions Is it producing what it should for its customers? Is this program worth the money?Can it perform better? Can it be done less expensively?

37

What/How much did we do?

#

Performance MeasuresPerformance Measures

How welldid we do it?

%

Quantity Quality

Effe

ct

Effo

rt

11 22

33 44

# %

Better Off

38

What Are The Steps To

Improve Our Performance?

39

1. Who are our customers? For which program?

2. How do we measure if we are making a difference? (customers are better off)

3. How do we measure if we deliver our services well? (quality measures)

4. What is the data telling us about how we are doing on these measures?

+ the trend line – better, worse or the same? For all customer groups especially where disparities exist

+ story behind the data – factors pushing up & pushing down ?

5. Which partners are needed to change the trend line?

6. What actions are powerful enough to address the factors? What works based on evidence, practice or experience?

7. What actions are we going to take as a group?

7 Performance Accountability Questions

40

People participating in the program or the activities to implement a strategy

children parents

1. Who are our customers?

Which program?

41

B = Behaviors Change

A = Attitudes/Opinions Shift

C = Circumstances are Better

K = Knowledge Increased

S = Skills Increased

“BACKS”

2. How do we measure if we are making a difference?

Ways people are “Better off”

42

T = Timeliness of service response-delivery A = Attendance rates and programs S = Satisfaction of how well trusted C = Costs per unit or amount spent S = Standards for services, staffing, etc.

“TASCS”

3. How can we measure if we deliver our services well?

Quality Measures

43

Current

History Forecast

Factors increasing trend

Factors decreasing trend

4. What is the data telling us about how are we doing on the performance measures?

What is the story?

Trend Line

? Better

? Same

? Worse

44

26542235

27772771

- 2006 - - 2007 - - 2008 - 2009

Shea Community Workforce Partner Number of Training Program Graduates with Jobs

45

Who is contributing?

Who is missing at the table? Who do we recruit to help move the work forward?

Which partners can expand their efforts?

5. Who are the partners?

46

What are the priority factors that are affecting the and of the trend line?

What do we think would be effective to make a difference – to improve the identified “better off” or “how well” performance measure?

What do we know from evidence, practice, or our experiences?

6. What actions are powerful enough to address the priority factors to improve the trend line?

47

What actions are we going to take?

Who is going to do what?

What is the timeline?

When will we collect the data to take stock of whether the actions are working?

7. What actions are we going to take?

48

Results

Indicators

Strategies

Performance Performance Measures forMeasures for Customers

Perfo

rman

ceThe Intersection of

Population & Performance AccountabilityP

op

ula

tio

n

49

The Contribution of Program Measures to Population Change

A Training Class

One Employment

Partner

New Haven Partners

Indicator: % of families employed

Result: New Haven families achieve financial stability and security.

Program Measure: # of re-entry residents employed

50

Taking Action to Improve the Trend Line

1. Identify your Performance Measure

2. Draw your trend line

3. List the factors that increase and

decrease the trend line

1. Use the priority factors to identify what actions to take to improve the trend line

2. Report out the actions to take

51

Group Report

Trend Line

Measure

Factors affecting the trend line --------------------------- ---------------------------

Partners --------------------------- ---------------------------

Three Actions – To Take --------------------------- --------------------------- ---------- No-cost / low-cost