Upload
brant
View
57
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Developing Performance Measures at UCEDD: Moving Forward with The Logic Model. Contact Information. Jon Desenberg Director Consulting Services, Performance Management The Performance Institute 1515 North Courthouse Road, Suite 600 Arlington, VA 22201 - PowerPoint PPT Presentation
Citation preview
Developing Performance Measures at UCEDD:
Moving Forward withThe Logic Model
Contact Information
Jon DesenbergDirector Consulting Services, Performance
Management
The Performance Institute 1515 North Courthouse Road, Suite 600
Arlington, VA 22201Phone: 703-894-0481 - Fax: 703-894-0482
www.performanceweb.org [email protected]
Measuring Performance for Results
If you don’t know where you are going, any road
will get you there. –Lewis Carroll
Why measure?
To Plan? To Comply?To Manage?To Optimize?To Innovate?
What gets measured gets done. –Peter Drucker
PERFORMANCE MEASURES ALLOW YOU TO:STRATEGIZECOMMUNICATEMOTIVATEMANAGE
Performance is not about mandates, it’s about management
Long-Term Move Towards Performance
Performance management is not a new phenomenon—50 years of work in the making to link resources with results
– Budget Accounting Procedures Act (BAPA) of 1950
– Planning-Programming-Budgeting System (PPBS), 1965-1971
– Management by Objectives (MBO) 1973-1974– Zero-Based Budgeting (ZBB), 1977-1981
Finally a Move in the Right Direction
• GPRA—(1993) Government Performance Results Act
• PMA—(2001) President’s Management Agenda
• PART—(2002) Program Assessment Rating Tool
• Mandates focusing on performance and accountability—bottom line results
Program Assessment Rating Tool (PART)
• OMB Tool to Evaluate Effectiveness of Program Performance in Budget Submissions for Individual Programs
• Designed to evaluate 20% of programs per fiscal year
• Methodological• Standardized• Evidence-based…and Transparent• Consistent with Government Performance Results
Act (GPRA), 1993
Four Sections of PART
• Program Relevance Mission and purpose of program
• Program PlanningFocus on program’s strategic objectives
• Program ManagementStewardship by front-line managers
• Program Results Program accountability to strategic objectives
PART Categories
• Competitive Grant Programs• Block/Formula Grant Programs• Regulatory-Based Programs • Capital Assets and Service Acquisition
Programs • Credit Programs• Research and Development Programs• Direct Federal Programs
FY06 Budget: Lessons Learned
Lessons Learned
• Root Cause of Failure: Poor Plans, Bad Measures, Weak Management– Lack of meaningful, results-oriented
performance measures– Failure to address management deficiencies
identified by GAO, IG’s, et al. – Ill-defined, conflicting, duplicative program
purpose
Lessons Learned
• Causes for Success– Good alignment towards Performance Plans– Effective management of strategic plans– Strong linkage between successful
implementation of strategic goals
Making the Grade:OMB’s weighting for each section
• Section One: 20%• Section Two: 10%• Section Three: 20%• Section Four: 50%
Section One: Relevance
• Clear agency mission• Unique contribution• Specific interest, problem or need• Optimally Designed to address specifics
Section Two: Planning
• Long-Term Performance Goals• Annual Performance Goals• Stakeholder Dialogue• Budget-Performance Integration
Section Three: Management
• Collection of timely, credible performance information
• Manager Accountability• Strict allocation of funds • Budget-Performance Integration
Section Four: Program Results
• Demonstrable progress towards goals• Achievement of annual goals• Improvement in efficiencies and cost
effectiveness • Favorable performance to comprable
programs
Performance Measurement Issues: Manageable by PART
• Outcomes are extremely difficult to measure• Are among many contributors to a desired
outcome• Have results that will not be achieved for many
years• Relate to deterrence or prevention of specific
behaviors • Have multiple purposes and funding that can be
used for a range of activities • Are administrative or process oriented
Performance Management in Non-profits translates a mission into reality and evaluates the results to all stakeholders
• Strategic Level– Measure Progress on Issues– Define & Validate Policy Strategies– Enhance Stakeholder Satisfaction
and Support
• Operational Level– Drive Change to Implement
Organizational Strategies– Ensure Compliance – Achieve Efficiencies– Improve Cycle Time
• Individual Level– Improved Morale/Retention– Achieve Clarity of Responsibilities
Transparency of Performance to:– Donors– Elected Leaders– Senior Management– Oversight Entities– Employees– Customers– Partners
Effective Performance Measures are SMART
S PECIFICM EASUREABLEA CCOUNTABLER ESULTS-ORIENTED (#1)T IME-BOUND
Applying SMART
End-Outcome: Reduce smoking-related deaths, illness and costsIntermediate Outcome: “Reduce the number of new youth smokers (10-18) by 2% each year”– Results-oriented: Youth smoking is where you can stop
the habit before it takes hold and has a lasting health impact
– Specific: “number of new youth smokers (10-18)”– Accountable: You have the ability to make it happen– Measurable: “reduce by 2%”– Time-Bound: “per year”
Selecting Performance Measures: The Doctor AnalogyOutcome Goal (End Outcome): Achieve and Maintain Proper Health and
Quality of Life– Measures: Dozens…Pulse, Years Lived;
Satisfaction Survey, etc.Objective Goals (Intermediate Outcomes):
-Achieve Appropriate Fitness for Age, Gender-Stop Pain in Y-Restore Function of X
– Measures: Thousands: Weight, Blood Analyses, Scans, etc.
Performance Measure Selection Criteria
Meaningful Reports tangible and significant accomplishmentsagainst objectives; useful for decision makers andother stakeholders
Clear Easily understand by managers, partners, otherstakeholders; tells clear story
Legitimate Accepted or legitimated by those who must usethe data
Consistent Clear definition and data collection methodologyacross populations
Reliable Captures what it purports to measure in anunbiased fashion
Granular Able to detect performance movement
Relevant Does not become obsolete too early: sets apattern or baseline of performance
(Step 4)
Performance Measure Selection Criteria
TechnicallyAdequate
Available data meets technical adequacy standards suchas accuracy, validity, timeliness
Responsible Does not have unintended and undesirableconsequences
Actionable Indicates what is good or bad, driving desired behaviorand the timing of action
Accountable Related to direct action or influence of an accountableand attributable entity
Balanced One of set of measures providing a clear picture of the fullrange of performance
Vital Measures are mutually exclusive and collectively exhaustive
Feasible Reasonable cost and accessibility of data that is notalready collected
(Step 4)
Performance measurement is a culture shiftFromFrom• ““These measures are These measures are
draining valuable draining valuable resources and are a data resources and are a data burden”burden”
• ““I can’t measure my I can’t measure my outcomes I can only outcomes I can only measure activities”measure activities”
• ““I need these measures I need these measures because my employees because my employees feel it is important”feel it is important”
• ““You can’t measure my You can’t measure my program.”program.”
ToTo • We are We are committedcommitted to to
tracking measures that tracking measures that matter most.matter most.
• We are We are accountableaccountable for for delivering our outputs and delivering our outputs and our intermediate outcomes.our intermediate outcomes.
• We are We are responsibleresponsible for our for our end outcomes.end outcomes.
Identifying Characteristics of Effective Performance Management Systems
8 Critical Success Factors for Effective Performance Management Systems
1. Defining and Aligning to Enterprise Strategy
2. Developing Meaningful Performance Measures
3. Increasing Data Availability
4. Maximizing Data Integrity
5. Enhancing Performance Reporting
6. Improving Evaluation and Analysis
7. Achieving Performance Integration
8. Driving Decision-Making
#1 Defining and Aligning to Enterprise Strategy
1.1 Has clearly defined its mission, vision and values
1.2 Has specific strategies in place to achieve organizational results (based on a SWOT or other strategic landscape analysis)
1.3 All structures (divisions, support functions) are fully aligned with enterprise-wide strategies
1.4 A formal strategic plan is clearly communicated to all employees at all levels of the organization
Survey Questions: 1, 2, 3, 4
#2 Developing Meaningful Performance Measures
2.1 Reliable measurement and reporting on Outcomes
2.2 Reliable measurement and reporting on Strategies
2.3 Organizational process metrics (Quality, Cycle Time, Efficiency)
2.4 Goals and measures enjoy support and buy-in from internal and external stakeholders
Survey Questions: 5, 6, 7, 8Module Two: Characteristics of Performance Management System
#3 Increasing Data Availability
3.1 Data sources are identified and readily accessible
3.2 Data burden is worth the information gleaned
Survey Questions: 9, 10Module Two: Characteristics of Performance Management System
#4 Maximizing Data Integrity
4.1 Data is collected, managed, and analyzed in a uniform and consistent manner
4.2 Data is validated or verified through sampling or independent means
PI Management Survey Question: 11, 12Module Two: Characteristics of Performance Management System
#5 Enhancing Performance Reporting
5.1 Internal reporting produces information for frontline managers and senior decision-makers on a “real time” basis.
5.2 Has a reporting system that produces comprehensive performance reports that include measures, analysis, trends, suggestions for improvement
PI Management Survey Questions: 13, 14
#6 Improving Evaluation and Analysis
6.1 For process measures, benchmarks and service levels are evaluated (1-2 year cycles)
6.2 For outcome and strategy measures, program performance is evaluated for “cause-effect” (2-5 year cycles)
PI Management Survey Questions: 15, 16
#7 Achieving Performance Integration
7.1 INTERNAL Integration: Support services’ contributions (HR, IT, Finance, etc.) to program performance is documented and managed
7.2 EXTERNAL Integration: Performance contributions of multiple contributors in samemeasurement area are tracked and compared
PI Management Survey Questions: 17
#8 Driving Decision-Making
8.1 Budgets and investments are made based on clear contributions to performance
8.2 Supply chain partners are held accountable for products and services
8.3 Employee bonuses and pay increases are linked to individual performance evaluations.
PI Management Survey Questions: 18, 19, 20Module Two: Characteristics of Performance Management System
Top Five/Bottom Five by “EXECUTION”
1. Publishing a strategic plan (1.4)
2. Has specific strategies in place to achieve organizational results (1.2)
3. Organizational process metrics (2.3)
4. Evaluation of process measures, benchmarks (6.1)
5. Budgets & investments are made based on contributions to performance (8.1)
1. All structures are fully aligned with enterprise-wide strategies (1.3)
2. Reliable measurement and reporting on strategies (2.2)
3. Internal integration of support service alignment to performance (7.1)
4. Data is collected, managed, and analyzed in a uniform manner (4.1)
5. Goals and measures enjoy support from internal/external stakeholders (2.4)
TOP FIVE INITIATIVES BOTTOM FIVE INITIATIVES
Module Two: Characteristics of Performance Management System
Top Five/Bottom Five by “IMPACT”
1. All structures are fully aligned with enterprise-wide strategies (1.3)
2. Reliable measurement and reporting on strategies (2.2)
3. Employee bonuses and pay increases are linked to individual performance (8.3)
4. Budgets & investments are made based on contributions to performance (8.1)
5. Internal reporting produces real-time data for decision-making (5.1)
1. Has clearly defined its mission, vision and values (1.1)
2. Data burden is worth the information gleaned (3.2)
3. Publishing a Strategic Plan (1.4)4. Comprehensive performance
reports (5.1)5. Data is validated through sampling
or independent means (4.2)
TOP FIVE INITIATIVES BOTTOM FIVE INITIATIVES
Module ThreeUnderstanding Logic Models
What is a logic model?1. Logical chain of events providing blueprint for mission
achievement2. Graphic representation that illustrates the rationale
behind a program or organization3. Depicts causal relationships between activities,
strategies, and end results4. Contains goals and performance measures 5. Integrates various program activities into a cohesive
whole6. Vehicle for dialogue, planning, program management
and evaluation
What does a logic model look like? • Graphic display of boxes and arrows;
vertical or horizontal– Relationships, linkages
• Any shape– Circular, dynamic– Cultural adaptations, storyboards
• Level of detail– Simple– Complex
• Multiple models
Logic modeling is based on mapping and defining linkages between what we do and why we do it .
I Work Out for One Hour
Each Day
I Will Burn More
Calories Than I
Consume
Lose Fat and Build
Muscle
Improve My Looks and Health
Have Better Image,
Feel Better & Live Longer
IF THEN IF THEN IF THEN IF THEN
INPUTS OUTPUTS OUTCOMES
Series of If-Then Relationships
Assumptions: improving looks = better self image Factors: Health History
InputsPeople and resources required to achieve outcomes
Activities/OutputsWhat the inputs produce
Immediate and Intermediate OutcomesChanges required to achieve end outcome
End Outcome
End goal or ultimate benefit
Clarifying the terms
Assumptions: beliefs or evidence that supports your IF-THEN logic
Factors: external influences beyond control that effect IF-THEN relationships
ToothpasteFlossTooth brush
Brush twice a day
Floss once a day
Remove plaque (decrease plaque in my mouth)
Have fewer
(ideally zero)
cavities
IF THEN IF THEN IF THEN
INPUTS OUTPUTS OUTCOMES
What is your goal at your annual dental check-up?
Assumptions: Plaque causes tooth decay Factors: Genetics
Logic Model “V”
Bottom-Line Investment
Top-Line Return
Alignment
Linkage
Measu
remen
t
Logic Model “V” Performance Dimensions
Distribute program grants
Child violence and abuse can be prevented and
detected
Child health and development can be
protected and maintained
Children can grow into productive
citizens and attain their intended
impacts on society
“So That”
“So That”
“So That”
Degree of Influence by Department
Ultimate Program IntentOutput
Intermediate Outcome
Intermediate Outcome
High-Level Outcome
Value Chain Diagram
Early Outcomes Later OutcomesLater ActivitiesEarly ActivitiesIf we do…
Outreach
Screening
ID of elevated kids
And we do…
Case mgmt of EBLL kids
Refer EBLL kids for medical treatment
Train family in in-home detection techniques
Assess environment of EBLL child
Refer environment for clean-up
Then….
EBLL kids get medical treatment
Family performs in-home techniques
Lead source identified
Environment gets cleaned up
Lead source removed
And then…
EBLL reduced
Developmental slide stopped
Quality of life improves
Global Logic Model: Childhood Lead Poisoning Program
Definition:
EBLL – Elevated Blood Lead
Levels Module Three: Using Logic Models
Inputs Activities Outputs End Outcomes
Most logic models incorporate the following elements.
Intermediate Outcomes
Attitudes Behaviors Conditions
WHY?
HOW
EFFECTCONTROL
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
• Resources used to support activities • Not performance measures• System integration: Link inputs through
process to goals and outcomes in:–Budget requests–Human resource plans–Information technology plans
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
• Processes and roles• What the program does• Subject of on-going process
improvement and strategy change• System integration through linkages to:
–Stakeholder satisfaction–Assessment quality–Desired outcomes–Efficiency measures
Drivers Inputs Activities Outputs Intermediate Outcomes
End Outcomes
• Products and services delivered (e.g. grants, audits, research studies, impact assessments, etc.)• Indicate strategy deployment• Foundation step for attainment of all types of outcomes• Often measured by low-level outcome types:
-Process vital signs-Customer satisfaction
• Good source of short-term, readily available results• 12 months-1 year
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
• Show cross-agency/program accountabilities• Can be one or several in number• Often measured by attitudes, behaviors and
conditions• Can show short- to medium-term change • 1-5 years
Connecting strategies, intermediate outcomes and measures
Module Four: Identifying outcomes and measures
U.S. Department of Labor Women’s Bureau
Strategy: Provide training in high-growth, demand-driven occupations to women
Intermediate Outcome: Increase hard skills in high-growth, demand-driven occupations for participants
Intermediate Outcome Performance Measure: % of women participants who successfully complete training, education or certification for high-growth, demand-driven occupations
*example only
Intermediate Outcomes
Drivers Inputs Activities Outputs Intermediate Outcomes
End Outcomes
• Shows ultimate benefit to tax payer• Often measured by long-term indicators
–Changes in economic, policy conditions • Captured in the strategic plan as end
goals• 5-10 years
Connecting mission, goals, end outcomes and measures
U.S. Department of Labor Women’s Bureau
Mission: Improve the status of wage earning women, improve their working conditions, increase their efficiency, and advance their opportunities for profitable employment.
Goal: Improve status of working women through better jobs.
End Outcome: Increase women’s employment in high-growth, demand-driven occupations.
End Outcome Performance Measure: % of women participants who find employment in high-growth, demand-driven occupation
End Outcomes
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
• Efficiency Measures• Ratio of outputs to inputs• Usually unit costs or service units per
FTE• Examples:
–Cost per assessment–Average value of grants–Investigations completed per FTE
Appreciating outcomes vs. outputsOutputs Outcomes
Number of housing units rehabilitated
Increases in equity (property value) of rehabilitated houses for low-income families as a results of targeted assistance
Number of businesses assisted through loans and training
Percent of businesses that remain viable 3 years after assistance
Number of people served by water/sewer projects
Increased percent of people with access to clean drinking water
Number of acres of agricultural lands with conservation plans
Percent improvement in soil quality; dollars saved in flood mitigation
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
Overall
Relating Logic Models to Strategic Plan Elements
Programs
Projects, tasks or initiatives designed to
contribute to end outcomes
Outcome Goals
Developed from statute based
mission$/FTE
Internal/External Environment
Statutory Authority
Based on strategic assessment
Mission
Values
Performance
Measures
Strategies
Changes in attitudes, behaviors
or conditions required to achieve
outcome goals
Drivers Inputs Activities Outputs Intermediate Outcomes
EndOutcomes
Control and Influence
Typical Program
Focus
Typical Agency Focus
Suggested Agency Focus
End Outcomes: Where is UCEDD?
• “More individuals…are independent and self-sufficient…participate in and contribute to the life of their communities.”
• “Ensure better access to Services”• # of individuals receiving direct services.• A leadership position in the developmental
disabilities.
Logic Models and Performance SystemsBeginnings
If your assumptions about the factors that influence
your issues hold true…
Intended Results
Should contribute to the results you expect based on this theory of change
Planned Work
Then, the activities you plan to do which build on these
assumptions…
Performance Reports
Strategic Plan
Management Plan
Planning & Design
Evaluation & Communication
Implementation
what we hope to dowhat we’ve done
how we will do what we said
Logic Model
Adapted from W.K. Kellogg Foundation, Logic Model Development Template
The Performance Logic Model
MEASURES
Introducing The Performance Logic Model
$FTE
$FTE
$FTE
What are the results of What are the results of specific strategies that will specific strategies that will contribute to achieving end contribute to achieving end outcomes? What changes in outcomes? What changes in attitudes, behaviors and attitudes, behaviors and conditions are required?conditions are required?
What activities can What activities can be performed and be performed and what products and what products and services can be services can be delivered to achieve delivered to achieve the outcomes? the outcomes?
What are the What are the ultimate benefits ultimate benefits to the public?to the public?
Performance Measures
Input Activities and Outputs Intermediate Outcomes End Outcomes
Input Activities and Outputs Intermediate Outcomes End Outcomes
What does The
Performance Logic Model do?
MANAGE
PLAN
MEASURES
COMMUNICATE
EVALUATE
$FTE
$FTE
$FTE
GoalsOutcomesOutputsActivities
GoalsOutcomesOutputsActivities
Prioritize Align
Measure Manage
Link
Metrics
Gap
Management Senior Leadership
Comment: This does not represent a physical breakout of the Logic Model, but rather an illustration to distinguish the differing level of key stakeholders (e.g., agency-
program, executive-manager, congress-agency, etc).
Attitudes
Behaviors
Conditions
Performance Logic Model Framework captures senior leadership and operational manager outcomes
Why use The Performance Logic Model?
• Brings detail to broad goals; helps in planning, evaluation, management, and communications.
• Builds understanding and promotes consensus about what the organization does and how it will work--builds buy-in and teamwork.
• Helps to clarify what is appropriate to evaluate, and when, so that evaluation resources are used wisely.
• Summarizes complex programs to communicate with stakeholders, funders, audiences.
Performance Logic ModelPerformance Logic Model
Input Activities and Outputs Intermediate Outcomes End Outcomes
Annual Performance Plan
Manager and Employee Performance Plans
$FTE
$FTE
$FTE
Activity-Based Costing/Performance BudgetingInformation Technology/E-Government Plan
Input Activities and Outputs Intermediate Outcomes End Outcomes
Human Capital Plan
Improved Financial Management
Competitive Sourcing/Contracting
Accountability and Performance Report
Strategic Plan
UCEED Intermediate Outcomes and Measures –
Focusing on the ABC’s of Change; Not End State
If your organization were successful, what would you SEE, HEAR, FEEL and DO?
More See Hear Feel Do
Less
Identifying outcomesEnd outcomes are grounded in mission and statute, assess progress toward strategic goals
Intermediate outcomes evaluate progress toward end outcomes, assess impact of strategies; measures changes in attitudes, behaviors or conditions required to achieve end outcomes
When goals and strategies are results-based, outcomes leap off the page.
Every outcome has a measureMeasures are the indicators of results. Good measures align activities and resources to achieve outcomes. Measures communicate if or to what extent activities have delivered the desired outcomes.
Outcome: Increase basic literacy skills in youth ages 14-17 who are basic skills deficient
Measure: % of youth enrolled in basic literacy skills training programs who increase basic skills by 1 educational functioning level
Performance Measures Definitions
• Performance Measures: Indicators, stats, metrics used to gauge program performance
• Target: Quantifiable characteristic that communicates to what extent a program must accomplish a performance measure
• Outcome Measures: Intended result of carrying out a program. Define an event or condition external that is a direct impact the public.
• Output Measures: Describes the level of activity that will be provided over a period of time
• Efficiency Measures: Measures that capture the skillfulness in executing programs, implementing activities, and achieving results
Current state of measurement• Too Many Measures• Wrong Kinds of Measures
– Too process and activity oriented– No clearly defined “Logic Model”– No measures of strategy– Few measures of end outcome
• Dumbing-down of Measures– Measuring only the things you can count rather
than things that are strategically important
Not everything that can be counted counts and not everything that counts can be counted. - Albert Einstein
Recommendations for Measures• Balance across three arms of performance
– Efficiency (e.g. reduced cost, reduced cycle time)– Effectiveness (e.g. improved customer access, awareness or satisfaction)– Quality (e.g. reduced error rate, increased compliance)
• Select measures for each outcome, output and activity (metrics)• Validate measures
– Do they pass the gut check – are they good measures?– Can you identify the data needed to calculate the measure?– Is the data readily available?
• Evolve measures over time as outputs change or as efficacy of measures decreases
Checking measures Is the measure results-oriented? Does it indicate achievement of
outcomes? Is this measure specific and meaningful? Can this be measured? (Consider how?) Will someone be held accountable for the
measure? Is the measure time bound? Does it indicate when the results will be
realized?
Identifying intermediate outcomes
Intermediate outcomes measure the results of strategies deployed to achieve the end outcome. Intermediate outcomes target the “center of gravity” of a particular problem to cause a change in the direction of the end outcome. Often problems addressed by the government and social services are the result of specific attitudes, behaviors, or conditions. Strategies and their intended outcomes (intermediate outcomes) target these attitudes, behaviors and conditions.
Attitudes Behaviors Conditions
Intermediate outcomes target the changes in attitudes, behaviors or conditions that are required to achieve end outcomesReducing teen smoking Attitudes: Alter the belief that “smoking is cool”
Behaviors: Decrease number of “new” smokers ages 12-15
Conditions: Reduce the amount of cigarettes sold to underage smokers
Identifying intermediate outcomes through Center of Gravity Analysis
Center of Gravity Analysis
1. What attitude, behavior or condition needs to change to achieve the end outcomes? (Target)
2. Identify who possesses the critical capability to cause the change or achieve the end outcomes. What must they do? (Who & What)
3. How can you get them to do that? (How?)
Module Six:7 Steps to Building a Performance Logic Model: Welfare to Work
Building a Performance Logic Model
Step 1: Identify end outcomes and their measures grounded in mission and values
Step 2: Identify intermediate outcomes and their measures informed by assumptions, factors
Step 3: Identify activities and outputs and their metrics required to achieve outcomes
Step 6: Allocate resources required to achieve outcomes
Step 4: Narrow cast and choose measures for management
Step 5: Set targets for chosen measures
Step 7: Clearly define, collect, analyze and report on measures
Step 1: Identify end outcome measures grounded in mission and customer values
1. What is the bottom line of your program? 2. If you had to defend your program’s value/benefit
before a grand jury, what 2-3 pieces of evidence would prove you were a success rather than a failure?
3. What is the end benefit to the taxpayer or society from your program?
4. How will you know you have been so successful that you can shut your program down?
5. How will you measure this outcome?
Clarifying the Logic of the Program (Step 1)
Logic Model TemplateThe mission of the
(Program)
Is to produce/provide (Products or Services)
To (Target of Change)
So that they can (Intermediate Outcome Change)
Resulting ultimately in… (End Outcome Goal)
Step 2: Identify intermediate outcome measures informed by assumptions, factors
Given the end outcomes you seek…1. What attitudes, behaviors or conditions must change
to achieve your end outcomes?2. What must increase, decrease or stay the same to
achieve your end outcomes?3. What must change in the status quo to create the
conditions necessary to achieve your end outcome? 4. How will you measure this outcome?
Step 3a: Identify activities, outputs and metrics required to achieve outcomes .
For each intermediate outcome… What specific things can this agency do to cause change
to happen? What specific things to influence that target of change? What products could you produce? What services could you provide? What is the actual workload that is to be handled?
(Note: Don’t include administrative items inside your program. Think of what things actually leave the four walls of your program.)
Separating Activities from Outputs (Step 3a)
Activity Definition TemplateThe purpose of
(Specific Program Work Activity)
Is to produce/provide(Output)
To (Target of Change)
So that they can (Intermediate Outcome Change)
Step 3b: Develop metrics for activities/ outputs
What metrics can and should be tracked to assess progress ofactivities conducted?
Metrics: assess progress of activities conducted, may be quantitative or qualitative; numerical value of outputs; also known as operational measures, efficiency measures, workload measures, productivity measures;
Examples: number of contracts processed, percent of contracts processed right the first time, unit cost per contract,
Module 6: Building a Performance Logic Model
Identifying metrics for activities/outputs (Step 3b)
Metrics: assess progress of activities conducted, may be quantitative or qualitative; also know as operational measures, efficiency measures, workload measures, productivity measures
Examples: number of contracts processed, percent of contracts processed right the first time, unit cost per contract
Checking metrics (Step 3b)
Do metrics reflect that which is most important to the customer?
Will these metrics track progress and activity completion?
Can this data be collected and analyzed?Is the data collection burden worth the
result?
Exit Opportunity
Proven Success
ResourcesAvailable
IMPORTANCE
PERFORMANCE
4
3
2
1
1 2 3 4
a
Factors
a = I4, P2
b = I3, P3
c = I2, P1
d = I1, P4
b
c
d
Attention Needed
Selecting Your MeasuresThe Program Performance Assessment Window™
Developing performance targetsTargets• numerical value of the performance measures• establish desired results within a specific timeframe
measures degrees of progress toward outcomes• established from baseline data--targets should NOT be
established without base line data.
Example: 55% of women participants will successfully complete training, education or certification for
high-growth, demand-driven occupations
(Step 5)
Agency Performance Measures
Department Performance Measures
ProgramPerformance Measures
Individuals Performance Measures
Crafting a System of Performance Measures
Cascading The Performance Logic Model
It is possible to
…use one model for all required measures from all funding sources…drill-down and roll-up logic models…logic model across functions and operations…capture both internal and external stakeholders
Benefits of The Performance Logic Model
Program Integration: Tracks and coordinates the contributions of multiple programs, bureaus, agencies, levels of government, and sectors of society (non-profits, for-profits, etc.)
Systems Integration: Allows for the comprehensive integration of different elements of the management agenda, such as IT, acquisition, human resources, etc.
Accountability: Links program goals to individual achievement and vendor/contractor performance
2005 UCEDD Logic ModelEnd Outcome – • More Individuals with developmental
disabilities are independent and self-sufficient. More individuals with developmental disabilities participate in and contribute to the life of their communities
Is this the vision? The Ultimate Goal? Can we measure it?
2005 UCEDD Logic ModelIntermediate Outcomes – • More models are field tested and expert knowledge relevant to the DD
field is gained and disseminated• More students & leaders are trained and remain in the field of DD.• More individuals with DD receive services from trained individuals.• More communities and policymakers are knowledgeable about DD issues.• More individuals with DD receive high quality services from trained
individuals and through improved access and expanded capacity
Are these outcomes, outputs or a little of both?
2005 UCEDD Logic ModelOutputs – • # of trainees who gained knowledge & skills• # of individuals in the community who gained• # who received services and support• # of research and evaluation activities conducted• # who remain in the field• # of products developed & disseminated• # of recipients of products disseminated
Do these overlap or align with the intermediate outcomes?