Final Presentations Wednesday April 27 th Anthony A – Peabody Jon – Dracut Kirsten – Hartford Sehrish – Manchester, NH Jeff – Beverly Lianna – Pittsfield

Final Presentations

Wednesday April 27th

• Anthony A – Peabody• Jon – Dracut• Kirsten – Hartford • Sehrish – Manchester, NH• Jeff – Beverly• Lianna – Pittsfield • Judy – Middlesex County

Wednesday May 4th

• Nina – Danvers • Mike – Lowell Career

Center • Keith – Cambridge • Anthony S – Haverhill • Amanda – Peabody Schools• Dustin – ?

Final Exam

Wednesday May 11th

At 6:00 pm

I email you the questions

Select 3/5 or 4/6

Before 9:00 pm

You email me your answers

Performance Measurement to

Performance Budgetingto

Performance Management

57.508-201

The Budget as a Policy, Planning and Information ToolMeeting 12 – April 20, 2011

Performance Measurement

A process through which an organization assesses how well its programs and services are achieving their desired results

The process must include both the organizational performance data and a benchmark that creates framework for analyzing that data

Organizational Performance

The quantity of work of an organization’s assets (employees and equipment) in converting inputs into

outputs and outcomes

and

The quality of the resulting outputs and outcomes

Why Performance Measurement?

Q: How many of the 8 reason can you name?


1. To Budget & adequately and appropriately allocate resources

2. To Evaluate how well is public agency performing.

3. To Control to ensure employees are doing the right thing

4. To Motivate and provide periodic sense of accomplishment

5. To Celebrate accomplishments

6. To Promote the organization and its employees

7. To Learn reasons behind both good & bad performance

8. To Improve and develop processes and approaches


If you don’t measure …

• How do you know what or where to improve?

• How do you know where to allocate money and people?

• How do you know how you compare with others?

• How do you know whether you are improving or declining?

• How do you know whether or which programs, methods, or employees are producing results supporting your strategic action plan?

Why Not Just Use…

• Managers’ Intuition?• Hunches?• Rumor?• Tradition?• Habits?• Rules of Thumb?• Guesses?• Anecdotes?

Relation to Strategic Management



Measurement Categories

1. Effectiveness

2. Efficiency

3. Quality

4. Timeliness

5. Productivity

• Safety, Access, Equity, Impact…

(TBD by the Organization)

Workload Measures

Provide information on how many units of service were provided… how many calls were answered, how many work orders were processed, how many buildings were maintained, how many training programs clients were served, etc.

But they tell us nothing about the efficiency with which these services were provided and nothing about the quality or effectiveness of the services

They give us some indication of the demand for a given service, but they offer little else of managerial value

Efficiency MeasuresThey relate outputs produced to resources consumed… the acres of lawn mowed per staff-hour have combined workload information (surface area mowed) with resource consumption information (cost of staff-hours) to create one type of measure

Other measures can focus instead on turnaround time or other aspects of process efficiency

The result in either case is a measure that reflects the stewardship of public resources to achieve a public purpose and provides important operational feedback that challenges managers to consider whether their operation’s level of efficiency is in line with leading organizations

Effectiveness Measures

They gauge the extent to which objectives are being met… promptness in responding to routine needs and emergencies, work completion rates, and rates of customer satisfaction

This category includes measures of service quality

Like efficiency measures, effectiveness measures provide feedback that challenges managers to assess the adequacy of their services is in line with leading organizations


“Leading and managing local governments requires understanding the performance of a jurisdiction across a range of criteria. Performance data helps in establishing goals and outcomes in budget decisions, setting targets of performance, comparing to and learning from other jurisdictions, and communicating service and financial improvements to citizens.”

Bernie McMullen, City Manager


“Tight budgets and demanding citizens put governments under increasing pressure to show that they are providing good value for money. Providing information about public agency performance can satisfy the public’s need to know, and is a very useful tool for public managers to evaluate their performance.”

OECD Policy Brief

From ICMA “Performance Matters”

Q: Is it worth so much focus on data reporting in this economy?

A: “With tax collections strained and furloughs common, some jurisdictions might be tempted to do less in the way of performance measurement and reporting. However, without the data gathered from performance measures – particularly those tied to strategic goals – it can be difficult to judge whether budgetary decisions are saving a few bucks or cutting into the desired community outcomes.”

Measurement Data Issues• Collecting not enough data

• Collecting too much data

• Collecting the wrong data

• Heavy reliance on summary data

• Emphasizing averages and discounts outliers

• Heavy reliance on historical patterns

• Heavy reliance on gross aggregates

• Heavy reliance on static (snapshot) analysis

• Little attention to time-based or value-added measures

Measurement Data Issues

“Most of us get into public service to make a difference, not just to work. We will never know if we are accomplishing something worthwhile if we don’t measure the results. Performance measurement without guidance may simply create more work, not better results.”

Dave Krings, former ICMA president

Measure All Meaningful Work • Desired performance outcomes must be established for all

meaningful work• Work that is not measured is deemed worthless regardless of its

outcomes• Work that is not measured cannot be managed because there is

no objective information to determine its value • Outcomes provide the basis for establishing accountability for

results rather than just requiring a minimum level of effort• Desired outcomes are necessary for proper work evaluation and

legitimate performance appraisal• Frequent reporting enables timely corrective action which is

needed for effective management and organizational success

Useless Performance Measures

• Does not provide good information – The measure is not valid, reliable, responsive... exactly what is being measured and how the results should be interpreted are not clear

• Does not illustrate progress toward achieving goals and objectives – The measure is not responsive and/or functional

• Does not have ownership – The measure is not functional, credible, and/or understandable and no one is responsible for this performance

• Will never have data available – The measure is not available… it would be either nearly impossible or cost-prohibitive to collect the necessary data

Attributes of Best Measurement Data

• A way to measure exists

• May be interpreted easily and uniformly

• Provide an agreed upon basis for decision making

• Are understandable by everyone

• Are precise in interpreting the results

• Apply broadly and comparably

• Are economical to apply

• Reflects the customer’s and organization’s needs

Different Levels of Information Needs

Public

Elected Officials

Chief Executive Officers

Department Heads

Program Managers and Supervisors

Other Employees

Peoria Arizona

• Performance information in annual budget document

• Use the data to inform budget decisions

• Departments supply information with budget requests

• Requests justified with performance measurement data

• Elected officials and citizens responded positively

• Budget process runs more smoothly

Getting Outcome Information

Step One: Prepare Your Organization

• Assess Organizational Readiness• Make a Commitment to Quality• Communicate a Mandate to Improve• Form a Benchmarking Team• Assemble Benchmarking Resources• Write an Distribute a Benchmarking Plan• Address Staff Resistance


Step Two: Determine What to Improve

• Analyze What May Need Improvement

• Identify Your Critical-to-Quality Outcomes

• Define and Assign Performance Measures


Step Three: Measure Performance

• Be Sure to Measure the Right Things

• Collect Internal Data to Set a Baseline

• Set Internal Benchmarks for Your Measures


Step Four: Learn What Works

• Select Benchmarking Partners

• Collect Relevant Data

• Analyze Performance Gaps

• Identify Best Practices

Using Outcome Information

Step Five: Implement Best Practices

• Create an Implementation Plan

• Create a Staff Communication Plan

• Implement the Best Practice

• Link Data to Annual Budget Process

What Kinds of Data?

• Program - Centered Outcomes

• Participant - Centered Outcomes

• Community - Centered Outcomes

• Organization - Centered Outcomes

Program - Centered Outcomes

• Reach– Percent of target constituency enrolled– Number of service requests per month/quarter– Percent constituents renewing

• Participation– Participation rates– Percent of capacity enrolled/registered– Average length of time in program

• Satisfaction– Number of favorable reviews/awards– Constituent satisfaction rate

Participant - Centered Outcomes

• Knowledge/Learning/Attitude– Percent increase in scores after program– Percent graduation to next level

• Behavior– Incidence rate– Relapse/recidivism rate– Percent that do not reenter the program/system

• Condition/Status– Percent who establish career/employment– Percent with reduced incidence of problem

Community - Centered Outcomes

• Policy– Percent of target constituents aware of issue– Number of people reached through communications

• Public Health/Safety– Percent of public taking precautions

• Civic Participation– Percent increase in turnout– Number of people volunteering

• Economic– Increased employment opportunities

Organization - Centered Outcomes

• Financial– Cost savings

• Management– Procedures and practices

• Personnel– Fewer disciplinary issues

• Governance– Organizational structure– By-laws

Outcome Definitions

• Outputs = the quantity of work activity completed. Outputs are expected to lead to desired outcomes, but by themselves do not tell anything about the outcomes.

• Intermediate Outcomes = Events or results that are expected to lead to the end outcomes, but are not themselves a measure of result. Also may include characteristics relating to the quality of the service provided to clients.

• Long Term Outcomes = The consequences/results of what the program did. These are likely to be aspects of the client’s condition or behavior that the program seeks to affect.

Who Should Collect the Data?

• Direct service providers should not bear the entire burden– Examine / endorse the relevant outcome data – Should be a major information source– Critical for explanations of key findings– Critical for help in interpreting the data

• Program managers are the most important in the process

• Managers responsible for examining and interpreting data

• If resources are available, help can be assigned part-time

• Use volunteers, university researchers, consultants

How to Collect the DataBasic data sources include:

• Organization records

• Records from other similar organizations

• Records from state or federal agencies

• Client / customer surveys

• Ratings by trained observers

• Tests of clients (knowledge, skills, etc.)

• Observations using equipment (tests for quality of water or air)

Survey Data

• Perception of the degree of improvement via program

• Ratings of service timeliness, effectiveness, quality

• Accessibility of staff and service facility

• Condition and safety of facilities

• Ratings of staff competence and courtesy

• Overall satisfaction with the services provided

• Reasons for any poor ratings given

• Suggestions for improvements

Survey Process

• Identify the specific information needed

• Develop the questionnaire

• Ask only for information related to program outcomes

• Determine how the survey will be administered

– Mail (Addresses? Literacy? Language issues?)– Telephone interviews – In-person interviews at the program’s offices– Some combination

• Determine when the survey will be admin

Comparing Outcome Data

Comparisons Across Time the Most Common

• Most recent year compared with the previous year

• Last month /quarter compared with the previous month/quarter

• Last month /quarter compared with the same in previous year

• Performance thus far this year compared with last year

• Performance over some period of time in the previous year

Establishing Targets

Is program meeting, exceeding or falling behind?

• Targets should not be set until at least one round of data collected

– Baseline data is basis for establishing realistic targets

– Actual outcome data first compared to baseline data

– Next the desired outcome targets are set

– Then outcome data compared against the targets

• Programs should have monthly /quarterly / annual targets

Benchmarks

Unless a program is unique, another agency is doing it

• Similar local organization• Federal or state agency • National associations• Performance indicator rates• National standards

Linking Data to Annual Budget

• Identify specific strategic organizational / program objectives

• Identify key outcome indicators to track progress

• Establish baseline values for each outcome indicator

• Set target values for each outcome indicator

• Work with staff to develop strategies for hitting targets

• Estimate the outcomes, cost and feasibility for each

• Select the most desirable strategies / action steps

• Require program managers to demonstrate how budget requests relate to hitting the selected targets

Positive not Punitive

“Performance measurement systems should be positive, not punitive. The most successful performance measurement systems are not ‘gotcha’ systems, but learning systems that help the organization identify what works.”

National Partnership for Reinventing Government

Three Case Studies

CASE #1

THE PUBLIC LIBRARY&

You Are the Library Director

Issue

• Each weekday at 3:00 pm, large numbers of school children fill up the local public library

• They occupy all the reading room chairs and tables, behave noisily, leave piles of books and litter scattered about, and trash the restrooms

• By 6:00 pm, the library is quiet again

Cause

When the children were questioned, it was learned that most were latchkey public school children with no one at home and no where else to go

The public library was being used as a de facto daycare center

What’s a Library Director to Do?

• Call all the parents?• Turn the kids away?• Complain to the school department?• Write to the newspaper about library use?• Establish rules prohibiting children?• Call a meeting of the library trustees?• Ask the town to fund more staff?

Likelihood of Success

What’s the library’s mission?

What the objective here?

• Is the action step doable?• Is the action step practical?• Is it within the organization’s principles ?• Is the action step likely to lead to success?

Thinking Creatively

• Get fully into the daycare business • Maybe a fee-for-service to supervise kids• Estimate direct costs of added staff• Include depreciation of building and equipment• Maybe a building addition to create more space• Mobilize seniors or volunteer parents to help

Policy Question

Is caring for these children within the mission of the town library?

Yes_____ No_____

Thinking Strategically

• Reschedule existing staff to cover 3:00-6:00 peak• Rearrange furniture & stacks to set up kids area• Establish study groups and reading hours• Invite in jugglers, puppeteers, animal trainers• Show movies and videos• Hold reading enrichment programs• Maybe start life-long love for books and libraries

Strategic Management

• Is a library just a place where books are kept and people come to read quietly?

• Or is a library a public place where seniors can gather, students can study, people can use computers, do-it-yourselfers can learn new skills, patrons can listen to music, etc.?


• The environment changed externally• The environment changed internally• Client needs have changed• There are no new resources to address change• A quality public library responds to the changes• The library’s services are more responsive• The library’s constituent response is positive

CASE #2

THE NEIGHBORHOODREDEVELOPMENT AGENCY

&

You Are the CDC Director

Issue• In a previously low to moderate income

neighborhood of mixed uses, small apartment buildings & duplexes, and locally owned shops & restaurants, now large commercial projects and high-rise condo and luxury apartments were starting to be built

• Rents and real estate prices were increasing

• Opportunities for 1st time home and business ownership were quickly being eliminated

Causes

The neighborhood’s access to the city’s commercial center had been improved by a new parkway and highway interchange

Real estate costs in the rest of the city had been increasing steadily

Once overlooked, the neighborhood had become an attractive location for new investment

Strategic Questions

• Is a neighborhood redevelopment agency simply a vehicle for new development and investment?

• Or is a redevelopment agency a vehicle for positive change in a neighborhood that has benefit for the neighbors, the city as a whole, the tax base and the private investors?

Policy Question

Isn’t it the mission of the redevelopment agency to encourage new development and new private sector investment?

Yes_____ No_____

Policy Question

Is it the only mission of the redevelopment agency to encourage new development and new private sector investment?

Yes_____ No_____

What’s a CDC Director to Do?

• Seek credit in the newspaper for the changes• Work with the new development interests to

completely alter the character of the area• Publicly condemn the private investment• Organize a neighborhood march on city hall• Strap yourself to the wrecking ball• Set fire to every new building at completion


What’s the CDC’s mission?



Thinking Strategically

• Use some of the new resources to hire more planners• Organize the neighborhood to participate in plans• Work affordable units into neighborhood projects• Insist that relocation plans be part of all new projects• Work with city hall to ensure representation• Use technical assistance monies for local training• Increase influence of local residents in all plans

CASE #3

THE CITY POLICE DEPARTMENT&

You Are the Chief

Issue

• In a large city of rapidly changing demographics with a growing minority, immigrant and non-English speaking population, crime is on the rise and respect and confidence in the police is down

• Command officers, some having served the city for over thirty years, are entrenched in their ways

• Police union is at odds with the city over lack of new contract so morale on the street is very low

Cause

Like many American cities, populations have become needier, lest trusting of government, more diverse and more violent

Changes have not been made over time to build a more diverse and representative police force

The old ways of policing focused on arrests and ignoring people’s needs wasn’t working

What’s a Chief to Do?

• Ask the mayor for more money for more cops

• Blitz the city with strict law enforcement

• Install curfews and limit hours at bars, etc.

• Redirect resources away from nuisance calls

• Focus on the highest crime neighborhoods

• Find the nearest doughnut shop


What’s the mission of the police department?



Policy Question

Is the mission of the PD simply arresting perpetrators and quelling public disturbances?

Or are they a public service?

If so, what service are they in?

Universal Question

What Business Are We In?

• Community organizing?• Neighborhood information exchange?• Fear (of crime and of police) reduction?• Victim assistance and follow-up?• Citizen (church, school, etc.) partnerships?• Crime prevention analysis?

Strategic Vision

• We cannot carry out our mission alone• We have to react to behavior emphasizing prevention• We have to behave appropriately ourselves • We have to reinforce the strengths of existing institutions• We have to encourage public input into our policies• We have to adopt formal cooperation strategies• We have to train and equip our officers for today’s job• We have to decentralize and be more visible


• The environment had changed dramatically• Client needs had changed• The police needed to respond to the changes• The chief could have resisted change• An effective chief seeks a win-win-win• The chief was aware that the PD’s constituents

included (1) the neighborhood citizens, (2) the city as a whole, and (3) his rank-and-file officers

Documents

Final Presentations Wednesday April 27 th Anthony A – Peabody Jon – Dracut Kirsten – Hartford Sehrish – Manchester, NH Jeff – Beverly Lianna – Pittsfield