11

Module 4:Module 4:Designing Performance Indicators Designing Performance Indicators

for Environmental Compliance for Environmental Compliance and Enforcement Programsand Enforcement Programs

22

Basic ConceptsBasic Concepts

33

Indicators…Indicators…

Measure and manage environmental compliance Measure and manage environmental compliance and enforcement program performance and enforcement program performance Ensure accountability to legislative bodies, Ensure accountability to legislative bodies, budget authorities, constituent groups, and the budget authorities, constituent groups, and the publicpublic Evaluate and improve effectiveness in achieving Evaluate and improve effectiveness in achieving program missionprogram mission

44

Types of IndicatorsTypes of Indicators

InputInput:: personnel, funds, and other resources that personnel, funds, and other resources that contribute to an activity (contribute to an activity (exampleexample: number of inspectors): number of inspectors)

OutputOutput:: quantitative or qualitative measures of activities or quantitative or qualitative measures of activities or work products (work products (exampleexample: number of inspections): number of inspections)

Intermediate OutcomeIntermediate Outcome:: changes in knowledge, behavior, changes in knowledge, behavior, or conditions that result from program activities (or conditions that result from program activities (exampleexample: : pounds of illegal discharge reduced)pounds of illegal discharge reduced)

Final OutcomeFinal Outcome:: the change in environmental quality the change in environmental quality attributable to program activities (attributable to program activities (exampleexample: improvement in : improvement in water quality)water quality)

55

Audiences & UsersAudiences & Users National/state policy makers and legislatorsNational/state policy makers and legislators Regulatory agenciesRegulatory agencies Industry sectors Industry sectors Stakeholder and interest groupsStakeholder and interest groups General publicGeneral public International bodiesInternational bodies

66

Benefits of IndicatorsBenefits of Indicators Targeting of program operationsTargeting of program operations Ability to set goals and adjust strategiesAbility to set goals and adjust strategies Decision-making for resource allocationDecision-making for resource allocation Ability to identify problems and correct Ability to identify problems and correct

performanceperformance Ability to motivate employeesAbility to motivate employees Communications with the publicCommunications with the public

77

Best PracticesBest Practices

88

Identifying IndicatorsIdentifying Indicators

Define Scope of Indicators Define Scope of Indicators Conduct Meetings with External Stakeholders and Conduct Meetings with External Stakeholders and

Internal StaffInternal Staff Apply the “Logic Model”Apply the “Logic Model” Develop Guiding PrinciplesDevelop Guiding Principles Select Criteria for Evaluating Potential Indicators Select Criteria for Evaluating Potential Indicators Develop Common Definitions for Key TermsDevelop Common Definitions for Key Terms Inventory Existing Data SourcesInventory Existing Data Sources Select an Appropriate Combination of IndicatorsSelect an Appropriate Combination of Indicators

99

Criteria for Evaluating IndicatorsCriteria for Evaluating Indicators

RelevantRelevant: to goals, objectives, and priorities : to goals, objectives, and priorities TransparentTransparent: promote understanding of program: promote understanding of program CredibleCredible: based on data that is complete and : based on data that is complete and

accurateaccurate FunctionalFunctional: easy to use: easy to use FeasibleFeasible: value to program outweighs cost: value to program outweighs cost ComprehensiveComprehensive: cover important operational : cover important operational

aspectsaspects

1010

Developing IndicatorsDeveloping Indicators

Select final indicators using criteria Select final indicators using criteria Establish internal work teams to:Establish internal work teams to:

Define measures in more detailDefine measures in more detail Review existing data for relevancyReview existing data for relevancy Develop new information collection and reporting processesDevelop new information collection and reporting processes Establish schedule for testing and implementing indicatorsEstablish schedule for testing and implementing indicators

Conduct pilot projects to test indicators and correct Conduct pilot projects to test indicators and correct problemsproblems

Resolve technical and methodological issuesResolve technical and methodological issues Execute implementation in phasesExecute implementation in phases

1111

Using IndicatorsUsing Indicators

Monitor Implementation with Regular Reports Monitor Implementation with Regular Reports Use Indicators to Analyze Performance of Use Indicators to Analyze Performance of

Organizational UnitsOrganizational Units Use Indicators to Review Effectiveness of Specific Use Indicators to Review Effectiveness of Specific

ProgramsPrograms Use Indicators to Report to External AudiencesUse Indicators to Report to External Audiences When Analyzing Data from Indicators, Understand When Analyzing Data from Indicators, Understand

the Data Limitationsthe Data Limitations Assess and Adapt IndicatorsAssess and Adapt Indicators

1212

Applying Indicators to Applying Indicators to Water GovernanceWater Governance

1313

Examples of Indicators Examples of Indicators of Effective Water Governanceof Effective Water Governance

Number of discharge permits revokedNumber of discharge permits revoked Total value of fines levied against violatorsTotal value of fines levied against violators Number of facilities ordered to cease operation until Number of facilities ordered to cease operation until

compliant with water regulationscompliant with water regulations Number of permits granted or updatedNumber of permits granted or updated Number of inspections/patrols conducted per monthNumber of inspections/patrols conducted per month Amount that discharges from facilities have been Amount that discharges from facilities have been

reducedreduced Exceedances addressed or returned to complianceExceedances addressed or returned to compliance

1414

A “Logic Model” of A “Logic Model” of Performance Measurement IndicatorsPerformance Measurement Indicators

InputsInputs

resourcesresources

OutputsOutputs

activitiesactivities

Intermediate Intermediate OutcomeOutcome

behavior changebehavior change

Final OutcomeFinal Outcome

environmental environmental ImpactImpact

PersonnelPersonnel

Funds for Funds for salaries, salaries, contracts, IT, contracts, IT, etc.etc.

Inspections Inspections conductedconducted

Enforcement Enforcement actions takenactions taken

Fines assessedFines assessed

Improved facility Improved facility management management practicespractices

  

Reduced illegal Reduced illegal dischargesdischarges

Greater Greater understanding of understanding of how to complyhow to comply

Improved ambient Improved ambient water qualitywater quality

Improved human Improved human and ecological and ecological healthhealth

1515

Exercise: Applying the “Logic Model”Exercise: Applying the “Logic Model”InputsInputs

resourcesresources

OutputsOutputs

activitiesactivities

Intermediate Intermediate OutcomeOutcome

behavior changebehavior change

Final Final

OutcomeOutcome

environmental environmental impactimpact

1616

ConclusionConclusion