Upload
scot-sharp
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
The Climate Change Evaluators‘ Community of Practice: Meta-Evaluations for Climate Change Mitigation; Tools and Usability
PresentationIPEN 2012
Astana, KazakhstanDr. Christine Wörlen
Outline
• The Climate-Eval Community of Practice• The tools for evaluating climate mitigation projects • --- and how they can be used for project design• The other opportunities for knowledge creation at the
community of practice
The Climate-Eval Community of Practice
History
• 2007 Astana Workshop• 2008 Alexandria Conference• 2009 Website• 2010 linked-in• 2011 Jordan; Durban• 2012 – Russian-language group in Linked-In
Functions: Knowledge repositoryAnalytical work
Important elements
• Andrew• Website• Linked-in• Webinars• Discussions (online, telephone)• Some face-to-face interaction
Analytical work of the Climate-Eval Community of Practice: The Mitigation Meta-Evaluation as an example
Typical climate change mitigation evaluation challenges (I)
– Baseline issues: counterfactual can be difficult
– Ultimate impact: GHG-emission reduction together with economic development (→ indicator and measurement challenges)
– is mostly not reached directly but through changes in behaviour (investment, utilization) of GHG emitting actors and their supply chain
Evaluation Framework (Tokle and Uitto (2009))
GHG emissionsreduction or avoidance
Sustainable market transformation for increased energy savings
or applications of renewable energy
Demonstrate creative project approaches
and technologies
Develop and disseminate
Information and knowledge
Develop business models and provide enterprise support
Develop financing instruments
and mechanisms
Develop enabling policies standards and certification
Innovation and technology diffused
Awareness created
Adequate business infrastructure
Adequate finance available
Enabling policies, strategies, standards and certification in place
RESU
LTS
PERF
ORM
ANCE
Impacts
Outcomes
Strategies
Evaluation framework
Typical climate change mitigation evaluation challenges (II)
– Not only one group of stakeholders plays a role in achieving that result, but a whole sector; consisting of users, suppliers, financiers and policy .
– But: many climate mitigation interventions affect only one group of stakeholders (e.g. users OR supply chain OR policy makers OR financiers).
– Issues with attribution and context complicate „usual“ measurement challenges – even for the evaluation of a single awareness or capacity building measure, the context and other initiatives need to be taken into account.
Keep developing this Framework Theory of Changeso that it will be able to :
– Reproduce „complete“ theory of change – not just the groups / capacities / factors / aspects that are the subject of the project
– Reflect sectoral context in a complete but „lean“ manner– Reflect relative importance of impeding / supportive factors for
intervention results– Allow for the development of (outcome) indicators across
stakeholders and interventions and GHG savings potentials– Be flexible and rigorous at the same time
Evaluation Framework (Tokle and Uitto (2009))
GHG emissionsreduction or avoidance
Sustainable market transformation for increased energy savings
or applications of renewable energy
Demonstrate creative project approaches
and technologies
Develop and disseminate
Information and knowledge
Develop business models and provide enterprise support
Develop financing instruments
and mechanisms
Develop enabling policies standards and certification
Innovation and technology diffused
Awareness created
Adequate business infrastructure
Adequate finance available
Enabling policies, strategies, standards and certification in place
RESU
LTS
PERF
ORM
ANCE
Impacts
Outcomes
Strategies
Evaluation framework
Policy makers
Financial sector
Financial sector
Supply chain and infrastructure
Theory of Non-Change: Barriers to energy-efficiency behavior
Every group of stakeholders can slow down the diffusion of new technologies / behaviors. Typically, they are impeded by
• IGNORANCE, • LACK OF MOTIVATION, • LACK OF ACCESS; • LACK OF EXPERTISE; • LACK OF BUSINESS MODELS, • LACK OF COST EFFECTIVENESS; • LACK OF AFFORDABILITY
Policy makers
Consumers / Users
Financial sector
Supply chain and infrastructure
Strategies to remove the barriers to energy-efficiency behavior
• IGNORANCE, • LACK OF MOTIVATION, • LACK OF ACCESS; • LACK OF EXPERTISE; • LACK OF BUSINESS
MODELS, • LACK OF COST
EFFECTIVENESS; • LACK OF AFFORDABILITY
• Information campaigns • Incentives (financial / nonfinancial) • Buildup of infrastructure • Capacity building• Business model development and
demonstration • Reduce cost (economies of scale,
economies of scope, subsidies)• Make financing available
•motivation•awareness• expertise•affordability
•awareness•expertise•affordability•Business model
•awareness•motivation • expertise •access•cost effectiveness• affordability
Potential barriers to market transformation and intervention strategies (TOC 2)
•awareness•expertise•access•business model•cost effectiveness•affordability
Consumers / users
Supply chain and
infrastructure
Financial sector
Policy makers
Necessary precondition for MT and immediate outcome ccm intervention
Stakeholder Group
Market transformation from GHG-emitting to GHG non-emitting behavior
GHG emission reduction / reduced climate change impact Overarching objective
Barrier Removal Strategies (incomplete)
Develop and disseminate Information and knowledge for awareness
Technical training
Financial assistance (investment subsidies , loans, financial risk guarantees)
Provision of external advice and best practice models
Develop enabling policies standards and certification
Develop locally adapted solutions (business models, contractual arrangements, technologies)
•Lack of motivation / interest•Lack of awareness of energy efficient technology (ignorance)•Lack of technical competence (expertise) for designing effective policies•Lack of fiscal means (affordability)
•Lack of technical competence for evaluating
technology (expertise)•Lack of business model (risk)•Lack of cost effectiveness
•Lack of liquidity (affordability)
•Lack of awareness of energy efficient technology (ignorance)•Lack of motivation / interest•Lack of technical competence
(expertise)•Lack of access to technology •Lack of cost effectiveness•Lack of investment capital
(affordability)
Stakeholder and potential barriers to market transformation
•Lack of awareness of energy efficient technology (ignorance)•Lack of technical competence (expertise)•Lack of access to technology for manufacturing or distribution•Lack of business model•Lack of cost effectiveness•Lack of working capital (affordability)
Market Transformation
GHG redu +econ. benefit
“Barrier Circle” – or “why things are NOT changing”
Red: “This barrier is a show-stopper for the market.”
Orange: “This is a significant barrier.”
Yellow: “Not a good situation, but no significant challenge.”
Green: “This potential barrier is not impeding market development.”
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business model
Lack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness
Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Households: Lighting and Refrigeration, 1992
Consumers/ UsersSupply Cain
and Infrastructure
Policy makers
local financiers
Market Transformation
GHG Mitigation plus Economic
Benefit
Simplification: leave off inner two circles (intermediate and ultimate objectives)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business model
Lack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness
Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Households: Lighting and Refrigeration, 1992
Consumers/ Users
Supply Cain and Infrastructure
Policy makers local financiers
Then: Overlay with project
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business model
Lack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness
Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Households: Lighting and Refrigeration, 1992
Consumers/ Users
Supply Cain and Infrastructure
Policy makers local financiers
Project strategies can be aligned with the respective barriers.
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
Awareness Campaign
Labels
Labels
Labels
bulk purchases
Consultancy
bulk purchases
TONC-Circle and Barrier Removal Strategies
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business model
Lack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers IgnoranceLack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness
Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectivenessLack of business model
Households: Lighting and Refrigeration, 1992
Consumers/ UsersSupply Cain
and Infrastructure
Policy makers
local financiers
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
Awareness Campaign
Labels
Labels
Labels
bulk purchases
Consultancy
bulk purchases
Approach of this study
- Test Theory of Non-Change (TONC) on a set of climate mitigation interventions from one sector (e.g. energy efficient products)
- Refine TONC, clarify terminology and barrier-strategy couples- Test on other sectors- …to develop generalized Sector Transformation TOC- (Develop indicators for the barriers. As barrier removal is the result of
the project, these can also be the result indicators for market transformation interventions.)
NB: Evaluation studies are the only possible source for this type of analysis. However, due to the variations in evaluation questions and other factors, not all of them are equally useful for this analysis.
Testing the TONC: Market Transformation through Demand Side Management in Thailand since 1992• Thai economy: 10.6% annual growth between 1986 and 1995• Energy demand increased in step• In 1992: Energy Conservation Law with obligatory energy reporting for large consumers and
other (softer) measures.• The national energy utility EGAT started • We look at 2 cases:
– energy efficient lighting in households and – energy efficiency in industrial and commercial facilities
• Crises: – Thai / Asian Financial Crisis in 1997– Privatization of EGAT in 2000-2002
• Test Question: can the model reflect market transformation successes and failures?
Three different markets
• Replacing T12 tubes by T8 tubes• Replacing light bulbs with energy savings bulbs (compact
fluorescent lamps, CFLs)• Replacing inefficient building chillers (large AC units) in
commercial and industrial buildings with efficient building chillers.
Market barrier cycle for efficient T8 light tubes in 2002 (Na Phuket; WB Post-IA)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business model
Lack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness
IgnoranceLack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
WB DSM + GTZ project activities for T8 light tubes(Na Phuket, Sulyma, WB)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business modelLack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
consensus building
Labels
Labels
consensus buildingConsultancy
Awareness Campaign
Labels
campaign
Labels
T8 market after DSM Project in 2000 (WB evaluations)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business modelLack of interest/ motivation
ignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectivenessIgnorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Market barrier cycle for energy savings bulbs (CFL)(Na Phuket; WB Post-IA)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business modelLack of interest/ motivationignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!
Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
local financiers
Project approach for efficient CLF (Na Phuket; WB Post-IA)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business modelLack of interest/ motivationignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordabilityLack of cost effectiveness
#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!Ignorance
Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
bulk purchase
bulk purchase
Consultancy, GTZ ENEP
Awareness Campaign
Labels
campaign
distribution through 7/11
bulk purchase
bulk purchase
Supply Chain and Infrastructure
local financiers
Market barrier cycle for efficient CLF at the end of project (2000) (WB ICR)
Ignorance
lack of expertise
Lack of cost effectiveness
Lack of business modelLack of interest/ motivationignorance
lack of expertise
lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
Lack of cost effectiveness#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!#BEZUG!IgnoranceLack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Interim test result
• Tool can reflect changes in market barriers and barrier removal strategies
• Easy to handle• It becomes clear that every “market” (in the sense of a GHG
emission reducing activity) needs its own set of analyses as the barriers are not of the same strength
• Currently only qualitative analysis possible – more standardization needed for it to have more predictive power
Industrial Energy Efficiency in 1992
4
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency
Overall economic boomRapid investment in new industrial and commercial facilitiesEnergy efficiency is an afterthought
Supply chain is happy to provide inefficient chillers
Policy makers know what to do but not how
Industrial Energy Efficiency in 1992: ENCON law, WB/GEF DSM project, GTZ project
Law requires large facilities to conduct Energy Audits
DSM project with EGAT as implementing agency to include industrial energy efficiency
From 1996: GTZ provides training to policy makers
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
add. Staff, training
GEF grant
consultancy
GTZ training
Energy Audit
law
energy audit
Industrial Energy Efficiency in 2000: no success, due to financial crisis
Energy Audits have been conducted by many but not implemented
EGAT as implementing agency is in the process of being restructured
Financial markets broke down, so that users can’t afford investments
They also have other worries…
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency
Industrial Energy Efficiency in 2001: WB GEF Building Chiller Replacement Project
Took three years to develop
EGAT unavailable
New implementing agency is a large bank
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of business model
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency EGAT
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
bank as implementation partnerzero-interest loan to bank
demonstration
Monitoring role in the project
Monitoring role in the project
outreach
call for LOI
provide loans through IAdemonstration
Industrial Energy Efficiency in 2001: WB GEF Building Chiller Replacement Project
Is removing a number of barriers
4
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivationlack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of access
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency
Industrial Energy Efficiency in 2001: WB GEF Building Chiller Replacement Project
Is removing a number of barriers
…but receives competition from EGAT once national funds are released
4
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivationlack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of access
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
revolving fund
revolving fund
…leading to a vibrant building chiller market in 2005.
4
F-Ignorancelack of expertiseLack of cost effectiveness
Lack of business model
PM-Lack of interest/ motivation
ignorance
lack of expertise
PM lack of affordability
Consumers Ignorance
Lack of interest/ motivation
lack of expertise
Lack of access
lack of affordability
C-Lack of cost effectivenessSC-Ignorancesc Lack of expertise
Lack of access
Lack of affordability
Lack of cost effectiveness
Lack of business model
IA-Ignorance
IA- lack of expertise
lack of affordability
iaLack of cost effectiveness
Lack of access
Consumers/ Users
Policy Makers
Supply Chain and Infrastructure
Local Financiers
Implementing Agency
Impact of 1997 Asian financial crisis on energy efficiency projects in Thailand
• Original motivation for energy efficiency efforts was economic growth, not climate change!
• Projects that were designed and implemented immediately after the financial crisis were affected more strongly than those that were under implementation during the financial crisis (example: Building Chiller Replacement, Energy Efficiency Revolving Fund).
• Impact of financial crisis on ongoing projects: extension in time, slowdown of privatization of public entities (EGAT), more difficulties to include private sector in projects, households were comparatively less affected.
Alternative display: comp table
• Color code allows to compare several projects in tables
• Here: case study Poland district heating
prior to project
2004prior to project
2004prior to project
2004prior to project
2002
Ignorance
Lack of expertise
Lack of access to technology
Lack of cost effectiveness
Lack of motivation / interest
Lack of affordability
Ignorance
Lack of expertise
Lack of access to technology
Lack of cost effectiveness
Lack of business model
Lack of affordability
Ignorance
Lack of expertise
Lack of cost effectiveness
Lack of business model
Ignorance
Lack of expertise
Lack of motivation / interest
Lack of affordability
Local Financiers
Policy Makers
Users
Barrier
Geothermal Coal to Gas BiomassDistrict heating
Supply Chain
Use of TONC in project design
• Theory of No Change: Analyzing barriers helps find out where the next project can push the envelope, cost-effectively (Climate Works evaluation, GEF EO impact study); it looks across different stakeholder groups
• Use of Tool for comparing different projects can help transfer lessons learned / useful project approaches / best practices from one project ot another.
• Use of Comparison Table allows for choice of most appropriate alternative behavior to be implemented in next project.
Overall, this analysis tells you what to do next.
How do people use the Climate-Eval Community of Practice?
Elements of the Community of Practice
• Mostly online• Members register, • Weekly (periodic – maybe twice or once a month) newsletter• Other social media: Twitter (@climateeval with 247 ‚followers‘)• Discussions through linked-in (language subgroups – French,
Spanish and soon, Russian)• Online resources:
– Database of evaluations (500+)– Multimedia content – recordings of webinar, etc– Blog posts– Job announcements
Membership
• Institutional evaluators • Project managers• Consultants / independent and consulting firms• Researchers in universities and think-tanks
Important elements
• Andrew• Website, newsletter• Linked-in (including a group in Russian)• Webinars• Discussions (online, telephone)• Some face-to-face interaction
How could you envision using it? What would make it more helpful to you?
Thank you for your attention.
• Further Questions?
• www.climate-eval.org• Climate-eval(at)climate-eval.org
• Christine Wörlen, woerlen(at)arepo-consult.com