Embed Size (px)
Citation preview
This article was downloaded by: [University of East Anglia Library]On: 24 September 2014, At: 02:45Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK
Environmental PoliticsPublication details, including instructions for authorsand subscription information:http://www.tandfonline.com/loi/fenp20
Climate policy innovation:developing an evaluationperspectiveMikael Hildéna, Andrew Jordanb & Tim Raynerb
a Finnish Environment Institute, Helsinki, Finlandb Tyndall Centre for Climate Change Research,University of East Anglia, Norwich, UKPublished online: 21 Jul 2014.
To cite this article: Mikael Hildén, Andrew Jordan & Tim Rayner (2014) Climate policyinnovation: developing an evaluation perspective, Environmental Politics, 23:5, 884-905,DOI: 10.1080/09644016.2014.924205
To link to this article: http://dx.doi.org/10.1080/09644016.2014.924205
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressedin this publication are the opinions and views of the authors, and are not theviews of or endorsed by Taylor & Francis. The accuracy of the Content shouldnot be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions,claims, proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly in connectionwith, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly
forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
Climate policy innovation: developing an evaluationperspective
Mikael Hildéna*, Andrew Jordanb and Tim Raynerb
aFinnish Environment Institute, Helsinki, Finland; bTyndall Centre for Climate ChangeResearch, University of East Anglia, Norwich, UK
A key dimension of the debate about policy innovation is the extent to whichnew policies achieve significant and lasting effects on the problems theypurport to address. However, little is known about such effects. We breaknew ground by investigating how far current evaluation practices in a policysystem with relatively ambitious climate policies – the European Union –identify the most effective (carbon-reducing) policies. We find that a smallnumber of policy instruments are projected to deliver the lion’s share ofemission reductions. Setting aside the special case of emissions trading,these instruments are not particularly innovative. If significant practicaland political obstacles can be addressed, more (detailed) evaluations couldenhance the evidence base and also the political prospects for deliveringdeeper emissions cuts through to 2050. An evaluation perspective could alsooffer a very different way to consider policy innovation dynamics.
Keywords: innovation; evaluation; climate policy; European Union
Introduction
Policy innovation can be interpreted from at least three different perspectives: thesource of new elements (‘invention’); their entry into wider use (‘diffusion’), andtheir subsequent effects (‘evaluation’; Jordan and Huitema 2014). Here, we focuson the third – and least developed – perspective: the effects of (new) policies, andthe systems that are in place for their evaluation.
In principle, policy evaluation can address many questions, including rela-tively narrow matters of effectiveness, such as emissions reductions achieved andthe degree of compliance with policy targets. But it can also address more wide-ranging issues, such as the contribution to longer-term problem solving andpolicy learning, through to process characteristics (legality, representativeness),efficiency, and distributional effects (the key political science questions of ‘whogets what, where, when and how?’; Vedung 2006, p. 400). In many cases,however, because ex ante and ex post policy evaluation systems (and any
*Corresponding author. Email: [email protected]
Environmental Politics, 2014Vol. 23, No. 5, 884–905, http://dx.doi.org/10.1080/09644016.2014.924205
© 2014 Taylor & Francis
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
coordination between them) are lacking in the environmental field (Mickwitz andBirnbaum 2009), policymakers seem to know relatively little about what theinstruments they might introduce will achieve or, once adopted, what they areachieving or have achieved (Mickwitz 2012). This gap appears particularlypronounced in climate policy (Haigh 2001, Kerr 2007), where formal, state-ledevaluation capacities are still very underdeveloped relative to the rapid pace ofpolicy development, and more informal evaluation activities by non-state actors(see Huitema et al. 2011) remain poorly equipped to fill the evident gaps.
Taking this situation as a starting point, we address three basic questions.First, how does the European Union (EU) formally regulate the evaluation ofclimate policy? Second, what do the evaluations that the formal system deliversindicate to be the most effective policies in EU member states? Third, what arethe political forces that affect regulation and practice related to the evaluation ofclimate change mitigation policies in this system?1 These questions are generallyimportant for developing the practice of evaluation, but have particular signifi-cance for policy innovations, since evaluations should establish whether innova-tions deliver expected effects. As a self-identified world leader in mitigatingclimate change, the EU is under particularly strong pressure to show reductionsin its greenhouse gas (GHG) emissions in accordance with its internationalcommitments, and that its policies effectively contribute to this – rather thanrelying on serendipitous factors such as industrial restructuring and energysupply switching (Kerr 2007, EEA 2013). Formally, since 1993, the EU hasoperated a climate policy Monitoring Mechanism (Council Decision 93/389/EEC), through which national-level emissions data are collected. Since 2004,Article 3.2 of the revised Mechanism (280/2004/EC) has obliged Member Statesto provide ‘…quantitative estimates of the effect of policies and measures onemissions by sources and removals by sinks of greenhouse gases’ (emphasisadded). In EU terminology, ‘policies and measures’ are often specific instru-ments, but can also be bundles of instruments such as subsidies, informationcampaigns, and regulatory adjustments or specific actions.2
In spite of these formal/legal requirements, the EU and its member states arestill far from having a developed ‘culture’ of formal or informal evaluation withregards to many climate policy questions (Huitema et al. 2011), including theability reliably to quantify emissions reductions per policy instrument (Hauget al. 2010). This is despite many calls for improved evaluation in all policyfields (EC 2007, 2013a, also Mickwitz 2012) and growing interest in policyeffects shown by scholars of policy invention and diffusion (Jordan and Huitema2014). The potential for policymakers and climate policy innovation scholarsalike to learn more by studying effects is significant, but for the most part largelyunrealised.
We begin by describing the EU’s relatively undemanding climate policyevaluation requirements as they stood until the adoption of the new MonitoringMechanism Regulation in 2013 (525/2013/EU). These requirements reflectmember state demands that the EU should only have light oversight. We then
Environmental Politics 885
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
outline our methods and empirical material, derived from reporting under theMonitoring Mechanism, which includes information on more than 600 policiesand measures, hitherto unexplored from this (‘evaluation’) or alternative (‘inven-tion’ and ‘diffusion’) perspectives. We then explore what can be extracted fromthe 2011 reporting cycle, focusing especially on those policies that promise thegreatest emission reductions.
Then we examine how political tensions underpinning monitoring and eva-luation practices have been addressed, before exploring possible future researchquestions and what a more fully developed evaluation perspective could add tothe academic debate about policy innovation. By highlighting these potentialcontributions, we begin to identify and explore new ways of integrating the sub-literatures on policy innovation, by working across the invention, diffusion, andevaluation perspectives. Finally, we summarise our main findings and identifyfurther research challenges and opportunities in this important but under-explored area of policy innovation research and practice.
In addressing these issues, we break new ground in at least two respects.First, while an ‘effects’ perspective can in principle cover a diversity of topics,we show that, in the EU, the formal evaluation system is strongly focused ondemonstrating that the bloc as a whole is fulfilling its international legal commit-ments. The relatively narrow scope reminds us how practices of evaluation,including the identification of effective policies, are inescapably political(Albaek 1996, Hildén 2014, Howlett 2014).
Second, we explore what an ‘effects’ or evaluation perspective is capable ofadding to the rapidly emerging literature on (climate) policy innovation. We doso first by investigating the extent to which those instruments reckoned to bemost effective (in reducing emissions) can also be considered ‘innovative’ fromthe ‘invention’ and/or ‘diffusion’ perspectives. As far as we can tell, no one hasexamined how far the policies that states expect to be most environmentallyeffective correspond to other criteria of innovativeness. Can they, for example, beclassified as new environmental policy instruments (NEPIs; Jordan et al. 2003) –whose advocates often make bold claims about their superior effectiveness – ordo they emerge out of a standard (for the EU, regulatory; Jordan et al. 2012)policy ‘toolbox’? NEPIs, including market-based instruments (such as emissionstrading or carbon taxation), or voluntary or negotiated agreements with societalactors (such as car manufacturers), reflect an attempt by states to share theirsteering capacity with other actors. Similarly, perceptions that an existing policyis underperforming can trigger calls for reform, dismantling, and perhaps even-tually new policy development (‘invention’). We build on these conceptualinterconnections to explore more radically different understandings of policyinnovation, in light of the more disruptive, structural changes that will probablybe required to deliver the highly ambitious emission reductions outlined in theEU’s ‘low carbon roadmap’ to 2050 (EC 2011c). Such disruptive changes areregarded by some scholars as the hallmark of true policy innovation (Lynn 1997,p. 96, Polsby 1984, p. 8).
886 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
The formal policy evaluation system in the EU
In this section, we show that the development of monitoring and evaluation ofthe policy measures that are expected to contribute to required emission reduc-tions has progressed slowly compared with the more technical development ofthe monitoring of GHG emissions. We suggest that this is due to the inherenttensions between the member states and any supranational body – in this case,the EU Commission and the United Nations Framework Convention on ClimateChange (UNFCCC) – when it comes to detailed evaluation of national policies.
International and EU rules and procedures
In determining the effectiveness of international climate policy, extensive effortshave been made to demonstrate that emissions are being – and will continue tobe – reduced. Globally, under the auspices of the UNFCCC, inventories of GHGemissions have been produced (UNFCCC 2006), informed by detailed meth-odologies from the Intergovernmental Panel on Climate Change (IPCC).3 Butwhile these offer a fairly reliable aggregate view of emission trends, they provideno information on the role of specific policies, let alone their innovativeness.Under a separate UNFCCC process, information on policies and their effective-ness is requested, but overall the guidance is modest (UNFCCC 2000).
Both individual member states and the EU as a whole are parties to theUNFCCC, and as such are required periodically to submit NationalCommunications, including information on emissions and removals of GHGs.4
From the outset in 1993, the EU’s Monitoring Mechanism has echoed UNFCCCguidelines. The material reported to the UNFCCC and the EU under thismechanism is expected to say something about the role of policies in reducingemissions. So far, analyses have focused on trends and key factors affectingemission levels (EEA 2011, 2012, 2013). Recently, however, greater effort hasbeen made to develop methodologies better able to identify the most effectivepolicy instruments (AEAT et al. 2009, Öko-Institut et al. 2012).
In keeping with UNFCCC guidelines, under Article 3(2)(a) of the 2004Monitoring Mechanism Decision,5 member states have been required to reportquantitative estimates of the effect of policies and measures on emissions bysources and removals by sinks of GHGs between the base year and subsequentyears, including 2005, 2010, 2015, and 2020. The 2004 legislation did notspecify whether ex post estimates should be provided. Similarly, reporting oncosts and benefits of specific measures is not mandatory, despite the emphasis oncost-effectiveness in high-level climate policy pronouncements. While indepen-dent reviews of National Communications are carried out, the UNFCCC has noformal power to demand revisions where reports are deemed unsatisfactory.
The recently approved Monitoring Mechanism Regulation (525/2013/EU)includes just three articles covering the reporting of policies and measures. At theirinsistence, it leaves considerable discretion to member states – as do other obligations
Environmental Politics 887
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
to report on climate-relevant policies andmeasures (see below). Reporting of policiesand measures is less systematically monitored than reporting of emissions, for whichthe European Environment Agency (EEA) has developed a scoring system to showhow well member states live up to their commitments.6 General reviews and synth-esis reports are, however, produced by the EEA (see Table 1).
The requirements of certain directives, notably those on Energy End-UseEfficiency and Energy Services (2012/27/EU) and Renewable Energy (2009/28/EC), have also spurred member states to evaluate the impacts of their policies.However, member states have pushed for, and been accorded, a large degree ofdiscretion in what to evaluate and how. Databases such as MURE for energyefficiency attempt to document successes, allowing studies such as the EnergyEfficiency Watch Project (Wuppertal Institute et al. 2013) to analyse states’actions and to reach judgements on the innovativeness of policies.
Table 1. Reporting obligations on policies and measures as required by the UNFCCCnational communication, the Monitoring Mechanism Decision (MMD), the RenewableEnergy Directive (RES), and the Energy Efficiency Directive (EED).
Reportingobligation
Focus of reporting on policiesand measures Publicly available information
UNFCCC NationalCommunications(UNFCCC 2000)
General overview of ‘all’climate-related policies andmeasures.
Individual country submissionsand independent reviews areavailable on the UNFCCCportal.14 General synthesis hasbeen provided (UNFCCC2011).
MMD (from 2013MMR)
Documentation of policies andmeasures contributing tomitigation. Mandatoryprojections for individualmeasures or group ofmeasures.
EEA maintains a database onpolicies and measures.Synthesis reports arepublished (EEA 2011, 2012).
RES (2009/28/EC) Progress in the use of renewableenergy and in policy measurespromoting transition torenewables.
Summarising and country-levelanalyses on progress towardsrenewable energy targets andpolicy measures are published(EC 2013b, 2013c, Ecofyset al. 2013).
EED (2012/27/EU) Progress in energy efficiencyand policies and measurespromoting energy efficiency.
Information on individualpolicies and measures isavailable in the MUREdatabase15 and national targetson the EU Commission’swebpages.16 A synthesis hasbeen provided in the EUenergy efficiency plan and itsimpact assessment (EC 2011a,2011b).
888 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
The ‘unpolitics’ of evaluation in the EU
Since the inception of a common EU climate policy in 1993, the whole matter ofmonitoring has been politically sensitive, with member states blocking open,political discussion of many topics. Pallemaerts and Williams (2006, p. 43)report that Member States were determined that the initial Mechanism woulddo little more than transpose UN reporting requirements, ‘with no objective ofharmonising national legislation’. The UK in particular resisted theCommission’s attempt to become involved in evaluating specific national poli-cies (Jordan et al. 2010, p. 77). By imposing such limits, member states wereable to confine the discussion of effects to what Crenson (1971) famously termedthe domain of ‘unpolitics’.
This ‘unpolitics’ can be highlighted by comparing the original proposal torevise the Mechanism (COM (2011) 789 final of November 2011) submitted bythe EU’s executive body, the Commission, with the Regulation finally approvedby states and the European Parliament in May 2013 (525/2013/EU). The com-parison shows, for example, that the Commission had to accept reporting bygroups of policies and measures instead of individual instruments. Likewise, ithad to accept qualifiers of ‘where available’ for information on ex post evalua-tions of effectiveness and costs and benefits of measures, rather than mandatesuch analyses. Its power to demand additional information from member statesusing delegated acts (where the political threshold for approval is generallylower) was restricted. Although, as we shall see, technical feasibility considera-tions lay behind some of the changes that were made, together they had the effectof restricting the ability of the Commission to facilitate comparative, EU-wideevaluations. Nor has the UNFCCC been able to improve practice greatly. Whileindependent reviews of National Communications have repeatedly highlightedthe dearth of quantitative ex post analysis of effectiveness of policies, reviewershave passed little comment on the adequacy of monitoring and evaluationsystems, which are often very cursorily treated.7
The reluctance of member states to support the Commission’s efforts tostrengthen evaluation requirements does not mean that they are uninterested init. Domestic requirements for evaluations related to climate policy developmenthave become more stringent in many states. In a review of national climatepolicy evaluation practice (AEAT et al. 2009), states have been categorised intothree groups: those with well-developed capacity and expertise in ex postevaluation (e.g. they had analysed impacts of their climate change strategy);those with some expertise but in a more ad-hoc way (e.g. they had experience inevaluating a few individual policies); and those with limited or no experience. Inbroad terms, the EU-15 (Member States before 2004) have tended to have moreexperience in ex post evaluations and more often had formalised monitoring andevaluation systems in place than the EU-12 (Member States acceding after 2004).As of 2009, the UK, Germany, Denmark, Austria, and the Netherlands had eachundertaken an ex post evaluation of their existing policies and measures. A
Environmental Politics 889
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
number of others had carried out bespoke ex post evaluations for an individualpolicy or collection of policies, but on a much smaller scale. Yet the debate in2012 over the new Monitoring Regulation showed that even countries withexperience of evaluating climate policies were unwilling to give theCommission an extensive mandate to determine how evaluations should becarried out. The main driving force appeared to be opposition to theCommission’s power to specify detailed information reporting on policies, ratherthan to evaluations as a practice per se. The side effect of this power struggle isto slow down progress towards harmonised and comparable evaluations ofclimate policies across the EU.
Data sources and methods
Despite the reluctance of member states to report on national climate policies,progress has been made in obtaining information on what policies and policyinstruments are expected to deliver. Here, we draw on the material submitted bystates under the Monitoring Mechanism in 2011, the most recent and compre-hensive source of quantitative information on policies and measures. Althoughclearly significant,8 we have excluded the direct effects of the emission tradingsystem (ETS), as the reductions it achieves are effectively predetermined andEU-wide (see Hildén 2014). We analyse those policies and measures for whichprojected quantitative emissions reductions have been provided for 2010, 2015,and 2020, in order to identify those expected to contribute the greatest reductionsoutside the ETS. In the analysis, it is essential to note the significance ofcombinations, or ‘bundles’, of policies and measures in delivering projectedemission reductions. Member states frequently provide information on suchbundles affecting a particular activity rather than detailed information on indivi-dual policies and measures. In the 2011 cycle, quantitative data on projectionswere reported for approximately 645 policies and measures or bundles thereof,although countries differed considerably with respect to how much quantitativeinformation they provided (see Supplemental material).
We focus on ex ante assessments of projected emissions reductions becauseex post evaluations of actually achieved reductions in terms of specific instru-ments are scarce in member states’ Monitoring Decision reports. Evaluations thatcould be compared across countries are also rare outside the formal evaluationsystem.9 We use projected emissions reductions for 2015 and, to check that suchan analysis does not miss significant policies that could become more effectiveover time, compare these with projected reductions for 2020. The high correla-tion (R2 = 0.86) between reductions expected to be achieved in 2015 and 2020suggests that focusing on the earlier date provides robust results on the policiesthat are expected to be effective.
In addition to the quantitative reduction, the material submitted providesdescriptive information, including a categorisation into specific instrumenttypes (economic, fiscal, voluntary, regulatory, information based). We have
890 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
used this categorisation to compare the relative projected emission reductionpotentials of the different types of measures. In order that instruments applied inthe highest emitting countries are not assumed to be most effective simply bydint of the high baseline they start from, we compare the reductions expectedwith the reported GHG-inventory data (2011) to gauge the proportionate effect.10
This allows us to identify the five most effective policies projected by the 24states that provided quantitative reduction estimates.
Policy effects: findings
Our analysis of the material delivered by the Monitoring Mechanism providesthe wider context in which the effects of any policy innovations can be judged.The results show considerable variation, but also that current reporting makes theeffects of innovations difficult to identify due to the generic categories in use andthe bundling of instruments.
Uneven emission reductions
The distribution of projected emission reductions suggests that there is greatvariation in the perceived effectiveness of policies and measures, with a limitednumber expected to achieve most of the reductions (Figure 1). Member states’
Figure 1. Cumulative reductions expected from policies and measures with quantita-tive data versus the share of reported policies and measures (excluding the ETS). Datasource: country-specific MMD data submitted in 2011.
Environmental Politics 891
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
top policies and measures are a diverse group (see Supplemental material), butrenewable energy is perceived to play a significant role in several countries,promoted by a combination of regulatory and economic instruments. Energysaving and energy efficiency are also deemed important, including a generalreduction in the use of fossil fuels in the electricity sector.
The top five policies and measures are generally expected to achievebetween 5% and 10% reductions by 2015, relative to national emissions in2011. Six countries (Belgium, Bulgaria, Greece, Lithuania, Slovenia, andSweden) stand out in that their top five are expected to reduce emissions bymore than 15% (Table 2), and, in Greece and Sweden, considerably more. Atface value, that around 30% of reported policies and measures might delivernearly all expected reductions could be taken to indicate significant
Table 2. Expected emission reductions achieved by 2015 by the five most importantpolicies and measures in EU member states for which quantitative information has beenprovided, ranked by percentage share, excluding the ETS (Gigagram CO2 equivalents,excluding LULUCF).
CountryTotal GHG emissions 2011 Gg
CO2 equivalentPercentage reduction by reportedfive top policies and measures
Sweden 61,449 38.6Greece 115,045 33.5Lithuania 21,612 19.4Slovenia 19,509 19.4Bulgaria 66,133 18.4Belgium 120,172 17.9Hungary 66,148 13.5Cyprus 9154 10.0Spain 350,483 9.5Netherlands 194,379 9.0Estonia 20,956 8.9Austria 82,842 8.7United Kingdom 556,458 7.7Luxemburg 12,098 7.6Denmark 57,748 7.4Slovakia 45,297 7.2Germany 916,495 7.1Finland 67,019 6.7Ireland 57,512 5.8France 491,497 5.7Czech Republic 133,496 5.2Malta 3021 5.2Italy 488,792 4.8Latvia 11,494 3.1Total 3,968,809 9.0
Data source: country-specific MMD data submitted in 2011 combined with statistical information onGHG emissions for 2011.
892 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
inefficiency in national climate-change policy, with numerous instrumentsachieving very little.11 However, this would be to ignore the possibility thatsome instruments are effective in creating the enabling conditions in whichothers can deliver significant emission cuts, for example by developing newtechnologies (e.g. procurement, research), developing the supply side, or iden-tifying where potential savings could be made (e.g. making information public,labelling, consensus-building activity). Member states may also report instru-ments that produce some emissions reduction as a minor side effect, forexample measures for rural development that may contribute to renewableenergy development or reduction of emissions from soils.
Reductions by policy and by sector
Member states often provide information on bundles of instruments affecting aparticular activity, such as the use of energy in housing, instead of trying todisentangle the effects of each. The reporting furthermore suggests that thegreatest projected reductions are expected to come from regulatory and economicpolicies and measures (Table 3). The dominance of regulatory and economicinstruments is also evident when the five most important are examined for eachcountry that has provided quantitative information (Table 4).
Reforming climate policy evaluation: opportunities and limits
A broad finding of our analysis is that relevant information on the quantitativeeffects of policy instruments is more difficult to obtain than might be expected.The reasons – partly technical, partly political – are explored further below.Another key finding is that projected emission reductions are mainly expected
Table 3. Projected emission reductions by type of policy or measure17 (excluding theETS).
Type of policy or measureCombined projected emission reduction in 2015
(Gg CO2 equivalent/year)
Regulatory 102,532Economic 101,749Economic + regulatory 49,278Economic + fiscal 36,136Fiscal 23,165Economic + regulatory + fiscal 20,742Economic + planning 19,068Planning 18,303Voluntary 17,463Other 17,160
Data source: country-specific MMD data submitted in 2011.
Environmental Politics 893
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
through combinations of regulatory and economic policies and measures. Welack exact quantitative data to prove it in detail – state concerns over excessiveadministrative burdens reduces the prospects of researchers ever having easyaccess to it – but our analysis indicates that innovativeness in terms of noveltydoes not necessarily make a policy or measure more effective compared to ‘old’policies and measures. We return to this in the next section.
The demand for better evaluations
As noted above, the data presented are projections, not actual emission reduc-tions (which will eventually need to be confirmed by ex post analyses, which atpresent the member states are not obliged to produce). A further caveat is thatsome countries may have given their adopted policies and measures innovativetwists, for example by combining them in some ingenious new way or byintroducing innovative elements. To reveal whether this is true requires moredetailed case analysis, of the type presented by Schaffrin et al. (2014). There isplenty of scope here for the more informal evaluation system – chiefly theacademic research community – to be more active in conducting independentevaluations to investigate issues such as these (Mickwitz and Birnbaum 2009,p. 105).
Obstacles to developing evaluation practice
In theory, in a polycentric governance system such as the EU (Ostrom 2010),evaluation can promote innovation and learning, as well as transparency andaccountability (Mickwitz 2012, p. 268). For this benefit of polycentricity tobe realised, however, systems and practices for monitoring and evaluationneed to be in place (Rayner and Jordan 2013). At present, a number of
Table 4. Number of times a specific type of policy or measure has been ranked among thefive most important in terms of projected GHG reductions in 2015 (excluding the ETS).
Number of times the policy or measure appears among the five mostimportant actions to reduce GHG emissions in member states
Primary type ofpolicy or measure
Mostimportant
Second mostimportant
Third mostimportant
Fourth mostimportant
Fifth mostimportant
1 regulatory 10 9 13 14 122 economic 10 10 6 7 73 fiscal 1 4 1 – 14 voluntary – – 4 – –5 planning 1 – – – 16 other 2 2 2 4 1
Data source: country-specific MMD data submitted in 2011.
894 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
barriers obstruct this. Some are technical, for example due to the mixednature of many policies and measures, as well as well-known attributionchallenges (Haug et al. 2010, Mickwitz 2012, pp. 268–269). The differencebetween fiscal and economic policies and measures is not always welldefined. As subsidies may take the form of tax exemptions or reductionsthat generate economic benefits for some, the combination of economic andfiscal instruments is expected to yield greater reductions than purely fiscalpolicies and measures (Table 3), but the effects are difficult to disentangle.Other barriers highlight how monitoring and evaluation, while ostensiblytechnical, remain inescapably political. First, they require resources, whichactors may be reluctant to allocate, particularly in times of austerity. Thus, innegotiating the 2013 Monitoring Mechanism Regulation, member statesrepeatedly stressed the need to minimise administrative burdens.
More fundamentally, there is deep and ongoing political opposition to the EUradically strengthening its role in evaluation, as well as in associated areas ofimplementation and enforcement (Jordan and Tosun 2012). Some actors may fearthat their favoured policy instruments could be undermined by rigorous assess-ments of effectiveness, and attempt to have available evidence framed in aspolitically convenient a manner as possible. Full disclosure may not be in theperceived interests of an individual state or sectors within it, and given the highdegree of subsidiarity in decision making, states have been able to resist pressurefor more systematic monitoring and evaluation (Stame 2008). Political tensionshave even emerged in the past over the precise function of the EEA, whosepotential role of collecting relevant data on institutional arrangements withinmember states and how they make EU legislation work remains undeveloped(Haigh 2001). So, far from pulling together, different actors in the EU’s poly-centric governance system may be working in a way that thwarts a morecoordinated approach to evaluation.
Earlier, we highlighted the constraints imposed by states on the revision ofthe Monitoring Mechanism. Even prior to the preparation of the 2013Regulation, the Commission rejected some of the more radical suggestionsproposed by those it had commissioned to conduct background analyses, pre-sumably in anticipation of member states’ opposition. These included sugges-tions to strengthen the links between ex post and ex ante assessment and thecomparability of analyses conducted across member states. To encourage regularand integrated policy evaluations and (re)appraisals, its consultants recom-mended introducing a reporting requirement for member states to prepare aNational Climate Change Policy Evaluation and Appraisal Report coveringtheir most important measures or a specified set of EU policies (AEAT et al.2009). It could have fostered a cyclical approach to appraisal and evaluation,whereby policies are reappraised in the light of findings of ex post analyses. Thefact that it was excluded from the 2011 proposal, despite its fit with theCommission’s own stated agenda for evaluation (EC 2007), again highlights‘unpolitics’ at work.
Environmental Politics 895
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
Possible ways to overcome or circumvent difficulties
Recognising that the current system contains inconsistencies, but aware ofpolitical sensitivities, the Commission has proceeded cautiously, using indirectand what are increasingly termed ‘open’ methods of policy coordination thatmimic the OECD technique of open peer review. For example, it has commis-sioned independent reviews of evaluation practice in the member states (AEATet al. 2009). Such projects have also aimed at developing common evaluationmethodologies – another tried and tested technique associated with the OpenMethod of Coordination.
One favoured approach to deal with the problem of incomplete data is tothink about evaluation in terms of three different tasks or ‘tiers’. Tier 1 usesroutinely available general statistical data, whereas Tier 3 focuses on specificdata collection for particular policies and measures. Tier 2 is an intermediatelevel of detail and data requirement (Öko-Institut et al. 2012). Crucially for ourpurposes, Tier 3 evaluations of policies and measures, while encouraged, are notmandatory in any of the current key reporting obligations (see Table 1).
One practical difficulty of the ‘Tier 3’ approach is its data intensity. Somestates are unwilling to commit themselves to costly, data-intensive reportingobligations. A mixed approach recommended by Öko-Institut et al. (2012)with alternation between tiers may encourage greater buy-in from member statesthat have the capacity to evaluate but have political concerns about addingexcessive new burdens or being too open.
The identification of policies and measures that should be subjected to theclosest evaluation is important for the tiered approach. The obvious choice, giventhat differences between policies and measures can be dramatic (Figure 1), is tofocus on the most significant in terms of projected emission reductions. Even ifthe general picture is partly an artefact that arises from the bundling of policiesand measures, it helps to identify areas where evaluations are needed. In additionto projected emission reductions, the existence of innovative elements (thedefinition of which could be determined by states themselves or EU bodies,but possibly including novel ways of ensuring emission reductions in ‘tradi-tional’ instruments) could be used as an additional criterion for focusing evalua-tions, and thus making the most of existing evaluation capacities. As alreadynoted, UNFCCC guidelines have encouraged reporting novelty and innovative-ness in the adopted policies and measures, but the review processes and sum-maries of national reports do not systematically follow up on this. The revisedMonitoring Mechanism does not even refer to innovativeness.
It is truly difficult to formulate detailed regulations on substance and processwhilst at the same time facilitating responsiveness to specific contexts and chan-ging circumstances (Knill and Lenschow 2003). Being overly prescriptive risksevaluations becoming resource-intensive rituals that do not provide useful inputinto policy formulation. More flexible and deliberative approaches may yield betterresults in terms of policy learning, but make comparative analyses more difficult.
896 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
An ‘evaluation perspective’ on policy innovation: elements of a new researchagenda
Significant obstacles stand in the way of a fuller appreciation of the effects ofparticular policies. However, if these can be addressed – and given the status quosummarised above, this is clearly a big ‘if’ – an evaluation perspective couldoffer a potentially very illuminating way to think about the three dimensions ofpolicy innovation.
Starting first with the classic ‘invention’ perspective, the top five performingpolicies and measures reported by member states do not generally appear toqualify as entirely new policy instruments. Indeed, the ‘old’ instrument ofregulation features prominently. When analysed using a standard Walker (1969,p. 881) definition, according to which an innovation is ‘a program or policywhich is new to the states adopting it, no matter how old the program may be orhow many other states may have adopted it’, such instruments are neither novelnor innovative (see Supplemental material). Indeed, the countries that expect toachieve the greatest relative reductions with their top policies and measures(Table 2) rely on ‘traditional’ policy instruments such as taxes, regulations, andsubsidies. This does not of course imply that there are no novel elements in them– this being a matter for more detailed qualitative research.
It is important to note that many member states’ respective top fives, such asmeasures promoting renewable energies, including biofuels, flow from EUpolicies (i.e. there may well be some coercive-diffusion effect at work – seebelow; Jordan and Huitema 2014). New quantitative targets and diffusion ofnovel approaches such as quantitative regulation for energy efficiency are foundamong the top five measures in a number of states, especially in the housingsector. This indicates the diffusion of an EU-wide approach, and the reportingsuggests that there is subsequent national adjustment (Supplemental material).Fine-tuning may also occur as an EU policy is tailored to national circumstancesduring implementation. For example, Malta expects to reduce emissions byintroducing more energy-efficient desalination. Countries have also targetedspecific emissions; Denmark and the Netherlands have identified N2O emissionsin parallel with carbon dioxide. Finally, we again highlight that Table 2 does notinclude the ETS, which has definitely marked a break from the EU’s regulatorypast (Jordan et al. 2012), but is far from having received a positive evaluation.
Nonetheless, the fact that member states prefer to rely on policies andmeasures that have been extensively used elsewhere, or in the country itself,for achieving major GHG reductions by 2015 or 2020, is noteworthy anddemands further research by policy innovation scholars. Does it highlight adegree of risk aversion and tendency among policymakers to avoid blameassociated with new but potentially unsuccessful initiatives (Howlett 2014)?Alternatively, does it simply reflect weak national evaluation capacities – orthe fact that more is known about the effects of regulations vis-à-vis some of thenewer instruments? Finally, one might assume that success in innovation is
Environmental Politics 897
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
everything (a bias noted by Jordan and Huitema 2014) – hence our focus on thetop fives. But studies of policies that are perceived (either narrowly or widely) tobe insignificant (for us, ‘the bottom fives’) could be just as revealing (Rogers2003, p. 113). Answers to these questions could be generated by subjecting ourresults to more detailed qualitative research.
Moving to a ‘diffusion’ perspective, Volden (2006) has claimed that theperceived success of a policy is important for wider adoption. Drawing on the‘attributes of innovation’ postulated by Rogers (2003, p. 219), work at theinterface between diffusion and evaluation is beginning to shed light on howfar policies’ relative advantage, performance and observability affects theirpropensity to diffuse. ‘Successful policies’ are said to diffuse more quicklyand/or more completely than less successful ones (but see Volden 2006, p.294). But is this true? These are old questions now requiring careful analysisin the new context of climate policy; thus far, their importance has been notedbut not systematically addressed (e.g. Huitema et al. 2011). They demand furthercase-study research (including, e.g., interviews with those who commissionevaluations and/or draw on their findings). Future research could usefully exam-ine how judgements of success are shaped, and the extent to which evaluationplays a role by constructing knowledge in certain ways. This should include theextent to which informal ex post evaluation studies can challenge interpretationsof policy effectiveness developed within the formal evaluation system, prevent-ing less effective (or counterproductive) policies from diffusing too far. Finally,whether good quality, independent evaluation can promote the diffusion ofeffective policies and prevent less effective ones spreading depends on howreceptive policy systems are to its findings, especially the potentially powerfulvested interests that commonly align behind particular policies. This is an arearesearchers should investigate as part of the wider study of the ‘innovationjourneys’ (Voß and Simons 2014) taken by particular policies and instruments.
On these matters, it will of course be important to draw on a rich seam ofrelated work on knowledge development and utilisation (Rich 1991), whichsuggests that a great deal of evaluation knowledge is not immediately utilisedand/or is twisted to fit prior purposes. This – coupled with the paucity ofindependently commissioned ex post evaluation studies – is more likely to leadto a situation in which policy advocates and entrepreneurs make partisan claimsthat are not evidence based.12 It is worth recalling that less than 10% of theentries in the 2011 reporting cycle included quantitative data based on ex postevaluations, with varying levels of transparency. Academic research could use-fully investigate whether the policies commonly projected to deliver the biggestsavings are those that are being the most rigorously evaluated ex post (or exdurante) by formal and/or informal policy evaluations. Of interest, might (again)be the six countries whose top five measures are expected to reduce emissions bymore than 15% (Table 2).
Finally, an ‘evaluation’ perspective on the actual effects of policy innovation(s)could raise some additional questions only partially addressed by the invention and
898 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
diffusion perspectives. Jordan and Huitema (2014) argue that the policy innovationliterature has been dominated by issues to do with invention and diffusion, whileeffects (not) generated by new/innovative policies have received far less attention.Rogers (2003, p. 453), following standard evaluation approaches, argued that theconsequences of innovation can be grouped into different types: desirable/undesir-able, direct/indirect, anticipated/unanticipated. Most innovations probably generatesome combination of all three (ibid., p. 444), but in their current state, formalclimate policy evaluation systems in the EU have not explored what these are. Theimmediate focus of most evaluation efforts has been on determining whether and/or when the EU is likely to fulfil its emission reduction targets. This may, however,turn out to be too limited a perspective. Biofuels policy offers a glaring example ofthe boundaries of analysis being drawn too narrowly to highlight serious sideeffects (Dunlop 2010), which strong advocates of the policy would prefer toremain in the realm of the ‘unpolitical’.
Given the ambition of the EU’s existing 2050 targets, policymakers willprobably need to go further than the policies that achieve quick emissionsreductions in the short term, to examine those that promise the disruptivechanges that will bring about long-term decarbonisation. The Commission’senergy efficiency plan (EC 2011a) recognises the need to explore new policiesin efforts to influence consumer behaviour, and the conclusions of the lowcarbon roadmap (EC 2011c) also hint at new policy initiatives. But neitherdocument proposes systematic evaluation as a way to improve longer-termproblem solving and policy learning. Whatever the obstacles, an identificationand evaluation of policy innovations would be in line with the Europe 2020strategy (EC 2010), which aims at boosting innovations not only in products andservices but also in policies and measures.
Probably the most significant additional aspect that an evaluation perspectiveon policy innovation can offer is to challenge more fundamentally the way inwhich policy innovations are usually thought about. For Polsby (1984, p. 8), theacid test of policy innovation is that it alters (or promises to alter) the lives ofpeople in ‘substantial and fairly permanent ways’. Transformative effects areregarded at least by some scholars as the hallmark of true policy innovation(Lynn 1997, p. 96, Polsby 1984, p. 8). For Lynn (1997, p. 96), they go beyondthe kind of incremental and/or continuous improvements that most evaluationpractices in the EU have focused on, to achieve an ‘original, disruptive, andfundamental transformation of … core tasks. Innovation changes deep structuresand changes them permanently’.
If a contribution to disruptive societal change is taken as the main criterionfor innovativeness, one can argue that several top policies and measures at leastaim to be innovative by striving to permanently change ‘deep structures’ in theenergy sector. But if the task of evaluations is the detection of disruptive change,then their focus needs to broaden from documentation of emissions reductions tooffering explanations of why and how reductions come about, or fail to
Environmental Politics 899
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
materialise. Suffice to say, this would push evaluation into areas currentlydeemed highly sensitive, and thus politically wholly ‘off limits’.
Conclusions
Policy innovation can be interpreted from several different perspectives. Of thethree perspectives noted by Jordan and Huitema (2014), the one focusing on theultimate effects of policy innovations remains least developed. In theory, anevaluation perspective on policy innovation can address relatively tightly definedquestions. But it can also raise and address much more wide-ranging issues, suchas contribution to longer-term problem solving, efficiency, as well as distribu-tional effects. But as the literature on climate policy evaluation has begun tohighlight, the EU and its member states are far from having a sufficientlydeveloped ‘culture’ of evaluation (linking the formal and informal systems) tooffer convincing answers to these and other questions.
Here, we have sought to identify future possibilities and current constraints inpolicy evaluation research by focusing on what the formal evaluation system inthe EU, centring on the Monitoring Mechanism, currently reveals about theeffects of policies. We have confirmed that current evaluation practices arecentrally concerned with the relatively narrow issue of compliance with aggre-gate targets. Some bigger – and often much more political – topics and questionshave been pushed aside. Jordan and Huitema (2014) argue that the policyinnovation literature has tended to ignore the politics of innovation. We haveshown that evaluation – a topic often wrongly perceived to be rather dry andtechnical – is potentially every bit as political as the invention and diffusion ofnew policies. It is just that many actors in the EU have preferred to keepevaluators on a tight leash. After all, it is not only the results of evaluationsthat are potentially very political; who should conduct monitoring and evalua-tion, how much effort should be invested, and how available evidence should beframed are equally political questions. That they are so deeply contested, even ina self-proclaimed ‘climate leader’ such as the EU, is a sobering reminder of howstrong is the pull of the status quo.
If some obstacles to more comprehensive and reflexive evaluation can beovercome, an evaluation perspective could offer a potentially very illuminatingperspective on policy innovation dynamics. Drawing on the most recent (2011)cycle of evaluation reports under the Monitoring Mechanism, we have shownthat a relatively small number of key instruments – not necessarily thosetypically portrayed as ‘innovative’ – are projected to deliver the lion’s share ofemissions reductions, if we exclude the ETS. How, why, and by whom thesepolicies were selected for attention deserves further research. After all, havingthe capacity to identify policy innovations that produce intended effects consti-tutes a vital sub-component of a state’s broader capacity to engage in policyinnovation, improvement, and learning over the longer term (Hildén 2014). Atpresent, state evaluation capacities appear to be markedly different, as does the
900 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
political desire to engage in more reflexive forms of evaluation (Huitema et al.2011). Here, attention can be turned to more informal non-state networks to pluggaps (Mickwitz 2012, pp. 281–283).13 Through the emerging debate about‘cultures’ of evaluation, the literature is finally beginning to explore why thismight be and what the implications for policymaking and, ultimately, problemsolving might be.
With those issues in mind, it seems even more important to ask just howwell equipped is the current EU system for the significant challenges laid out inthe Commission’s roadmap to 2050. To date, strategies for adopting newpolicies and measures have often been developed with plenty of ‘designspace’ to spare (Howlett 2011). As long as the targets are – as they have tendedto be thus far (Jordan et al. 2010, p. 265–266) – perceived to be relativelyeasily achievable, actors will be under less pressure to design policies carefullyand/or put their faith in unfamiliar (i.e. innovative) policies and measures.Although the EU has got used to working like this, it sits uncomfortably withEurope 2020 strategy’s bold announcement that ‘Europe faces a moment oftransformation’ (EC 2010, p. 5). Moving this beyond rhetoric suggests a needfor innovative policies and measures, in order for ambitious 2050 emissionsreduction targets to be achieved.
At present, some hope rests with informal evaluations at the level of memberstates. These, however, have been shown to be variable in the number conducted,quality, and, perhaps more surprisingly, willingness to challenge existing policygoals and targets (Huitema et al. 2011). Nonetheless, there is clearly a need forindependent critical examination of the expected – and delivered – emissionreductions. The cases of Greece and Sweden, which expect to be able to cutemissions by more than 30% with the help of just five bundles of policies andmeasures, surely merit detailed analysis. Independent analyses may also criticallyexamine side effects from policies such as biofuels, and prevent policies backedby strong vested interests gathering too much momentum (Dunlop 2010). Suchevaluations are necessarily highly political, but could be very important ifunfounded projected emissions reductions repeatedly inflate a bubble of falsehope that is only later revealed by aggregate GHG emissions inventories. Bythen, precious time would have been lost; time which could have been used todevelop ‘truly’ innovative (i.e. effective) policies.
AcknowledgementsWe thank three anonymous referees for insightful comments and suggestions on an earlierdraft. All authors acknowledge support from the EU FP7 Responses project (Grant No.244092). Andrew Jordan acknowledges support from a major research fellowship of theLeverhulme Trust (Grant No. F00204AR) and the INOGOV COST Action (No. IS1309).Mikael Hildén was supported by the Finnish Academy Greenpol project (Grant No.259929). We are grateful to Hans-Paul Velema for help in extracting information fromMonitoring Mechanism reports.
Environmental Politics 901
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
Supplemental dataThe supplemental material for this article can be accessed at http://www.syke.fi/en-US/Research__Development/Support_for_climate_policy/Projects
Notes1. We lack the space and empirical evidence to focus here on process characteristics
and how they are handled in evaluations.2. A clear distinction between national policy instruments and those deriving from the
EU is often hard to make, and is not attempted here. We will note, however, howseveral significant member state instruments have been adopted to implement EUlegislation.
3. The 2006 IPCC guidance on GHG inventories has been published in five volumes:http://www.ipcc-nggip.iges.or.jp/public/2006gl/index.html
4. Under the 1997 Kyoto Protocol, the 15 then-member states (EU-15) committed toreduce their aggregate emissions to 8% below 1990 levels by 2008–2012.Subsequently, the EU unilaterally committed to reduce emissions by 20%, increasethe use of renewable energy sources by 20%, and improve energy efficiency by 20%by 2020; member states are allocated varying targets in the interests of equitableburden sharing. As a whole, the EU is on track towards the targets, but individualcountries face very different challenges with respect to specific policy targets (e.g.on the deployment of renewable energy; EEA 2012, 2013).
5. Decision No 280/2004/EC of the European Parliament and of the Council of 11February 2004 concerning a mechanism for monitoring Community greenhouse gasemissions and for implementing the Kyoto Protocol.
6. http://www.eionet.europa.eu/dataflows/pdf2012/flow_summary?flow = AE2 (VisitedJuly 30 2013).
7. UNFCCC guidelines are in effect ‘soft law’, and reporting obligations can variouslybe ‘compulsory’, ‘expected’, or ‘voluntary’. Interestingly, while they suggest that ingathering information on policies and their effectiveness, priority should be given tothe evaluation of ‘innovative’ policies (UNFCCC 2000b), this is not echoed in theEU equivalent. The concept of innovativeness is in any case left undefined in theUNFCCC document.
8. At least in terms of emissions reduction sought and achieved (EEA 2013) comparedto other policies.
9. In the 2011 reporting cycle, less than 10% of the entries included quantitative databased on ex post evaluations, with varying levels of transparency. In a survey of fivestates’ (‘formal’ and ‘informal’) evaluations carried out for the ADAM project (seeHaug et al. 2010), while the UK had a high number of ex post quantifications (22 of72 studies surveyed presented a quantification of the emission reductions achievedby a particular instrument or package of instruments), Germany had 8 out of 56;other member states had none.
10. Greenhouse Gas Inventory Data – Detailed data by Party: http://unfccc.int/di/DetailedByParty/Event.do?event=go
11. We are grateful to an anonymous reviewer for highlighting this aspect.12. Hence calls for greater ‘pluralism’ in evaluation (e.g. Mickwitz and Birnbaum
2009).13. See especially the work of the European Environmental Evaluators Networking
Forum.14. http://unfccc.int/national_reports/annex_i_natcom/submitted_natcom/items/4903.
php
902 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
15. MURE (Mesures d’Utilisation Rationnelle de l’Energie) http://www.muredatabase.org/
16. http://ec.europa.eu/energy/efficiency/eed/reporting_en.htm17. The types are as defined in FCCC/CP/1999/7 page 85. Planning was added, as this
type frequently occurs in country reports. The fiscal category includes measuresaffecting taxes; the economic category, measures such as infrastructure programmes,subsidies, investment programmes, feed-in tariffs, loans/grants, trading schemes;regulatory, binding standards and regulations; information, measures such as label-ling, awareness raising, voluntary standards; education, measures such as trainingprogrammes, capacity building, education of multipliers; research, research pro-grammes, demonstration projects; planning, measures such as waste management,transport planning and urban planning.
ReferencesAEAT, Ecofys, Fraunhofer ISI, NTUA, 2009. Quantification of the effects on greenhouse
gas emissions of policies and measures: Final Report. Didcot: AEA Group, FinalReport No. AEA/ED05611/Final Report. ENV.C.1/SER/2007/0019.
Albaek, E., 1996. Why all this evaluation? The Canadian Journal of Program Evaluation,11 (2), 1–34.
Crenson, M., 1971. The un-politics of air pollution. Baltimore: Johns Hopkins Press.Dunlop, C.A., 2010. The temporal dimension of knowledge and the limits of policy
appraisal: biofuels policy in the UK. Policy Sciences, 43 (4), 343–363. doi:10.1007/s11077-009-9101-7
EC, 2007. Communication to the Commission from Ms Grybaukaité in agreement with thePresident: Responding to strategic needs: reinforcing the use of evaluation. Brussels,European Commission, SEC (2007) 213.
EC, 2010. Europe 2020 - A strategy for smart, sustainable and inclusive growth. Brussels,European Commission, COM (2010) 2020.
EC, 2011a. Energy efficiency plan 2011. Brussels, European Commission, COM (2011)109 final.
EC, 2011b. Energy efficiency plan 2011 - Impact assessment. Commission Staff WorkingDocument SEC (2011) 277 final. Brussels, European Commission.
EC, 2011c. A Roadmap for moving to a competitive low carbon economy in 2050.Brussels, European Commissionm, COM (2011) 112 final.
EC, 2013a. Strengthening the foundations of Smart Regulation – improving evaluation.Brussels, European Commission, COM (2013) 686 final.
EC, 2013b. Commission staff working document accompanying the Renewable energyprogress report. Brussels, European Commission, SWD (2013) 102 final.
EC, 2013c. Renewable energy progress report. Brussels, European Commission, COM(2013) 175 final.
Ecofys et al., 2013. Renewable energy progress and biofuels sustainability. Brussels,European Commission, No. ENER/C1/463-2011-Lot2.
EEA, 2011. Greenhouse gas emissions in Europe: a retrospective trend analysis for theperiod 1990–2008. Copenhagen, EEA, EEA Report 6/2011.
EEA, 2012. Greenhouse gas emission trends and projections in Europe 2012- Trackingprogress towards Kyoto and 2020 targets. Copenhagen, EEA, EEA Report 6/2012.
EEA, 2013. Greenhouse gas emission trends and projections in Europe 2012- Trackingprogress towards Kyoto and 2020 targets. Copenhagen, EEA, EEA Report 10/2013.
Haigh, N., 2001. Foreword in: Reporting on environmental measures: are we beingeffective?. Copenhagen: European Environment Agency.
Environmental Politics 903
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
Haug, C., et al., 2010. Navigating the dilemmas of climate policy in Europe: evidencefrom policy evaluation studies. Climatic Change, 101 (3–4), 427–445. doi:10.1007/s10584-009-9682-3
Hildén,M., 2014. Evaluation, assessment and policy innovation: exploring the links in relationto emission trading. Environmental Politics, 23, 5. doi:10.1080/09644016.2014.924199
Howlett, M., 2011. Designing public policies. New York: Routledge.Howlett, M., 2014. Why are policy innovations rare and so often negative? Blame
avoidance and problem denial in climate change policy-making. GlobalEnvironmental Change, (in press). doi:10.1016/j.gloenvcha.2013.12.009
Huitema, D., et al., 2011. The evaluation of climate policy: theory and emerging practicein Europe. Policy Sciences, 44 (2), 179–198. doi:10.1007/s11077-011-9125-7
Jordan, A.J. et al., eds., 2010. Climate change policy in the European Union. Cambridge: CUP.Jordan, A. and Huitema, D., 2014. Innovations in climate policy: The politics of
invention, diffusion and evaluation. Environmental Politics, 23, 5. doi:10.1080/09644016.2014.924209
Jordan, A. and Tosun, J., 2012. Policy implementation. In: A.J. Jordan and C. Adelle, eds.Environmental policy in the European Union. 3rd ed. London and Sterling, VA:Earthscan, 247–266.
Jordan, A., et al., 2012. Environmental policy: governing by multiple instruments? In:J. Richardson, ed. Constructing a policy making state. Oxford: OUP, 104–124.
Jordan, A., Wurzel, R.K.W., and Zito, A.R., 2003. New’ instruments of environmentalgovernance: patterns and pathways of change. Environmental Politics, 12 (1), 1–24.doi:10.1080/714000665
Kerr, A., 2007. Serendipity is not a strategy: the impact of national climate programmes ongreenhouse gas emissions. Area, 39 (4), 418–430. doi:10.1111/j.1475-4762.2007.00773.x
Knill, C. and Lenschow, A., 2003. Modes of regulation in the governance of the EuropeanUnion. European Integration online Papers (EIoP), 7, 1–20.
Lynn, L., 1997. Innovation and the public interest. In: A. Altshuler and R. Behn, eds.Innovation in American Government. Washington, DC: Brookings Institute, 83–103.
Mickwitz, P., 2012. Policy evaluation. In: A.J. Jordan and C. Adelle, eds. Environmentalpolicy in the European Union. 3rd ed. Routledge: London, 267–286.
Mickwitz, P. and Birnbaum, M. 2009. Key insights for the design of environmentalevaluations. In: M. Birnbaum and P. Mickwitz, eds. Environmental program and policyevaluation: addressing methodological challenges. New Direction for Evaluation, 122,105–112.
Öko-Institut, et al.,2012. Ex-post quantification of the effects and costs of policies andmeasures CLIMA. Berlin, Öko-Institut, A.3/SER/2010/0005 Final Report.
Ostrom, E., 2010. Polycentric systems for coping with collective action and global environ-mental change. Global Environmental Change, 20 (4), 550–557. doi:10.1016/j.gloenvcha.2010.07.004
Pallemaerts, M. and Williams, R., 2006. Climate change: the international and Europeanpolicy framework. In: M. Peeters and K. Deketalaere, eds. EU climate change policy.Cheltenham: Edward Elgar, 22–50.
Polsby, N., 1984. Political innovation in America. New Haven: Yale University Press.Rayner, T. and Jordan, A., 2013. The European Union: the polycentric climate policy
leader? WIRES Interdisciplinary Reviews: Climate Change, 4 (2), 75–90.Rich, R., 1991. Knowledge creation, dissemination, and utilization. Knowledge: The
International Journal of Knowledge Transfer and Utilization, 12 (3), 319–337.Rogers, E., 2003. Diffusion of Innovations. 5th ed. New York: Free Press.Schaffrin, A., Sewerin, S., and Seubert, S., 2014. The innovativeness of national policy
portfolios: climate policy change in Austria, Germany and the UK. EnvironmentalPolitics, 23, 5. doi:10.1080/09644016.2014.924206
904 M. Hildén et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
Stame, N., 2008. The European project, federalism and evaluation. Evaluation, 14 (2),117–140. doi:10.1177/1356389007087535
UNFCCC, 2000. Guidelines for the preparation of national communications by Partiesincluded in Annex I to the Convention, Part II. UNFCCC guidelines on reporting andreview. FCCC/CP/1999/7.
UNFCCC, 2006. Updated UNFCCC reporting guidelines on annual inventories followingincorporation of the provisions of decision 14/CP.11. Nairobi Kenya, UNFCCC, No.FCCC/SBSTA/2006/9, SBSTA.
UNFCCC, 2011. Compilation and synthesis of fifth national communications (No. FCCC/SBI/2011/INF.1/Add.1). Bonn: UNFCCC.
Vedung, E., 2006. Evaluation research. In: B.G. Peters and J. Pierre, eds. Handbook ofpublic policy. London: Sage, 397–417.
Volden, C., 2006. States as Policy Laboratories: Emulating Success in the Children’sHealth Insurance Program. American Journal of Political Science, 50 (2), 294–312.doi:10.1111/j.1540-5907.2006.00185.x
Voß, J.-P. and Simons, A., 2014. Instrument constituencies and the supply side of policyinnovation: the social life of emissions trading. Environmental Politics, 23 (5).doi:10.1080/09644016.2014.923625
Walker, J., 1969. The diffusion of innovations among the American states. AmericanPolitical Science Review, 63 (3), 880–899. doi:10.2307/1954434
Wuppertal Institute, Ecofys, ESV - O.Ö. Energiesparverband, 2013. Improving andimplementing national energy efficiency strategies in the EU (Final Report).EEWPublication. (Energy Efficiency Watch-project).
Environmental Politics 905
Dow
nloa
ded
by [
Uni
vers
ity o
f E
ast A
nglia
Lib
rary
] at
02:
45 2
4 Se
ptem
ber
2014
