Forest law enforcement through district blacklisting in ......Forest law enforcement through district blacklisting in the Brazilian Amazon Elías Cisneros 1, Sophie Lian Zhou 2, Jan

Forest law enforcement through district blacklisting in the Brazilian Amazon

Elías Cisneros1, Sophie Lian Zhou2, Jan Börner3

1Center for Development Research - University of Bonn, Walter-Flex-Str. 3, 53113 Bonn, Germany.

2Institute for Food and Resource Economics - University of Bonn, Nussallee 21, 53115 Bonn, Germany.

3Center for Development Research - University of Bonn, and Center for International Forestry Research (CIFOR), Walter-Flex-Str. 3, 53113 Bonn, Germany. Corresponding author: [email protected]

Abstract

Deforestation in the Brazilian Amazon has dropped substantially after a peak at over 27 thousand square kilometers in 2004. Starting in 2008, the Brazilian Ministry of the Environment has regularly published blacklists of critical districts with high annual forest loss. Farms in blacklisted districts face stricter registration and environmental licensing rules. In this paper, we quantify the impact of blacklisting on deforestation. We first use spatial matching techniques using a large set of covariates to identify appropriate control districts. We then explore the effect of blacklisting on change in deforestation in double difference regression analyses using panel data covering the period from 2002-2012. Several robustness checks are conducted including an analysis of field-based enforcement missions as a potential causal mechanism behind the effectiveness of the blacklist. We find that the blacklist has considerably reduced deforestation in the affected districts even after controlling for in situ enforcement activities.

Keywords: deforestation, impact evaluation, matching

JEL codes: Q32, Q15, Q38, Q5, H43

1

1. Introduction

Brazil stands out as one of the few countries in the world, where tropical deforestation rates have

dropped over the past decade (Hansen et al. 2013). Emerging evidence from semi-experimental

evaluation studies on the effectiveness of Brazil’s post-2004 strategy to combat Amazon

deforestation unambiguously suggests that environmental policy has come to play a major role in

determining land use decisions in the region (J. Assunção et al. 2012; CEPAL-IPEA-GIZ 2011;

Hargrave and Kis-Katos 2013). Apart from a substantial expansion of the region’s protected area

network, field-based law enforcement operations targeted to deforestation hot-spots by using

improved remote sensing technologies have been among the major short-term success factors

(Juliano Assunção et al. 2013a). Between late 2007 and early 2008, Brazil has introduced two

additional measures to reinforce in situ enforcement action. Resolution 3.545 published in 2008

by the Brazilian Monetary Council (Conselho Monetário Nacional) limits credit access for farms

that are non-compliant with the Brazilian Forest Code and establishes best-practice rules for

offenders to re-access credit flow. Assuncao et al. (Juliano Assunção et al. 2013b), estimate that

this measure has avoided 2700 square kilometers of deforestation between 2009 and 2011. The

Presidential Decree 6.321 (December 2007) created the legal basis for a list of priority

municipalities, henceforth districts, with outstanding historical deforestation rates. In

“blacklisted” districts, stricter rules with regard to the authorization of forest clearing applied and

defined administrative targets (see details below) had to be fulfilled to qualify for removal from

the list.

Both decrees essentially operate as cross-compliance measures, where access to public services

or administrative rights at farm or district level is made conditional on compliance with forest

law. In this paper we apply semi-experimental evaluation techniques to gauge the role that

district blacklisting has played in the overall contribution of Brazil’s policy mix to combat

Amazon forest loss. We find that, on average, blacklisted districts have experienced distinctly

larger reductions in deforestation than comparable non-listed districts and produce evidence that

this difference is partially a genuine effect of blacklisting.

The paper is structured as follows. Below, we describe key elements of the Brazilian blacklisting

strategy. We also discuss the potential mechanisms and pathways through which blacklisting

might have contributed to reducing deforestation beyond the combined effect of other policy

2

instruments (theory of change). Section 2 summarizes our empirical strategy to estimate the

effect of blacklisting on deforestation. Section 3 documents our data sources and section 4

presents main results and robustness checks. In section 5 we discuss potential caveats of our

analysis in the context of the emerging literature evaluating conservation programs and section 6

provides conclusions and implications for conservation policy design.

History and impact logic of the Brazilian district blacklist

Decree 6.321, published in December 2007, clearly defines the objective of the blacklist as a

strategy to monitor and control illegal deforestation and prevent land degradation. It states that

the list is to be updated annually based on official deforestation statistics and specifies the

complementary roles of IBAMA and the National Institute for Agrarian Reform (INCRA) in

monitoring and registering landholdings in the blacklisted districts. Three criteria are put forward

as being used (without further specification) to compose the blacklist, namely:

1. The total deforested area

2. The total deforested area of the preceding three years

3. The increase of deforestation of minimum three out of the past five years

Figure 1 schematically depicts how the blacklist has evolved since the publication of Decree

6.321.

[Figure 1. History of district blacklisting and blacklist criteria. Positive numbers in parentheses depict additions to the blacklist. Negative numbers depict removals. ]

In January 2008, the first blacklist was published covering 36 districts. Seven districts were

added in each of the years 2009 and 2011. Only six districts were removed until 2012. Removal

was conditioned on registering at least 80% of the eligible area (mostly privately claimed land)

under the CAR. Moreover, annual deforestation had to be kept below 40 sqkm.

3

District blacklisting probably qualifies as the most innovative element in Brazil’s multi-

instrument conservation policy mix. To our knowledge no other country has yet applied a similar

institutional cross-compliance mechanism in the forestry sector. The impact pathway of

blacklisting is still unclear and very little research on blacklisting as a governance mechanism

exists. Jacobs and Anechiarico (1992) argue that contractor blacklisting is a sensible and

ethically justifiable strategy to protect government organizations from fraud. China has

experimented with an environmental disclosure policy including the publication of lists of

violators of environmental regulations. A recent study found that this blacklisting strategy has

helped in engaging civil society stakeholders in environmental governance (Tan 2014). The

study, however, concluded that effects on behavioral change have been limited due to the

country’s authoritarian structure. In 2010, a synthesis report by the Transparency and

Accountability Initiative found that transparency and accountability policies have considerable

potential to make a improve governance in sectors, such as public service delivery, natural

resource governance, and donor aid (McGee and Gaventa 2010). Similar findings on public

disclosure policies are

2. Empirical Strategy

The methodological challenge of evaluating the effect of the blacklist on deforestation in the

blacklisted districts consists of identifying an appropriate counterfactual scenario of what would

have happened in the absence of the blacklist (Khandker et al. 2010). From the previous section,

we know that blacklisting was not random. Instead, regulators have used defined selection

criteria that were all linked to historical deforestation. Regression Discontinuity Design (RDD) is

a commonly used evaluation technique for interventions were the selection mechanism is known

(Hahn et al. 2001). Unfortunately, the exact approach used to arrive at the published blacklists

was never made public. Although past deforestation highly correlates with selection, it is not

possible to reproduce the first list of 38 districts based on the three published selection criteria

alone. We can thus only speculate, which other criteria could have played a role in composing

the blacklist. Moreover, our sample of treated districts is too small for informative local linear

regression analyses in an RDD.

4

A frequently used quasi-experimental evaluation technique in the presence of unknown selection

mechanisms is matching (Andam et al. 2008; Gaveau et al. 2009; Ho et al. 2007; Honey-Rosés et

al. 2011; Paul R. Rosenbaum and Rubin 1983). Matching relies on propensity scores or other

distance measures that are derived from observed characteristics of treated and non-treated

observations (here districts). Treated observations are paired with “similar” non-treated (or

control) observations to reduce the bias in treatment effect estimations. A strong assumption of

the matching estimator is unconfoundedness, i.e. one assumes that no other than the observed

criteria were relevant in selecting districts into the blacklist. Moreover, matching requires that

there is a considerable region of overlap in the distance measures or propensity scores of treated

and non-treated observations of the sample. While we are able to control for a large number of

potential selection criteria (see below), our sample of non-blacklisted districts is unlikely to be a

satisfactory pool of potential controls, because most blacklisted districts have indeed been among

the highest deforesting districts in the Brazilian Amazon region before the blacklist was enacted.

Matching can, nonetheless, help us to identify an appropriate set of control observations and thus

represents a sensible preprocessing step in our evaluation strategy (Ho et al. 2007).

Since the group of potential control districts is likely to exhibit lower pre-treatment levels in

deforestation than the treated districts we will rely on the double difference method to ultimately

estimate the treatment effect of blacklisting (Hargrave and Kis-Katos 2013; Khandker et al.

2010). A critical assumption of the double difference method is that treated and control

observations exhibit parallel time trends in the outcome variable (time invariant heterogeneity).

In other words, absent blacklisting, we assume that treated and control districts would have had

the same change in deforestation over time even though they exhibit different absolute levels in

forest loss. While we cannot test whether this assumption holds, it is possible to explore the

implications of some forms of violations in robustness tests (see below).

Following (Jalan and Ravallion 1998), we derive the double difference estimator for our purpose

as follows. Using log deforestation as outcome variable, the panel fixed effect can be written as:

ittiiititit utZXBDef +++++= ηαδγβ '''ln Eq. (1)

where itB is treatment variable indicating whether the municipality � has been blacklisted in a

given year t , itX the vector for time-varying covariates, iZ the vector for time-invariant

5

covariates (or the so-called “initial conditions”), iα is the municipality-specific fixed effect, tη

the year-specific treatment effect, and itu the error term. Initial conditions are interacted with the

time variablet . We are interested in the average treatment effectβ .

Both fixed effect and first difference estimators can be used, but we proceed with the first

difference estimator that is less prone to serial correlation (Verbeek 2012, pg. 349). Taking first

differences, Eq. 1 becomes:

ittiititit uZXBDef ∆+∆++∆+∆=∆ ηδγβ '''ln Eq. (2)

where the municipality-specific fixed effect is canceled out and the initial conditions stay in the

equation as time-invariant covariates ( 1=∆t ). Other than in (Jalan and Ravallion 1998), where a

single time trend is assumed for all periods, we assume year-specific fixed effects.

Our timeframe of analysis covers all years between 2002 and 2012. Deforestation is measured

over the period from August and July and we adjust all explanatory variables accordingly.

Treatment indicators have to account for the fact that blacklists were released at different points

in the year. The first list of 38 districts was published in February 2008. Hence, we set treatment

itB to 0.5 to represent the six months during which blacklisting could have affected deforestation

in 2012. The second and third blacklists were published in April 2009 and May 2011 and the

respective treatments are set to 0.25 and 0.17 (see Eq. 3 below). The 4th list was published in

October 2012 and thus outside our analytical timeframe.

=otherwise

ngblacklisti after yearssubsequent all and 2nd in

ngblacklisti of year1st in

0

1

)1,0(

itB Eq. (3)

The treatment coefficient β measures the average change in deforestation due to blacklisting for

all years after treatment. Hence, we initially assume a constant influence of blacklisting

throughout the timeframe of analysis, but will later also analyze dynamic effects.

Confounding factors that could affect deforestation are considered in the covariates vectors itX

and iZ of Eq. 2. Our choice of covariates is based on previous empirical work on tropical

6

deforestation in the Amazon region and beyond (Aguiar et al. 2007; Andersen 1996; Araujo et al.

2009; Arima et al. 2007; Hargrave and Kis-Katos 2013; Kaimowitz and Angelsen 1998; Pfaff

1999). We broadly distinguish between (1) time invariant, (2) time varying, and (3) mechanisms

with potential effects on our outcome variable (deforestation). Since clouds represent a

significant source of measurement error in remotely sensed deforestation data, we include cloud

cover in all regression analysis and report the respective coefficient estimates. Descriptive

statistics of all variables used in regressions are reported in Table 2 below.

Among time invariant covariates, we consider various measures of deforestation and forest cover

up until the beginning of our 2002-2012 time frame and control for district size and population

density. Moreover, we control for farm characteristics, indicators of agricultural intensification,

average land values, and average travel distance to district cities, which have shown to be

important predictors of deforestation in previous studies (Angelsen and Kaimowitz 2001; Pfaff

1999).

Among time varying predictors, we consider GDP per capita, timber and soy prices (zero in

districts without soy production) (see also, Hargrave and Kis-Katos 2013), and the area of

settlements, protected areas, and indigenous territories in each district. All these tenure

categories have been found to affect deforestation rates in previous studies (Ezzine-de-Blas et al.

2011; Soares-Filho et al. 2010). In addition, we control for political factors by introducing

dummy variables indicating whether districts are governed by the Brazilian Workers Party

(dominating political party at federal level during most of the studied time frame). As

mechanism through which the blacklist could have affected deforestation we consider the

number of field-based inspections by the environmental protection agency registered in each

district per year.

The panel data models are implemented in R using the function “plm” from the “plm” package

(Croissant and Millo 2008; R Core Team 2012). For post-matching regressions we run a placebo

analysis to test whether results are simply an artifact of selection bias. Placebo tests are reported

in Appendix (SI Table 1.1).

Details on the approaches used to analyze, dynamic treatment effects, spatial spillovers, and

causal mechanism effects are provided in the respective subsections below.

7

3. Study area and data

Our study area is the Legal Brazilian Amazon, an area of approximately five million square

kilometers that extends into nine Brazilian states. Figure 2 depicts the study area highlighting

changes in average deforestation in blacklisted and non-blacklisted districts after the cut-off

point in 2008, when the Decree 6.321 was enacted.

[Figure 2 here]

From Figure 2 it becomes clear that the blacklisted districts have experienced the largest

reductions in average annual deforestation from the period 2003-2007 to the period 2008-2012.

Large increases in average deforestation almost exclusively occurred in non-blacklisted districts,

but many of the latter also experienced reductions in forest loss.

Table 1 summarizes the data sources used in this study. Table 2 presents descriptive statistics for

all variables used in for the empirical analysis. The Brazilian Legal Amazon district database

from the Brazilian Institute for Geography and Statistics (IBGE) covers 771 districts. To avoid

bias introduced by districts with no or negligible forest cover, e.g. in the Amazon/Cerrado

ecotone, we exclude 312 districts (none of which was blacklisted) by restricting the sample to

districts with a minimum initial forest coverage of 10% in 2002.

[Tables 1 + 2 here]

4. Results

Descriptive analysis and baseline regressions

Figure 3 depicts average deforestation (left panel) and average year-to-year increase in forest

loss for blacklisted and non-blacklisted districts during our study period. Average deforestation

in blacklisted districts exhibits a much faster decrease than deforestation in untreated districts,

but substantial decreases already occurred before the blacklist was enacted in 2008, for example

between 2004 and 2005. The right panel of Figure 3 shows that average year-to-year percentage

changes in deforestation were constantly lower in blacklisted than in control districts after 2006.

8

[Figure 3]

We, nonetheless, start our analysis with all observations in a series of baseline models using the

specification in Eq. 2 and gradually adding covariate groups (Table 3).

[Table 3 here]

All models are balanced panels, but due to missing values in some time invariant variables (see

Table 2) some observations are dropped in models (2-4). In model (3) we have to omit the year

2012 and model 4 discards both the years 2012 and 2002.

All four models yield similar results with large and highly significant average treatment effects.

The two-sided Durbin Watson test for serial correlation indicates serial correlation only for

model (1). Not shown in Table 3: Year effects are negative with the exception of 2008 and 2010

and among the time invariant covariates cumulated deforestation in 2002, tractor density, and

land value per hectare are negatively associated with change in deforestation. Among time

varying covariates, the timber price is negative and the settlement area positively associated with

deforestation. Hargrave and Kis-Katos (Hargrave and Kis-Katos 2013) report similar results with

regard to timber prices and argue that high value timber could boost long-term investment in

forest and therefore contribute to lower deforestation.

Model 4 includes the lagged number of field inspections as a potential external mechanisms

through which blacklisting could have affected deforestation. The coefficient is insignificant, but

the role of field inspections as a causal mechanism will be further investigated below.

Post-matching regressions

As discussed above, regression models tend to be less prone to misspecification and selection

bias when data is preprocessed using matching techniques. As matching covariates we use the

official blacklisting criteria reported by the Brazilian authorities (Figure 1) and further include a

large group of variables (including size and land use variables, economic conditions, and

conservation policies). Matching is implemented in R using the “Matching” package (Sekhon

2011) and the Mahalanobis distance measure. A comparison of the covariate balance before and

after matching is provided in Table SI 2.1 (Appendix). For almost all variables, the standard

9

mean difference has greatly improved after matching. However, significant imbalances still exist

and thus a simple comparison between the means of blacklisted and matched non-blacklisted

groups would likely be biased. Figure SI 2.1 in the Appendix compares average year-to-year

change in deforestation and average deforestation trends separately for blacklisted, matched non-

blacklisted, and un-matched non-blacklisted districts. After matching, treated and control

districts do exhibit very similar pre-blacklist deforestation trends, which makes us confident that

the critical assumptions for our subsequent double difference regression are likely to hold.

We use the matched dataset to re-estimate baseline models (3) and (4) in Table 3, which we

consider the most adequate specifications. Results are presented in Table 4. Note again that

model (4) only considers the years from 2003-2011.

[Table 4 here]

Using the matched dataset, both models show improved goodness of fit. At the same time, all

time invariant covariates cease to be significant. Among time varying covariates only timber

prices (negative sign) and the indicator variable for the district mayor’s term (positive) are

significant in both models. Both models also now suggest the same average treatment effect,

which corresponds to a 29% decrease in annual deforestation in blacklisted district as a result of

blacklisting.

We test our main results using alternative matching techniques. We compare the results from our

preferred matching approach to (1) a one-to-one matching on propensity scores, (2) a one-to-two

matching on the Mahalanobis distance, and (3) a one-to-one matching on the Mahalanobis

distance using only the official selection criteria to the blacklist. The blacklisting effect stays

significant are slightly higher in size (Results not presented here). Our preferred one-to-one

matching with replacement on the Mahalanobis distance with an extended set of covariates turns

out to be the most conservative version to estimate the effect of blacklisting.

Dynamic treatment effects

As discussed previously, several blacklists were published over time and some districts were

removed from the lists in the process. Laporte and Windmeijer (Laporte and Windmeijer 2005)

10

show how delayed response to treatment can lead to substantial differences in treatment effects

in the post-treatment periods. In this section we allow for dynamic treatment effects. We do not

consider anticipation effects prior to treatment though, because the period lying between the

publication of Decree 6,321 and the first blacklist was too short to have resulted in significant

effects on deforestation as measured by the INPE-PRODES program (see data sources Table 1).

To account for dynamic treatment effects we split the blacklisting dummy into several dummies

as follows:

ittiitititititit uZXBBBBDef ∆+∆++∆+∆+∆+∆+∆=∆ +++ ηδγββββ ''''''ln 3322110

Eq. (4)

itB is between 0 and 1 as for the year of blacklisting and zero for all subsequent years. The

treatment variables 1+itB , 2+itB and 3+itB are set to one only in the first, second and third year after

blacklisting respectively, for each blacklisted district. We thereby capture the effect of

blacklisting over the years. The treatment coefficients 0β to 3β can be interpreted as the average

effect of blacklisting on deforestation for the respective year after blacklisting. Results for

models (3) and (4) are shown in Table 5. Model (3) suggest that blacklisting has significant

effects on deforestation in all subsequent years. In model (4) only the coefficients for the second

and third year after blacklisting are significant. For the first year after blacklisting in model (4),

clustering standard errors at district level increases the p-value from 0.02 to 0.11. Overall, the

dynamic treatment effect are rather stable over time, i.e. considering standard errors individual

year effects are not significantly different from each other.

[Table 5 here]

Spatial spillover effects

Spatial spillover effects, such as leakage or deterrence, could bias our treatment effect

estimation. In our sample, 129 out of the 408 non-blacklisted districts have had at least one

blacklisted neighbor district. Leakage could take place if the blacklist encouraged deforestation

agents to move to neighboring non-blacklisted districts. However, it is also possible that the fact

of having a blacklisted neighbor district deters land users in non-blacklisted districts from

11

deforesting. In the case of leakage from blacklisted to neighboring non-blacklisted districts we

would overestimate the effect of blacklisting on deforestation, especially if these districts are part

of our matched set of control districts. If deterrence effects of blacklisting were leading to more

conservation in neighboring districts, we would underestimate the effect of blacklisting both in

blacklisted districts and at the regional scale.

We account for spatial leakage effects from blacklisting by introducing a neighboring treatment

effect as follows:

ittiitititit uZXNBBDef ∆+∆++∆+∆+∆=∆ ηδγϕβ ''''ln Eq. (5)

Our main interest lies in the effect of blacklisting on neighboring districts that have not been

blacklisted, ϕ . The neighbor effect itNB is set equal to one when a district is not blacklisted and

has at least one blacklisted neighbor and becomes zero otherwise. Consequently itB and itNB

are mutually exclusive, i.e. they can only be jointly zero but not jointly one. Table 6 reports

results for model specifications (3) and (4) as in the previous section for both the pre and the

post-matching data sets.

[Table 6 here]

We find evidence pointing to a significant spillover effect on non-blacklisted neighbors of

blacklisted districts in the unmatched sample (see left columns in Table 6). The negative sign of

the newly introduced neighbor dummy variable suggests that blacklisting a district may have

deterrence effects on deforestation also in neighboring non-blacklisted districts. The spillover

effect, however, ceases to be significant when we run the same model with the matched data set

(right columns in Table 6). The post-matching regression models do not capture the spillover

effect, because the matched control group consists predominantly of direct neighbors of

blacklisted districts. Most districts that are neither blacklisted nor have one or more blacklisted

neighbors are dropped in the process of matching. This finding suggests that the treatment effects

estimated in Tables 4 and 5 are probably biased, because matched control districts tend to be

neighbors of blacklisted districts. The bias, however, leads us to under rather than overestimate

the effect of blacklisting in blacklisted districts and the size of the coefficient for the “Neighbor

blacklisted” variable in Table 6 gives us an indication of the size of the bias.

12

Blacklisting and field-based enforcement

Above we have produced evidence that the drop in deforestation after 2007 was much more

pronounced in blacklisted districts than in other Amazonian districts. However, our analysis does

not allow for conclusions with respect to the causal mechanism behind the effect of blacklisting

on deforestation. In section 1 we have discussed potential impact channels or mechanism that

could have played a role in reinforcing the effectiveness of blacklisting. One of these

mechanisms is the practice of in situ field inspections that were shown to have played an

important role in Brazil’s efforts to reduce Amazon deforestation (Juliano Assunção et al. 2013a;

Hargrave and Kis-Katos 2013).

While we have controlled for the number of field inspections in model (4), our estimator may

still be biased if field inspections were actually affected by blacklisting (Paul R Rosenbaum

1984). To avoid this bias we need an empirical approach that allows us to determine (1) what the

number of field inspections would have been in the absence of blacklisting and (2) what the

effect of blacklisting would have been, had there not been any effect on field inspections. Based

on a method proposed by Flores and Flores-Lagunes (Flores and Flores-Lagunes 2009), Ferraro

and Hanauer (Ferraro and Hanauer 2014) have recently addressed similar questions in the

context of protected areas. The isolated effect of a mechanism is called the mechanism average

treatment effect (MATT). The remaining effect of blacklisting is called the net average treatment

effect (NATT).

Beyond the assumptions made up to this point, two additional assumptions are necessary to

estimate MATT and NATT: (1) Expectations that blacklisting will increase the density of field

inspections in blacklisted districts have not influenced selection onto the list, and (2), changes in

the number of field inspections have the same effect in districts where blacklisting has affected

the number of field inspections and in districts were it has not. The second assumptions could

theoretically be violated, for example, if field inspections in blacklisted districts would somehow

have been of a different nature than inspections in other districts. Our data does not contain any

information in that regard.

To gauge the potential mechanism effect of field inspections, we estimate the NATT with the

mechanism effect blocked (i.e. holding field inspections at the counterfactual level). The

13

difference between the overall average treatment effect measured above and the NATT is then

the mechanism effect of field inspections.

The implementation involves three steps:

1. We restrict the sample to the 50 blacklisted districts and run Model 5 for the post-

treatment period (2008-2011). This gives us a set of coefficients including the effect of

lagged fines on deforestation. I.e., in order to avoid a potential reverse causality between

deforestation and the number of issued fines, we use fines from the previous year (t-1).

2. Second, we set the number of field inspections in the blacklisted districts to

counterfactual levels, i.e. the number of fines from the matched paired non-blacklisted

districts to the blacklisted districts. All other variables keep their original values. With the

new values and the point estimates from step (1), we predict the counterfactual

deforestation level for the blacklisted districts. Under the assumptions made above, the

counterfactual deforestation represents the level of deforestation had there been no

change in field inspections as a result of blacklisting.

3. We re-estimate model (5) with the matched data set used in Table 4 and deforestation as

well as fine levels modified as described in step (1) and (2) to arrive at the NATT of

blacklisting.

This last step is done for both immediate treatment effect and the dynamic treatment effect

model.

Results are reported in Table 7.

[Table 7 here]

As expected, the net treatment effect estimates for blacklisting in Table 7 are significant (with

p=0.104 for the variable “blacklisted in t=0”) and smaller than the “gross” treatment effects in

Tables 5 and 6. Due to the size of the standard errors, however, we cannot safely conclude that

there is a significant post-treatment mechanism effect of field inspections that would bias our

average treatment effect estimations. While we acknowledge the possibility of such an effect, we

believe it is unlikely that it dominates in the overall effect of blacklisting.

14

5. Discussion

We have found a robust and strongly significant negative effect of district blacklisting on

deforestation. As we discuss in the introduction, there are several potential pathways, through

which we could theoretically explain this result. Given data limitations, we were only able to

formally test for the role of field-based enforcement missions as a potential causal mechanism

behind blacklist effectiveness. Inspections, however, turned out to be less important in

explaining deforestation reductions in the blacklisted districts than we had expected.

Administrative disincentives, reputational risk, and positive external support thus remain as

potentially jointly effective causal mechanisms behind the effect of the Brazilian blacklist that

could be explored in further research.

As potential rival explanation for our findings we have to consider the credit restriction imposed

by the Brazilian Monetary Council in the same year, in which the first blacklist was published

(Juliano Assunção et al. 2013b). Note however, that the credit policy covered the whole

Brazilian Amazon biome, where also most of our matched control districts are located. Only 5

control districts (and 7 blacklisted districts) extend into the part of the Legal Brazilian Amazon

that is not considered Amazon biome.

Like any quasi-experimental evaluation, our analysis remains prone to unobservable bias. One

important potential source of bias would, however, lead us to under rather than overestimate the

conservation effect of blacklisting. Since blacklisting is endogenously determined by

deforestation, a naïve comparison of deforestation rates (see Figure 3) clearly suggests higher

deforestation in blacklisted than in non-blacklisted districts. Due to limited common support, this

bias could not be fully corrected for by matching, which is why we only rely on matching as a

pre-processing technique (Ho et al. 2007).

On the other hand, we would indeed overestimate the negative effect of blacklisting on

deforestation, had blacklisted districts exhibited a faster decrease in deforestation in the

unobserved counterfactual scenario than the non-listed control districts (parallel time trend or

time invariant heterogeneity assumption). A related common evaluation pitfall, termed

“Ashenfelter’s or pre-program dip”, can occur if selection is affected by unusual pre-program

changes in the outcome variable (Heckman and Smith 1999). In our case, a pre-blacklist peak in

15

deforestation could hypothetically have resulted in a selection of districts that would have

exhibited much faster decreases in deforestation - even in the absence of blacklisting - than any

potential control district. While we cannot completely rule out such a phenomenon, we argue that

it is unlikely to play a major role in explaining our findings. First, because we control for past

increases in deforestation rates in our matching exercise and both pre and post-matching

differences in the number of increases in deforestation in the period 2002-2007 are rather small.

Second, because the blacklist was enacted five years after average deforestation had peaked in

the blacklisted districts (see Figure 3). In the two years prior to the publication of the blacklist,

deforestation trends had instead been remarkably similar in treated and control districts. And

third, the blacklisted districts have been leading deforestation rankings even prior to our

observation period. Hence, and as supported by our placebo treatment analysis (Table SI 1.1), the

substantial drop in average forest loss in these districts after 2008 can hardly be attributed solely

to normalization after an unusual peak.

We are thus confident that our analysis correctly identifies the blacklist as an environmental

governance measure that made a substantial complementary contribution to bringing

deforestation down in the Brazilian Amazon region.

6. Conclusions

In this study we have used a quasi-experimental evaluation design to gauge the potential

contribution of district blacklisting to the drop in deforestation rates in the Brazilian Amazon.

Blacklisting has been used in other environmental governance contexts (McGee and Gaventa

2010), but we are unaware of any attempt at quantifying the effect of blacklisting through

counterfactual-based evaluation.

We find that the average effect of blacklisting on deforestation in blacklisted districts ranges

between roughly 14-36%. Based on the average own treatment effect estimated by model (3) in

Table 5, this corresponds to an absolute reduction in deforestation of roughly 4500 sqkm

between 2008-2012. While this is less than the cumulated effects of improved field-based

enforcement calculated by (Juliano Assunção et al. 2013a; Hargrave and Kis-Katos 2013), it is

more than the amount of avoided deforestation (2700 sqkm) that (Juliano Assunção et al. 2013b)

attribute to the credit restrictions that were enacted in the same year as the blacklist.

16

In other words, until 2012, the decision to bolster the Brazilian anti-deforestation campaign by

district blacklisting has avoided almost 80% of one year’s deforestation in the whole Brazilian

Amazon, where annual deforestation rates have been fluctuating around 5600 sqkm since 2011.

At federal level, the incremental administrative costs of maintaining the blacklist have probably

been low. However, the blacklist has reportedly induced a substantial amount of local level

transaction costs and operational expenses by supporting NGO and state-level government

organizations. Putting a price tag on the Brazilian blacklisting experience is thus not a

straightforward exercise.

Given the scarce evidence on the effectiveness of transparency and accountability measures, our

results should nonetheless encourage experimentation with blacklisting as a complementary

forest conservation measure. Clearly, a country’s administrative structure is likely to affect

outcomes in significant ways. For example, Brazilian districts (i.e. municipalities) do not have

environmental policy mandates as opposed to the more decentralized governance structure in

other tropical forest countries, such as Indonesia (Luttrell et al. 2014). The effectiveness of the

diverse potential impact channels of blacklisting may thus differ substantially depending on the

ability of local stakeholders to organize towards the goal of being removed from a blacklist.

From a national government’s point of view, including in the context of an international

mechanism to Reducing Emissions from Deforestation and Degradation (REDD+), blacklisting

would appear as a low-cost and no-regret option to increase compliance with forest law.

Acknowledgements

We thank the participants of the International Workshop “Evaluating Forest Conservation

Initiatives: New Tools and Policy Needs”, 10-12th December 2013 in Barcelona, Spain, for

discussions and comments that have helped in developing this study. Comments from three

anonymous reviewers for the ICAE 2015 have helped to improve this manuscript.

References

Agropecuário, IBGE Censo (2006), 'Instituto Brasileiro de Geografia e Estatística'.

17

Aguiar, Ana Paula Dutra, Câmara, Gilberto, and Escada, Maria Isabel Sobral (2007), 'Spatial statistical analysis of land-use determinants in the Brazilian Amazonia: Exploring intra-regional heterogeneity', Ecological Modelling, 209 (2–4), 169-88.

Andam, Kwaw S., et al. (2008), 'Measuring the effectiveness of protected area networks in reducing deforestation', Proceedings of the National Academy of Sciences, 105 (42), 16089-94.

Andersen, Lykke E. (1996), 'The Causes of Deforestation in the Brazilian Amazon', The Journal of Environment & Development, 5 (3), 309-28.

Angelsen, A. and Kaimowitz, D. (2001), Agricultural technologies and tropical deforestation (New York, Oxon: CIFOR, CABI Publishing ).

Araujo, Claudio, et al. (2009), 'Property rights and deforestation in the Brazilian Amazon', Ecological Economics, 68 (8–9), 2461-68.

Arima, Eugenio Y., et al. (2007), 'FIRE IN THE BRAZILIAN AMAZON: A SPATIALLY EXPLICIT MODEL FOR POLICY IMPACT ANALYSIS*', Journal of Regional Science, 47 (3), 541-67.

Assunção, J. , Gandour, C. C., and Rocha, R (2012), 'Deforestation Slowdown in the Legal Amazon: Prices or Policies?', (Rio de Janeiro: Climate Policy Initiative, Pontifical Catholic University of Rio de Janeiro).

Assunção, Juliano, Gandour, C. C., and Rocha, Romero (2013a), 'DETERring Deforestation in the Brazilian Amazon: Environmental Monitoring and Law Enforcement', (Climate Policy Initiative, Rio de Janeiro, Brazil).

Assunção, Juliano, et al. (2013b), 'Does Credit Affect Deforestation? Evidence from a Rural Credit Policy in the Brazilian Amazon', in Climate Policy Initiative (ed.), (Rio de Janeiro, Brazil).

CEPAL-IPEA-GIZ (2011), 'Avaliação do Plano de Ação para a Prevenção e Controlo do Desmatamento da Amazônia Legal - PPCDAm - 2007-2010', (Brasilia: CEPAL, IPEA, GIZ).

Croissant, Yves and Millo, Giovanni (2008), 'Panel data econometrics in R: The plm package', Journal of Statistical Software, 27 (2), 1-43.

Eleitoral, Tribunal Superior 'Tribunal Superior Eleitoral - Estatísticas Eleitorais'. Ezzine-de-Blas, Driss, et al. (2011), 'Forest loss and management in land reform settlements: Implications

for REDD governance in the Brazilian Amazon', Environmental Science & Policy, 14 (2), 188-200. Ferraro, Paul J and Hanauer, Merlin M (2014), 'Quantifying causal mechanisms to determine how

protected areas affect poverty through changes in ecosystem services and infrastructure', Proceedings of the National Academy of Sciences, 111 (11), 4332-37.

Flores, Carlos A and Flores-Lagunes, Alfonso (2009), 'Identification and estimation of causal mechanisms and net effects of a treatment under unconfoundedness', (IZA Discussion Papers).

Gaveau, David L. A., et al. (2009), 'Evaluating whether protected areas reduce tropical deforestation in Sumatra', Journal of Biogeography, 36 (11), 2165-75.

Hahn, Jinyong, Todd, Petra, and Van der Klaauw, Wilbert (2001), 'Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design', Econometrica, 69 (1), 201-09.

Hansen, M. C., et al. (2013), 'High-Resolution Global Maps of 21st-Century Forest Cover Change', Science, 342 (6160), 850-53.

Hargrave, Jorge and Kis-Katos, Krisztina (2013), 'Economic Causes of Deforestation in the Brazilian Amazon: A Panel Data Analysis for the 2000s', Environmental and Resource Economics, 54 (4), 471-94.

Heckman, James J and Smith, Jeffrey A (1999), 'The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies', The Economic Journal, 109 (457), 313-48.

Ho, Daniel E., et al. (2007), 'Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference', Political Analysis, 15 (3), 199-236.

18

Honey-Rosés, Jordi, Baylis, Kathy, and Ramírez, M. Isabel (2011), 'A Spatially Explicit Estimate of Avoided Forest Loss', Conservation Biology, 25 (5), 1032-43.

IBAMA 'Instituto Brasileiro do Meio Ambiente e dos Recursos Naturais Renováveis - Sistema Compartilhado de Informacoes Ambientais'.

IBGE-PAM 'Instituto Brasileiro de Geografia e Estatística - Produção Agrícola Municipal'. IBGE-PEVS 'Instituto Brasileiro de Geografia e Estatística - Produção da Extração Vegetal e da

Silvicultura'. IBGE 'Instituto Brasileiro de Geografia e Estatística - Índice Nacional de Preços ao Consumidor Amplo -

IPCA'. --- 'Instituto Brasileiro de Geografia e Estatística - Geociencias'. --- 'Instituto Brasileiro de Geografia e Estatística - Produto Interno Bruto dos Municípios '. --- (2000), Censo Demografico. INCRA 'Instituto Nacional de Colonização e Reforma Agrária - Avervo Fundiário'. INPE-PRODES 'Instituto Nacional de Pesquisa Espaciais / Projeto PRODES - Monitoramento da floresta

amazônica brasileira por satélite'. Jacobs, James B and Anechiarico, Frank (1992), 'Blacklisting public contractors as an anti-corruption and

racketeering strategy', Criminal Justice Ethics, 11 (2), 64-76. Jalan, Jyotsna and Ravallion, Martin (1998), 'Are there dynamic gains from a poor-area development

program?', Journal of Public Economics, 67 (1), 65-85. Kaimowitz, D. and Angelsen, Arild (1998), Economic Models of Tropical Deforestation: A Review (Bogor,

Indonesia: Center for International Forestry Research). Khandker, S.H., Koolwal, G.B., and Samad, F.A. (2010), Handbook on impact evaluation: quantitative

methods and practices (Washington DC The World Bank). Laporte, Audrey and Windmeijer, Frank (2005), 'Estimation of panel data models with binary indicators

when treatment effects are not constant over time', Economics Letters, 88 (3), 389-96. Luttrell, Cecilia, et al. (2014), 'The political context of REDD+ in Indonesia: constituencies for change',

Environmental Science & Policy, 35, 67-75. McGee, Rosemary and Gaventa, John (2010), 'Synthesis report: Review of impact and effectiveness of

transparency and accountability initiatives', Available at SSRN 2188139 (Cambridge, UK: Transparency and Accountability Initiative), 56.

Nelson, A. 'Travel time to major cities: A global map of Accessibility.', Global Environment Monitoring Unit - Joint Research Centre of the European Commission. .

Pfaff, Alexander S. P. (1999), 'What Drives Deforestation in the Brazilian Amazon?: Evidence from Satellite and Socioeconomic Data', Journal of Environmental Economics and Management, 37 (1), 26-43.

R Core Team (2012), 'R: A Language and Environment for Statistical Computing', (Vienna, Austria: R Foundation for Statistical Computing).

Rosenbaum, Paul R (1984), 'The consquences of adjustment for a concomitant variable that has been affected by the treatment', Journal of the Royal Statistical Society. Series A (General), 656-66.

Rosenbaum, Paul R. and Rubin, Donald B. (1983), 'The central role of the propensity score in observational studies for causal effects', Biometrika, 70 (1), 41-55.

Sekhon, J. (2011), 'Multivariate and propensity score matching software with automated balance optimization: The matching package for R.', Journal of Statistical Software, 42 (7), 1-52.

Soares-Filho, Britaldo, et al. (2010), 'Role of Brazilian Amazon protected areas in climate change mitigation', Proceedings of the National Academy of Sciences, 107 (24), 10821-26.

Tan, Yeling (2014), 'Transparency without Democracy: The Unexpected Effects of China's Environmental Disclosure Policy', Governance, 27 (1), 37-62.

19

Uniao, Diário Official da 'Decree 6.321/2007 and Portaria No 28/2008, Portaria No 102/2009, Portaria No 66/2010 , Portaria No 138/2011, Portaria No 139/2011, Portaria No 175/2011, Portaria No 187/2012'.

Verbeek, Marno (2012), A Guide to Modern Econometrics (John Wiley & Sons, Chichester).

20

Tables and Figures

Table 1: Data sources Variable Year(s) Source Blacklist additions and removals

2008-2012 Decree 6.321/2007 and Provision 28/2008, Provision 102, 203/2009, Provision 66,67,68/2010 , Provision 138, 139, 175/2011, Provision 187,322,323,324/2012 (Uniao)

Deforestation and clouds 2002-2012 INPE-PRODES (INPE-PRODES) Municipality list and borders 2007 IBGE (IBGE) Protected areas 2002-2012 IBAMA (IBAMA) Indigenous areas 2002-2012 IBAMA (IBAMA) Settlement areas 2002-2012 INCRA (INCRA) Mayors’ party affiliation 2002-2012 TSE (Eleitoral) IPCA price deflator 2002-2012 IBGE (IBGE) Soy prices 2002-2012 IBGE-PAM (IBGE-PAM) Timber prices 2002-2012 IBGE-PEVS (IBGE-PEVS) GDP 2002-2011

IBGE (IBGE) IBGE Agricultural Census (Agropecuário 2006)

Number of farms 2006 Share of land owners 2006 Land value per ha 2006 Number of tractors 2006 Cattle stocking rate 2006 Population 2007 IBGE Demographic Census(IBGE 2000) Average distance to district center

Nelson (Nelson 2008)

Field-based law enforcement inspections

2002-2010 Hargave and Kis-Katos (Hargrave and Kis-Katos 2013) and Börner et al. (submitted - this issue)

Table 2: Descriptive statistics of variables (2002-2012) used in empirical analyses

Variable N Mean St. Dev.

Min Max

Dependent

ln deforestation 5,038 2.21 1.49 0.00 7.18

Treatment

blacklisted 5,038 0.04 0.19 0.00 1.00

Correction for measurement errors

ln clouds (sqkm) 5,038 1.88 2.70 0.00 10.89

21

Time invariant

ln deforested area in 2002 (sqkm) 5,038 19.92 2.68 0 23.05

ln district total area (sqkm) 5,038 8.29 1.34 4.16 11.98

ln forest area in 2002 (sqkm) 5,038 7.42 1.84 3.37 11.92

ln area under farms in 2005 (ha) 4,950 11.4 1.36 4.73 14.19

ln population density in 2007 (persons/sqkm)

5,038 1.43 1.41 -2.36 7.05

ln farm density in 2005 (farms/sqkm)

4,950 -1.57 1.49 -6.13 2.51

ln share of small farms in 2005 (%)

4,950 -0.41 0.39 -3.8 -0.01

ln tractors in 2005(units per district)

4,950 0.12 0.23 0 2.18

ln stocking rate in 2004 (heads/ha of pasture land)

4,917 0.15 0.81 -6.95 3.46

ln share of land owners in 2005 (%)

4,950 4.23 0.45 1.5 4.61

ln land value in 2005 (BRL/ha) 4,928 6.83 0.8 4.38 8.92

ln average distance to district center (hours)

4,950 6.19 0.97 2.61 8.51

Time varying1

GDP per capita (BRL/capita) 4,580 94.72 110.02 13.14 1,501.61

Soy price (BRL/ton) 5,038 0.001 0.003 0 0.02

Timber price (BRL/cubic meter) 5,038 0.97 0.9 0 9.6

ln indigenous area (skqm) 5,038 2.84 3.57 0 11.43

ln multiple use protected area (skqm)

5,038 2.76 3.51 0 10.77

ln strictly protected area (skqm) 5,038 1.46 2.91 0 10.5

ln settlement area (skqm) 5,038 4.91 2.68 0 10.21

party affiliation (binary) 5,038 0.11 0.3 0 1

Mechanisms

ln field inspections in t-1 (Number)

4,122 1.55 1.38 0.00 6.32

22

1Monetary figures are in 2012 Brazilian Reais (BRL), 1 BRL corresponded to USD 0.56 on

average in 2012 (www.oanda.com).

Table 3: Deforestation and blacklisted municipalities, full sample first difference regressions

Dependent variable:

log of deforestation

(1) (2) (3) (4)

Blacklisted -0.821*** -0.576*** -0.597*** -0.540***

(0.098) (0.105) (0.123) (0.124)

Log of cloud area 0.031*** -0.029*** -0.022*** -0.022***

(0.007) (0.006) (0.006) (0.006)

Year effects YES YES YES YES

Time invariant covariates YES YES YES

Time variant covariates YES YES

Number of field inspections YES

Observations 5,038 4,460 4,014 3,568

Adjusted R2 0.408 0.135 0.146 0.159

2-sided Durbin-Watson-Statistic 1.171 2.393 2.401 2.471

DW test p-value

23

Year effects YES YES

Time invariant covariates YES YES


Number of fines YES

Observations 900 800

Adjusted R2 0.300 0.302

2-sided Durbin-Watson-Statistic 2.566 2.623

DW test p-value 0.479 0.479

Note: The table reports first difference estimates with the dependent variable being the change in the log of yearly newly deforested area. Standard errors, clustered at district level, are reported in parentheses. Time invariant and variant covariates include first differences of the variables reported in Table 3. Observations are selected by a 1:1 closest neighbor matching on the Mahalanobis distance measure, with replacement. *,**,*** denote significance at the 10/5/1% level.

Table 5: Dynamic effects of blacklisting

Dependent variable:


(3) (4)

blacklisted in t+0 -0.495* -0.473

(0.290) (0.296)

blacklisted in t+1 -0.341** -0.328**

(0.140) (0.143)

blacklisted in t+2 -0.586*** -0.555***

(0.150) (0.155)

blacklisted in t+3 -0.375** -0.338

(0.189) (0.207)

Log of cloud area -0.495* -0.473

(0.290) (0.296)




Number of fines YES


Adjusted R2 0.308 0.309



24


Table 6: Spatial neighbor effects before and after matching

Dependent variable:


Before matching After matching

(3) (4) (3) (4)

Blacklisted -0.696*** -0.635*** -0.375* -0.368*

(0.128) (0.129) (0.199) (0.202)

Neighbor blacklisted

-0.229*** -0.214*** -0.091 -0.091

(0.069) (0.069) (0.169) (0.170)

Log of cloud area

-0.023*** -0.022*** 0.007 0.007

(0.006) (0.006) (0.013) (0.013)

Year effects YES YES YES YES

Time invariant covariates

YES YES YES YES

Time variant covariates

YES YES YES YES

Neighborhood characteristics

YES YES YES YES

Number of fines YES YES

Observations 4,014 3,568 900 800

Adjusted R2 0.150 0.163 0.301 0.302

2-sided Durbin-Watson-Statistic

2.422 2.481 2.570 2.628

DW test p-value 0.479 0.479 0.479 0.479


25

Table 7: Net average treatment effect of blacklisting

Dependent variable:


(5a) (5b)

Blacklisted -0.269*

(0.150)

Blacklisted in t+0

-0.479

(0.295)

Blacklisted in t+1

-0.294**

(0.142)

Blacklisted in t+2

-0.533***

(0.154)

Blacklisted in t+3

-0.311

(0.205)

Log of clouds 0.009 0.011

(0.013) (0.013)




No. of fines at counterfactual level YES YES


Adjusted R2 0.300 0.308




26

Figures

Figure 1: History of district blacklisting and blacklist criteria. Positive numbers in parentheses depict additions to the blacklist. Negative numbers depict removals.

27

Figure 2: Change in average deforestation (sqkm) in blacklisted and non-blacklisted districts comparing the years 2008-2012 and 2003-2007. Together the districts shown in the map represent the Legal Brazilian Amazon.

Figure 3: Deforestation trends in blacklisted (black lines) and non-blacklisted (grey lines) districts.

28

Appendix S1: Placebo regression

For placebo regressions we re-code the treatment variable as if blacklisting had started in 2006 as opposed to 2008. We then use models (3) and (4) with post-matching data to estimate the placebo treatment effect. As expected the placebo treatment variable is insignificant.

Table SI 1.1: Placebo post-matching first difference regressions

Dependent variable:


(3) (4)

blacklisted (as if in t-2) 0.013 0.043

(0.163) (0.170)

Log of cloud area 0.007 0.008

(0.013) (0.013)




Number of fines YES


Adjusted R2 0.297 0.299




29

Appendix S2

Table SI 2.1: Covariate balance before and after matching

Covariates Standardized differences in means

Before matching After matching

Official criteria total deforested areas 2007 1.189 0.759 deforestation 2005 0.992 0.741 deforestation 2006 0.899 0.695 deforestation 2007 0.746 0.575 def. Increase in past 5 yrs 0.784 0.261 Size and land use municip. area (mil. Km2) 0.384 0.276 % forest coverage 2002 0.840 0.559 % settlement area 2007 -3.811 -0.505 settlement area 2007 (km2) 0.325 -0.024 farm area 0.258 -0.135 popula. density (1000/km2) 0.268 -0.055 Economic and agricultural conditions

GDP per capita 2005 -2.953 -0.559 GDP per capita 2006 -0.426 -0.321 GDP per capita 2007 1.160 0.609 No. farms per km2 0.128 0.133 % small farms 0.236 -0.142 distance to nearest city 0.209 -0.065 land value (in BRL/ha) -0.929 -0.179 % farms w/ legal title 0.340 0.142 cattle stocking rate 0.321 0.047 No. tractors per farm -0.315 0.091 Protected areas

% indigeneous 2007 -1.434 -0.109 % strictly protected 2007 -0.640 -0.153 % multiple use 2007 0.255 0.205 indigeneous 2007 (km2) 0.081 0.031 strictly protected 2007 (km2) 0.036 -0.042 multiple use 2007 (km2) 0.101 0.202 Fines

No. fines 2005 0.613 0.306 No. fines 2006 0.707 0.428 No. fines 2007 0.676 0.336

30

Figure SI 2.1: Average deforestation and change in deforestation after matching. Black lines represent blacklisted districts, blue lines represent matched control districts, and grey lines represent unmatched control districts.

Documents

Forest law enforcement through district blacklisting in ......Forest law enforcement through district blacklisting in the Brazilian Amazon Elías Cisneros 1, Sophie Lian Zhou 2, Jan