Research Evaluation Papers

7/31/2019 Research Evaluation Papers

1/12

1

Research Evaluation Papers

Arjun Patel

Qualitative Paper: Violence, Identity and Poverty,by Amartya Sen

Research Question

The research question prompting this article is not explicitly stated, but it engages with the wider

debate of How is violence caused? In doing this Sen examines two current theories that attempt to

explain the outbreak of violence. A second fitting research question, then, is How well do cultural

approaches and approaches of political economy explain the outbreak of violence in contemporary

global society?

Philosophical and Methodological Frameworks

The very idea of an evaluation suggests the need for a coherent if not neutral - criterial framework

from which to make the evaluation. I do not attempt address constructivist or interpretivist concerns in

this paper. Instead this paper seeks to make an evaluation from a perspective within the scientific

paradigm. A cursory glance at Sens arguments, however, shows that the project of evaluating Sens

work from this perspective is itself not entirely congruous. Violence, Identity and Poverty lacks

features that some would argue are essential to scientific research; an explicit research question, a

clearly defined methodology, commensurable empirical analyses, and testable hypotheses are not

included. Some of the evidence cited is little more than anecdotal. Sen, furthermore, does not present

the article as a piece of scientific research, simply framing the article as a discussion.

This paper argues, however, that Sens arguments can be brought into a philosophical framework of

science through dividing his arguments along the lines of Poppers Science: Conjectures and

Refutations (1963). Indeed, the article has much to contribute to a scientific approach to understanding

violence causation. Many of the arguments are based on empirical observation, and almost all are

shaped by logics of either deduction or induction. Reading the article from a Popperian perspective of

the scientific method, the arguments can be divided into those of refutation, and those of conjecture and

theory-building. Sen makes a deductive refutation of the hate at first sight hypothesis (2008, p. 6), for

instance, by arguing that many historical figures have actively sought an understanding of other

cultures without resorting to violence. The argument that Those trying to eradicate poverty in the

world are, naturally enough, tempted to seek support from the apparent causal connection that ties

violence to poverty, on the other hand, is sheer conjecture. However, from Poppers perspective such

non-scientific theory (2002, p.50) can play an important role in developing scientific knowledge by

leading to testable predictions.

Within the hypothetico-deductive understandings of science, such as those of Popper, there are more

prescriptive methodological frameworks for developing research design, such as those set out by King


2/12

2

(1994), or Babbie (2004). It is clear that Violence, Identity and Poverty does not progress far beyond

the initial theory-building stages, when read in light of these. Sens arguments do not fit into the four

stages of research proposed by King, nor do they fit in with the linear model of Babbie. Whereas such

models seek to encapsulate an entire process of inquiry within the design of a single project, Sens

article points to a much more collaborative approach. He refutes and builds upon the theories of others,

and also points future research to continue the inquiry in an empirical direction.

Conceptualization

Perhaps the main stumbling blocks to progressing further within such formal frameworks are the

problems of conceptualization and operationalization. In many senses, these are not problems peculiar

to this article, but permeate this academic debate in general- concepts such as violence, poverty,

inequity, and cultural divisions are rarely undisputed.

One of the main aims of this article is to explain violence. To advance such a goal, a clear

conceptualization of the variables discussed is paramount. In some instances this is done successfully,

It is argued that the concept of distinct civilizations - a premise of the Clash of Civilizations theory -

should instead account for the many constructive historical interactions and exchanges across borders

(p. 6). In lieu of a solitarist (p. 6) conceptualization of identity, Sen also puts forward a more nuanced

view, whereby an individual may hold several different identities at the same time without necessary

contradiction. These conceptualizations by themselves do much to restructure the debate away from

reductionist cultural explanations of violence.

The upshot of a clearer understanding of the concepts identity and civilization, however, is a more

blurred understanding of violence causation itself. Clear-cut operationalizations of inequality, such as

the Gini-coefficient, are critiqued for being too narrow - Purely economic measures of inequality do

not bring out the social dimensions of the disparity(p. 15), such as race. These dimensions are

required, it is argued, to explain instances such as the Paris riots of 2005. It seems as though the more

precisely the variables are conceptualized (and operationalised), the less explanatory power they have

on their own, and our understanding of the causal mechanism becomes muddier.

Surprisingly, perhaps, a specification of violence itself is never explicitly discussed Sen depends

upon an implicit, shared understanding of what it might mean. In one argument, violence is

operationalised by homicide rates in cities around the world. Yet this operational definition is neither

justified nor defended. As King argues If the method and logic of a researchers observations and

inferences are left implicit, the scholarly community has no way of judging the validity of what was

done (1994, p. 8).

A more public specification of different types of violence (e.g. homicide, violence during robbery,

violence as part of a war, rape) might be disputable, but it would be helpful in the overall project. By

outlining a clearer specification of what violence entails, it might, for example, emerge that different

forms of violence need to be explained through markedly different causal mechanisms, and that

violence causation, as well as being analyzed from society-level structures of inequality and ethnic


3/12

3

divisions, might also need to be explained through group- and actor-level processes, that take into

account motivations and the psychology of grievances. The obstacle to this, of course, is that if the

different forms of violence have incommensurable causal mechanisms, and in terms of Gerrings idea

of trade-offs it loses resonance, parsimony, theoretical utility and field utility the word violence

itself becomes redundant, as it always requires further definition (1999). Clearly there is a balance to be

struck between broad and universally accepted concepts that can be useful for empirical research.

Operationalization and Validity

These difficulties contribute to the fact that there is only one instance in the article, where a concept is

operationalised to produce a concrete measure the homicide rate in various cities is used as a measure

of violence. The face validity and criterion-related validity of this is high - it seems reasonable to

assume that homicide necessarily implicates (and predicts) violence. As the only measure of violence

put forward, the construct and content validity of the measure is more problematic much of violence

is not covered by this operational definition, yet we necessarily have to equate homicide with violence,

and to the corresponding theoretical arguments. The remedy, therefore, would be not to remove

homicide rates, which constitute valuable evidence in this argument, but to supplement it with other

measures of violence.

Reliability

Sen writes that the data for the different cities have been collected from the respective municipal and

national publications and official sources (p.9). Assuming these measures based upon the total

numbers of recorded homicides in each city, they can be assumed to be fairly representative of the total

numbers killed, as sampling and estimation is not required. However, they would also then be

dependent on the accuracy of these records, which would likely vary from source to source.

Time Dimension

The years in which the measures were taken are the closest year (p.9) to 2005 for which there is

available data, presumably due to insufficient data. However, a longitudinal study looking the average

number of homicides in each city per year, within a fixed period of time, would have helped to sidestep

this issue, and provide a much more reliable measure.

Randomization and Sampling Bias

In terms of the selection of cities, there has been no systematic process of randomization. This is

perhaps particularly pertinent as Sen picks chooses his home city Calcutta, and then uses the findings

to discuss at length about the reasons for the low levels of violence here. Randomisation would have

been useful in making sure the cities selected were not subject to personal bias or preconceived

theories.


4/12

4

Hypothesis Building

The main shortcoming of this article, evaluated as a scientific paper, is that it does not progress very far

beyond theoretical argument. The four propositions in the articles conclusion are not formulated into

clear and testable hypotheses, despite the papers calls for more empirical research. The difficulty in

conceptualizing and then operationalizing concepts such as violence, inequality, and poverty, seem to

be the barriers to forming such testable hypotheses. This is possibly because Sen seems reluctant to

accept the loss of detail in the concepts he has carefully explored. If we are to accept Poppers calls for

risky propositions, however, this is a necessary step.

Possible Improvements

In summary, then, this paper identifies the following areas for improvement in the research design.

Firstly, an explicit research question would help provide a clear structure and point of reference for the

arguments made. Secondly, whilst this paper recognizes the difficulties in conceptualizing terms such

and violence and poverty, they need to be given clear operational definitions, so the concepts can be

empirically observe, and to provide a common language for the discussion. The effect of the inevitably

low validity of such measures might be mitigated through techniques of triangulation. In terms of the

particular study of homicide rates in global cities, this could be improved by performing a longitudinal

study over a number of years, and also by employing a systematic process of randomization. Finally,

the propositions at the end of the paper should be formulated into testable hypotheses, in order to truly

facilitate the further empirical studies that are required.


5/12

5

Research Evaluation Paper

Quantitative Paper:Ethnicity, Insurgency and Civil War,by

J. Fearon and D. Laitin

Research Questions and Structure

Ethnicity, Insurgency and Civil War broadly aims to answer the following question: What explains the

recent prevalence of violent civil conflict around the world? (Fearon and Laitin, 2003a, p.75).

Specifically, it poses the following questions: 1. Is it due to the end of the Cold War and associated

changes in the international system, or is it a longer-term trend? 2. Why have some countries had

civil wars while others have not? and 3.Why did the wars break out when they did? (2003a, p.75).

The first section in the article, Civil War since 1945, is a longitudinal, descriptive study looking to

examine trends in civil war prevalence between 1945 and 1999, it successfully refutes the proposition

in question 1. that the end of the Cold War is responsible for the recent prevalence of civil war. The

trend analysis is an efficient method of looking at possible effects of changes in the international

system, without having to engage with the concepts and mechanisms of such a process in depth. I

discuss the measures of the dependent variables later, as these are also used in the following two

sections. These sections are respectively titled Ethnicity, Discrimination and Grievances and

Insurgency. They constitute most of the study, and seek to answer questions 2 and 3 the rest of the

paper focuses on these.

Theories, and hypothesis formulation

To answer these two questions, the article formulates hypotheses drawn from several different theories

about the causes of civil conflict. This comparative approach is a great strength of the paper, but, as

discussed below, it also leads to its greatest difficulties. The research uses an empirical approach to

assess the merits of various theories, by looking for their observable implications in the real world. The

hypotheses are testable, and provide clear and comparable information about how well various theories

might explain empirical observations. On the basis of the evidence provided, perennialist and

modernist theories are critiqued on account of their lack of explanatory power for civil war causation.

Dependent Variables 1

The principal measures of the dependent variable - civil war onset and prevalence - are compiled from

various commonly-used datasets on war, and are used in all three sections. The authors use data from

all the 127 civil wars they identify post-1945, so the sample probably constitutes the bulk of the total

population of civil war occurrences in this period.1 Due to space constraints, this paper does not discuss all of the measures in this study. The appendix tables, however, provide a

reference of the key measures in the paper.


6/12

6

In terms of the validity of the different measures of civil war, the chief argument raised is whether wars

of independence in former colonies should be included under the concept of a civil war. The authors

addressed this issue appropriately by including analyses both including and excluding such cases.

Another definitional issue is the cut-off point of at least 1,000 persons killed (2003a, p 76). This is an

admittedly arbitrary, but also a commonly accepted definition.

Establishing how many people were killed in each war, however, is important for the classification of

civil war, and this would have been estimated. Whilst this impacts the reliability of the data, an

approximate measure of the numbers killed is sufficient to draw the inferences made in this study. To

make sure of this, the authors compute their results using two other datasets of war, and the findings

are similar in both cases (p.88).

Independent Variables

The measures of the independent variables vary in type considerably, and each face their own

problems. For some measures, such as the ELF index, the main problems are those of validity. For

others, such as rough terrain, there are problems of sufficient data. For yet others, precision and

reliability are an issue, such as the measure that seeks to identify discriminatory policies.

Independent Variables: Operationalisation of grievance-based variables

Such reductions seem minor, however, when compared to the operationalizations of ethnic grievances,

where, for example, the various personal, emotional, and historical dimensions of ethnic divisions are

reduced to an Ethno-Linguistic Fractionalization index. Such an index, for all its complexity, is a

highly abstracted measure of ethnic diversity, and this in turn is a particular operationalisation of the

concept of ethnic division. The authors use triangulation here and elsewhere to remedy the low

validity of the measures, but at best this has a limited effect, and it is not demonstrable. In the findings,

none of measures indicate a significant relationship with civil war onset this is given as strong

evidence to show that ethnic grievance related explanations have little predictive power (2003a, p. 84).

It remains unclear, however, whether this is due to a lack of valid measures, or due to a genuine lack of

causal connection, as argued.

Other hypotheses based upon ethnic-grievance explanations also face problems in terms of the low

validity of their measures: there are shortcomings in the measures for discrimination (discussed below),

and the religious diversity measure uses a parallel index to the ELF.

Independent Variables: Operationalisation of the conditions that favour insurgency

With the exception of the measures of democracy (which are included for both grievance-based and

insurgency-based hypotheses), structural factors such as population, oil revenue, and rough terrain

are clearly defined, quantifiable, and show in several instances a correlation to civil war onset. They

therefore act in several instances as useful predictors of civil war, though admittedly there are

difficulties in quantifying these factors. Such difficulties are overwhelmingly due to insufficient data of

the structural factors, e.g. the exact area of rough terrain in countries is not available, and the measure


7/12

7

is then proxied by the area of mountainous terrain. The authors also use multiple imputation to fill in

missing data values for per capita income measures (Fearon and Laitin 2003b, p.5) to rectify this

problem.

Methods and Bias

As the dependent variables have binary outcomes, and many of the independent variables have

unknown distributions, the use of logistic regression is appropriate in determining the relationship

between the two in many instances.

However, quantitative studies are inefficient in establishing relationships using variables such as those

outlined in the previous section; on the other hand, they naturally display a bias to identifying

relationships between easily quantifiable concepts, which constitute most of the insurgency-related

variables. These concepts, in addition, are apposite in cost-benefit analyses and other rationalist

decision-making processes. Rational-actor models, or structural explanations that utilize these concepts

as explanatory variables, therefore may predict outcomes much better than other non-quantifiable

mechanisms that might lead to the same outcome. For this reason, this evaluation regards the method as

inherently biased against finding relationships for the independent variables that fall under the

grievance categorization.

Unit of Analysis

Another difficulty in this study is the unit of analysis used for the hypotheses drawn from different

theories. All the data about the civil wars is collated at the level of country years e.g. the per capita

income for a particular country in a particular year. Again, this seems to have the effect of unfairly

stretching the operationalization of various grievances. Many of the operationalizations, as a result, are

skewed towards an understanding through state-level mechanisms. Discrimination, for example, is

proxied by indices of democracy and civil liberties, and measures of discriminatory policies. More

informal processes of discrimination, and those at group-level, which may also contribute to civil war

onset, are given little space in such measures.

Data Collection and Sourcing

The data collection methods used are often unspecified. Many of the measures used are from secondary

sources such as Freedom House and the World Bank, and this is not ideal in terms of randomization -

measures, for instance, of per capita income all tend to rely on similar sources. However, this

outsourcing, is a more efficient way to deal with issues of reliability and validity, due to the sheer

quantity of data used in this study to conduct a study of this scale using only primary evidence would

be an inefficient use of resources.

Broadly speaking, a principal finding regarding correlations to economic and grievance-based variables

is shown to be reproducible in studies by Collier and Hoeffler this provides some indication of the

inter-observer reliability of these results. This, in turn, suggests that the measures employed were

appropriate.


8/12

8

Inferences

This evaluation argues that there is a bias in the methods employed that fail to fully engage with the

various theories being tested. A particular difficulty is in the operationalisation of various concepts that

fall under the category of grievances. The authors acknowledge the difficulties in measuring such

concepts, and most of the conclusions reflect this these are qualified, and made on a pragmatic mid-

level that demonstrates observable predictors of civil conflict and policy implications.

But if, as the research questions suggest, the paper seeks to go further, and explain civil war

prevalence, then the findings are rather less useful for establishing causal mechanisms. The theoretical

implications and causal mechanisms must lie outside the scope of the project we may be able to

predict civil conflicts better through these findings, but we still cannot explain them. For this reason,

Fearon and Laitins assertion that civil wars of the [post-Cold War] period have structural roots (p.88)

is, therefore, an inference too far.

Possible improvement

The authors could perhaps best explain causal mechanisms, by supplementing their quantitative study

with comparative case studies. This would be useful in highlighting the differences and consistencies

in the mechanisms of each conflict, at different units of analysis, and the thick descriptions would be

better suited to capture a fuller conceptualization of ethnic grievances, as well as structural factors.


9/12

9

Appendix: Tables of Dependent and Independent Variables inEthnicity,

Insurgency and Civil Wars

Dependent variable Measure/ Proxies Units/Coding

Civil War Ongoing civil war prevalence, Figure 1 Number of countries with ongloing

conflict; Percentage of all countries withongoing conflict

Civil War Probability of Civil War, Figure 2 Probability per 5 year period

Civil War Civil war onset, Table 1: Model 1 Country-years in which civil wars started

coded as "1"; all other country years

coded as "0"

Civil War "Ethnic" war onset,(Conflicts in which the

fighters were mobilized primarily along ethnic

lines) Table 1: Model 2

Country-years in which civil wars started


coded as "0"

Civil War Civil war onset, Table 1: Model 3 Country-years in which civil wars started

coded as "1"; all other country yearscoded as "0"

Civil War Civil war (plus Empires), Table 1: Model 1 Country-years in which civil wars started


coded as "0"

Civil War Civil War (COW), Table 1: Model 1 Country-years in which civil wars started


coded as "0"

Concepts: Independent variables (Proxies in brackets) Measures

CIVIL WAR SINCE 1945

The End of the Cold War End date

"Major international systemic change"p.77 Not explicitly operationalised, though see

above

Decolonization Not explicitly operationalised

ETHNICITY, DISCRIMINATION AND GRIEVANCES

Ethnic diversity Ethno-linguistic Fractionalization Index

(ELF)

Ethnic diversity Population proportion of largest ethnic group

Ethnic diversity The number of languages spoken by groups

exceeding 1% of the population

Religious diversity Religious Fractionalization Index, parallelsELF

Ethnic or religious diversity, controlled for by per capita income Per Capita income (GDP in USD 1985)

interacted with above four measures

Grievance; Discrimination (Political Democracy) Polity IV; Przeworkski et al. 2000 democracy

measures

Grievance; Discrimination (Civil Liberties) Freedom House Index

Ethnic Grievance; Discrimination Policies that discriminate in favour of a

particular language or religion

Grievance; Economic Inequality Gini coefficient

Ethnic Diversity A minority group comprising at least 5% of

population


10/12

10

INSURGENCY

Rough (Mountainous) terrain Mountainous Terrain

Foreign, cross-border sanctuaries Measures unavailable

A co-operative local population Measures unavailable

Local knowledge (having a rural base) Measures unavailable

Government capability and reach in rural areas; a state's overall financial,

administrative, military and police capabilities (Per capita income)

Per capita income

New states Countries in first and second years of

independence are marked

Political Instability Dummy variable indicating whether a country

had a change on the Polity IV >3 for the three

years before the country-year in question

Anacrocy Countries between -5 and +5 marked

Large country population Country Population, based on World Bank

figures

A territorial base separated from the states center by water or distance Dummy variable for states with

noncontiguous territory

Support of foreign governments/ diaspora Observable policies from foreign govts. (only

2 cases)

Land that supports the productions of high-value, low weight goods Measures unavailable

State revenue derived from oil exports Country-years in which fuel exports exceeded

one-third of export revenues


11/12

11

Appendix: Web Links to Articles

Sen, A. 2008. Violence, identity and poverty.Journal of Peace Research. 45(1): 5-15. [Online]. 2008.

Available from:http://sfx.ucl.ac.uk/sfx_local?sid=Elsevier:Scopus&issn=00223433&isbn=&volume=45&issue=1&spage=5&epage=15&pages=5-

15&artnum=&date=2008&title=Journal+of+Peace+Research&atitle=Violence%2c+identity+and+poverty&aufirst=&auinit=A.&aui

nit1=A&aulast=Sen&_service_type=getFullTxt[Accessed: 4th May 2011].

Fearon, J. and Laitin, D. Ethnicity, Insurgency and Civil War. American Political Science Review.

91(1): 75-90. [Online]. 2003. Available from:http://journals.cambridge.org/action/displayFulltext?type=1&fid=142718&jid=PSR&volumeId=97&issueId=01&aid=142717

[Accessed: 4th May 2011].


12/12

12

Bibliography

Qualitative Article

Sen, A. 2008. Violence, identity and poverty.Journal of Peace Research 45(1): 5-15.

Babbie, Earl. 2004. The Practice of Social Research. Belmont, CS: Thompson-Wadsworth.

Gerring, John. 1999. What Makes a Concept Good? A Critical Framework for Understanding Concept

Formation in the Social Sciences. Polity 31(3): 357-393.

King, Gary C., Robert Keohane and Sidney Verba. 1994.Designing Social Inquiry: Scientific Inference

in Qualitative Research. Princeton: Princeton University Press.

Popper, Karl. 1963. Science: Conjectures and Refutations. In Philosophy of Science: The Central

Issues. New York: W.W. Norton and Co.

Quantitative Article

Fearon, J. and Laitin, D. 2003a. Ethnicity, Insurgency and Civil War. American Political Science

Review. 91(1): 75-90.

Fearon, J. and Laitin, D. 2003b. Additional Tables for Ethnicity, Insurgency, and Civil War.

[Online]. Available from: http://www.stanford.edu/ group/ethnic/. [Accessed 4th May 2011].

Documents

Research Evaluation Papers