Embed Size (px)
Citation preview
www.ssoar.info
Media Bias Towards African-americans Before andAfter the Charlottesville RallyLeschke, Julia C.; Schwemmer, Carsten
Erstveröffentlichung / Primary PublicationKonferenzbeitrag / conference paper
Empfohlene Zitierung / Suggested Citation:Leschke, J. C., & Schwemmer, C. (2019). Media Bias Towards African-americans Before and After the CharlottesvilleRally. In Proceedings of the Weizenbaum Conference 2019 "Challenges of Digital Inequality - Digital Education, DigitalWork, Digital Life" (pp. 1-10). Berlin https://doi.org/10.34669/wi.cp/2.25
Nutzungsbedingungen:Dieser Text wird unter einer CC BY Lizenz (Namensnennung) zurVerfügung gestellt. Nähere Auskünfte zu den CC-Lizenzen findenSie hier:https://creativecommons.org/licenses/by/4.0/deed.de
Terms of use:This document is made available under a CC BY Licence(Attribution). For more Information see:https://creativecommons.org/licenses/by/4.0
Media Bias towards African-Americans before and after the Charlottesville Rally
Julia C. LeschkeLondon School of Economics
and Political ScienceLondon, England
Carsten SchwemmerUniversity of Bamberg Bamberg, Germany
Abstract
African-Americans are still experiencing racial discrimination rooted in structural bias in USAmerican society. Research has shown that this behaviour can be reduced if individuals aremade conscious of their bias, but little is known about these mechanisms on a societal level.Envisaging the white-supremacist Charlottesville rally in 2017 as an event that rendered Amer-ican society conscious of its racism, we scrutinise whether racial bias in the digital media haschanged, comparing levels of pre- and post-Charlottesville bias. We fit word embedding mod-els to a broad sample of largely US media and quantify bias by calculating cosine similaritiesbetween terms for black or white actors and positive or negative character traits. We findno differences in positive character traits after Charlottesville. However, African-Americansare associated substantially less with negative character traits post-Charlottesville, while whiteactors are semantically closer to negative traits.
Keywords
Media bias; Ethnic studies; Automated text analysis; Word embeddings; Charlottesville
PROCEEDINGS OF THE WEIZENBAUM CONFERENCE 2019 CHALLENGES OF DIGITAL INEQUALITY
DIGITAL EDUCATION | DIGITAL WORK | DIGITAL LIFE
DOI: 10.34669/wi.cp/2.25
to scrutinise the external validity of previousfindings, we envisage the white-supremacistrally in Charlottesville in 2017 as such a starkreminder of existent racism, which renderedAmerican society conscious of its structuraldiscrimination. More specifically, we testwhether there are any substantial changesin implicit racial bias in a broad sample ofUS and UK online media by comparing thelevels of pre- and post-Charlottesville bias.The sample we use contains 97,542 articlesfrom 47 media outlets, combines tabloid andbroadsheets, online bogs and satirical maga-zines, and spans from the extreme right tothe left of the political spectrum. To op-erationalise racial bias we resort to the lit-erature on the logic of Implicit AssociationTests (IAT) that were developed to empir-ically test racial bias via word group asso-ciations. Drawing on this idea, we mea-sure the association between specific wordgroups in written media. We fit word embed-ding models to pre- and post Charlottesvillemedia samples and calculate cosine similar-ities between lists of words denoting blackor white actors as well as positive and neg-ative adjectives for character traits. This al-lows us to quantify the change in media biastowards African-Americans before and afterthe rally, compared to the bias towards whiteactors. We find that after the rally, thereis no considerable change in positive biastowards white or black actors, while post-Charlottesville African-Americans are associ-ated considerably less with negative charactertraits. Our findings suggest that media biastowards marginalized groups can temporar-ily shift after exogenous shocks such as theCharlottesville rally.
1 Introduction
The African-American population is still con-fronted with racial discrimination, which originates from a negative structural bias of American society towards black people. A recent and extreme example of the dis-criminatory behaviour, with which African-Americans are confronted, is the high number of fatal shootings of unarmed black men by white police officers across the United States. Digital media, online news and blogs play a central role in the persistent phenomenon of racial discrimination, as they serve as a pri-mary source of important information on cur-rent events but also, more generally, inform and shape the attitude and worldview held by the population. Due to its crucial role in opinion formation and updating, the im-plicit (as well as explicit) positive or negative bias towards minority groups spread by on-line media can reinforce biases in individuals, which can lead to discriminatory behaviour. In this sense, biases spread by media sources across the political spectrum of broadsheets, tabloids and blogs can be regarded as a proxy for public bias. In this study, we provide ev-idence in support of the idea that biases can be found across a broad spectrum of news sources and that these biases are likely to shift over time. We particularly focus on on-line media as a proxy for public opinion as well as the public’s strength and direction of bias. Related to this issue we also ask how the biases that persist in digital societies emerge from the individual level.Several studies have shown that making peo-ple conscious of their racial bias can pave the way to a significant reduction in discrimina-tory behaviour (Devine et al. 2012, Amodio, Devine, and Harmon-Jones 2007). Yet, can an intervention that reduces bias in individu-als under laboratory conditions also work on digital societies in the real world? Setting out
2 Interventions to ReduceRacial Bias
In spite of a general empirical tendencyshowing that racial bias is gradually dwin-dling since the 1960s (Gaertner and Do-vidio 1986, Schuman et al. 1997), African-Americans are still suffering from structurallyunequal treatment, such as poor quality in-teractions (McConnell and Leibold 2001),limited employment opportunities (Bertrandand Mullainathan 2004) or smaller chancesof receiving life-saving medical treatment(Green et al. 2007). Racially prejudicedbehaviour is believed to originate from im-plicit biases (Devine 1989, Gaertner and Do-vidio 1986), which produces discriminatorybehaviour (McConnell and Leibold 2001).These biases are reproduced in inter-personalinteractions but also in collective means ofcommunication, such as newspaper articles,fake news or blog entries. Past research hasshown that media content produces negativedispositions towards such minority groups(Boomgarden and Vliegenthart 2009) andcan be amplified by respective exogenousshocks (Czymara and Schmidt-Catran 2017).Results from earlier studies also indicate thatracist bias is not static, but can be reducedtemporarily or even in the long-term (Galin-sky and Moskowitz 2000, Devine et al. 2012).For an individual to reduce their racial bi-ases, the first step is to grow conscious oftheir bias, which is linked to the evocation ofconcern and guilt (Devine 1989), which mo-tivates self-regulation to discontinue biasedbehaviour (Amodio, Devine, and Harmon-Jones 2007). Long-term de-biasing effectswere achieved if this was coupled with biaseducation programmes designed to evoke gen-eral concern about implicit biases (Devine etal. 2012). Presenting an individual with feed-back of their racial bias, thereby rendering
them conscious of their bias and evoking con-cern about the racist biases held, can thuspave a way into decreasing racial prejudice.Implicit association tests (IAT) is a methoddeveloped to lay bare socially significant asso-ciative structures and can be used to measur-ing evaluative associations that underlie im-plicit, e.g. racially biased, attitudes (Green-wald, McGhee, and Schwartz 1998). In theseIAT, in essence, participants are made to an-swer to certain words with other words, e.g.names perceived to be typically white or blackhave to be replied to with words that fall un-der the category pleasant or unpleasant. If forinstance an association between an exampleof each of the categories white and pleasant isstronger, this indicates an underlying positivebias towards the category white. The logic ofIAT has been used in the application of wordembeddings to text to track stereotypes ongender and minorities (Garg et al. 2018). Wemake use of this application of word embed-dings to quantify negative and positive sen-timent towards African-Americans and whiteAmericans.
3 Media Bias Pre- andPost Charlottesville
Our example looks at the case of persis-tent racism by the US population towardsAfrican-Americans. In this example, wegauge public opinion by a broad range of me-dia sources and compare the implicit bias to-wards the black and white population beforeand after a march of white-supremacists whoexpressed their overtly racist stances. Do-ing this, we use the Charlottesville rally asan event that serves like a nation-wide inter-vention of conscious-rendering. In this ar-gument we draw on the mechanisms fromthe social psychology literature, which is fo-cused on the effects of de-biasing on indi-
viduals. Analogously to social psychologists,which examine the effects on a group of par-ticipants, we examine a potential effect of col-lective conscious-rendering on public opinion,for which we use a large sample of media out-lets and popular news blogs as proxy. Al-though the consumption of biased news caninform racially biased or racist believes, ourfocus in this research preliminary lies on themedia as a proxy for public opinion and de-bate in society.On 11 August 2017 the Unite the Rightrally took place in Charlottesville, Vir-ginia. The march consisted largely of whitemen, who self-identified as alt-right, neo-Confederates, neo-fascists, neo-Nazis, whitenationalists and supremacists. The marcherschanted racist and anti-Semitic slogans,carried swastika and torches. Althoughthe Charlottesville rally was previously an-nounced, the level of racist slander, violenceand the homicide committed by a rally mem-ber on the early morning of 12 August cameas an as-good-as external shock, laying barethe perilous racism and its violent potentialexisting in US society. The intensity of overtracism and violence must have also evokedgeneral concern among (at least a large part)of the public and caused a nation-wide de-bate. The Charlottesville rally serves as atreatment of collective feedback and evoca-tion of general concern. If the bias-breakingmechanisms put forward by the psychologi-cal literature were to hold in the US case,we would expect a neutralisation of biasestowards black actors. Indicators of such aneutralisation would entail that we wouldfind neither a substantially more positive nora substantially more negative bias towardsblack people in the post-Charlottesville me-dia. We could also expect a new biasing ef-fect, in which in a post-Charlottesville world,there would be more negative bias towardsCaucasians.
4 Data and Method
Our media sample uses 97,542 online news-paper articles and blog entries from a va-riety of US and UK sources, spanning theperiod of 10 May to 11 November 2017.The sample includes mainstream and well-established newspapers such as CNN or TheGuardian, fake news blogs and newspapersand satire sources as well as hyper-partisanpolitical outlets such as Breitbart (Horne etal. 2018). The British media was includedinto the sample as the UK also has a largepopulation, which faces structural negativebias, but also to increase article numbers. Weremoved sources with a very low publicationoutput and processed articles by commonmethods of automated text analysis (Grim-mer and Stewart 2012). In this process, wealso removed short articles with fewer than50 terms. This eventually resulted in a newssample of predominantly right-wing media,which will only allow us to infer shifts in biasin more conservative-oriented public opinion.Future research should use a more diversifiedsample across the political spectrum. Thesources and number of articles for our finalsample can be found in Table 1.Figure 1 further displays the term frequencyof the most frequent words of all articlesthat stem from the post-rally sample and in-clude the term Charlottesville. While termsrelated to political actors and the rally it-self appear very frequently, it is also appar-ent that the post-Charlottesville reports fre-quently discuss racism and violence. For thisreason, we should unsurprisingly find an in-creased association between ethnicity of ac-tors and terms such as violence, such thatCaucasians are more closely associated withthese terms after the rally.
Table 1. News sources and report counts.
Source No. of articles
True Pundit 7313Washington Examiner 6382Breitbart 5964BBC 5183Drudge Report 4427CNN 3391New York Post 2920The Huffington Post 2705National Review 2610Salon 2485The Daily Beast 2321RedState 2310Politicus USA 2226CBS News 2071Daily Mail 2035The Gateway Pundit 2017Bipartisan Report 2011CNBC 1870TheBlaze 1757Freedom Daily 1756Vox 1711The New York Times 1706New York Daily News 1671NPR 1589RT 1585The Political Insider 1523NewsBusters 1498ThinkProgress 1342The Guardian 1332USA Today 1294Conservative Tribune 1286Infowars 1282Natural News 1259The Duran 1211The Atlantic 1208The Daily Caller 1205CNS News 1151Counter Current News 1133Fox News 1061Media Matters for America 1050The D.C. Clothesline 1036PBS 1033Talking Points Memo 1030Yahoo News 1012Daily Kos 997The Right Scoop 925The Conservative Tree House 658
However, in this work, we instead scrutinizewhether the Charlottesville rally affected dif-ferences between ethnic groups not for termsdirectly related to racism and violence, butinstead for positive and negative character
traits. This allows us to examine racial biasfor terms that are not directly related to therally. We argue that we should only see anincreased association between positive or neg-ative character traits and ethnic actors post-Charlottesville if the rally affected the racialbias.Thus, a stronger association of e.g. black andthe word friendly after the rally would denotean increase of positive racial bias towardsAfrica-Americans. To operationalise bothethnic groups, we compile two sets of dictio-naries, i.e. list of terms. We compile this listdrawing on previous work that uses terms re-lated to either African-Americans (black) orCaucasians (white) (Kozlowski, Taddy, andEvans 2018).We then select terms that occur in our cor-pus using pre-existing dictionaries for char-acter traits commonly perceived as positive(e.g. friendly) and negative (e.g. unreliable)(Gunkel 2019).We examine bias towards African Americanswith an automated text analysis approach re-lying on doc2vec, a recent variant of wordembeddings (Mikolov et al. 2013, Le andMikolov 2014). In comparison to bag of wordsapproaches, which do not take into accountthe syntax of language (Grimmer and Stew-art 2012), word embeddings can capture morecomplex semantic relations. Given enoughtraining data this allows word embeddingmodels to solve analogy tasks. For instance,the analogy problem man is to woman as kingis to ? can be solved with the arithmeticoperation king - man + woman applied tovectors learned from an embedding model,which would return the result queen (Ko-zlowski, Taddy, and Evans 2018). This pow-erful method is increasingly acknowledgedby scholars and is, for instance, utilised tostudy the development of societal stereotypes(Garg et al. 2018) that are captured by al-gorithms trained on textual data (Bolukbasi
Figure 1. Word cloud of term frequency of all post-Charlottesville articles on the rally.
et al. 2016). In our paper, we instead focuson short- and mid-term developments of biasrather than stereotypes. We begin by train-ing separate doc2vec models on articles pub-lished in two time periods, one three monthsbefore and three months after the rally. Foreach period, we train 20 models on boot-strapped samples of articles from the respec-tive periods. The articles used to train eachmodel are drawn at random with replace-ment. This allows us to not only examine bi-ases before and after the Charlottesville rallybut also to quantify uncertainty in our esti-mates (Kozlowski, Taddy, and Evans 2018).For each model, we project ethnicity on a po-larity scale (Caucasian vs. African-Americandimension) based on the ethnic dictionary.We then compute cosine similarities betweenethnicity and positive as well as negativetraits. This enables us to analyse the changein media bias towards African-Americans incomparison to Caucasians, as well as beforeand after the rally. In terms of research de-sign, we are aware of the limitations of the
causal claims we can make. We cannot ran-domly assign the treatment of the rally to asubset of the news outlets and therefore can-not control for possible confounders. How-ever, our design still allows us to determineshifts in semantic associations between eth-nicity and character traits before and afterthe rally.
5 Results
To examine whether the Charlottesville rallycould have affected biases towards African-Americans, we visualise the results of theword embedding models using cosine similar-ities in Figure 2 (negative traits) and Figure3 (positive traits). Due to space constraintswe only visualise ten terms for each figure al-though the findings for negative and positivetraits also apply to the remaining terms inour more comprehensive dictionaries.Both Figures include the ethnicity dimension,where the left-hand side (negative values)is associated with Caucasian and the right-
Figure 2. Cosine similarities with bootstrapped 90% intervals between ethnicity dimension and negative character traits.
Figure 3. Cosine similarities with bootstrapped 90% intervals between ethnicity dimension and positive character traits.
hand side (positive values) is associated withAfrican-American. Values for each term de-note cosine similarities between the ethnicitydimension and the character trait. To give anexample for the interpretation of the results,Figure 2 includes cosine similarities betweenthe ethnicity dimension and negative term
silly, both before and after the Charlottesvillerally. Before Charlottesville silly was seman-tically closer to African-Americans. After therally, the negative racial bias shifts, so thatalso the term silly is more closely related toCaucasians. This change in overall negativebias towards white actors in the post-rally
sample can also be observed with regards tothe character traits in our dictionary that arenot displayed in Figure 2. Unlike the nega-tive character traits, the shift in positive biasbetween pre- and post-rally media reportingis minor, as can be seen in Figure 3. Boot-strapped intervals for the positive trait intel-ligent and other terms before and after therally overlap, indicating minor or no differ-ences.Altogether, our findings do not suggest thatthere are any meaningful changes for associ-ations between ethnicity and positive charac-ter traits. The similarities for positive termsin articles published after the Charlottesvillerally are mostly in line with the similaritiesfrom articles published before the rally. How-ever, the change in bias for negative termsis substantially larger. African-Americansare associated substantially less with negativecharacter traits post-Charlottesville, whilewhite actors are semantically closer to neg-ative traits. These results suggest that atleast for a short time period after the Char-lottesville rally, articles in the digital mediacontained fewer associations between nega-tive traits and African-Americans, i.e. a de-crease in negative bias towards black people.
6 Conclusion
In this paper we set out to scrutinise whetherthe white-supremacist rally in Charlottesvillein 2017 could have brought about any shiftof positive and negative racial biases towardsblack and white Americans in the media.This research draws on social psychologicalconcepts and mechanisms, such as IATs andbias-breaking interventions, and tests themon the aggregate level of American media.Theoretically, we could expect a de-biasingeffect after the rally, meaning that therewould be no notable difference between posi-
tive and negative associations of white andblack people. Such a neutralisation couldbe the result of a successful bias-breakingintervention that renders individuals, or inour case the media, conscious of their pre-viously held negative bias towards African-Americans and positive bias towards whiteAmericans. To measure racial bias we com-pare the pre- and post-rally similarities of as-sociations between ethnic terms and wordsfor character traits. We find that there is nodifference in pre- and post-rally media sam-ples for positive associations, meaning Char-lottesville did not seem to have had an effectof positive bias towards whites and blacks.However, we can observe a substantive over-all shift in negative media bias towards blackand white people after the rally. After themarch, black people are associated less withnegative terms than prior to the rally, whilewhite actors are associated more with neg-ative character traits post-rally. This holdsdespite of the fact that our sample comprisesmore right-wing than left-wing or centristsources, so that we would expect to see sim-ilar but stronger effects in a more balancedsample of American news. Future researchcould build upon our work by looking intohow and whether different segments of newsoutlets across the political spectrum adapttheir implicit bias after such politically dis-rupting events. Scholars could also use theapplication of word embeddings in an exper-imental or quasi-experimental framework toisolate clear causal effects and test the the-ory of bias-breaking on the aggregate levelof societies. Despite the shortcomings of ourwork, we seek to contribute to the literatureon marginalized groups in digital societies,showing that media biases towards marginal-ized groups can temporarily decrease in thelight of exogenous shocks.
7 Acknowledgments
Earlier versions of this paper were presentedat the GESIS Summer School on Methods forComputational Social Science and the Euro-pean Symposium on Societal Challenges inComputational Social Science. We thank theparticipants at these events for their usefulcomments. In particular, we thank DamianTrilling who initiated this project and JamesEvans for his many helpful suggestions.
8 References
Amodio, David M., Patricia G. Devine, andEddie Harmon-Jones (2007). “A dynamicmodel of guilt: Implications for motiva-tion and self-regulation in the context ofprejudice”. In: Psychological Science 18,E542–E530.
Bertrand, Marianne and Sendhil Mul-lainathan (2004). “Are Emily and GregMore Employable Than Lakisha and Ja-mal? A Field Experiment on Labor Mar-ket Discrimination”. In: American Eco-nomic Review 94.4, pp. 991–1013.
Bolukbasi, Tolga et al. (2016). “Man isto Computer Programmer As Woman isto Homemaker? Debiasing Word Embed-dings”. In: Proceedings of the 30th Inter-national Conference on Neural Informa-tion Processing Systems. NIPS’16. USA:Curran Associates Inc., pp. 4356–4364.
Boomgarden, Hajo G. and Rens Vliegen-thart (2009). “How news content influ-ences anti-immigration attitudes: Ger-many, 1993-2005”. In: European Journalof Political Research 48.4, pp. 516–542.
Czymara, Christian S and Alexander WSchmidt-Catran (2017). “Refugees Unwel-come? Changes in the Public Acceptanceof Immigrants and Refugees in Germanyin the Course of Europe’s ‘Immigration
Crisis’”. In: European Sociological Review33.6, pp. 735–751.
Devine, Patricia G. (1989). “Stereotypes andprejudice: Their automatic and controlledcomponents.” In: Journal of Personalityand Social Psychology 56.1, pp. 5–18.
Devine, Patricia G. et al. (2012). “Long-term reduction in implicit race bias: Aprejudice habit-breaking intervention”. In:Journal of Experimental Social Psychol-ogy 48.6, pp. 1267–1278.
Gaertner, Samuel and John F. Dovidio(1986). “The aversive form of racism”. In:Prejudice, discrimination, and racism. Or-lando: Academic Press, pp. 61–89.
Galinsky, Adam D. and Gordon B. Moskowitz(2000). “Perspective-taking: Decreasingstereotype expression, stereotype accessi-bility, and in-group favoritism.” In: Jour-nal of Personality and Social Psychology78.4, pp. 708–724.
Garg, Nikhil et al. (2018). “Word embed-dings quantify 100 years of gender andethnic stereotypes”. In: Proceedings ofthe National Academy of Sciences 115.16,E3635–E3644.
Green, Alexander R. et al. (2007). “ImplicitBias among Physicians and its Predictionof Thrombolysis Decisions for Black andWhite Patients”. In: Journal of GeneralInternal Medicine 22.9, pp. 1231–1238.
Greenwald, Anthony G, Debbie E McGhee,and Jordan L K Schwartz (1998). “Mea-suring individual differences in implicitcognition: the implicit association test.”In: Journal of personality and social psy-chology 74.6, p. 1464.
Grimmer, Justin and Brandon M Stewart(2012). “Text as Data: The Promise andPitfals of Automatic Content AnalysisMethods for Political Texts”. In: PoliticalAnalysis 21.617, pp. 267–297.
Gunkel, Patrick (2019). 638 Primary Per-sonality Traits. url: http : / / ideonomy.
mit . edu/ essays / traits . html (visited on02/08/2019).
Horne, Benjamin D. et al. (2018). “Sam-pling the News Producers: A Large Newsand Feature Data Set for the Study ofthe Complex Media Landscape”. In: arXivpreprint 1803.10124.
Kozlowski, Austin C., Matt Taddy, andJames A. Evans (2018). “The Geometryof Culture: Analyzing Meaning throughWord Embeddings”. In: arXiv preprint1803.09288.
Le, Quoc V. and Tomas Mikolov (2014).“Distributed Representations of Sentencesand Documents”. In: arXiv preprint1405.4053.
McConnell, Allen R. and Jill M. Leibold(2001). “Relations among the Implicit As-sociation Test, Discriminatory Behavior,and Explicit Measures of Racial Atti-tudes”. In: Journal of Experimental SocialPsychology 37.5, pp. 435–442.
Mikolov, Tomas et al. (2013). “DistributedRepresentations of Words and Phrasesand their Compositionality”. In: Advancesin Neural Information Processing Systems26. Curran Associates, Inc., pp. 3111–3119.
Schuman, Howard et al. (1997). Racial atti-tudes in America: Trends and interpreta-tions. Cambridge, MA: Harvard Univer-sity Press.
