1

PoliticalScience817EmpiricalMethodsofPoliticalInquiry

UW-Madison•Fall2017•Wednesdays3:30-5:25pm•422NorthHall

ContactInformation Professor: Barry C. Burden Email: bcburden@wisc.edu Phone: 608-263-6351 Office: 101 A/B North Hall Office Hours: Mondays 2-4pm and by appointment TheCourse Political science, like all mature academic fields, advances by scholarly research. This course introduces graduate students to the issues that commonly arise in qualitative and quantitative empirical political science research designs and their execution. Because no single approach to a research question is ideal, we will learn about the trade-offs involved in selecting one design over another. As a researcher and a consumer of others’ research, you must be self-conscious about the strengths and weakness of each methodology. Requirements The readings are a combination of books, book chapters, and articles. Some are general sources intended to acquaint you with a set of issues; others are applications that exemplify a methodological approach. The two required books are Gerring’s Social Science Methodology: A Unified Framework (2nd ed.) and King, Keohane, and Verba’s Designing Social Inquiry (KKV), both available for purchase online and at the University Book Store. Other readings will be distributed electronically on the Learn@UW course website. Students are expected to do all of the required reading and come to class meetings ready to discuss the material. I will give advance notice when the readings are changed. Class meetings will mostly be in a seminar format interrupted by occasional mini lectures. You should expect to participate in the dissection of each week’s required readings. There will be five short homework assignments scattered throughout the semester. These exercises will allow you to develop a specific skill, evaluate competing research designs, and see how theories work in practice. Each assignment will take time to execute, but should not run longer than 5 double-spaced pages. Please deliver these to me in hard copy. A ★ indicates when the assignment will be distributed. It will generally be due 7-10 days later and should be uploaded to the appropriate dropbox folder.

2

For the final paper each student will develop an original research design. The paper should efficiently synthesize the relevant literature and outline the hypotheses, arguments, or theory to be examined. In more detail, it should propose an empirical research project, a sort of blueprint for a study that depends only on resources that are typically available to a graduate student. I hope that the research design will eventually lead to a paper in another seminar, a conference presentation, or part of a dissertation. An abstract of at least one page is due in the dropbox by Monday, November 20. Final papers are due in the dropbox by Thursday, December 21. Most papers will be about 20 pages long and should not be much longer unless you are combining the paper with another course or actually undertaking the research in the paper. Please consult with me if want to be considered for these options. Grades Final grades are based on a weighted average of letter grades from three sources: contributions to class participation (25%), homework assignments (35%), and the final paper (50%). Please talk with me anytime during the semester if you have questions about the material, requirements, or your performance. Introduction&Inference:TheGeneralandtheParticular(September6and13) Required Gerring [Chapters 1-3] KKV [Chapters 1 & 2] Friedman, Milton. 1953. “The Methodology of Positive Economics.” In Essays in Positive

Economics. Chicago: The University of Chicago Press. Geertz, Clifford. 1973. The Interpretation of Cultures. New York: Basic Books. [Chapter 1] Recommended Abbott, Andrew. 2004. Methods of Discovery: Heuristics for the Social Sciences. New York:

W.W. Norton. [Chapter 4] Allwood, Carl Martin. 2012. “The Distinction between Qualitative and Quantitative Research

Methods is Problematic.” Quality & Quantity 46:1417-29. Brady, Henry E., and David Collier, eds. 2010. Rethinking Social Inquiry: Diverse Tools, Shared

Standards. 2nd ed. Lanham, MD: Rowman and Littlefield Clarke, Kevin A., and David M. Primo. 2012. A Model Discipline: Political Science and the

Logic of Representations. New York, NY: Oxford University Press. Fenno, Richard F., Jr. 1986. “Observation, Context, and Sequence in the Study of Politics.”

American Political Science Review 80:3-15. Mahoney, James, and Gary Goertz. 2006. “A Tale of Two Cultures: Contrasting Quantitative and

Qualitative Research.” Political Analysis 14:227-249. Oliver, Jack E. 1991. The Art of Discovery. New York: Columbia University Press. [Chapter 2] Sil, Rudra. 2000. “The Division of Labor in Social Science Research: Unified Methodology or

‘Organic Solidarity?’” Polity 32:499-531.

3

MeasurementI:ConceptOperationalization(September20) Required Gerring [Chapter 5] Abdelal, Rawi, Yoshiko M. Herrera, Alastair Iain Johnston, and Rose McDermott. 2006.

“Identity as a Variable.” Perspectives on Politics 4:695-711. Fisman, Ray, and Edward Miguel. 2007. “Cultures of Corruption: Evidence from Diplomatic

Parking Tickets.” Journal of Political Economy 115:1020-48. Recommended Collier, David, and James E. Mahone, Jr. 1993. “Conceptual Stretching Revisited: Adapting

Categories in Comparative Analysis.” American Political Science Review 87:845-55. « Assignment 1: Measuring Educational Quality MeasurementII:ReliabilityandValidity(September27) Required Gerring [p. 80-6 & Chapter 7] KKV [p. 150-68] Adcock, Robert, and David Collier. 2001. “Measurement Validity: A Shared Standard for

Qualitative and Quantitative Research.” American Political Science Review 95:529-46. Coppedge, Michael, et al. 2011. “Conceptualizing and Measuring Democracy: A New

Approach.” Perspectives on Politics 9:247-267. Recommended Althaus, Scott L., et al. 2011. “Assumed Transmission in Political Science: A Call for Bringing

Description Back In.” Journal of Politics 73:1065-80. Grimmer, Justin, and Brandon M. Stewart. 2013. “Text as Data: The Promise and Pitfalls of

Automatic Content Analysis Methods for Political Texts.” Political Analysis 21:267-297. Munck, Gerardo L., and Jay Verkuilen. 2002. “Conceptualizing and Measuring Democracy:

Evaluating Alternative Indices.” Comparative Political Studies 35:5-34. Theories,Hypotheses,andInference(October4) Required Gerring [Chapters 6 & 8] Fearon, James D. 1991. “Counterfactuals and Hypothesis Testing in Political Science.” World

Politics 43:169-95. Popper, Sir Karl. 1998. “Science: Conjectures and Refutations.” Both in Introductory Readings

in the Philosophy of Science, 3rd ed., ed. E.D. Klemke, Robert Hollinger, and David Wÿss Rudge. Amherst, NY: Prometheus Books.

Stinchcombe, Arthur L. 1968. Constructing Social Theories. New York: Harcourt, Brace, & World. [p. 15-38]

4

Recommended Clarke, Kevin A., and David M. Primo. 2007. “Modernizing Political Science: A Model-Based

Approach.” Perspectives on Politics 5:741-53. Esaray, Justin, and Nathan Danneman. 2015. “A Quantitative Method for Substantive

Robustness Assessment.” Political Science Research and Methods 3:95-111. Kitcher, Philip. 1998. “Believing Where We Cannot Prove.” In Introductory Readings in the

Philosophy of Science, ed. 3rd ed. E.D. Klemke, Robert Hollinger, and David Wÿss Rudge. Amherst, NY: Prometheus Books.

Nuzzo, Regina. 2014. “Scientific Method: Statistical Errors.” Nature 506:150-2. « Assignment 2: Durkheim’s Hypotheses CollectingData(October11) Required Gerring [p. 74-80 & 86-95] KKV [Chapter 6] Hurley, Dan. 2005. “Divorce Rate: It’s Not as High as You Think.” New York Times. April 19. Lustick, Ian S. 1996. “History, Historiography, and Political Science: Multiple Historical

Records and the Problem of Selection Bias.” American Political Science Review 90:605-18.

McDonald, Michael P., and Samuel Popkin. 2001. “The Myth of the Vanishing Voter.” American Political Science Review 95:963-74.

Robinson, W.S. 1950. “Ecological Correlations and the Behavior of Individuals.” American Sociological Review 15:351-7.

Recommended Kritzer, Herbert M. 1996. “The Data Puzzle: The Nature of Interpretation in Quantitative

Research.” American Journal of Political Science 40:1-32. Soroka, Stuart N., Christopher Wlezien, and Iain McLean. 2006. “Public Expenditure in the UK:

How Measures Matter.” Journal of the Royal Statistical Society A 168:255-71. Wooley, John T. 2000. “Using Media-Based Data in Studies of Politics.” American Journal of

Political Science 44:156-73. « Assignment 3: Fact Finding SpecificationandSpuriousness(October18) Required KKV [p. 168-84 & 199-207] Caiazza, Amy. 2002. “Does Women’s Representation in Elected Office Lead to Women-

Friendly Policy?” Institute for Women’s Policy Research Publication I910. <http://www.iwpr.org/pdf/i910.pdf> Leamer, Edward E. 1983. “Let’s Take the Con Out of Econometrics.” American Economic

Review 73:31-43. Licklider, Roy. 1995. “The Consequences of Negotiated Settlements in Civil Wars, 1945-1993.”

American Political Science Review 89:681-90.

5

Recommended Clarke, Kevin A. 2006. “The Phantom Menace: Omitted Variable Bias in Econometric

Research.” Conflict Management and Peace Science 22:341-352. NoClassOctober25Causation,Necessity,andSufficiencyinSmall-NStudies(November1) Required Gerring [Chapters 11 & 12] KKV [Chapter 3] Goertz, Gary, and Harvey Starr. 2003. “Introduction: Necessary Condition Logics, Research

Design, and Theory.” In Necessary Conditions, ed. Gary Goertz and Harvey Starr. Lanham, MD: Rowman & Littlefield.

Hill, Sir Austin Bradford. 1987. “The Environment and Disease: Association or Causation?” <http://www.edwardtufte.com/tufte/hill>

Talbot, Margaret. “The Placebo Prescription.” The New York Times Magazine. January 9, 2000. Recommended Gerring, John. 2007. “Review Article: The Mechanistic Worldview: Thinking inside the Box.”

British Journal of Political Science 38:161-79. Imai, Kosuke, Luke Keele, Dustin Tingley, and Teppei Yamamoto. 2011. “Unpacking the Black

Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” American Political Science Review 105:765-89.

Keele, Luke, 2015. “The Statistics of Causal Inference: A View from Political Methodology.” Political Analysis 23:313-35.

LabandFieldExperiments(November8) Required Gerring [Chapter 9 & p. 256-73] KKV [p. 185-99] Gerber, Alan S., and Donald P. Green. 2001. “Do Phone Calls Increase Voter Turnout? A Field

Experiment.” Public Opinion Quarterly 65:75-85. McDermott, Rose. 2002. “Experimental Methods in Political Science.” Annual Review of

Political Science 5:31-61. Smith, Gordon C.S., and Jill P. Pell. 2003. “Parachute Use to Prevent Death and Major Trauma

Related to Gravitational Challenge: Systematic Review of Randomised Controlled Trials.” BMJ 327:1459-61.

Recommended Druckman, James N., Donald P. Green, James H. Kuklinski, and Arthur Lupia. 2006. “The

Growth and Development of Experimental Research in Political Science.” American Political Science Review 100:627-35.

6

Gaines, Brian J., and James H. Kuklinski. 2011. “Experimental Estimation of Heterogeneous Treatment Effects Related to Self-Selection.” American Journal of Political Science 55:724-36.

Gerber, Alan S., and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York, NY: W.W. Norton.

Hainmueller, Jens, Daniel J. Hopkins, and Teppei Yamamoto. 2013. “Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” Political Analysis 22:1-30.

Lehrer, Jonah. 2010. “The Truth Wears Off.” The New Yorker. December 13. Monogan, James E., III. 2005. “Research Preregistration in Political Science: The Case,

Counterarguments, and a Response to Critiques.” PS: Political Science & Politics 48:425-429.

Mutz, Diana. 2011. Population-Based Survey Experiments. Princeton, NJ: Princeton University Press.

« Assignment 4: Designing an Experiment Quasi-Experiments(November15) Required Gerring [p. 273-90] DellaVigna, Stefano, and Ethan Kaplan. 2007. “The Fox News Effect: Media Bias and Voting.”

Quarterly Journal of Economics 122:1187-234. Campbell, Donald T., and H. Laurence Ross. 1968. “The Connecticut Crackdown on Speeding:

Time Series Data in Quasi-Experimental Analysis.” Law & Society Review 3:33-53. Dunning, Thad. 2008. “Improving Causal Inference: Strengths and Limitations of Natural

Experiments.” Political Research Quarterly 61:282-93. Kern, Holger Lutz and Jens Hainmueller. 2009. “Opium for the Masses: How Foreign Media

Can Stabilize Authoritarian Regimes.” Political Analysis 17:377-99. Recommended Achen, Christopher R. 1986. The Statistical Analysis of Quasi-Experiments. Berkeley, CA:

University of California Press. Angrist, Joshua, and Alan B. Krueger. 2001. “Instrumental Variables and the Search for

Identification: From Supply and Demand to Natural Experiments.” Journal of Economic Perspectives 15:69-85.

Diamond, Jared, and James A. Robinson, ed. 2011. Natural Experiments in History. Cambridge, MA: Harvard University Press.

Druckman, James N., Donald P. Green, James H. Kuklinski, and Arthur Lupia, eds. 2011. Cambridge Handbook of Experimental Political Science. New York, NY: Cambridge University Press.

Dunning, Thad. Natural Experiments in the Social Sciences: A Design-Based Approach. New York, NY: Cambridge University Press.

Gerber, Alan S., and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York, NY: W.W. Norton.

Sovey, Allison J., and Donald P. Green. 2011. “Instrumental Variables Estimation in Political Science: A Readers’ Guide.” American Journal of Political Science 55:188-200.

7

CaseSelection,CaseMethods,Sampling,andGeneralization(November22) Required KKV [Chapter 4] Ebbinghaus, Bernhard. 2005. “When Less is More: Selection Problems in Large- N and Small- N

Cross-National Comparisons.” International Sociology 20:133-52. Geddes, Barbara. 1991. “How the Cases You Choose Affect the Answers You Get: Selection

Bias in Comparative Politics.” Political Analysis 2:131-50. Gisselquist, Rachel M. 2014. “Paired Comparison and Theory Development: Considerations for

Case Selection.” PS: Political Science & Politics 47:477-84. Recommended Altonji, Joseph G., Todd E. Elder, and Christopher R. Taber. 2005. “Selection on Observed and

Unobserved Variables: Assessing the Effectiveness of Catholic Schools.” Journal of Political Economy 113:151-

Beach, Derek, and Rasmus Brun Pederson. 2013. Process-Tracing Methods: Foundations and Guidelines. Ann Arbor, MI: University of Michigan Press.

Bennett, Andrew, and Colin Elman. 2006. “Qualitative Research: Recent Developments in Case Study Methods.” Annual Review of Political Science 9:455-76.

Berinsky, Adam J., Gregory A. Huber, and Gabriel S. Lenz. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk.” Political Analysis 20:351-36.

Cohen, Jacob. 1994. “The Earth is Round (p < .05).” American Psychologist 49:997-1003. George, Alexander L., and Andrew Bennett. 2005. Case Studies and Theory Development in the

Social Sciences. Cambridge, MA: MIT Press. Gerring, John. 2007. Case Study Research: Principles and Practices. New York: Cambridge

University Press. Slantchev, Branislav, Anna Alexandrova, and Erik Gartzke. 2005. “Probabilistic Causality,

Selection Bias, and the Logic of the Democratic Peace.” American Political Science Review 99:459-62.

« Assignment 5: Selecting a Sample QualitativeInterviewsandParticipantObservation(November29) Required Fenno, Richard F. Jr. 1978. Home Style: House Members in Their Districts. Appendix: Notes on

Method: Participation Observation. [p. 249-95] Morris, Zoë Slote. 2009. “The Truth about Interviewing Elites.” Politics 3:209-17. Rivera, Sharon Werning, Polina M. Kozyreva, and Eduard G. Sarovskii. 2002. “Interviewing

Political Elites: Lessons from Russia.” PS: Political Science & Politics. (Also see other articles in the symposium in the same issue.)

Rubin, Herbert J., and Irene S. Rubin. 1995. Qualitative Interviewing: The Art of Hearing Data. Thousand Oaks, CA: Sage Publication. [Chapter 2]

8

Scott, James C. 1987. Weapons of the Weak: Everyday Forms of Peasant Resistance. Yale University Press. [Chapters 1 & 2]

Recommended Feldman, Martha S., Jeannine Bell, and Michele Tracy Berger, eds. 2003. Gaining Access: A

Practical and Theoretical Guide for Qualitative Researchers. Walnut Creek, CA: Altamira Press.

Goffman, Alice. 2014. On the Run: Fugitive Life in an American City. Chicago, IL: University of Chicago Press. [Appendix]

Mosley, Layna, ed. 2013. Interview Research in Political Science. Ithaca, NY: Cornell University Press.

Schatz, Edward, ed. 2009. Political Ethnography: What Immersion Contributes to the Study of Power. Chicago, IL: University of Chicago Press.

Soss, Joe. 2006. “Talking Our Way to Meaningful Explanations: A Practice-Centered Approach to In-Depth Interviews for Interpretive Research.” In Dvora Yanow and Peregrine Schwartz-Shea, ed. Interpretation and Method: Empirical Research Methods and the Interpretive Turn. New York, NY: M.E. Sharpe.

SurveyResearch(December6) Required Davis, Darren W., and Brian D. Silver. 2003. “Stereotype Threat and Race of Interviewer Effects

in a Survey on Political Knowledge.” American Journal of Political Science 47:33-45. Gonzalez-Ocanatos, Ezequiel, et al. 2012. “Vote Buying and Social Desirability Bias:

Experimental Evidence from Nicaragua.” American Journal of Political Science 56:202-17.

Mondak, Jeffrey J., and Mary R. Anderson. 2004. “The Knowledge Gap: A Reexamination of Gender-Based Differences in Political Knowledge.” Journal of Politics 66:492-512.

Schaeffer, Nora Cate, and Stanley Presser. 2003. “The Science of Asking Questions.” Annual Review of Sociology 29:65-88.

Recommended Prior, Markus, and Arthur Lupia. 2008. “Money, Time, and Political Knowledge: Distinguishing

Quick Recall and Political Learning Skills.” American Journal of Political Science 52:169-83.

Rosenfeld, Bryn, Kosuke Imai, and Jacob N. Shaprio. 2016. “An Empirical Validation Study of Popular Survey Methodologies for Sensitive Questions.” American Journal of Political Science 60:783-802.

Zaller, John R., and Stanley Feldman. 1992. “A Simple Theory of the Survey Response: Answering Questions versus Revealing Preferences.” American Journal of Political Science 36:579-616.

9

CommunicatingResearch(December13) Required Data Access and Research Transparenty (DART) web site: www.dartstatement.org Gerber, Alan, and Neil Malhotra. 2008. “Do Statistical Reporting Standards Affect What Is

Published? Publication Bias in Two Leading Political Science Journals.” Quarterly Journal of Political Science 3:313-26.

Kastellec, Jonathan P., and Eduardo Leoni. 2007. “Using Graphs Instead of Tables in Political Science.” Perspectives on Politics 5:755-71.

Lupia, Arthur. 2000. “Evaluating Political Science Research: Information for Buyers and Sellers.” PS: Political Science & Politics 33:7-13. [and see other essays in the symposium for further reading]

Maisel, L. Sandy, and Walter J. Stone. 1998. “The Politics of Government-Funded Research: Notes from the Experience of the Candidate Emergence Study.” PS: Political Science & Politics 31:811-7.

Sigelman, Lee. 1999. “Publication Bias Reconsidered.” Political Analysis 8:201-10. Recommended King, Gary and other authors. 1995. Essays on replication. PS: Political Science & Politics

28:444-99. King, Gary, Michael Tomz and Jason Wittenberg. 2000. “Making the Most of Statistical

Analyses: Improving Interpretation and Presentation.” American Journal of Political Science 44:347–61.

Lehrer, David, Janin Leschke, Stefan Lhachimi, Ana Vasiliu, and Brigitte Weiffen. 2007. “Negative Results in Social Science.” European Political Science 6:51-68.

Tufte, Edward. 2006. [Reflection on his 1975 APSR article.] American Political Science Review 100:686-7.

Wicherts, Jelte M., et al. 2006. “The Poor Availability of Psychological Research Data for Reanalysis.” American Psychologist 61:726-28.