Embed Size (px)
Citation preview
Dr Juliet Hassard
Deputy Director, Centre for Sustainable Working Life
Lecturer in Occupational Health Psychology
* Secondary Data Analysis: An Introduction
*Overview of presentation
*What is secondary data analysis?
*Types and sources of data
*Opportunities, limitations, and challenges
*Ethics
*Thinking forward: funding and publishing.
*Secondary data analysis: why
and what it is?
*The use of secondary data, or existing data that are freely available to researchers who were not involved in the original study, has a long and rich tradition in the social sciences [1].
*Sociology, economics, etc.
*Why collect new data, given the wealth of existing data sets that can be used to answer important questions?
*Longitudinal & large sample sizes.
* Traditionally, the field of psychology (any many of those within it) have dismissed the importance and value of studies using secondary data.
*But times are changing…….
*Why use secondary data?
*To ask and answer important questions. For example,
*To understand the longitudinal nature of relationships.
*To understand group differences, trends over time?
*To explore new and emerging social phenomena.
*Why secondary data analysis?
*More data (and types of data) are being collected (and available!) then ever before.
*There is a unique opportunity to explore this ever growing source(s) of data, and to ask important research questions.
*Types and sources of data….
*Let’s get creative……..
*In small groups of 3-5. Discuss and outlines 4-5 different types of data/ types of information that could be used to investigate an important psychological research question.
Online support groups
Second life
App technology
Blogs Chat forums
Published business reports
*Where do I find data?
*The UK Data Service
* https://www.ukdataservice.ac.uk/
* Census data
* International macrodata
* Longitudinal studies
* Qualitative/mixed methods
* UK surveys
*The National Data Service
* http://www.nationaldataservice.org/about/
* Individual studies may have different access points.
* E.g., Whitehall II Study, UCL.
*Secondary data is everywhere –
some in the public forum.
*Examples
*Online support groups:
* COULSON, N.S., 2015. Exploring patient's engagement with web-based peer support for Inflammatory Bowel Disease: forums or Facebook? Health Psychology Update. 42(2), 3-9.
*Longitudinal data (Whitehall II survey)
* Kouvonen, A., et al . (2011). Negative aspects of close relationships as a predictor of increased body mass index and waist circumference: the Whitehall II study. American journal of public health, 101(8), 1474-1480.
*Twitter, Instragram…..
* Whiting, R., & Pritchard, K. (2015). “Big Data? Qualitative Approaches to Digital Research", Qualitative Research in Organizations and Management: An International Journal, Vol. 10 Iss: 3, pp.296 - 298
*Advantages, Limitations, &
Challenges
Low response
rate
Small sample size
Reliance on convenience
samples
Access to high quality
measures
Limited money &
resources to collect
primary data
Limited scope for extensive
comparative research (across
groups or internationally)
Correlation does not equal causation
‘Traditional’ Challenges in Psychological
Research
High attrition rates
*Advantages
*The data has already been collected.
*Save time – primary researcher does not have to design study and collect new set of data.
*The types of data that are typically collected tend to be higher quality than could be obtained by individual researchers.
* Typically longitudinal, have large sample sizes that have been obtained using elaborate sample plans.
Ref: Trzesniewski et al., 2011
*Advantages
*Learning how to work with, manage and analyse secondary data can provide individual researchers with the raw materials to make important contributions to the scientific literature
*… using data sets with impressive levels of external validity.
Ref: Trzesniewski et al., 2011
*Advantages
*Open-source approach to research
*Replicate findings using similar analyses
*Encourages careful reporting and justification of analytical decisions.
*Allows researchers to test alternative explanations and competing models.
*Encourages transparency, which in turns help facilitates good science.
Ref: Trzesniewski et al., 2011
*Disadvantages
*The data has already been collected!!!
*You may not have all the information on how or why certain types of information was collected.
*You may not know of any particular problems that occurred during data collection.
*Sometimes you are left wanting more …..
*Disadvantages
*The temptation: a statistical fishing trip.
*Great research is driven by a good research question that is strongly underpinned and shaped by theory.
*The purpose of analysing data is to refine the scientific understanding of the world and to develop theories by testing empirical hypotheses.
* “Mo Money Mo Problems” - Mo Data, Mo Temptations ?
*A note about statistical power. Ref: Trzesniewski et al., 2011
*Disadvantage
*Considerable time and effort:
* is invested by the researcher to understand the nature and structure of a data set.
* is needed by the researcher to explain and justify the theoretical and analytical approached used.
*Although, I would argue there is real advantages to the time invested in doing this.
Ref: Trzesniewski et al., 2011
*Disadvantage
*Measures in these datasets are often abbreviated. Often because the projects themselves were designed to serve multiple purposes and to support a multidisciplinary team.
* Shortened measures, mix-levels of data, and single items measures.
*These datasets often have impressive levels of breadth (many constructs are measured), but often with an associated cost in terms of depth of measurement.
*Therefore, measurement issues are ~ therefore ~ one of the major issues in secondary data analysis
* These issues often require quite a bit of conceptual consideration & defending in the peer-review process.
Ref: Trzesniewski et al., 2011
*Challenges
*A good grounding in psychometrics and Classic Test Theory.
*You need to carefully consider and evaluate the trade-offs in reliability and validity.
*You need to defend your position when writing up.
*You need to understand how measurement issues frame your findings; and, in turn, your interpretation of your findings. Ref: Trzesniewski et al., 2011
*Practical & Methodological
Challenges
*Creating and managing data files
* Data inventory
* Research journal
*Approach to missing data and data screening procedures
*Use of and/or development of constructs
*Use of proxy variables
*Development & testing of composite measures
* Single item measures
*Accounting for the data structure in your analysis
*Case study: An Example
*MODELLING GENDER-RELATED DIVERSITY IN PSYCHOSOCIAL PROCESSES
AND WORK-RELATED WELLBEING: PATHWAYS AND MECHANISMS
*The aim of the doctoral thesis was to develop and test a theoretical model seeking to describe the aetiological role of psychosocial processes, in and out of the workplace, in predicting gender-related diversity issues in men’s and women’s health at a structural/population level.
*An iterative multi-stage methodology was utilised to develop and test the proposed theoretical model.
*Case Study: Methodology
Stage one
• Literature review – Theoretical framework
Stage two
• Identification of suitable source of data
Stage
three
• Data review (data inventory)• Measurement development and testing• Data cleaning
* Case study
*European Working Conditions Survey
*Pan-European cross sectional survey of working conditions, worker’s health and safety, and living conditions (n = over 40, 000 workers)
*Now on the 6th wave of data collection.
*The survey as evolved over time asking more questions.
*Survey items are informed and based on contemporary theory
*The measures used are not always based on a validated psychometric measures
*Single items vs. composite measures?
*Case study: Single item
measures
*The vast majority of latent conceptual constructs are complex and multifaceted in nature.
*Consequently, the use of a single item as a theoretical concept may not yield an accurate, comprehensive, and reliable measurement of the given construct of interest.
*Case study: Measurement
error
*The guiding premise by many in the scientific community is that multiple responses reflect the “true” response more accurately than does a single response.
*Imprecision in measurement is one of the key causes (although not the sole cause) of measurement error.
*Measurement error creates ‘noise’ to the observed variables.
*Case study: Implications of poor measurement
*Inaccurate and unreliable measurement of a concept results in key concerns regarding the overall validity and reliability of the hypotheses tested using this (or these) given measurement(s).
* It is generally agreed/ suggested that research findings that are valid, reliable and generalizable, are built on a solid foundation of accurate and consistent measurement.
*Composite measures
*The primary objective of creating a series of summated (or composite) scales is to avoid the exclusive use of, or dependence on, single item constructs where possible.
*The use of several variables as indicators provides an opportunity to represent differing facets of a given concept, with the aim of yielding a more well-rounded perspective and, arguably, a better measurement of the given concept
*Thinking about ethics
*A note about ethics
*Researchers need to ask: how was consent obtained in the original study? Where sensitive data is involved, we cannot/ should not assume informed consent.
*Given that it is usually not feasible to seek additional consent, a professional judgement may have to be made about whether the use of secondary data violates the contract made between subjects and the primary researchers.
*Growing interest in secondary data make it imperative that researchers in general now consider obtaining consent, which covers the possibility of secondary analysis as well as the research in hand.
* This is consistent with professional guidelines on ethical practice
Heaton, J (1998). Secondary analysis of qualitative data. Social Research Update (issue 22). See: http://sru.soc.surrey.ac.uk/SRU22.html
*Some thoughts on writing up
*Can you publish secondary data analysis – yes!
*Never forget: the central role of theory.
*Be detail orientated!
* Justifying your research question is important, but you also need to be prepared to justify and outline the logic of your analysis framework and approach.
*Understand and reflect on how the research design or any experienced methodological issues of your secondary data may impact or frame the interpretation of your results.
*Conclusion
*Secondary data analysis is an important and useful research methodology.
*There are many benefits and strengths to using secondary sources of data.
*But there also important pragmatic and methodological challenges that face researchers.
*Suggested reading
* Trzesniewski, K. H., Donnellan, M., & Lucas, R. E. (2011). Secondary data analysis: An introduction for psychologists. American Psychological Association.
* Vartanian, T. P. (2010). Secondary data analysis. Oxford University Press.
* Heaton, J. (2008). Secondary analysis of qualitative data: An overview. Historical Social Research/Historische Sozialforschung, 33-45.
* Hinds, P. S., Vogel, R. J., & Clarke-Steffen, L. (1997). The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qualitative Health Research, 7(3), 408-424.
