Qualitative Statistics

Observation

• Can be used to infer correlation, but not causation– Therefore use different statistical tests• Experiments-look to see how different two things might

be• Observational Studies-look to show how closely two

variables relate to one another

• Clarity of what is being observed is essential

Types of Observation• Laboratory Observation

– Conducted in lab setting for control– Allow for ease in conducting, lower validity

• Ethological Observation– In natural environment– High validity, hard to conduct (observation can effect behavior)

• Participant Observation– Joining the group you are studying– Ethically questionable/Hard to keep records

• Epidemiology– Used most often in medical and environmental health research– A form of indirect observation using complicated statistical analysis to

find correlations

Collecting Data in Observational Research

• 2 difficulties in collecting this data– Selective Perception

• We focus on things we are interested in and lose a great deal of info in the process

– Anticipatory Schema• We also must deal with our tendency to look for that which we

already expect to see

• Precautions to avoid these pitfalls– Structure form of observation carefully

• Behavioral Coding• Time-Sampling

Behavioural Coding

• A system that uses a series of categoried for behavior, and fitting all observations that are made into one of these categories

• Usually worked out before starting through similar studies or pilot tests

• Categories must be reasonably comprehensive, but also limited for managing purposes

• Video recording can be done to allow coding to be done later, but camera does not pick up all that the eye can

Time-Sampling

• Often used with behavioural coding to record what is happening with a group over a period of time

• Set intervals are assigned and observations are only made during these intervals

• Can give a reasonably accurate picture of typical activities

Multiple Observers

• Uses independent observers• 2 advantages

– One observer may notice something another has missed– Allows researcher to measure the amount of agreement

between observers• Any data both agree on can be seen as reasonably valid

• Can also measure inter-observer reliability• Correlation between reports of different observers

– Above .8 reliable and acceptably accurat– If lower could be for 2 reasons

» Too much going on for one person to take in» Observers influenced by personal bias or expectations

Ethics of Observational Research

• Should always have full consent of person being observed• Feedback about observations purpose and findings needs

to be provided• Covert Studies need to follow ethical principles

– Utilitarian, cost-benefit criteria– The effects of the participants– Issues of individual integrity

• Passive Observations must be understood to have some effect on behavior– Active research can help to alleviate some of the behavioral

changes

Questionnaire Studies

• Allow researcher to collect info from large numbers of people

• Used as a starter for many studies to help generate Hypotheses

• Can be used to determine how widespread a phenomena is

• Vulnerable to response bias-people say what they feel is the right answer

Uses of Questionaires

• Info on– Facts and Knowledge– Past Behavior– Likely Future Behavior– Motives– Opinions and Attitudes

Design

• A good questionnaire must – Account for differences in population– Make itself crystal clear to every member

• Need to expect the unexpected response and design with that in mind

• Avoiding questionnaire fallacy is also important– Assumption you have found answer to behavior

when it may be limited scope of options to choose

Stages of Design• Working out the aims

– What is main point and hypothesis• Selecting the appropriate question styles

– How free is the response?– Does it fit analysis needs?

• Designing the questions– Need to be structured and phrased with aims in mind

• Piloting – Check for lack of clarity

• Revising– Go for perfection and check to make sure you can score appropriately

• Administering– Choose most suitable technique for topic you are researching

• Analysis– Report all valid statistical data, both Quantitative and Qualitative

• Reporting– Standard format

Common Mistakes

• Begin by writing out questions• Making the scope to broad– Creates indifference in respondents

• Not understanding purpose for analysis before administering

Designing Questions• Sequencing

– Arranging common questions together is important• Priming helps respondents think more clearly

– Funneling-Broad to Specific– Reverse Funneling-Specific to Broad

• Writing– Issues

• Leading Questions• Ambiguous Questions• Multiple-Content Questions• Implicit Questions• Over-Complex Vocab• Over-elaborate phrasing• Patronising Tone

• Complexity– Open Questions

• Pros-High Ecological Validity and Open up New Possibilities• Cons-Hard to analyze

– Require Content Analysis- Looking for common trends or qualitative analysis

– Closed Questions• Pros- Answers pre-specified, makes for easy analysis• Cons- Low validity

Administering• Sampling Technique

– Want Representative Samples• Best ways to do this

– Random Sampling-Every member has an equally likely chance of being selected

• Other Ways– Quota sampling-Selecting people who fit into certain categories and setting quotas that match criteria– Stratified Sampling- Subgroups identified and then random sampling applied to get members from

each subgroup– Opportunity Sampling-Whoever is available, most likely to become biased– RDD-Random Digit Dialing, privacy issues

• Distribution Method– 4 main ways

• Face to face interviewing• Handout Questionnaires• Postal Questionnaires• Telephone Questionnaires

Improving Response Rates

• Prior Contact• Ease in Responding• Small Rewards

Ethical Issues

• How are they administered– No Coercing – Maintain Privacy (phone interviews)

• What is done with the findings– Confidentiality Key

Psychometrics

• 2 types – Attitude scales– Psychometric Tests

Attitude Scales

• Construction more tightly specified and much more rigorous than questionnaires

• Constructed to avoid response bias– Can set up a lie scale-questions scattered

throughout to show if someone is trying to portray themselves in a certain fashion

– Scale should be balanced between yes and no responses

Examples of Attitude Scales• Direct-Likert Scale

– Steps to using• Decide on topic of investigation• Get several people together and generate broad set of statements about topic• Discard repetitive or badly worded statements• Present list to separate group to determine if statements are positive or negative with

regards to topic• Discard ambiguous statements• Select equal number of positive and negative statements and arrange in random order• Allocate 5 point scale• Combine into full Likert Questionaire

• Indirect- Semantic Differential– Asks the respondent to express how the target of the attitude measure would

rate on a number of dimensions– Tries to capture the depth of someone’s attitude more than can be seen in

conventional ways using bipolar adjectives

Assumptions of Attitude Scales

• Attitudes can be expressed in verbal statements

• Same statement has the same meaning for all participants

• Attitudes expressed as verbal statements can be measured and quantified

Psychometric Tests• Used to gain insight into aspects of human

psychology that are normally hidden, or not immediately apparent

• Information can be used to help individual make more informed choices

• 2 types– Idiographic-help therapist gain insight into person’s

ideas or problems– Nomothetic-used to compare people with one

another

Test Construction• Very rigorous including several pilot stages• Construction based on:

– Reliability• Test-retest-same test administered twice to same participants on different occasions

– -1 to 1– .8 acceptable– Disadvantage-people may remember responses from first time

• Split-half-test split in half and given to participants at two times– Spilt is either arbitrary or cumbersome

• Alternative forms-2 equivalent tests generated and given on 2 occasions– Disadvantage- transfer of training and familiarity

– Validity• Face validity- based on appearance

– Not good for psychometric tests

• Criterion validity- comparing to some other standard which already exists– 2 types used in psychology

» Concurrent validity-comparing scores to those on an existing test» Predictive validity—using scores to predict future success

• Construct validity-does test reflect theoretical constructs on which it was based– Questions in subsets scrutinized to see if they correlate to one another more than to questions from other subsets

• Ecological Validity-does test reflect real life equivalents

– Standardization• Procedures must be followed• Population norms must be established• Test outcomes need to fit criteria of other tests like it

Ethical Issues of Psychometric Measurement

• Deception used to keep people from fully understanding intent

• Technical Instrument– Lie scales

• Uses can be unethical socially• Feedback can be falsified• Test must be administered by a professional

Interviews

• Six Types– Hostile-interrogation– Limited Survey-Market Research interview– Rapport-personal experience interview– Asymmetrical Trust-Doctor-Patient interview– Depth-Anthropological Interview– Phenomenological-Specialist Qualitative Research

Interviews

Interviewer Effects

• Non-verbal signals- can greatly skew an interview

• People adjust answers for what they feel are appropriate responses for the interviewer based on social knowledge

• These are called interviewer effects and must be considered in any study

Interview Skills• Requires Sensitivity• Must develop a rapport• Requires Verbal and Nonverbal skill sets

– Verbal• May require standardization of responses to keep verbal influences to a minimum (can be seen as artificial)• Speech registers must be noted, how something is being said before responding with a standardized response

– 5 Types of Registers» Frozen» Formal» Consultative» Casual» Intimate

• Reflection, sounding back, is an important skill• Amplifying, broadening the scope is a must• Non-commital agreement, pulling out more without giving your views is essential

– Non-Verbal• Must pay attention to details (All Important)

– Appropriate Eye contact– Posture– Gestures– Tones of voice

Conducting Interviews

• Establish Range of Information being sought• Determine what type of information needs to

be obtained and which type of method will best help you determine your answer

• Consider the needs of your interview schedule

Structured Interviews

• Very similar to a questionnaire, but with interviewer present

• Can use probe questions to redirect statements to where interviewer is looking, but are highly standardized

• Require same level of planning as developing a questionnaire

• Negative is that its rigidity and lack of spontaneous interaction cause changes in interviewee

Semi-Structured Interviews

• Allow more freedom to ask questions and to answer them

• Contains a number of closed questions which require specific answers

• Clinical Interviews-semi-structured interviews used in a specific setting for research purposes

Open-Structured Interviews

• Seem more like a natural conversation• Allows interviewee to answer questions as

they feel while being directed towards answers

• Primer questions often used to warm people up

• Requires Tape Recorder (Can raise ethical issues)

Analysing Interview Data• Structured- very easy to generate statistics for since choices are limited• Semi-Structures- Requires content analysis

– Info broken into themes– Indications are made of every them mentioned (frequency) and descriptive

statistics can be generated• Number of mentions does not always show importance of a theme

– Can also look for emotively loaded words• Requires answers collected word for word• Can give clues to attitudes underlying what people say

– Can also use salient quotes• Again requires word for word collection• Look for quotations relevant to topic being discussed

• Open-Structure Interviews– Can best utilize emotively loaded words and salient quotes– Very time consuming and complicated

Stages of Interview Research

• Define Aims• Decide form interview will take• Plan content of the interview• Interview Schedule Planned– Same as questionnaire– Requires pilot testing

• Conduct the interviews• Analysing and Summarizing the data

Ethical Concerns

• Putting self outside of professional competence

• Gain informed consent– Recording– Reporting– Using Quotes

• Insure confidentiality of interviewee and anyone mentioned

Case Studies

• Focus on single cases and explore them using more than one research method

• Allow far more detail in exploration• Can be used to provide theoretical insights to

stimulate other research• Regarded throughout history as “unscientific”– Completely false• Much had to do with behaviourist influence• Also feared the dangers of anecdotal evidence

Uses of Case Study

• Used to explore aspects of human experience• Insight into how psychological processes may

be operating• Not used to develop general laws about

behavior

Advantages of Case Studies

• Stimulating New Research• Contradicting Established Theory• Giving new insight into phenomena or

experience• Permitting investigation of otherwise

inaccessible situations

What is a case?

• Does not have to mean a single person• 5 Types– Person– Group– Location– Organization– Event

Methods used in a Case Study

• Multiple methods can be used besides interviewing– Tasks– Repertory grid– Diary

Triangulation

• Using several different research methods to explore the same phenomenon

Systems Analysis Approach

• Looks at 4 major dimensions of the system– Elements-The separate parts which make up the

system– Order-coherence between elements ex. patterned

interactions, mutual understandings and expectations

– Process-Changes over time, or transactions or exchanges

– Functions-the goals or outcomes of activity within the system

Psychological Field

• A way of expressing the complexity of social experience by organizing it into different dimensions– Most important to psychologists is psychological dimension

• Involves aspects of individual experience and identity

– Other important dimensions• Spatial Dimension-relates to the places or locations within which a

particular experience is set• Cultural Dimension-symbols and social rituals involved in its

occurrence• Historical Dimension-how do previous or related events influence

how this one is perceived or dealt with• Social Dimension-involves relationships, lifestyles and social

networks

Vignettes

• Ways of summarizing observations or experiences succinctly

• Can be used to draw comparisons and see what others might not see– Requires Thematic Qualitative Analysis

Repertory Grids

• Allow for the research of idiographic measures– Exploring the distinctive qualities of the individual

• Based on the idea of personal constructs– We all have our own subjective view of the world

based on our own theories about how the world works

• The Grid enables a researcher to explore the personal constructs of an individual– Allows for distinguishing of core constructs and

subordinate constructs

Laddering

• Interviewing technique which uses questioning to move from subordinate constructs to their core constructs

• Style needs to be kept light• Can raise ethical concerns

Disadvantages of the Case Study

• Uniqueness leads to low reliability ratings• Subjective feelings of researcher can influence

case study• Selectivity of reporting• Retrospective case studies can be incorrect in

assumptions• Can not generalize results

Ethical Issues of Case Study

• Confidentiality• Labeling and attribution• Stepping out of professional competence

Ethnography

• Concerned with life as it is lived from day to day

• Key Aspects– Data collected through range of sources– Behavior studied in every day contexts– Early stages of data gathering unstructured by the

researcher– Exploration conducted in depth

Analyzing Documents

• Uses evidence provided by documents to understand human experience

• Can also be used to analyze what psychologists have learned so far

The Diary Method

• Respondents required to keep notes about a particular type of experience or event

• Can be structured or open-ended• Ultimately must make repeated recordings

over time to give clues regarding:• Issues of development• Change• Recurrent experieince

Advantages of Diary Method

• Familiarity• Cost-Effective• Data Sequencing• Intimacy• Exploration• Spontaneity• Historicity

Disadvantages of the Diary Method

• Control of Content• Dropout• Poor Recruitment– Artefacts-distortions in data relating to

uncontrolled variables or artificial situations• Veracity and verification-authenticity• Reactance-change in behaviors due to diary

Designing Diary Studies

• Usually pen and pencil, but can take other forms• Are there set questions? (Good for Quantitative Analysis)

– Makes it easier to compare data• Is there room for personal reflection? (Better With

Qualitative Analysis) – Provides more genuine answers

• Many times diary studies use open-ended format– Tend to be more idiographic (focus on uniqueness) than

nomothetic (focus on comparisons)• Instructions must be clear and unambiguous

– Provide examples/may contaminate responses

Reminders in Diary Studies

• Keeping in contact with participants can reduce dropout numbers in diary studies

• Some form of incentive program like in questionnaire studies has been used by some researchers

Diaries and Documents

• Using diaries and historical documents that were not developed for psychological studies is called document analysis– Goal: to see what they reveal about underlying or

recurrent psychological processes and mechanisms

• Various forms of media can be considered documents for analysis

Problems with pre-existing material

• May not provide full info the researcher needs• Material may have been re-worked keeping

out full subjective experience of the individual• Actual documents from past with intentions of

being personal may include shorthand or codes that make it difficult for a researcher to fully understand

Meta-Analysis

• Documents consist of published research findings

• Looks to make sense of multiple replications of the same study and various findings that are not exactly the same

• Helps to summarize overall implications of research findings to date

• Takes narrative analysis a step further to eliminate subjectivity

Meta-Analysis vs. Narrative reviewNarrative Review Meta-Analysis

Selective, not comprehensive Incorporates every known study on topic

Focus only on statistical significance of studies

Takes into account effect size as well as statistical significance

Difficulty dealing with conflicting results Deals with conflicting results by identifying sources of error and artefact

Conclusions reflect aims and concerns of individual researcher (difficult to replicate)

Conclusions are replicated by other researchers

Valuable in developing new theoretical insights

Used only for summarizing existing research findings

Conducting a Meta-Analysis

• Select relevant independent and dependent variables• Identify and obtain all relevant studies• Identify characteristics which may be pertinent to the

outcome of the study• Calculate effect size estimates for relevant independent

and dependent variables• Calculate mean and standard deviation of the effect

size across the studies• Compare the effect size summaries with characteristics

identified, identify connections or correlations

Effect Size and Statistical Significance

• ES=Mean of Experimental- Mean of Control Standard Deviation of Control Group

Problems of Meta-Analysis

• What are boundaries of a research topic?• What about poorly designed studies?• File Drawer Problem for failed studies– Can be addressed by Rosenthal’s formula which

shows how many failed studies would need to exist to cancel out the effect

Descriptive Statistics• Allow us to describe our data• Can only be used if we covert Qualitative data into a Quantifiable Medium• Can explain via:

– Numbers• Measures of Central Tendency (All should line up if data is not skewed)

– Mean-average – Median- Score which falls in the center of the data set– Mode-most commonly occurring score in the data set

• Measures of Dispersion (Tell us about how scores are distributed)– Standard Deviation- Lets us calculate how different a score is from the mean

– Graphical Images• Graphs- demonstrate how a particular measure varies in response to some other variable• Bar Charts-Used with data that belongs to different categories to compare how often each

category is represented in a set of scores• Pie Chart- shows us proportions for an entire set of scores• Scattergrams- enable us to see correlations and their direction