View
219
Download
0
Category
Preview:
Citation preview
Research Methods in Social Relations
Professor Mike Gallivan
Georgia State University
Atlanta, Georgia, USA
Class 5: June 22, 2009
Overview of Class 5 today Remind students about tomorrow’s assigned reading We will discuss some material that I did not have time
to present last Thursday / Friday Different modes of data collection Scale uni-dimensionality vs. multi-dimensionality
New material for today: Multiple regression analysis
select only certain groups “dummy coding” for nominal variables
Different types of measurement scales summative (Likert) scales are most popular also Differential, Guttman and Semantic differential
Explain how qualitative research process differs
Date Readings in 6th ed.
red book
Readings in 7th ed.
green book
Special comments
June 22
7: 146-169(omit 7: pp. 159-162)
7: 158-177(omit 7: pp. 167-170)
Finish data analysis team project. Focus on Likert scales (Semantic, differential and Guttman scales are used less)
June 23
6: 128-142article about response rate on website You can omit:p. 360 (formulas)p. 369-370 (Adv. CFA and SEM)
8: 181-195article about response rates on website. You can omit:p. 360 (formulas)p. 369-370 (Adv. CFA and SEM)
Paper from JAIS by Sivo, Saunders et al. (2006). Discuss issues related to response rate. Student will read and critique a study about its response rate
June 24
19: 463-47619: 457-460 (repeat)
20: 521-53619: 483-491
Writing academic research papers (skip section about meta-analysis in 7th ed.)
Questions you asked about U.S.A. What is your favorite NBA sports team?
My hometown is Boston So, my favorite team is Celtics!
Are all American people good at dancing? No, definitely not! Many Americans are very bad dancers!
Review of last week’s classes Please take 1 minute to write one idea or
lesson you learned Thursday or Friday that you will remember when course is finished!
Review of Discriminant Validity in SPSSExploratory factor analysis (EFA)
This shows which items go together In this method, items that load onto the same
factor can prove convergent validity In this method, items that load onto separate
factors can prove discriminant validity Convergent + discriminant validity = construct validity
Performing EFA using SPSS Procedure is called “Data reduction – Factor” Results are easier to understand if you do:
select “Options” and then “sorted by size” also “suppress values with absolute value < 0.40 also select Rotation: “Varimax” or “Direct oblimin” difficult to interpret if construct is multi-dimensional
Review of Multi-dimensionalityMulti-dimensional vs. uni-dimensional
Some constructs have > 1 dimensionFamily’s socio-economic status
Amount of salary income (from formal job) Amount of investment income (from bank) Amount of other income (from gambling, eBay) Level of parents’ education
Individual creative style construct Originality (number of original ideas generated) Rule-conformity (willing to follow rules) Efficiency (perform tasks in time-efficient way)
Multi-dimensional constructs in data set Section #9 – Psychological and physical strain Items about psychological strain factor separately
Topics we did not cover in Class 3Summary of Contents from Ch. 10
(Ch. 5 in green book, pp. 96-110)
Modes of direct questioning Paper-and-pencil questionnaire Face-to-face (in-person) interviews Telephone interviews Web-based surveys on the Internet Experience sampling (“diary methods”)
Other modes of measurement (indirect) Collateral data (another person also gives data)
Observing others (it may be unobtrusive) We will not discuss these methods in detail
Topics we did not cover in Class 3Summary of Contents from Ch. 10
(Ch. 5 in green book, pp. 96-110)
Paper-and-pencil questionnaire Advantages
Authors claim that paper surveys are inexpensive Minimize interviewer (researcher) bias Less time pressure for subjects Subject has feeling of being anonymous
Disadvantages Subjects cannot ask a question if they are confused Often a very low response rate (e.g., < 10%) You may not know how non-respondents are similar to or different
from the survey respondents Perhaps the respondents are 60% female, 40% male, but the non-
respondents are 30% female, 70% male
Topics we did not cover in Class 3Summary of Contents from Ch. 10
(Chapter 5 in green book) Face-to-face (in-person) interviews
Advantages Can “probe” the subject further for more information Subject can ask the researcher for clarification This may create a higher quality of information Can provide visual aids (maps, photos, examples)
Disadvantages Potential for large interviewer effect (bias) Especially for sensitive or personal questions May not want to give an honest answer to question
“Have you ever had an unwanted pregnancy?” This is called social desirability bias
Some subjects might want to impress the interviewer “How much alcohol do you normally drink in 1 week?”
Both of these effects might hurt the information quality
Data collection modes Topics we did not cover in Class 3Summary of Contents from Ch. 10
(Chapter 5 in green book) Telephone interviews Advantages
Combines advantages of both surveys + interviews Can probe subjects for further information Cost is low, compared to face-to-face interviews
Disadvantages More expensive than paper-and-pencil surveys Chance for response bias
People with phones might differ from those without Famous example in American presidency in 1960s Especially for sensitive or personal questions
If you rely on a telephone directory In U.S., telephone directories do not list mobile phones Young people only have mobile phones (no land phone) So, you might have an age bias effect in your results The solution to this problem is random-digit dialing
Data collection modes Topics we did not cover in Class 3Summary of Contents from Ch. 10
Web-based surveys on Internet Advantages
1: Avoid the problem of transcript errors 2: Very, very cheap to do 3 Easy to conduct 4: No effort required to mail survey back 5: More voluntary than a face-to-face 6 Faster, easier and “more fun” for subjects
Disadvantages Response rates are often very low (e.g., <1%) How many subjects did you send survey to? This information is required to compute response rate Not a good idea to just post survey to a website unless you know how many
total people will see the survey
Experience sampling (“diary methods”) Can use a pager to remind subjects to record data These methods can be used in medical research
Summary of Contents from Chapter 3: pp: 53-66
(Ch. 4 in green book, pp. 85-90 and Ch. 7: 152-158)
Single-item vs. multiple item scales Single-item is acceptable sometimes
Some examples of single-item measures How old are you in years?
Do we need to also ask “how old are you in months?” What is your sex? (male or female) What is your major? What is the city where you were born?
Advantages to multiple-item scales multiple items required to calculate internal reliability journal reviewers prefer multiple-item scales you can compute amount of measurement error
measurement error = 1 – Cronbach a
Summary of Contents from Chapter 7: pp: 146-169
(Ch. 7 in green book, pp. 158-177)
Different types of rating scales Graphic rating scales Itemized rating scales Comparative rating scales
Source of the ratings Self-ratings (individual)
Often the individual is best judge of their own opinions
Parent ratings of child’s behavior This is common in medical surveys for young children
Peer or co-worker or supervisor ratings Sometimes more accurate data from another person
Summary of Contents from Chapter 7: pp: 146-169
(Ch. 7 in green book, pp. 158-177
Problems with all measurement Social desirability bias – try to give answer subject
thinks the interviewer wants to receive “Do you ever drink > 3 drinks at dinner or party?” “Have you had premarital sex?” The direction of bias effect might differ depending:
If the researcher is a peer member (e.g., same age) If the researcher is much older or a medical doctor
Sometimes the wording creates bias
My doctor asked me last month: “You always wear sunscreen, don’t you?” What would be a better way to ask question?
Summary of Contents from Chapter 7: pp: 146-169
(Ch. 7 in green book, pp. 158-177
Remember definition of measurement in Class 2: Any measurement item is comprised of 3 parts:
The construct of interest (general intelligence) Other constructs not of interest (English language skill) Random measurement error (mistakes in answer)
Observed score (e.g., each survey item) = True score + systematic error + random error Variable of interest + variable not of interest + random error
Multiple item scales help calculate random error Halo bias (generosity error) is a systematic error
Example of personal attractiveness (an attractive person is often considered smarter, more competent, trusted)
Example of website attractiveness (a website that is more visually attractive is often trusted more by users)
Summary of Contents from Chapter 7: pp: 146-169
(Ch. 7 in green book, pp. 158-177
Developing multiple-item scales Domain sampling
start with very broad range of items have some positive-worded items
“I like Presidential candidate Obama”
also have some negative-worded items “I don’t trust Presidential candidate Obama”
Process of pilot testing the survey Small sample to identify wording problems Modify or delete confusing items Then test with a much larger sample of subjects
Example of Survey Development: Personal Innovativeness with IT
Initial survey items:
Measuring multi-item scales Examine Cronbach a (internal reliability)
Delete any items that reduce a < 0.70 For all items retained in the final scale:
Average the items together
Examine Exploratory Factor Analysis This is a more complex method
Some researchers prefer this method of “factor scores” Be sure to select “Options”
Missing values: exclude cases “pairwise” “Suppress absolute values < 0.40”
Then choose “Scores” and “Save as Variables” This allows each item to have a different weight
(e.g., item with 0.84 factor loading will have twice as much weight as another item with 0.42 factor loading)
Example of multi-item scales using Psychological & Physical Strain items
The next 2 slides show items from the survey
Section #9: During the past week…
(Circle one number per item)
1. I was bothered by things that usually don't bother me.
2. I did not feel like eating; my appetite was poor.[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
3. I felt that I could not shake off the blues even with help from my family or friends.[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
4. I had trouble keeping my mind on what I was doing.
[1]Rarely or None of the
Time (Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
[1]Rarely or None of the
Time (Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
[1]Rarely or None of the
Time (Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
5. I felt depressed.
[1]Rarely or None of the
Time (Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
6. I felt I that everything I did was an effort.
[1]Rarely or None of the
Time (Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
9. I talked less than usual.[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
10. I felt lonely.[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
11. I had crying spells.[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
12. I felt sad.[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
13. I could not “get going.”[1]
Rarely or None of the Time
(Less than 1 Day)
[2]Some of the Time
(1-2 Days)
[3]Occasionally or a
Moderate Amount of Time(3-4 Days)
[4]Most or All of the Time
(5-7 Days)
8. My sleep was restless.
Results from Factor AnalysisThis is first result table showing the number of factors
Initial Eigen-values
Compo-nent
Total % of Variance
Cumulative %
1 5.931 45.627 45.627
2 1.087 8.359 53.986
3 1.026 7.892 61.878
4 .905 6.963 68.841
5 .703 5.405 74.246
6 .676 5.200 79.447
Exploratory Factor Analysis resultsThis is a very messy 3-factor resultMost items load on Factor 1 and 2
A few items load separately on Factor 3
Rotated Component Matrixa
.814
.812
.736
.661 .426
.658
.529 .522
.501 .477
.885
.512 .638
.509
.744
.716
.449 .534
WEK_MIND
WEK_NOGO
WEK_EFRT
WEK_DEPR
WEK_CRY
WEK_BLUS
WEK_LONE
WEK_SLEP
WEK_SAD
WEK_BOTH
WEK_EAT
WEK_FEAR
WEK_TALK
1 2 3
Component
Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.
Rotation converged in 5 iterations.a.
Review Data Analysis AssignmentWhich surveys items are Nominal?
Other names for “nominal” items Categorical or dichotomous (if just 2 categories) Chi-squared (“crosstabs”) analysis that Dr. Doug
Rice had you do Friday was a dichotomous item Items? (section # and item #)
Which surveys items are Ordinal? Items? (section # and item #)
Recommended