View
225
Download
0
Category
Tags:
Preview:
Citation preview
1
Chapter 2
Methods and Statistics
in I-O Psychology
Roy
alty
-Fre
e/C
OR
BIS
2
Module 1: Science
• What is science?
• Science has common methods• Science is a logical approach to investigation
– Based on a theory, hypothesis or basic interest
• Science depends on data– Gathered in a laboratory or the field
3
Common Methods (cont'd)
• Research must be communicable, open, & public
– Research published in journals, reports, or books
1) Methods of data collection described
2) Data reported
3) Analyses displayed for examination
4) Conclusions presented
4
Common Methods (cont'd)
• Scientists set out to disprove theories or hypotheses– Goal: Eliminate all plausible explanations
except one
• Scientists are objective– Expectation that researchers will be objective &
not influenced by biases or prejudices
5
Role of Science in Society
• Expert witnesses in a lawsuit– Permitted to voice opinions about practices
– Often a role assumed by I-O psychologists
6
Daubert Challenge
• Challenging testimony of an expert on the grounds it is not scientifically credible
• Daubert v. Merrill-Dow, 1993– Resulted in introduction of a method for
distinguishing between “legitimate science” & “junk science”
7
Scientific Testimony in Court
• Theories presented in court must:– Be recognized by particular scientific area as
worthy of attention– Be peer reviewed or subjected to scientific
scrutiny– Have a known “error rate”– Be replicable or testable by other scientists
8
Module 1 (cont'd)
• Why do I-O psychologists engage in research?– Better equip HR professionals in making
decisions in organizations– Provide an aspect of predictability to HR
decisions
9
Module 2: Research
• Research design– Experimental
• Random assignment of participants to conditions
• Conducted in a laboratory or the field
– Quasi-experimental• Non-random assignment of participants to conditions
10
Research Design (cont'd)
• Non-Experimental– Doesn’t include “treatment” or
assignment to different conditions
– 2 common designs:• Observational design• Survey design
11
Methods of Data Collection
• Quantitative methods– Rely on tests, rating
scales, questionnaires, & physiological measures
– Yield results in terms of numbers
C. Borland/PhotoLink/Getty Images
12
Methods of Data Collection
• Qualitative methods– Include procedures like observation, interview,
case study, & analysis of written documents– Generally produce flow diagrams & narrative
descriptions of events/processes
13
Quantitative & Qualitative Research
• Not mutually exclusive
• Triangulation, (Rogelberg & Brooks, 2002)
– Examining converging information from different sources (qualitative and quantitative research).
14
K Lewin
• B = f (p*e)– Behavior is a function of
• Person X environmental influences
15
Experimental v. Corr research
• “I” side: focus on Individual differences– Person attributes:
• E.g. Personality, behaviors, cognitive ability
• “O” side: focus on Environmental influences– Situation variables:
• E.g. work conditions, leadership style, pay for performance
16
I v. O
• Which is most likely to use – Experimental designs?– Correlational designs?
• Why?
17
Generalizability in Research
• Application of results from one study or sample to other participants or situations– Benefit of using theory
• Every time a compromise is made, the generalizability of results is reduced
18
Sampling Domains for I-O Research
Figure 2.1: Sampling Domainsfor I-O Research
19
Observational Unit
• Worker
• Team
• Department
• Organization
• Industry
• Others?
20
Measurement Unit(one of something)
• Identify a measurement unit for:– Worker’s performance score– Years of experience– Absenteeism– Motivation– Sales performance– Cognitive ability
21
Control in Research
• Experimental control– Influences that make results less reliable or
harder to interpret are eliminated
• Statistical control– Statistical techniques used to control for
influences of certain variables
22
Ethics
• Ethical standards of the APA
• Collection of 61 cases endorsed by SIOP– Illustrates ethical issues likely to arise in I-O
psychology (Lowman, 1985a,1998)
23
Module 3: Data Analysis
• Descriptive statistics– Summarize, organize, describe sample of data
Frequency Distribution:– Horizontal axis = Scores running low to high– Vertical axis = Indicates frequency of
occurrence
24
Describing a Score Distribution
• Measures of central tendency
• Mean• Mode• Median
Ryan McVay/Getty Images
25
Describing Score Distribution (cont'd)
• Variability– Standard deviation
• Lopsidedness or skew
Ryan McVay/Getty Images
26
Descriptive Statistics:Two Score Distributions (N = 30)
Figure 2.2 Two Score Distribution (N=30)
27
Two Score Distributions (N = 10)
Figure 2.3
28
Inferential Statistics
• Aid in testing hypotheses & making inferences from sample data to a larger sample/population
• Include t-test, F-test, chi-square test
29
Statistical Significance
• Defined in terms of a probability statement
• Threshold for significance is often set at .05 or lower– p < .05 (likelihood of this effect size would
occur less than 5 times in a hundred)
30
Statistical Power
• Likelihood of finding statistically significant difference when true difference exists
• Smaller the sample size, lower the power to detect a true or real difference
31
Concept of Correlation
Positive Linear Correlation
Figure 2.4Correlation betweenTest Scores andTraining Grades
32
Concept of Correlation (cont'd)
• Scatterplot– Displays correlational relationship between 2
variables
• Regression– Straight line that best fits the scatterplot
33
Correlation Coefficient
• Statistic or measure of association
• Reflects magnitude (numerical value) & direction (+ or –) of relationship between 2 variables
34
Correlation Coefficient
• Positive correlation → High values of one variable are associated with high values in the other variable (& vice versa)
• Negative correlation → High values of one variable are associated with low values in the other (& vice versa)
35
Figure 2.6: Scatterplots of Various Degrees of Correlation
36
Curvilinear Relationship
• Although correlation coefficient might be .00, it can’t be concluded that there is no association between variables
• A curvilinear relationship might better describe the association (eta η)– SPSS can provide η with F test
37
Curvilinear Correlation
Figure 2.7An Example ofa CurvilinearRelationship
38
Multiple Correlation
• Multiple correlation coefficient– Overall linear association between
several variables & a single outcome variable (R) • R2 = Proportion of variance in DV (outcome)
accounted for all preditors (several vars)
39
Meta-Analysis
• Statistical method for combining results from many studies to draw a general conclusion
• Statistical artifacts– Characteristics of a particular study that distort
the results– Sample size is most influential
40
Module 4: Interpretation
• Reliability– Consistency or stability of a measure
– Test-retest reliability• Calculated by correlating measurements
taken at Time 1 with measurements taken at Time 2
41
High and LowTest-Retest Reliability
Figure 2.8Examples of High and LowTest-Retest Reliability: Score Distributions of IndividualsTested on Two DifferentOccasions
42
Reliability (cont'd)
• Equivalent forms reliability– Calculated by correlating measurements
from a sample of individuals who complete 2 different forms of same test
• Internal consistency (Cronbach alpha α)
– Assesses how consistently items of a test measure a single construct
43
Reliability (cont'd)
• Inter-rater reliability– Can calculate various statistical indices to show
level of agreement among raters• Intraclass correlation ICC• Rwg
– Generalizability theory• Simultaneously considers all types of error in
reliability estimates
44
Validity
• Whether measurements taken accurately & completely represent what is to be measured
• Predictor– Test chosen or developed to assess identified abilities
• Criterion– Outcome variable describing important aspects or
demands of the job
45
Figure 2.9: Validation Process from Conceptual and Operational Levels
Figure 2.9
46
Criterion-Related Validity
• Correlate a test score with a performance measure (validity coefficient)
• Predictive validity design– Time lag between collection of test data &
criterion data– Test often administered to job applicants
47
Criterion-Related Validity (cont'd)
• Concurrent validity design– No time lag between collection of test data &
criterion data– Test administered to current employees,
performance measures collected at same time– Disadvantage: No data about those not
employed by the organization
48
Content-Related Validity
• Demonstrates that content of selection procedure represents adequate sample of important work behaviors & activities or worker KSAOs defined by job analysis
49
Construct-Related Validity
• Investigators gather evidence to support decisions or inferences about psychological constructs
• Construct - concept or characteristic that a predictor is intended to measure; examples include intelligence and extraversion
50
A Model for Construct Validity
Figure 2.10A Model forConstruct Validity
51
Construct Validity Model of Strength and Endurance Physical Factors
Figure 2.11
Recommended