Upload
trinhmien
View
221
Download
2
Embed Size (px)
Citation preview
1
Computer Adaptive
Personality Scales:
A Military and Civilian Application
Walter C. Borman PDRI, an SHL Company
and University of South Florida
Tracy Kantrowicz
PreVisor, an SHL Company
Outline
• Description of the “adaptive test”
concept
• Development of NCAPS and GPI-
Adaptive
• Initial validation results
• Conclusions and next steps
2
Description of the
“Adaptive Test” Concept
• Initial ability domain application
• Our work began in the performance
domain
• Forced-choice Computer Adaptive
Rating Scales (CARS)
• Laboratory study showed lower
standard error measurement and higher
validity and accuracy for CARS
CARS Performance Estimation
Process
Very Somewhat Very Ineffective Ineffective Effective Effective
First Item Pair 1 2 3 4
Second Pair 1 2 3 4
Third Pair 1 2 3 4
Fourth Pair 1 2 3 4
Final Estimated
Performance 1 2 3 4
3
Development of NCAPS
• Idea was to apply CARS concept to
non-cognitive testing
• 19 potential personality constructs were
identified
• Psychologists rated their importance for
79 Navy jobs
• 10 constructs selected based on means
and SDs
Development of NCAPS (Cont.)
• PDRI generated 1725 items at all trait
levels
• Construct and trait level “retranslation”
was conducted
• 1494 items survived, 106-199 per
construct
4
Example Items for
Social Orientation
• It is easy for me to find something in
common with any person I meet (M =
6.36)
• It takes real effort for me to hide my
impatience with people who aren’t very
bright (M = 1.49)
• I am able to make friends when I put
some effort into it (M = 3.63)
Validation Results
• Concurrent validation study designed
with 110 first tour Navy Sailors
• NCAPS and a conventional personality
inventory administered and supervisor
performance ratings gathered on nine
“Navy-wide” dimensions
• Composite of the ratings was reliable
(interrater r = .53)
5
Validation Results (Cont.)
• Unit-weighted composite of 10 NCAPS
conventional against composite overall
performance (r = .13 uncorrected;
r = .18 corrected)
• Unit-weighted composite of 10 NCAPS
adaptive against composite overall
performance (r = .27 uncorrected;
r = .37 corrected)
Validities Associated With Different Numbers of Item-Pairs
(Dependability)
0.00
0.05
0.10
0.15
0.20
0.25
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Number of Item-Pairs
Va
lid
ity
Co
eff
icie
nt
6
Civilian Version:
Global Personality Inventory-Adaptive
• Identified 13 dimensions for item
development
• PDRI and PreVisor generated 2679
items for 13 dimensions
• Construct and trait level retranslation
was again conducted
• 2519 items “survived the cut,” 150-200
per construct
Constructs Identified for GPI-Adaptive
compared to NCAPS Constructs
GPI-Adaptive
• Achievement Striving
• Social Orientation
• Cooperation
• Composure
• Adjusting to Change
• Positive Self-Concept
• Responsibility
• Thoroughness
• Sense of Duty
• Self-Sufficiency
• Learning Orientation
• Directing and Influencing
• Innovation
NCAPS
• Achievement
• Social Orientation
• Stress Tolerance
• Adaptability/Flexibility
• Attention to Detail
• Dependability
• Dutifulness/Integrity
• Self Reliance
• Willingness to Learn
• Vigilance
7
Validation Results
• Concurrent Validity Study Against Supervisor
Performance Ratings (N=1340)
• Unit-Weighted Composite Against Overall
Performance (r=.28, corrected for criterion
unreliability)
• Validities Asymptote at 5-8 Item Pairs
• Testing Time ½ that of Conventional
Inventory