Upload
truongdat
View
216
Download
0
Embed Size (px)
Citation preview
6/3/2014
1
The New WJ IV Battery: Introduction and Overview
Dr. Kevin McGrewInstitute for Applied Psychometrics (IAP)
WJ IV Presentation Menu: Main
Introduction and Overview
New Tests (COG/OL)
Cognitive Battery (GIA & CHC clusters)
Oral Language Battery
Achievement Battery
Variations and Comparisons
General Technical Information
Tech info.
Scholastic Aptitude clustersNew Ga
Tech info.
Tech info.
WJ IV Presentation Menu: Sub
Design principle: Cognitive complexity
Ga: No longer the Rodney Dangerfield of CHC abilities
Tech: Norm sample
Tech: Norms and score construction
Beyond CHC: CHC + Info. Proc. + AC
Hot off the press: Analyses in process
6/3/2014
2
Contemporary CHC broad and narrow ability content coverage by WJ‐IV Cognitive, Oral Language, and
Achievement batteries.General
Intelligence (g)
Mathematical achievement
(A3)
Reading decoding (RD)
Reading comprehension
(RC)
Reading speed (RS)
Spelling ability (SG)
English usage (EU)
Writing ability (WA)
Writing speed (WS)
General verbal information (K0)
Lexical knowledge (VL)
Listening ability (LS)
Induction (I)
General sequential
reasoning (RG)
Quantitative reasoning (RQ)
Memory span (MS)
Working memory capacity (WM)
Associative memory (MA)
Meaningful memory (MM)
Speed of Lexical access (LA)
Naming facility (NA)
Visualization
(Vz)
Visual Memory (MV)
Spatial Scanning
(SS)
Phonetic coding (PC)
Quantitative Knowledge
(Gq)
Reading & Writing (Grw)
Comp ‐Knowledge
(Gc)
Fluid Reasoning (Gf)
Short‐Term Wk Mem. (Gsm)
Long‐Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing
Speed (Gs)
Perceptual speed (P)
Number Facility (N)
General science info. (K1)
Knowledge of culture (K2)
Domain‐Specific Knw.
(Gkn)
Geography ach. (A5)
Shading designates proposed changes in CHC model based on analysis of WJ IV COG, OL. ACH norm data (see Chapter 1 and Appendix A in WJ IV Technical
Manual)
Language development
(LD)
Attentional Control (AC)
Memory for Sound Patterns
(UM)
Word Fluency (FW)
Verbal (print) lang. comp. (V)
Gsm has been renamed Gwm at the round table of cognitive CHC abilities
During the past two decades, and the last decade in particular, cognitive neuroscience has indicated that the more narrow Gsmdefinition was outdated and incorrect (Dehn, 2008).
Working memory refers to a dynamic, temporary storage system that allows information to be held immediate awareness and be manipulated.
Working memory refers to individual differences in both the capacity(size) of primary memory and to the efficiency of attentional control mechanisms that manipulate information within primary memory.
Short‐term memory refers to tasks that involve significant storage but only minimal processing or manipulation.
Attentional Control (Gwm‐AC). The ability to focus on task‐relevant stimuli and ignore task‐irrelevant stimuli. The ability to regulate intentionality and direct cognitive processing. Sometimes referred to as spotlight or focal attention, focus, control of attention, executive controlled attention or executive attention.
Memory for Sound Patterns (Ga‐UM). Ability to retain (on a short‐term basis) auditory codes such as tones, tonal patterns, or speech sounds.
Word Fluency (Glr‐FW). Ability to rapidly produce words that share a phonological (e.g., fluency of retrieval of words via a phonological cue) or semantic feature (e.g., fluency of retrieval of words via a meaning‐based representation). Also includes the ability to rapidly produce words that share non‐semantic features (e.g., fluency of retrieval of words starting with the letter “T”).
Speed of Lexical Access (Glr‐LA). Ability to rapidly and fluently retrieve words from an individual's lexicon; verbal efficiency or automaticity of lexical access.
Proposed changes/additions to CHC narrow ability taxonomy
6/3/2014
3
OM
PC US UM U8 URU1U9
UP UL
Vz SR MV CS SS CF IM PI LE IL PN
Sensory‐M
otor Domain‐
Specific Abilities
Sensory
The CHC Periodic Table of Human
Abilities
Adapted from Schneider & McGrew (2012) and McGrew, LaForte and Schrank (2014)
I RG RQ
WM MS AC
R3 PT MT
P N R9
R1 R2 R4 R7 IT
MA MM M6 FI FA FE SP F0 NA FW LA FF FX
Ideas Words Figures
Domain‐Independent
Cap
acities
Glr‐Learning efficiency
Glr‐Retrieval fluency
Broad ability
Narrow ability
© Institute for Applied Psychometrics (IAP)Dr. Kevin McGrew 4‐17‐14
KM A3
LD VL K0 LS CM MY
KL K1 A5 MK KF LP BC
V RD RC RS WA SG EU WS
Acquired Knowledge
Systems
K2
PI P2 P3 P4 P6 P7 P8
U1U9
UP UL
Motor
A1
KM A3
LD VL K0 LS
K1 A5
V RD RC RS WA SG EU WS
PC UM
Vz MV SS
Acquired
Knowledge
System
sSensory‐M
otor Domain‐
Specific Abilities
I RG RQ
WM MS AC
P N
MA MM NA FW LA
Ideas Words Figures
Domain‐Indep
enden
t Cap
acities
K2
Broad and Narrow Abilities Measured by the WJ IV COG, OL and ACH
FI
© Institute for Applied Psychometrics (IAP) Dr. Kevin McGrew 4‐11‐14
6/3/2014
4
Gc Gwm Gs Ga Glr GvGf
Vocab.(2)
(These two clusters require one test each from Cognitive and Oral Language easels)
Aud. Mm. Sp.(2)
Organization of WJ IV Cognitive (COG) and Oral Language (OL) batteries
General Intellectual Ability – g (7)
Brief Intellectual Ability (3)
Gf+Gc Composite (4)
Clusters available from Standard Cognitive
easel(10 tests)
(#) = # tests
Clusters available from Oral Language
easel(12 tests)
(#) = # tests
ST Wk. Mem.(2)
Cmp. Know.(2)
Fld. Reas.(2)
Cognitive Efficiency (2)
Other clusters available by
combining Standard Cognitive tests with select tests from
Extended Cognitive easel (8 tests)
(#) = # tests
Cog. Pr. Spd.(2)
LT Retrieval(2)
Visual. Proc.(2)
Aud. Proc.(2)
ST Wk. Mem.(3)
Cmp. Know.(3)
Fld. Reas.(3)
Qnt. Reas.(2)
Perc. Spd.(2)
Number Facility (2)
Cognitive Efficiency (4)
Reading, Math and Writing Scholastic Aptitudes (each comprised of a mix of 4 different CHC abilities)
Broad Oral Lang (3)*
Oral Expression (2)
Oral Lang. (2)*
Listening Comp. (2)* (* English or Spanish)
Phon. Cod.(2)
Spd. Lx. Acc.(2)
6/3/2014
5
WJ IV COG and OL tests by CHC factor domains
Oral Vocabulary
(VL)
Number Series
(RQ)
General Information
(K0)
Object‐Num.
Sequencing
(WM)
Pair Cancellation
(P/WM‐AC)
Memory for Words
(MS)
Analysis‐Synthesis
(RG)
Concept Formation
(I)
Numbers Reversed
(WM)
Picture Recognition
(MV)
Visual‐Aud.
Learning
(MA)
Story Recall
(MM)
Number‐Pat.
Matching
(P)
COG
OL
New or sign. change
Verbal Attention
(WM/AC)
Letter‐Pat. Matching
(P)
Nonword Repetition
(PC/UM‐MS)
Phonological Processing
(PC/Glr‐LA)
Visualization
(Vz)
Picture Vocabulary
(VL/LD)
Oral Comp.
(LS)
Under.
Directions
(WM/Gc‐LS)
Sentence Repetition
(MS)
Retrieval Fluency
(FI/LA)
Rapid Picture Naming
(NA/LA)
Sound Awareness
(PC)
Sound Blending
(PC)
Segmentation
(PC)
CHC Factors
Comp ‐Knowledge
(Gc)
Fluid Reasoning (Gf)
Long‐Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing
Speed (Gs)
Short‐Term
Wrk Mem(Gwm)
WJ IV ACH tests by CHC factor domains
Grw ‐Reading
CHC Factors
New or sign. change
Grw ‐Writing
Gq Gc/Gkm
Letter‐Word
Identification
(RD)
Passage Comp.
(RC)
Editing
(EU)
Social Studies
(K0/K2)
Sentence Wr. Fluency
(WA/WS)
Reading Vocabulary
(RC/Gc‐VL)
Writing Samples
(WA)
Calculation
(A3)
Humanities
(K0/K2)
Spelling
(SG)
Science
(K0/K1)
Applied Problems
(A3/RQ)
Word Attack
(RD/Ga‐PC)
Sentence Rdg. Fluency
(RC/Gc‐RS)
Spelling of Sounds
(SG/Ga‐PC)
Math Facts Fluency
(A3/Gs‐N)
Reading Recall
(RC/Glr‐MM)
Word Rdg.
Fluency
(RC/Gs‐RS)
Oral Reading
(RD/V)
Number Matrices
(Gf‐RQ)
Grw‐W
Grw‐R Gq Gc/Gkn
Phon.‐Graph. Know. (2)
Rdg. Cmp.(2)
Other clusters available by combining Standard ACH tests with select tests from Extended ACH easel (9
tests)
(#) = # tests
Rdg. Cmp.(3)
Rdg. Rate(2)
Bas. Wr. Sk.(2)
Math Pr.Slv.(2)
Acd. Knw.(3)
Clusters available from Standard Acheasel (11 tests)
(#) = # tests
See Table 1‐3 for COG test‐cluster interpretive
cluster composition
Organization of WJ IV
Achievement (ACH) battery
Bas. Rdg. Sk.(2)
Brd. Rdg.(3)
Rdg. Flu.(2)
Rdg.(2)
Wr. Exp.(2)
Brd. Wr. Lg.(3)
Wr. Lg.(2)
Math Cal.Sk.(2)
Brd. Math.(3)
Math.(2)
Academic Skills (3)
Academic Applications (3)
Academic Fluency (3)
Broad Achievement (9)
Brief Achievement (3)
6/3/2014
6
Organizational overview of the WJ IV variations and comparisons
Dr. Kevin McGrewInstitute for Applied Psychometrics (IAP)
Comparison Options
• GIA/Achievement
• Scholastic Aptitude/Achievement
• Gf‐Gc/Achievement/other abilities
• Broad Oral Language/Achievement
• Academic Knowledge/Achievement
Five ability/achievement difference score procedures to help compare ability to current
levels of achievement.
[Procedures account for regression‐to‐the mean (and how it varies by age)]
Variation Options
• Intra‐cognitive based on COG Tests 1—7
• Intra‐achievement
– Based on ACH Tests 1—6
– Based on Academic Skills, Academic Fluency, and Academic Applications clusters
• Intra‐oral language based on OL Tests 1—4
Four variation procedures to help document an individual’s pattern of strengths and weaknesses.
6/3/2014
7
Note: All score distributions represent real scores for all 9‐13 year old norming subjects from
WJ IV co‐normed sample. Actual
prediction models vary by age or grade (developmentally shifting prediction models).
Note: The SD of predicted and difference score distributions are not 15. They would only be 15 if GIA/Brd. Rdg. correlation
was perfect (1.0).
Illustration of procedures used to develop ALL WJ IV variation/comparison difference (standard score) norms (GIA‐Broad Reading ACH example)
-E
(minus)
B Actual Broad Reading scores
in WJ IV norm data
40 80 120 160100 14060
‐1 to +1 SD
SD = 15
AGeneral Intellectual
Ability (GIA) scores in WJ IV norm data (predictor score)
40
80
120
160
100
140
60
‐1 to +1
SDSD
= 15
=(equals)
FDistribution of Actual‐Predicted GIA/Broad Reading Difference Scores in WJ IV norm data
(SD of difference score
distribution allows for
specification & evaluation of significant S/W with SD
and PRmetrics)
‐40 ‐30 ‐20 ‐10 0 10 20 30 40
‐1 to +1SD
SD = 10.8
D Predicted (criterion)
Broad Reading scores in WJ IV norm data
40 80 120 160100 14060
‐1 to +1 SD
SD = 11.6
General Intellectual Ability (GIA) cluster (Predictor score)
General Intellectual Ability (GIA) /Achievement comparison procedure
Gwm Gs Ga Glr GvGcGf
CHC Cognitive Abilities and WJ IV COG and OL Clusters
Oral Language
Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp.(& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Phn.‐Grp.Kn.
Math.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
(Possible criterion or predicted target scores)Broad Oral Language
Oral Expression
Listening Comprehension
CHC Achievement Abilities and WJ IV
clusters
Regression‐based prediction models that account for regression‐to‐the‐mean (and how it varies by age)
Predicted Target
Cluster Score
Actual Target Cluster Score(minus) (equals)
Difference Score
SD and PR for specifiedSD cut‐off score
(Compare to distribution of difference scores in WJ IV norm sample to determine
significant strength or weakness)
CHC Cognitive Abilities and WJ IV COG and OL Clusters
Gwm Gs Ga Glr GvGcGf
Oral Language/Achievement comparison procedure
SD and PR for specified
SD cut‐off score
(Compare to distribution of difference scores in WJ IV norm sample to determine
significant strength or weakness)
Broad Oral Language
(Predictor score)
Sp. Lx. Acc.Phon. Cod.
(Possible criterion or predicted target scores)
Regression‐based prediction models that account for regression‐to‐the‐mean (and how it varies by age)
Predicted Target Cluster
Score
Actual Target Cluster Score(minus) (equals)
Difference Score
Academic Skills
Academic Fluency
Academic Applications
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp.(& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Phn.‐Grp.Kn.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
Reading Math.
Grw Gq
CHC Achievement Abilities and WJ IV ACH clusters
(Possible criterion or predicted target scores)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
6/3/2014
8
Phon. Cod. Sp. Lx. Acc.
ST Wk. Mem.(&Ext)
Cog .Pr.Spd. LT Retrieval Visual Proc.Aud. Proc.
Perc. Spd.
Cognitive Efficiency (& Ext)
Aud. Mm. Sp.
Other CHC broad/narrow cognitive and orallanguage abilities and WJ IV clusters
Gwm Gs Ga Glr Gv Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp.(& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Phn.‐Grp.Kn.
Math.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
CHC achievement abilities and WJ IV clusters
Gf‐Gc Composite(Predictor score)
GcGf
WJ IV Gf+Gc cognitive Ability cluster
Gf‐Gc “hybrid” variation/comparison procedure
Num. Fac.
Predicted Target Cluster
Score
Actual Target Cluster Score(minus) (equals)
SD and PR for specifiedSD cut‐off score
Difference Score
(Possible criterion or predicted target scores)
(Compare to distribution of difference scores in WJ IV norm sample to determine significant
strength or weakness)
Regression‐based prediction models that account for regression‐to‐the‐
mean
(See Table 1‐12for the specific tests and cluster information)
Phon Cod Sp Lex Acc
ST Wrk Mem(&Ext)
Cog Prc Spd LT Retrieval Visual ProcAud Proc
Perc Spd
Cognitive Efficiency (& Ext)
Aud Mem Sp
Other broad/narrow cognitive abilities and processing abilities and WJ IV clusters
Possible S/W predicted targets available for
evaluation
Gwm Gs Ga Glr Gv
Gf‐Gc Composite(Predictor)
GcGf
Gf+Gc cognitive abilityand WJ IV cluster
WJ IV Gf+Gc strength and weakness comparison procedure and options
Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd Rdg
Rdg Flu
Bas Rdg Sk
Rdg Cmp(& Ext)
Rdg Rate
Wr Lng
Brd Wr Lng
Bas Wrt Sk
Wr Exp
Phn‐Grp Kn
Math
Brd Math
Math Calc Sk
Math Pr Solv
Achievement domains and WJ IV clusters
Possible S/W
predicted targets
available for evaluation
Regression‐based predictions that account for regression‐to‐the‐
mean and produce “real discrepancy norms” to identify possible patterns of significant strengths and weaknesses
Grw Gq
Brief Achievement
Academic Skills
Academic Fluency
Academic Applications
Broad Achievement
Reading
Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Cmp.(& Ext)
Rdg. Rate
Wr. Lng.
Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Math.
Brd. Math
Math Cal.Sk.
Math Pr. Slv.
CHC achievement abilities and WJ IV clusters
Academic Knowledge(Predictor score)
Academic Knowledge/Achievement comparison procedure
Predicted Target Cluster
Score
Actual Target Cluster Score(minus) (equals)
SD and PR for specifiedSD cut‐off score
Difference Score
(Possible criterion or predicted target scores)
(Compare to distribution of difference scores in WJ IV norm sample to determine significant
strength or weakness)
Gwm Gs Ga Glr GvGcGf
CHC cognitive abilities and WJ IV cognitive and oral language clusters
Sp. Lx. Acc.Phon. Cod.
(See Table 1‐8 for the specific tests and cluster information)
Regression‐based prediction models that account for regression‐to‐the‐mean
6/3/2014
9
Gc Gwm Gs Ga Glr GvGf
WJ IV Scholastic Aptitude Cluster Organization
Visualization(Vz)
NumbersReversed(WM)
OralVocabulary(LD/VL)
Analysis‐Synthesis(RG/RQ)
PairCancellation
(P/EF)
Visualization(Vz)
Oral Vocabulary(LD/VL)
NumberSeries (RQ)
Num PatternMatching
(P)
PhonologicalProcessing
(PC)
OralVocabulary(LV/VL)
VerbalAttention(WM)
Num PatternMatching
(P)
PhonologicalProcessing
(PC)
OralVocabulary(LD/VL)
VerbalAttention(WM)
Num PatternMatching
(P)
PhonologicalProcessing
(PC)
OralVocabulary(LV/VL)
Story Recall(MM)
Num PatternMatching
(P)
PhonologicalProcessing
(PC)
OralVocabulary(LD/VL)
Concept Formation
(I)
ReadingBroad ReadingReading CompReading Comp‐ExtReading FluencyReading Rate
Basic Rdg Skills
WritingBroad WritingWritten Expression
Basic Writing Skills
MathBroad MathMath Calc Skills
Math Prob Solving
WJ IV Ach Clusters
Grw/Gq domain general
Grw domain specific
Gq domain specific
Reading Brd. Rdg.
Rdg. Flu.
Bas. Rdg. Sk.
Rdg. Rate
Wr. Lng. Brd. Wr. Lg.
Bas. Wr. Sk.
Wr. Exp.
Math. Brd. Math
Math Cal.Sk.
Math Pr. Slv.
GrwGwm Gs Ga Glr GvGcGf Gq
Rdg. Cmp.
Rdg Cmp. Ex
Rdg. Apt. A(Predictor Score)
Rdg. Apt. B(Predictor Score)
WL. Apt. A(Predictor Score)
WL. Apt. B(Predictor Score)
Math Apt. A(Predictor Score)
Math Apt. B(Predictor Score)
Regression‐based prediction models that account for regression‐to‐the‐mean
CHC cognitive abilities and WJ IV Scholastic Aptitude cognitive clusters CHC achievement abilities and WJ IV clusters
Predicted Target Cluster
Score
Actual Target Cluster Score(minus) (equals)
SD and PR for specifiedSD cut‐off score
Difference Score
(Compare to distribution of difference scores in WJ IV norm sample to determine significant strength or
weakness)Scholastic Aptitude/Achievement comparison procedures
(See Table 1‐10 for the specific tests and cluster information)
Copyright, Institute for Applied Psychometrics, K.McGrew; 01‐23‐14
Scholastic Aptitude Clusters: Potential Uses
• Designed to predict near term academic performance
• Time efficient referral‐focused selective testing
• Time efficient academic domain‐specific screening
• Time efficient annual review evaluations
• Gifted and talented screening – domain‐specific talents?
• Potentially useful in estimating quickness of response to intervention
• Provide information regarding the concept of “expected underachievement”
• Formulation of differential academic domain expectations
• ?
6/3/2014
10
One of the WJ IV guiding design objectives was for the WJ IV to retain a focus on psychometric quality that has been associated with the previous editions of Woodcock‐Johnson batteries:
•By providing a new, large, and nationally‐representative standardization sample of the US population;
•By updating items and simplifying test administration and interpretation procedures;
•By augmenting the underlying scaling of speeded tests; and
•By utilizing state‐of‐the art data collection, test development, and data analytic methods as models to facilitate progress in the field of applied test development.
WJ IV Gs tests based on new Rasch “rate‐based” item calibration method: WJ III Visual Matching / WJ IV
Letter‐Pattern Matching example
WJ III Visual Matching (3 minutes)• 60 items• 22 Possible “artificial” bonus points• 82 Possible raw score points (27 % of total scale was “artificial” bonus points)
WJ IV Letter‐Pattern Matching (3 minutes)• 84 items (24 more “real” items to eliminated need for bonus points)• Test scores based on count of number of correct responses in each of thethree minute intervals. Thus, three different rate‐based “item” scores
• Rasch rating scale model (Andrich, 1978) use to obtained difficulty estimatefor each minute of testing
• Need for artificial bonus points eliminated• Method rewards examinees who work quickly and accurately• Method penalizes examinees who respond quickly and not cautiously
(Note: IRT scaling of speed tests has historically been problematic due to violations of IRT assumptions. IRT methods stilltypically used for practical purposes, recognizing the assumption violations. See McGrew, Werder & Woodcock, 1991)
National Norm Sample
• 7,416 participants
• Preschool (664)
• K‐12 (3,891)
• College/University (775)
• Adult (2,086)
• Ages 2‐90+ years, Grades K.0‐18.0
• 100 geographically diverse communities from 46 states and the District of Columbia
• The Standards for Educational and Psychological Testing (aka., the Joint Standards) (American Psychological Association, American Educational Research Association, & National Council on Measurement in Education, 1999; in press) guided the norming and technical analyses
6/3/2014
11
Median reliabilities (across
all technical age groups) for WJ IV COG
clusters
General Intellectual Ability 0.97Brief Intellectual Ability 0.94Gf-Gc Composite 0.95Comprehension-Knowledge 0.93Comprehension-Knowledge- Extended 0.94Fluid Reasoning 0.94Fluid Reasoning- Extended 0.96Short-Term Working Memory 0.91Short-Term Working Memory- Extended 0.93Cognitive Processing Speed 0.94Auditory Processing 0.92Long-Term Retrieval 0.97Visual Processing 0.86Quantitative Reasoning 0.94Auditory Memory Span 0.90Number Facility 0.90Perceptual Speed 0.93Cognitive Efficiency 0.95Cognitive Efficiency-Extended 0.95Reading Aptitude A 0.93Reading Aptitude B 0.93Math Aptitude A 0.94Math Aptitude B 0.95Writing Aptitude A 0.92Writing Aptitude B 0.93
Median reliabilities (across
all technical age groups) for WJ IV OL clusters
Oral Language 0.90Broad Oral Language 0.92Oral Expression 0.89Listening Comprehension 0.90Phonetic Coding 0.95Speed of Lexical Access 0.89Vocabulary 0.93
Copyright; Institute for Applied Psychometrics; K.McGrew 02‐5‐14
Reading 0.95Broad Reading 0.97Basic Reading Skills 0.95Reading Comprehension 0.93Reading Comprehension-Extended
0.96
Reading Fluency 0.96Reading Rate 0.96Mathematics 0.96Broad Mathematics 0.97Math Calculation Skills 0.97Math Problem Solving 0.95Written Language 0.94Broad Written Language 0.95Basic Writing Skills 0.95Written Expression 0.92Academic Skills 0.97Academic Applications 0.96Academic Fluency 0.97Academic Knowledge 0.95Phoneme-Grapheme Knowledge 0.94Brief Achievement 0.97Broad Achievement 0.99
Median reliabilities (across
all technical age groups) for WJ IV ACH
clusters
6/3/2014
12
Sources of validity evidence presented
•Representativeness of the WJ IV Test Content, Process, and Construct Coverage
•Developmental Patterns of WJ IV Ability Clusters
•Internal Structure and Relations within the WJ IV
•Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement
•Performance of Clinical Samples on WJ IV Measures
Stage 3: Confirmatory structural
model cross‐validation phase
Stage 2: Exploratory structural model
generation (MG) and evaluation
Ages 3‐5 A(n = 209)
Ages 3‐5 B (n = 208)
Ages 6‐8 A(n =412)
Ages 6‐8 B(n = 411)
Ages 9‐13 A(n = 785)
Ages 9‐13 B(n = 787)
Ages 14‐19 A(n = 842)
Ages 14‐19 B(n = 843)
Ages 20‐39 A(n = 625)
Ages 20‐39 B(n = 626)
Ages 40‐90+ A(n = 571)
Ages 40‐90+ B(n = 575 )
Model Development
(MD) samples (A)
Model Cross‐Validation
(MCV) samples (B)
Stage 1: Split‐sample
random sample generation
Three‐stage internal/ structural validity procedures for WJ IV battery
+
+
Cluster analysis(CA)
Multidimensional scaling analysis
(MDS)
Principal component analysis(PCA)
Stage 2AExploratory structuralanalysis in Sample A
Review of prior WJ,
WJ‐R, WJ III structural validity research
Specify initial confirmatory factor analysis model (CFA)
Review of contemporary
CHC and cognitive
neuroscience research
CFA of MG model result from MD (A) sample in MCV (B)
sample(s) Yes
Most plausible and best fitting odel?
Model adjustment(s)
No
Stage 2BSpecify initial CFA
models
Stage 2CEvaluate and adjust
CFAmodel
Wards cluster analysis of ages 9-13 year sample (model development sample A; n=785)
NUMREVOBJNUMVRBATN
MEMWRDSENREP
NWDREPUNDDIR
MEMNAMWRDATKSPLSND
SNDAWRSEGMNTPHNPROSNDBLN
VALVISUAL
ANLSYNCONFRM
VISCLOPICREC
RPCNAMRETFLU
ORLCMPVRBANL
HUMPICVOC
ORLVOCGENINF
SOCSCI
STYRECWRTSMPRDGRECNUMSENAPPROB
CALCNUMSERNUMMAT
SPELLLWIDNT
PSGCMPRDGVOC
EDITORLRDGMTHFLUWRDFLUSNRDFLSNWRFLNUMPATLETPATPAIRCN
Gc
Glr-MM/Grw
Gq/Gf-RQ
Grw
Gs
Gs-ACH
Gs-COG
Acquired knowledge systems
Gv
GfGf+Gv
Glr-LA
Gwm-WM
Gwm-MS Gwm
Ga-PC
Ga-PC/Grw
Ga
Ga+Gwm (sequential/serial processing)
Cognitive Operations: Controlled/deliberate/system 2
General intelligence (g)
Bold font = Broad or general CHC abilities
Underlined font= Possible intermediate level stratum dimensions
Groupings to the left of the vertical dashed lines represent the groupings interpreted as representing valid CHC abilities. Higher‐order groupings to the left of the dashed line are potentially interesting hypothesized intermediate ability dimensions.
Glr-NA
Glr (Retrieval Fluency)
Cognitive Operations: Automatic/system 1
6/3/2014
13
-2 -1 0 1 2
-2
-1
0
1
2
NUMSER
VRBATN LETPAT
PHNPRO
VAL VISUAL
GENINF
NUMREV
NUMPAT
NWDREP
STYREC
PICREC
ANLSYN
OBJNUMPAIRCN
MEMWRD
SEGMNT
RPCNAM
SENREP
UNDDIR
SNDBLN
RETFLU
SNDAWR
LWIDNT
SPELL
PSGCMP
CALC
WRTSMP
ORLRDG
MTHFLU
SNWRFL
RDGREC
NUMMAT
EDIT
WRDFLU
SPLSND
RDGVOC
SCI
MEMNAM
VRBANL
VISCLO
Gs‐Cog
Gs‐Ach
Glr‐LA
Ga
Grw
Gv
Glr‐MM
Gc
Gq/Gf‐RQ
Gwm‐WM
Gwm‐MS
WRDATK
MDS (Guttman Radex) of ages 9-13
year sample (model
development sample A; n=785 )
GfCONFRM
Test EFAOther exp. methods CFA Cross‐validation
WJ III No No Yes No
SB5 No No Yes No
DAS‐II No No Yes No
KABC‐II No No Yes (in “exploratory manner”)
No
WAIS‐IV No No Yes No
WISC‐IV Yes No Yes Yes: Factor score (7) congruence in random CV
sample of 440
WJ IV Yes Yes: MDS and cluster analysis
Yes: Exploratory model generating CFA (MD samples) and CV CFA (CV
samples)
Yes: Randomly split samples at six age groups. CV of final MD models in
CV sample
Structural validity method comparisons across 7 intelligence batteries
15 independent WJ IV concurrent validity
studies differentiated by age/grade
WJIV COG compared to:
• WISC‐IV, WAIS‐IV, WPPSI‐III, KABC‐II, DAS‐II
WJIV OL compared to:
• CELF‐F, PPVT‐4, CASL, OWLS
WJIV ACH compared to:
• KTEA‐II, WAIT‐III, OWLS‐WE
6/3/2014
14
Clinical Validity Group Inclusion Criteria
Learning Disability (LD):
6–17 years Documented learning disability in reading Reading LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category
Learning Disability (LD): Math
6–17 years Documented learning disability in math Math LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category
Learning Disability (LD): Writing
6–17 years Documented learning disability in writing Writing LD must be the primary diagnosis or eligibility category Currently receiving special education services under SLD category
Head Injury 7–17 years Documented traumatic brain injury TBI must be the primary diagnosis Currently receiving special education services
Language Delay 7–13 years Documented language delay Language delay must be the primary diagnosis Currently receiving special education services
Autism Spectrum Disorder (ASD)
6–17 years Documented Autism Spectrum Disorder1
ASD is the primary diagnosis Currently receiving special education services
Attention Deficit Hyperactivity
Disorder (ADHD)7–12 years
Documented ADHD2
ADHD is the primary diagnosis Currently receiving special education services and/or 504 plan
accommodations
Gifted3 4–9 years Currently participating in high ability/gifted and talented school curriculum Currently receiving gifted services
Mental Retardation 6–12 years Documented mental retardation Mental retardation is the primary diagnosis Currently receiving special education services
WJ IV clinical validity evidence reported for 9 clinical groups
What is cognitive complexity?
Cognitive complexity
CHC factor breadth
Degree of g‐loading
Complicated
(Does not necessarily equal)
Factorial complexity
Content/stimulus dimension (Extension of BIS and Brunswick Symmetry Model)
Cognitive knowledge domains/systems
Cognitive operations
Cognitive control
Cognitive efficiency
Sensory functions
Motor functions
Abilty domain dim
ension
Type I
Processing
Type II
Processing
Note: CHC taxonomy is embedded in the ability domain dimension
McGrew (2010) – First formal introduction of cognitive complexity in “Beyond CHC” test development /evaluation framework(Neuropsych. Conference in Fremantle, Australia)
Cognitve complexity dimension
High
Medium
Low
© Institute for Applied Psychometrics; Kevin McGrew 05-21-14
This is a heuristic framework and is not intended to represent a theoretical model of intelligence
6/3/2014
15
• Larger number of cognitive component processes(factorial complexity)
• Accumulation of speed component differences
• Increased demands of attentional control (AC) & working memory
•More demands on adaptive functions (assembly, control, and monitoring – executive functions)
• More important component processes (e.g., inference; eduction of relations)
Parameters of cognitive efficiency in info. proc. models
Cognitively complex design characteristics (Lohman & Lakin, 2011)
Approach 1. Increasing the cognitive complexity of a test is often accomplished by making the test a mixed measure of more than one narrow CHC ability (factorially complex mixed CHC measures)
Approach 2. A second approach is to increase the complexity of information processing demands of the tests within a specific narrow CHC domain (Lohman & Larkin, 2011; McGrew, 2012). This second form of cognitive complexity, not to be confused with factorial complexity, places greater demands on cognitive information processing (cognitive load), requires greater allocation of key cognitive resources (working memory or attentional control), and invokes the involvement of more cognitive control or executive functions (Arend, Colom, Botella, Contreras, Rubio, & Santacreu, 2003; Jensen, 2011; Lohman & Larkin, 2011; Marshalek, Lohman, & Snow; 1983). This second approach to increasing test cognitive complexity was a primary design principle for the WJ IV.
One design objective in the WJ IV was to increase the cognitive complexity requirements for selected tests and clusters to provide greater ecological validity and
interpretive relevance of the measures.
Most contemporary CHC designed individual tests have focused on developing relatively pure measures of each cognitive ability
(mental pulley)
Gf Gwm Ga Gv
Gc Gs Glr Gs
Analogy: Think of general intelligence (g) as a system of relatively independent cognitive abilities (relatively construct
“pure” pulleys) working together to deal with a specific cognitive task load
6/3/2014
16
In this approach a test is designed to be a mixed measure of two (or more) cognitive abilities (mental pulleys; Gf + Gv)
Gf + Gv
Gf + Gv Gwm Gs Ga Glr Gv Gs
Approach 1 to developing cognitively complex tests:Construct factorially complex measures (a system of pulleys from 2 or more domain functions working in
combination).
Wechsler Arithmetic test
Gf + Gv
Gf + Gwm + Gc + Gq Ga Glr Gv Gs
Approach 1 example
Ga Gv Gwm Gs Gf Glr Gv Gs
Approach 2 is to increase the complexity of information processing demands of the tests within a specific CHC cognitive functional domain. Tasks are still as relatively pure a measure of the CHC domain as possible but there is a deliberate increase in the number of “mini‐pulleys” (cognitive information component complexity) that work together within the CHC domain. This was the primary approach used for certain WJ IV tests.
6/3/2014
17
Working Memory
Long‐Term Memory
Storage Retrieval
Central Executive (Executive functions or control?
• Inhibit•Shift
•Update
PerceptionSensation
DecisionsOutput
????
Conceptual explanation of parameters of cognitive efficiency
Focus of Attention
Recently activated concepts (Blue‐new; Red‐old)
0.4 0.5 0.6 0.7 0.8g-loading
-0.10
0.45
1.00
1.55
2.10
MD
Sre
lativ
e C
C (
rCC
)
ORLVOC‐Gc
SNDAWR‐Ga
ORLCMP‐Gc
VRBATN‐Gwm
PHNPRO‐Ga
OBJNUM‐Gwm
NUMSER‐GfCONFRM‐GfUNDDIR‐Gwm
NUMREV‐Gwm
PICVOC‐Gc
STYREC‐Glr
SENREP‐Gwm
SEGMNT‐Ga
ANLSYN‐GF
MEMWRD‐Gwm
SNDBLN‐Ga
VISUAL‐Gf
RETFLU‐GlrLETPAT‐Gs
NWDREP‐GaNUMPAT‐Gs
PAIRCN‐Gs
RPCNAM‐Glr
PICREC‐Gv
Relationship between g‐loadings and MDS‐based relative cognitive complexity (rCC) for WJ IV COG and OL tests
Correlation = .93; but correspondencediverges as test become higher in g and
cognitive complexity
• COG: Verbal Attention
• COG Number Series
• COG Letter‐Pattern Matching
• COG Visualization
• COG Phonological Processing
• COG Nonword Repetition
• OL Segmentation
New COG OL Tests
6/3/2014
18
COG Test 2: Number Series
• Was in WJ III Diagnostic Supplement• Gf‐RQ (Quantitative Reasoning)• Not a “controlled learning” test as are ConceptFormation (Gf‐I) and Analysis‐Synthesis (Gf‐RG)
• More Gf “in the wild” – without examiner provided scaffolding
• Extensive history as a premier Gf test in thepsychometric measurement of intelligence
• High in cognitive complexity and g. Best single testpredictor of achievement. Best indicator of Gf factor.
• In GIA, BIA, Gf‐Gc Composite, Gf, Gf‐Ext, QuantitativeReasoning (RQ), and one Math Aptitude clusters.
COG Test 7: Visualization
• Measure of Gv‐Visualization (Vz)
•Visualization consists of two subtests that each measure Gv‐Vz (visualization) via tasks that vary on task complexity and degree of “minds eye” (mental rotation) manipulations
• Within Gv, highest on cognitive complexity, g, Gv factor, and prediction of achievement
• In GIA, Gv and both Math Aptitude clusters
COG Test 3: Verbal Attention
• Measure of Gwm (working memory‐WM; attentionalcontrol‐AC)
• More ecological “real world” valid measure ofworking memory
• High in cognitive complexity and g. Within Gwm, themost cognitively complex, one of bestindicators of Gwm factor, and best predictorof achievement
• In GIA, BIA, Gwm, Gwm3, Cognitive Efficiency, and oneReading and 1 Written Language Aptitude clusters.
6/3/2014
19
COG Test 4: Letter‐Pattern Matching
• Measure of Gs (perceptual speed) and orthographicprocessing
• This speeded test (all WJ IV speeded tests) is based on a newrate‐based method of scaling the scores that eliminates theneed for bonus points
• Within Gs, it matches Number Pattern Matching in g, Gs factor loading, and prediction of achievement. Is more cognitively complex than Number Pattern Matching
• In GIA, Gs, Perceptual Speed (P), Cog. Eff. and Cog. clusters
COG Test 5: Phonological Processing
• Ga (PC) / Glr (LA/FW)•3 subtests (Word Access; Word Fluency; Substitution• Measures three aspects of speech sound processingthat requires the efficiency construction of sound‐based lexical representations
• High in cognitive complexity and g. Best single Gatest predictor of achievement. High loading on Gaand secondary low loading on Gc (accessing thelexicon). Also loaded on narrow LA factor inbroard+narrow bottom‐up CFA models.
• In GIA, Ga, and all reading and writing scholasticaptitude clusters
Nonword repetition tasks
Examinee listens to a nonsense word and then mustrepeat the word exactly.
Requires temporary storage of phonological segments in immediate awareness.
Significant body of research has found such tasks tobe significantly related to (and be possible “markers”of) reading disabilities, dyslexia and SLI (specific language impairment)
6/3/2014
20
OL Test 3: Segmentation
• Ga (PC)
• Examinee listens to words and identifies word parts
• In OL Phonetic Coding (PC) cluster
• Highest loading test on Ga factor across all ages
• A moderate measure of g and predictor of ach. across all ages; much more so (and more cognitively complex) than Sound Blending.
• Such tasks have been reported to be strong predictors of early reading (Bouwmeester et al, 2011; Geuden & Sandra, 2003)
Contemporary CHC broad and narrow ability content coverage by WJ IV Cognitive (as per WJ IV authors)
General Intelligence (g)
Mathematical achievement
(A3)
Reading decoding (RD)
Reading comprehension
(RC)
Reading speed (RS)
Spelling ability (SG)
English usage (EU)
Writing ability (WA)
Writing speed (WS)
General verbal information (K0)
Lexical knowledge (VL)
Listening ability (LS)
Induction (I)
General sequential
reasoning (RG)
Quantitative reasoning (RQ)
Memory span (MS)
Working memory capacity (WM)
Associative memory (MA)
Meaningful memory (MM)
Speed of Lexical access (LA)
Naming facility (NA)
Visualization
(Vz)
Visual Memory (MV)
Spatial Scanning
(SS)
Phonetic coding (PC)
Quantitative Knowledge
(Gq)
Reading & Writing (Grw)
Comp ‐Knowledge
(Gc)
Fluid Reasoning (Gf)
Short‐Term Wk Mem. (Gsm)
Short‐Term Wk Mem. (Gsm)
Long‐Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing
Speed (Gs)
Perceptual speed (P)
Number Facility (N)
General science info. (K1)
Knowledge of culture (K2)
Domain‐Specific Knw.
(Gkn)
Geography ach. (A5)
Shading designates abilities measured by WJ IV COG battery
Language development
(LD)
Attentional Control (AC)
Memory for Sound Patterns
(UM)
Word Fluency (FW)
Verbal (print) lang. comp. (V)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
The WJ IV COG is not your fathers intelligence test!
The WJ IV COG GIA is a much more cognitively complex(and high g) measure of intelligence
How did we do this?
What evidence do we have to support this conclusion?
This image cannot currently be displayed.This image cannot currently be displayed.
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
6/3/2014
21
WISC‐IV WAIS‐IV WPPSI‐III KABC‐II SB‐5 DAS‐II
FS IQ FS IQ FS IQ FCI FS IQ GCA
(n=174) (n=177) (n=99) (n=50) (n = 50) (n = 49)
WJ IV g‐measures
General Intellectual Ability (GIA)
0.86 0.84 0.72 0.77 0.80 0.83
Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79
Gf‐Gc Composite 0.83 0.78 0.71 0.82
Note. Correlations in italic represent correlations with a pseudo‐WJ IV GIA score computed from 7
WJ IV tests (WJ IV COG does not provide an GIA‐Edv cluster)
Select concurrent validity evidence: Correlations of WJ IV primary COG g‐scores with external measures
Conclusion: The WJ IV GIA, BIA and Gf‐Gc composite clusters demonstrate strong validity evidence as measures of general intelligence when the criterion are the global composite/total
scores from other major IQ batteries in the field
WJ III GIA other IQ score correlations
were from .67 to .76
GIA ‐Standard
GIAOral Vocabulary
General Information
Number Series
Concept Formation
Verbal
Attention
Numbers Reversed
Story Recall
Visual‐AudLearning
Visualization
Picture Recognition
Letter‐Pat. Match.
Pair
Cancellation
Phonological
Processing
Nonword
Repetition
Primary WJ III and WJ IV COG tests and Clusters
Tests in WJ IV COG clusters
Tests in WJ III COG Clusters
WJ IV GIA
WJ III GIA‐Standard
* Visual Matching is renamed Number‐Pattern Matching in WJ IV
Comparison of composition of primary WJ III and WJ IV COG CHC and GIA clusters
(WJ III Spatial Relations is half
of WJ IV Visualization)
(WJ IV Oral Vocab. was part
of WJ III Verbal Comp.)
Auditory Attention
Decision
Speed
Analysis‐Synthesis
Retrieval
Fluency
Spatial
RelationsVerbal Comp.
Memory for Words
Sound
Blending
Visual
Matching *
General Intelligence (g)
Comp ‐Knowledge
(Gc)
Fluid Reasoning (Gf)
Long‐Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing
Speed (Gs)
Short‐Term
Wrk Mem(Gwm)
© Institute for Applied Psychometrics; Kevin McGrew 01‐23‐14
Four primary design principles have resulted in the WJ IV COG GIA being significantly different in composition than the WJ III COG GIA‐Std.
The WJ IV COG GIA cluster tests should:
1. Be the best factor indicators of each CHC broad domain
2. Be the best predictors of achievement from each CHC broad domain
3. Be the most cognitively complex indicators from each CHC broad domain
4. Be the best measures of g (general intelligence) from each CHC broad domain
5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural)
(Note: Trade‐offs between the four design objectives were sometimes necessary)
6/3/2014
22
© Institute for Applied Psychometrics; Kevin McGrew 02‐05‐14
This image cannot currently be displayed.
The Cognitive tests were evaluated on the basis of four (of five total) quantifiable COG design criteria
Data augmented by Siskel and Ebert informal rating system
Average CHC factor loadings
Average achievement correlation across domains
Average degree of g‐loadings
Average degree of relative cognitive complexity
(Average across ages 6‐90+)
Comprehension – Knowledge(Gc)
Lexical Knowledge(VL)
General Information(K0)
Language Development (LD)
ORLVOC* GENINFPICVOC
* Test in GIA
Author/expertCHC narrow
factor classifications
(Gc cluster)(Voc. Cluster‐
LD/VL)
Induction(I)
General SequentialReasoning (RG)
QuantitativeReasoning (RQ)
ANLSYNCONFRMNUMSER* (Gf cluster)(Gf3
cluster)
(Quantitative Reasoning Cluster‐RQ)
* Test in GIA
Fluid Reasoning(Gf)
Author/expertCHC narrow
factor classifications
6/3/2014
23
Short‐Term WorkingMemory (Gwm)
Working Memory(WM)
Memory Span(MS)
Attentional Control(AC)
MEMWRD
SENREP
(Auditory MemorySpan Cluster‐MS)
OBJNUMVRBATN* NUMREV
* Test in GIA
(Gwm cluster)
(Gwm3 cluster)
Author/expertCHC narrow
factor classifications
© Institute for Applied Psychometrics; Kevin McGrew 2-04-14
Cognitive Processing Speed(Gs)
Attentional Control
PerceptualSpeed (P)
PAIRCNNUMPATLETPAT*
* Test in GIA
Short‐Term Working Memory (Gwm)
(Gs cluster)
(Perceptual Speed cluster‐P)
Author/expertCHC narrow
factor classifications
Long‐Term Retrieval(Glr)
AssociativeMemory (MA)
MeaningfulMemory (MM)
* Test in GIA
VALSTREC* (Glr cluster)
Speed of LexicalAccess (LA)
RETFLURPCNAM (Speed of Lexical Access
cluster‐LA)
Author/expertCHC narrow
factor classifications
6/3/2014
24
AuditoryProcessing (Gs)
PhoneticCoding (PC)
NWDREPPHNPRO*
* Test in GIA
Short‐Term Working Memory (Gwm)
Memory for Sound Patterns (UM)
SEGMNT SNDBLN
(Ga cluster)
(Phonetic Coding cluster‐PC)
Author/expertCHC narrow
factor classifications
© Institute for Applied Psychometrics; Kevin McGrew 2-04-14
This image cannot currently be displayed.
The WJ IV Auditory Processing (Ga) cluster is not your fathers Ga measure.
WJ IV still has the Oldsmobile Ga (Phonetic Coding) in OL: COG now has more cognitively
complex Gameasures
The WJ IV has taken a broader contemporary view of the domain of Ga
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
Auditory Processing (Ga) abilities, when properly measured, should have a
prominent chair at the round table of cognitive CHC abilities
© Institute for Applied Psychometrics; Kevin McGrew 02‐01‐14
6/3/2014
25
Visual Processing(Gv)
Visual Memory(MV)
Visualization(Vz)
* Test in GIA
PICRECVISUAL* (Gv cluster)
Author/expertCHC narrow
factor classifications
COG/OL Mdn.Clusters r r2General Intellectual Ability 0.76 0.58Brief Intellectual Ability 0.73 0.53Gf-Gc Composite 0.68 0.46
Comprehension-Knowledge (Gc) 0.58 0.34Comprehension-Knowledge-Ext (Gc3) 0.58 0.34Fluid Reasoning (Gf) 0.64 0.41Fluid Reasoning-Ext (Gf3) 0.59 0.35Short-Term Working Memory (Gwm) 0.53 0.28Short-Term Working Memory-Ext (Gwm3) 0.52 0.27Cognitive Processing Speed (Gs) 0.49 0.24Auditory Processing (Ga) 0.51 0.26Long-Term Retrieval (Glr) 0.43 0.18Visual Processing (Gv) 0.38 0.14Quantitative Reasoning (Gf-RQ) 0.59 0.35Auditory Memory Span (Gsm-MS) 0.47 0.22Number Facility (Gs-N) 0.62 0.38Perceptual Speed (Gs-P) 0.58 0.34Cognitive Efficiency (Gsm+Gs) 0.55 0.30Cognitive Efficiency-Ext (Gsm+Gs) 0.66 0.44
Oral Language 0.55 0.30Broad Oral Language 0.57 0.32Oral Expression 0.54 0.29Listening Comprehension 0.55 0.30Phonetic Coding (Ga-PC) 0.44 0.19Speed of Lexical Access (Glr-LA) 0.37 0.14Vocabulary (Gc-LD/VL) 0.60 0.36
WJ IV COG and OL cluster correlations
with WJ IV ACH clusters:
Correlations across 15 ACH clusters (ages
6‐90+)
WISC‐IV WAIS‐IV WPPSI‐III KABC‐II SB‐5 DAS‐II
FS IQ FS IQ FS IQ FCI FS IQ GCA
(n=174) (n=177) (n=99) (n=50) (n = 50) (n = 49)
WJ IV g‐measures
General Intellectual Ability (GIA)
0.86 0.84 0.72 0.77 0.80 0.83
Brief Intellectual Ability (BIA) 0.83 0.74 0.76 0.79
Gf‐Gc Composite 0.83 0.78 0.71 0.82
Note. Correlations in italic represent correlations with a pseudo‐WJ IV GIA score computed from 7
WJ IV tests (WJ IV COG does not provide an GIA‐Edv cluster)
Select concurrent validity evidence: Correlations of WJ IV primary COG g‐scores with external measures
Conclusion: The WJ IV GIA, BIA and Gf‐Gc composite clusters demonstrate strong validity evidence as measures of general
intelligence when the criterion are the global composite/total scores from other major IQ batteries in the field
WJ III GIA other IQ score correlations
were from .67 to .76
6/3/2014
26
WISC‐IV WISC‐IV WISC‐IV WISC‐IV
VCI PRI WMI PSI
WJ IV Measures (Gc) (Gf/Gv) (Gwm) (Gs)
CHC Factor Clusters
Comprehension‐Knowledge (Gc) 0.79Fluid Reasoning (Gf) 0.70
Short‐Term Working Memory (Gwm) 0.72
Processing Speed (Gs) 0.55
Auditory Processing (Ga)Long‐Term Retrieval (Glr)
Visual Processing (Gv) 0.55
Narrow Ability & Clinical ClustersQuantitative Reasoning (RQ) 0.65
Auditory Memory Span (MS) 0.52
Number Facility (N) 0.57Perceptual Speed (P) 0.56
Note: Bold font values represent correlations between best comparable CHC broad composites.Red bold font are interesting correlations given the Wechsler composites composition.
WISC‐IV/WAIS‐IV PRI and WMI composites are not comparable‐‐different mixtures of CHC abilities. (see next slide)
WJ IV COG /WISC‐IV CHC composite select score correlations ( n = 174)
Block Design‐Gv‐VzPic Cocnepts – Gf‐IMatrix Reasoning – Gf‐I
Digit Span – Gwm‐MS/MWLetter‐Num Seq – Gwm‐MW
WAIS‐IV WAIS‐IV WAIS‐IV WAIS‐IVVCI PRI WMI PSI
WJ IV Measures (Gc) (Gv/Gf) (Gwm/Gq) (Gs)
CHC Factor ClustersComprehension‐Knowledge (Gc) 0.74Fluid Reasoning (Gf) 0.57Short‐Term Working Memory (Gwm) 0.67Processing Speed (Gs) 0.44Auditory Processing (Ga)Long‐Term Retrieval (Glr)Visual Processing (Gv) 0.57
Narrow Ability & Clinical ClustersQuantitative Reasoning (RQ) 0.54 0.53Auditory Memory Span (MS)Number Facility (N) 0.65 0.52Perceptual Speed (P) 0.61
Note: Bold font values represent correlations between best comparable CHC broad composites.Red bold font are interesting correlations given the Wechsler composites composition.WISC‐IV/WAIS‐IV PRI and WMI composites are not comparable‐‐different mixtures of CHC abilities. Noted Wechsler correlations with tests classifications based on Flanagan et al., 2013.
WJ IV COG / WAIS‐IV CHC composite select score correlations ( n = 177)
Block Design‐Gv‐VzVisual Puzzles – Gv‐VzMatrix Reasoning – Gf‐I
Digit Span – Gwm‐MS/MWArithmetic – Gwm‐WM/Gf‐RQ (Gq?)
KABC‐II KABC‐II KABC‐II KABC‐II KABC‐IIKnow./ Plan./ Sim./ Seq./ Lrng./
WJ IV Measures GcIndex
GfIndex
Gv Index
GsmIndex
GlrIndex
CHC Factor ClustersComprehension‐Knowledge (Gc) 0.82Fluid Reasoning (Gf) 0.46Short‐Term Working Memory (Gwm)
0.42
Processing Speed (Gs)Auditory Processing (Ga)Long‐Term Retrieval (Glr) 0.64Visual Processing (Gv) 0.37
Note: Bold font values represent correlations between best comparable CHC broad composites.Low to moderate corresponding Gf, Gv and Gwm correlations most likely reflect narrow ability content differences in composites (classifications based on Reynolds et al., 2007 and Flanagan et al., 2013).
WJ IV COG / KABC‐II composite select score correlations (n = 50)
Pattern Reasoning ‐ Gf‐I/Gv‐Vz Story Completion – Gf‐RG
Number Recall – Gsm‐MSWord Recall – Gsm‐MS/WM?
Rover – Gv‐SSTriangles – Gv‐Vz
6/3/2014
27
SB‐5 SB‐5 SB‐5 SB‐5 SB‐5Fluid Quant. Vis.‐Spatial Working
Know. Reas. Reas. Proc. MemoryWJ IV Measures (Gc) (Gf) (Gf‐RQ) (Gv) (Gwm)
CHC Factor ClustersComprehension‐Knowledge (Gc) 0.68 0.75 0.72 0.72Fluid Reasoning (Gf) 0.67 0.56 0.66 0.66Short‐Term Working Memory (Gwm) 0.62 0.69Cognitive Processing Speed (Gs)Auditory Processing (Ga) 0.68 0.73 0.72Long‐Term Retrieval (Glr)Visual Processing (Gv) 0.40
Note: Bold font values represent correlations between best comparable CHC broad composites.Red bold font are interesting correlations given the SB‐5 composites composition.The convergent/divergent validity of the SB‐5 CHC composite scores has been seriously questioned (Canivez, 2008; DiStefano & Dombrowski, 2006; Keith & Reynolds, 2010) as well as very high composite score intercorrelations in the SB‐5 technical manual (.65 to .75)
WJ IV COG / SB‐5 CHC composite select score correlations (n = 50)
• 7 new tests• Oral Reading, Reading Recall, • Word Reading Fluency• Number Matrices• Science, Social Studies, Humanities
• 8 new clusters• Reading, Reading Comprehension‐Extended, Reading Fluency, Reading Rate
• Written Language• Mathematics• Brief Achievement, Broad Achievement
Achievement: What’s New?
• Conormed with WJ IV Tests of Cognitive Abilities andWJ IV Tests of Oral Language
• Qualitative Observation Checklists for Tests 1‐11– Located in Test Record– Helps document important information about how examinee performed on the task
– Includes data on percentage of age mates at each rating
Achievement: What’s New?
6/3/2014
28
Test 1: Letter‐Word Identification
Test 2: Applied Problems
Test 3: Spelling
Test 4: Passage Comprehension
Test 5: Calculation
Test 6: Writing Samples
Test 7: Word Attack
Test 8: Oral Reading‐‐NEW
Test 9: Sentence Reading Fluency (Renamed)
Test 10: Math Facts Fluency (Renamed)
Test 11: Sentence Writing Fluency (Renamed)
Tests of AchievementStandardBattery
WJ IV Tests of Achievement‐‐‐Extended Battery
Test 12: Reading Recall‐‐NEW
Test 13: Number Matrices‐‐EXPANDED
Test 14: Editing
Test 15: Word Reading Fluency‐‐NEW
Test 16: Spelling of Sounds
Test 17: Reading Vocabulary‐‐REVISED
Test 18: Science‐‐EXPANDED
Test 19: Social Studies‐‐EXPANDED
Test 20: Humanities‐‐EXPANDED
Provides clusters for:•ReadingComprehension
•Reading Comprehension‐Extended
•Reading Rate•Math ProblemSolving
•Basic Writing Skills•Academic Knowledge
• Includes 8 tests for evaluating different aspects of reading– Letter‐Word Identification, Passage Comprehension, Word Attack, Sentence Reading Fluency, Reading Vocabulary
– Oral Reading, Reading Recall, Word Reading Fluency
• Provides 7 clusters for a comprehensive evaluation of reading performance• Reading, Broad Reading, Basic Reading Skills, Reading Comprehension
• Reading Fluency, Reading Rate, Reading Comprehension‐Extended
Summary: Reading
3 new tests
3 new clusters
6/3/2014
29
• Includes 4 tests measuring various aspects of math achievement:– Applied Problems, Calculation, Math Facts Fluency, and Number Matrices
• Provides 4 clusters for a comprehensive evaluation of math performance:– Mathematics, Broad Mathematics, Basic Math Skills, Math Problem Solving
Mathematics: Summary
• Includes 5 tests measuring various aspects of written language:– Spelling, Writing Samples, Sentence Writing Fluency, Editing, and Spelling of Sounds
• Provides 4 clusters for a comprehensive evaluation of writing performance:– Written Language, Broad Written Language, Basic Writing Skills, Written Expression
• Contributes to Phoneme‐Grapheme Cluster (Spelling of Sounds + Word Attack)
Written Language: Summary
• Includes 3 tests measuring various aspects of academic knowledge (comprehension‐knowledge [Gc])
• Test 18: Science• Test 19: Social Studies• Test 20: Humanities
• Provides 1 cluster: Academic Knowledge– Serves as the ability score in an ability/achievement comparison procedure
Academic Knowledge
6/3/2014
30
• A cluster that includes tests from different academic domains (reading, writing, math)
• 7 cross‐domain clusters are available– Brief Achievement and Broad Achievement
– Academic Knowledge
– Academic Skills
– Academic Fluency
– Academic Applications
– Phoneme‐Grapheme Knowledge
Cross‐Domain Clusters
Two options
Intra‐Achievement using Tests 1‐6•Test 1. Letter Word Identification• Test 2. Applied Problems•Test 3. Spelling• Test 4. Passage Comprehension• Test 5. Calculation• Test 6. Writing Samples
Intra‐Achievement using 3 clusters: •Academic Skills•Academic Fluency•Academic Applications
Intra‐Achievement Variation Procedures
• Additional tests can be added to procedures
• Compares performance on one test or cluster to average performance on other tests or clusters
• Provides information about examinee’s strengths and weaknesses within academic areas
• Helps pinpoint a pattern of strengths and weaknesses
Intra‐Achievement Variation Procedures
6/3/2014
31
Academic Skills/Academic Fluency/Academic Applications
When ACH Tests 1 through 6 and 9 through 11 are administered, a cross‐domain comparison is made.
• Academic Skills cluster is compared to a predictor score from the Academic Applications and Academic Fluency clusters;
• Academic Applications cluster is compared to Academic Skills and Academic Fluency clusters;
• Academic Fluency cluster is compared to a predictor score from the Academic Skills and Academic Applications clusters.
Median reliabilities (across
all technical age groups) for WJ IV
ACH clusters
Reading 0.95Broad Reading 0.97Basic Reading Skills 0.95Reading Comprehension 0.93Reading Comprehension-Extended 0.96Reading Fluency 0.96Reading Rate 0.96Mathematics 0.96Broad Mathematics 0.97Math Calculation Skills 0.97Math Problem Solving 0.95Written Language 0.94Broad Written Language 0.95Basic Writing Skills 0.95Written Expression 0.92Academic Skills 0.97Academic Applications 0.96Academic Fluency 0.97Academic Knowledge 0.95Phoneme-Grapheme Knowledge 0.94Brief Achievement 0.97Broad Achievement 0.99
KTEA‐II KTEA‐II KTEA‐II KTEA‐II KTEA‐II
WJ IV Measures Oral. Lang. Rdg. Math Wr. Lang Comp. Ach.Oral Language ClustersOral Language 0.64Oral Comprehension 0.74Oral Expression 0.64Listening Comprehension 0.74Reading Achievement ClustersReading 0.94Broad Reading 0.92Basic Reading Skills 0.93Reading Comprehension 0.85Reading Fluency 0.86Reading Speed 0.78Math Achievement ClustersMathematics 0.94Broad Mathematics 0.91Math Calculation 0.83Math Problem Solving 0.82Written Language ACH ClustersWritten Language 0.81Broad Writing 0.84Basic Writing Skills 0.91Written Expression 0.65Cross Domain ACH ClustersBrief Achievement 0.88Broad Achievement 0.91
WJ IV ACH OL / KTEA‐II composite select score correlationsAges 8‐12 (n = 49)
6/3/2014
32
KTEA‐II KTEA‐II KTEA‐II KTEA‐II KTEA‐IIOral Wr. Comp.
WJ IV Measures Lang. Rdg. Math Lang Ach.Oral Language ClustersOral Language 0.74Oral Expression 0.64Listening Comprehension 0.58Reading Achievement ClustersReading 0.83Broad Reading 0.78Basic Reading Skills 0.81Reading Comprehension 0.68Reading Fluency 0.79Reading Speed 0.65Math Achievement ClustersMath 0.87Broad Math 0.86Math Calculation 0.82Math Problem Solving 0.69Written Language ACH ClustersWritten Language 0.87
Broad Writing 0.85
Basic Writing Skills 0.89
Written Expression 0.66
Cross Domain ACH ClustersBrief Achievement 0.91Broad Achievement 0.88
WJ IV ACH OL / KTEA‐II composite select score correlationsAges 13‐18 (n = 50)
WJ IV ACH OL /WIAT‐III composite select score correlations– Grades 1‐8 (n=51)
WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐IIIOral Total Basic Rdg. Cmp. Math Wr. Total
WJ IV Measures Lang. Rdg. Rdg. & Flu. Math. Flu. Expr. Ach.Oral Language ClustersOral Language 0.82Oral Expression 0.57Listening Comprehension 0.60Reading Achievement ClustersReading 0.93 0.92 0.82Broad Reading 0.89 0.87 0.82Basic Reading Skills 0.93 0.94 0.78Reading Comprehension 0.88 0.85 0.81Reading Fluency 0.87 0.84 0.80Reading Speed 0.67 0.62 0.64Math Achievement ClustersMath 0.92 0.57Broad Math 0.90 0.79Math Calculation 0.82 0.85Math Problem Solving 0.83 0.53Written Language ACH ClustersWritten Language 0.78Broad Writing 0.77Basic Writing Skills 0.79Written Expression 0.62Cross Domain ACH ClustersBrief Achievement 0.93Broad Achievement 0.90Bold font designates correlations >= .70
WJ IV ACH OL /WIAT‐III composite select score correlations—Grades 9‐12 (n=49)
WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III WIAT‐III
Oral Total Basic Rdg. Cmp. Math Wr. Total
WJ IV Measures Lang. Rdg. Rdg. & Flu. Math. Flu. Expr. Ach.Oral Language ClustersOral Language 0.79Oral Expression 0.68Listening Comprehension 0.75Reading Achievement ClustersReading 0.78 0.75 0.74Broad Reading 0.79 0.72 0.80Basic Reading Skills 0.86 0.86 0.74Reading Comprehension 0.54 0.50 0.60Reading Fluency 0.78 0.72 0.77Reading Speed 0.66 0.61 0.72Math Achievement ClustersMath 0.84 0.42Broad Math 0.76 0.68Math Calculation 0.58 0.72Math Problem Solving 0.79 0.45Written Language ACH ClustersWritten Language 0.73Broad Writing 0.76Basic Writing Skills 0.77Written Expression 0.52Cross Domain ACH ClustersBrief Achievement 0.85Broad Achievement 0.85Note. font designates correlations >= .70
6/3/2014
33
Contemporary CHC (Schneider & McGrew, 2012) broad and narrow ability content coverage by WJ‐IV Oral Language
General Intelligence (g)
Mathematical achievement
(A3)
Reading decoding (RD)
Reading comprehension
(RC)
Reading speed (RS)
Spelling ability (SG)
English usage (EU)
Writing ability (WA)
Writing speed (WS)
General verbal information (K0)
Lexical knowledge (VL)
Listening ability (LS)
Induction (I)
General sequential
reasoning (RG)
Quantitative reasoning (RQ)
Memory span (MS)
Working memory capacity (WM)
Associative memory (MA)
Meaningful memory (MM)
Speed of Lexical access (LA)
Naming facility (NA)
Visualization
(Vz)
Visual Memory (MV)
Spatial Scanning
(SS)
Phonetic coding (PC)
Quantitative Knowledge
(Gq)
Reading & Writing (Grw)
Comp ‐Knowledge
(Gc)
Fluid Reasoning (Gf)
Long‐Term Retrieval (Glr)
Visual Processing (Gv)
Auditory Processing
(Ga)
Processing
Speed (Gs)
Perceptual speed (P)
Number Facility (N)
General science info. (K1)
Knowledge of culture (K2)
Domain‐Specific Knw.
(Gkn)
Geography ach. (A5)
Shading designates abilities measured by WJ IV OL battery
Language development
(LD)
Attentional Control (AC)
Memory for Sound Patterns
(UM)
Word Fluency (FW)
Verbal (print) lang. comp. (V)
ST Working
Mem (Gwm)
© Institute for Applied Psychometrics; Kevin McGrew 01-23-14
Median reliabilities (acrossall technical age groups) for
WJ IV OL clusters
Oral Language 0.90Broad Oral Language 0.92Oral Expression 0.89Listening Comprehension 0.90Phonetic Coding 0.95Speed of Lexical Access 0.89Vocabulary 0.93
CELF‐4CELF‐4CELF‐4 CELF‐4 CELF‐4 CELF‐4 PPVT‐4WJ IV Measures Core Rec. Expr. Lang. Lang. Work.
Lang. Lang Lang. Con. Struc. Mem.Oral Language Clusters
Oral Language 0.63 0.59 0.64 0.67 0.67 0.50 0.74Oral Expression 0.74 0.64 0.72 0.67 0.73 0.58 0.70Listening Comprehension 0.64 0.62 0.63 0.63 0.68 0.59 0.69Speed of Lexical Access 0.31 0.37 0.32 0.35 0.38 0.57 0.43Bold font correlations > = .60
WJ IV OL / CELF‐4 and PPVT‐4 composite select score
correlations—Ages 5‐8 (n = 50)
6/3/2014
34
CELF‐4 CELF‐4CELF‐4CELF‐4 CELF‐4 CELF‐4 PPVT‐4WJ IV Measures Core Rec. Expr. Lang. Lang. Work.
Lang. Lang Lang. Con. Struc. Mem.Oral Language ClustersOral Language 0.75 0.46 0.69 0.49 0.65 0.17 0.76Oral Expression 0.83 0.56 0.79 0.49 0.74 0.33 0.62Listening Comprehension 0.76 0.50 0.68 0.42 0.69 0.30 0.55Speed of Lexical Access 0.42 0.19 0.36 0.18 0.39 0.23 0.14. Bold font correlations > = .60
WJ IV OL / CELF‐4 and PPVT‐4 composite select score correlations—Ages 10‐18 (n = 56)
CASL OWLS
Core Basic Sent. Syntax Parag. Prag. Oral List. OralWJ IV Measures Comp. Conc. Comp Constr Comp Judge Comp Comp Exp
Oral Language ClustersOral Language 0.60 0.56 0.57 0.47 0.41 0.49 0.60 0.42 0.61Oral Expression 0.48 0.43 0.51 0.42 0.41 0.31 0.46 0.31 0.49Listening Comprehension
0.58 0.60 0.53 0.49 0.44 0.46 0.57 0.44 0.55
Speed of Lexical Access
0.48 0.41 0.36 0.43 0.36 0.40 0.24 0.07 0.33
Note. n = 50 for all measures except WJ IV Phonological Processing (n = 48). All correlations with WJ IV Phonological Processing have n = 48 sample size.
WJ IV OL / CASL & OWLS composite score correlations in ages 3‐6 sample
WJ IV Oral Language ClustersOral Oral Oral Listening Speed of
CASL/OWLS Measures Language Comp. Expression Comp. Lex. Acc.CASL MeasuresCore Composite 0.85 0.82 0.72 0.76 0.57Antonyms 0.82 0.78 0.64 0.75 0.39Synonyms 0.71 0.73 0.55 0.70 0.32Sentence Completion 0.76 0.73 0.66 0.68 0.51Idiomatic Language 0.82 0.80 0.75 0.72 0.42Syntax Construction 0.71 0.68 0.57 0.64 0.47Paragraph Comprehension 0.66 0.70 0.54 0.75 0.56Grammitcal Morphemes 0.63 0.60 0.53 0.56 0.47Sentence Comprehension 0.59 0.60 0.58 0.57 0.47Grammaticality Judgement 0.67 0.65 0.70 0.58 0.57Nonliteral Language 0.67 0.63 0.64 0.54 0.40Meaning from Context 0.77 0.74 0.63 0.67 0.56Inference 0.70 0.70 0.62 0.67 0.42Amibgious Sentences 0.72 0.73 0.58 0.70 0.45Pragmatic Judgement 0.52 0.47 0.34 0.42 0.48
OWLS MeasuresOral Composite 0.68 0.68 0.62 0.64 0.41Listening Comprehension 0.53 0.57 0.50 0.56 0.25Oral Expression 0.67 0.64 0.60 0.59 0.45
Bold font are correlations >= .70
WJ IV OL / CASL & OWLS score correlations in ages 7‐17 sample (n = 50)