Deficit or Difference?bearcenter.berkeley.edu/sites/default/files/IIR... · 48 2.5 Item Category Count Mean θ Item Category Count Mean θ (logits) (logits) Item 1 0: No Integration

Deficit or Difference?Cycling through the 4-Building-Blocks

to Assess the Power of Context in Narrative Comprehension in Autistic and Typically

Developing Adolescents

Comic vs. Text

1

About Us

2

About Alexander• BA Psychology

• MA Special Education

• High School English Teacher - Special Day Class

• PhD Special Education - Joint Doctoral Program

• Measurement in the Social Sciences - I, Graduate Student Instructor (2018)

• Adjunct SFSU Special Education Department

• Project ALLIES - Instructor & Advisor

3

About James• BSc Computer Science 1996

• MEd 2011

• High School Mathematics Teacher

• PhD Candidate - Quantitative Methods and Evaluation

• Measurement in the Social Sciences - I, Graduate Student Instructor (2014)

4

Our Mission

5

inspired by the neurodiversity movement.

Challenging Assumptions

6

Challenging assumptions about autism using a lens of access,

Repositioning Autism

7

Deficit? Or difference?

… rather than as a way to assert control and harm vulnerable groups.

First, do no harmTo use measurement as a way to help people and advance science…

8

Recognizing Autism

9

Diagnostic Statistical Manual of Psychiatric

Disorders - DSM 5 (American Psychiatric

Association, 2013)

Areas of Impact

10

•Frith (1989)

•Baron-Cohen, Richler, Bisarya, Gurunathan, and Wheelwright (2003)

•Baron-Cohen and Wheelwright ( 2004)

Meaning Making & Narratives

11

•Nuske & Bavin (2011)

•White, Hill, Happe & Frith (2009)

Happe and Frith, (2006)

Visual Processing Disposition

12

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi dapibus felis quis consectetur sodales. Curabitur consectetur velit neque, in tincidunt dui semper id. Nunc sagittis, ex at molestie elementum, ipsum augue posuere neque, at hendrerit nunc odio tincidunt ex. Nunc tempor urna ac mollis aliquet. Quisque lobortis augue eu venenatis consequat. Donec sodales urna tellus, ut varius lacus vehicula non. Nullam aliquam velit a mauris hendrerit ornare. Fusce auctor porta luctus. Ut aliquam odio ac nisl fringilla blandit. In hac habitasse platea dictumst. Nulla congue ligula quis ante ultricies vestibulum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec venenatis ligula vitae magna condimentum, nec scelerisque ligula posuere.

Vivamus volutpat tellus nisl, ac aliquam lacus aliquet ac. Pellentesque vestibulum, nulla a posuere ultricies, lorem felis eleifend est, interdum aliquet nibh tellus semper nisl. Nam malesuada scelerisque ante sit amet vulputate. Etiam bibendum viverra ipsum, et vehicula metus blandit vel. Nunc non mi nec diam bibendum luctus vel vel nibh. Integer sodales bibendum velit, eget ultricies arcu varius ac. In et urna sagittis, dapibus neque ac, bibendum lectus. Mauris volutpat quam at lacus laoreet, et sollicitudin lectus molestie. Proin at consequat magna. Cras rhoncus eleifend rutrum.

Phasellus bibendum ut ex quis lobortis. Proin ultricies arcu erat, sed placerat risus ullamcorper vitae. Curabitur pharetra elit eleifend, tristique felis et, convallis turpis. Curabitur in sagittis neque, condimentum accumsan risus. Praesent efficitur purus quis commodo ullamcorper. Pellentesque suscipit ac quam non venenatis. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Fusce sed volutpat elit. In pharetra, tortor ut maximus pellentesque, enim lorem convallis dolor, non consectetur mi purus ac eros. Sed consectetur, nunc luctus iaculis aliquet, felis turpis vestibulum nunc, sed mollis nisi quam quis erat.

Fusce dapibus nisi vitae eros iaculis, sed convallis sapien accumsan. Quisque egestas interdum tortor sit amet mollis. Phasellus quis diam molestie, facante vitae, auctor enim. Nulla ultricies, sem egestas eleifend commodo,urna lectus semper libero, nec pharetra erat urna at dui. Orci varius natoqpenatibus et magnis dis parturient montes, nascetur ridiculus mus. Phafringilla massa sed leo convallis dapibus. Interdum et malesuada fameac ante ipsum primis in faucibus. Phasellus congue, nibh vel ultriciesfinibus, nibh risus eleifend nibh, volutpat ultrices nulla lacus vel ligul

Comics and Access

13

Comic+text vs Text-Only

14

Research Question• When individuals, both neurotypical and autistic, engage with narratives,

are their challenges better explained by a deficit perspective or an access perspective? – Do their challenges result from an internal deficit within their

repertoire of narrative comprehension processing? – Alternatively, is their performance more of an issue of access, in

which all three facets of the prevailing construction-integration models (i.e., the reader, text, and context) interact to create situation specific challenges?

15

Slow Down!

16

Construct Map

17

ItemsDesign

Outcome Space

Measurement Model

BEAR Assessment SystemFour Building Blocks (Wilson, 2005)

My First Construct Map

18

1

Coherence

My Next Construct Map

19

1

Inferential Thinking Bias

Items for Inferential Thinking Bias

20

1

➔ Motivational Inference“Why do you think Tom let Jimmy keep the toy car?”

➔ Meta-Reasoning“What made you think of that answer?”

➔ Evaluative Inference“What is the most important lesson someone can learn from this comic?”

➔ Meta-Reasoning“What made you think of that answer?”

• As I made decision rules to relate it back to the construct....

Outcome Space

21

1

A portion of the outcome space

Measurement Model: PCM

22

• Notice the banding of the thresholds

1

? But wait...

23

Something’s not quite right…• What about those responses with a

combination of text-implicit and script-implicit?

• No strong theory to hold levels together.

2 Is Integration the Key?

24

Can Integration be Ordinal?

25

• less integration

2

• more integration

Integrative Construct Map

26

2

Same Items Design

27

2

same items...

• Bay Area Elementary School• Grades 3–6• N=72

Participants

28

Number of Participants by

Age

Age N

8 6

9 11

10 21

11 20

12 14

Scoring

29

• Only scored the “why” question for an initial analysis

2

2 Wright Map

30

• This includes only the “why” questions

Scoring, again...

31

• Scored all the item types• Dropped one story

– Everyone scored into the same category

2.5

32

Text-Implicit QAR

Script-Implicit QAR

Script-Implicit & Text-Implicit QAR

Exp

licit

Inte

grat

ion

2.5

Mean Location

33

2.5

Item Category Count Mean θ Item Category Count Mean θ(logits) (logits)

Item 1 0: No Integration 1 -2.94 Item 7 0: No Integration 5 -2.07

1: Text-Implicit 30 -0.32 1: Text-Implicit 31 -0.35

2: Script-Implicit 37 0.4 2: Script-Implicit 34 0.64

3: Combination 4 0.54 3: Combination 2 1.95








3: Combination 0 3: Combination 6 1.22













Latent Regression

34

Exp

licit

Inte

grat

ion

Regressor Coefficient Std Err

Age* 0.282* 0.122

* Significant at the .05 level

2.5

Multidimensional

35

2.5

.767

.760

.834

Construct Map

36

3

• We are confident in the IIR construct at this point

• Made only minor revisions to the construct map

• Now, we’re ready to use IIR as an outcome measure in a study

Demographics

37

Gender Age

11 12 13 14 15 16 17 Total

M 2 20 7 6 13 4 12 64

F 4 23 1 1 5 9 10 53

Missing - - - - - - - 13

Total 6 43 8 7 18 13 22 130

Study Design

38

• Two samples (Autistic & Typically Developing)• Two “treatments” (comic+text vs. text-only)• Random assignment within each sample

Autistic groups:

• A (Autistic + Comic)

• B (Autistic + Text)

Typically Developing sample:

• C (Typically Dev. + Comic)

• D (Typically Dev. + Text)

Study Design — Comparability

39

• Text-explicit QARs (before “main” items):– Check that all students were able to decode – Both text and text+comic formats

(counterbalanced)– Questions required text-explicit QARs

Comparability Results

40

Comic+text Text-only t-test

Autistic 5.95/6 5.35/6 t=3.27, df=19, p=0.004

Neurotypical 5.85/6 5.82/6 t=0.37, df=124, p=0.714

t-test t=0.570, df=143, p=0.570 t=-2.67, df=143, p=0.009

Items Design — Text-Only

41

1. Billy loved bikes. He was riding his bike to the beach one day, when he came across the most beautiful bike he had

ever seen. He said to the boy that was riding the beautiful bike, “wow, that’s such a beautiful bike!” “Thanks!” said

the boy.

2. “Hey, how much did you pay for that bike?” asked Billy. “It was $300”, answered the boy.

3. Billy rode his bike home. He thought to himself, “hmmm, maybe my parents will buy me that bike for my birthday”.

4. Billy got home. He says to his parents,“hey mom and dad. For my birthday I would like a new bike pretty please!”.

“No Billy. You already have a nice bike and you don’t need a new expensive one”, said his father.

5. Billy felt sad, and mad. So he took his bike, and rode it to an ice cream shop.

6. Billy walked out of the ice cream shop with ice cream, and found a table to sit outside. “Ice cream, you always make

me feel better” Billy said to himself. Next to him was another table, where a lady was sitting.

7. As Billy was eating his ice cream, the lady left and forgot her wallet. Billy went and grabbed her wallet. He opened

her wallet. “Wow, is this really $300? With that money, I could buy that red bike!” Billy said.

8. The lady that was sitting at the table next to him came back, looking for her wallet. She said, “hello there, have you

seen my wallet? I can’t find it anywhere.”

9. “No, I don’t know where it is”, said Billy.

3

Items Design—Text+Comic

42

3

panels on

following

slides…

Panels 1–3

43

3

Panels 4–6

44

3

Panels 7–9

45

3

3 Outcome Space

46

Level 3—Original • He is a liar and I have seen

greedy kids do this stuff before.

• Desperate people do desperate things when they find money like Billy did.

• New conjunctions used

Level 3—New

Wright Map

47

3

Mean Location

48

2.5

Item Category Count Mean θ Item Category Count Mean θ(logits) (logits)

Item 1 0: No Integration 0 Item 7 0: No Integration 4 -0.48










2: Script-Implicit 85 0.24 2: Script-Implicit 75 -0.03







1: Text-Implicit 118 0.01 1: Text-Implicit 33 -0.32

2: Script-Implicit 2 -1.04 2: Script-Implicit 50 0.15






Model Selection

49

3

Model 1 Model 2 Model 3 Model 4

xx

Main Effects

50

3

Predictor Variable Coefficient Std. Error

Age 0.005 (.024)Comic (Format Type) -0.171 (.099)Girl (Gender)Autistic (Diagnosis)Self-Rated Comic ExperienceSpeech-to-Text tool

0.291*-0.534*0.186* -0.164

(.101)(.165)(.061)(.122)

Model 1

Significant Predictors Only

51

3

Predictor Variable Coefficient Std Error

Comic (Format Type) -0.182* (.099)Girl (Gender) 0.294* (.101)Autistic (Diagnosis) -0.553* (.161)Self-Rated Comic Experience 0.175* (.061)

Model 2

Interpretation

Impact of local processing disposition on students’ inferential reasoning

• This disposition translates into particular types of student thinking; namely attention to details, chaining propositions, and the creation of enumerative rather than integrative summaries of text.

• Text-implicit QARs (Pearson & Jonson, 1978) favor this line of thinking since they do not require the integration of world knowledge, but rather operate within the propositional facts of the narrative.

• In addition, relying on details in the narrative to serve as the evidentiary basis for making an inference may seem to be safer, in terms of predictability and risk-avoidance.

52

3 Model 2

Two-Way Interactions

53

3

Predictor Variable Coefficient Std Err

Comic (Format)Female (Gender)Autistic (Diagnosis)Perceived Comic ExperienceComic x Perceived Comic ExperienceAutistic x Comic ExperienceAutistic x Comic Format

-0.802*0.256*-0.934*-0.1060.400*0.375*-0.615*

(.211)(.092)(.325)(.080)(.111)(.128)(.289)

Model 3

Analysis á Trois

54

3

Predictor Variable Coefficient Std Err

Format (Comic)Gender (Female)Diagnosis (Autistic)Self-Rated Comic ExperienceFormat (Comic) x Self-Rated Comic ExperienceDiagnosis (Autistic) x Self-Rated Comic ExperienceDiagnosis (Autistic) x Comic FormatDiagnosis (Autistic) x Comic Format x Self-Rated Comic Experience

-0.520*0.262*-0.268-0.0350.2340.036

-1.884*0.671*

(.230)(.090)(.413)(.082)(.124)(.185)(.535)(.249)

Model 4

Group Mean Locations

55

3

Gender(F=1)

Autistic(Yes=1)

Format(comic=1)

Comic Experience

Group Mean IIR

1 1 0 0 0.1211 1 0 1 0.1221 1 0 2 0.1231 1 0 3 0.1241 1 1 0 -2.2831 1 1 1 -1.3771 1 1 2 -0.4711 1 1 3 0.4351 0 0 0 0.3891 0 0 1 0.3541 0 0 2 0.3191 0 0 3 0.2841 0 1 0 -0.1311 0 1 1 0.0681 0 1 2 0.2671 0 1 3 0.466

Gender(F=1)

Autistic(Yes=1)

Format(comic=1)

Comic Experience

Group Mean IIR

0 1 0 0 -0.141000

111

000

123

-0.139-0.138-0.14

0 1 1 0 -2.5450 1 1 1 -1.6390 1 1 2 -0.7330 1 1 3 0.1730 0 0 0 0.1270 0 0 1 0.0920 0 0 2 0.0570 0 0 3 0.0220 0 1 0 -0.3930 0 1 1 -0.1940 0 1 2 0.0050 0 1 3 0.204

Model 4

56

3

Model 4

Discussion Points

57

Deficit is Situated

• On the one hand it is a deficit since their mean location on the IIR scale is significantly lower.

• But is the deficit constitutional (i.e., built into the autistic students) or experiential (i.e., suggesting a lack of opportunity to immerse themselves in the medium)?

• If an individual not familiar with baseball played for the first time and did not perform well, we would not conclude that they have a baseball deficit.

• Clearly, if that person continued to play baseball, he would improve.

• Can the same lens be applied to cognition and particularly forms of integration?

58

Autism & Integration

• Van der Hallen et al., (2015) found through their meta-analysis that when changing the context for visual integration, such as the outcome measure (e.g., response time vs. accuracy) the global differences don’t necessarily present themselves.

59

Autism & Integration

• Auditory integration has also been investigated as a context, or modality by Mottron and colleagues (2000)

– Had autistic and typically-developing kids listen to two different pieces of music, one at a time, and then were asked if they had the same or different rhythm.

– The pieces of music were manipulated at different levels (local vs. global), with rhythm being the highest global representation.

– They found that all participants used the rhythm as a discrimination cue, which is attenuation at the global level.

– Even when their were changes at the local level, such as pitch, it did not interfere with them processing the rhythm.

60

A Road Map for a Situated Target

• IIR can serve as a road map to position students’ responses on the IIR scale, further taking note of what material and context brings out their IIR, while at the same time building off of their responses and scaffolding them up the scale.

• Since capturing disposition is somewhat of a moving target, the IIR taxonomy (i.e., IIR construct map) serves as a guide in illuminating the positionality of the respondent for that given context.

61

• IIR also suggests the value of trying to capture students’ dominant dispositions, rather than simply achievement.

• Educators are often surrounded by high stakes testing, where achievement and accountability are the coin of the realm.

• If students do well, they are deemed to have more ability and schools can receive more funding.

A Road Map for a Situated Target

62

Context vs Learner

• This can lead to teachers teaching to a test, being consumed with an achievement mindset.

• In which case, any time a learner cannot do something, they are painted with a deficit lens.

• Rather, seeing learning in the context of situated dispositions, educational materials can be developed with the idea of bringing out one’s potential by situating their learning, and focusing on the context rather than the learner as the key issue.

63

64

Blum, Mason, Kim, & Pearson (under review)

Text-Implicit QAR

Script-Implicit QAR

Script-Implicit & Text-Implicit QAR

Exp

licit

Inte

grat

ion

Pearson & Johnson (1978)

QQ A

● Text-Explicit● Text-Implicit● Script-Implicit

Raphael & Au (2005)

Basaraba et al. (2013)

A Final Word

65

Repositioning AutismA NeuroDiversity Perspective

• Let us reconsider and expand our thinking about the thinking of autistic learners.

• The findings of this study imply that autistic individuals have the same inherit drive as neurotypical peers to integrate their world knowledge with a narrative

– Although their capacity for integration appears to be situated differently.

• Cognition not as a static set of skills, but rather as a set dynamic, situated operations.

• With practice comes experience, with experience comes knowledge, with knowledge comes deeper comprehension, which in turn promotes cognition.

• This line of research provides alternate frameworks, outside of the deficit model and within a neurodiversity lens, for thinking about autism and narrative meaning making.

66

…

Doing Measurement WrightThe importance of collaboration between measurement and substantive research

• Began with substantive question/experiment needing an outcome measure(here measurement is a tool for scientific inquiry)

• The effort of constructing a measure contributes to substantive theory:– In learning how to measure inferencing, we were actually learning

what inferencing is.– Constructing measures is a journey not a destination:

• trial-and-error → new questions → more research • The substantive and measurement questions become entwined and inseparable

(here measurement is scientific inquiry)

67

…

Thank You

[email protected]

[email protected]

68

Thanks especially to:

● David Pearson● Pamela Wolfberg● Karen Draney● Mark Wilson● Berkeley Community

Documents

Deficit or Difference?bearcenter.berkeley.edu/sites/default/files/IIR... · 48 2.5 Item Category Count Mean θ Item Category Count Mean θ (logits) (logits) Item 1 0: No Integration