Monitoring cognitive processes during reading: Is there

converging evidence from fMRI studies of metacognitive control?

Linda BakerDepartment of Psychology

University of Maryland, Baltimore County

Maryland Metacognition SeminarMay 10, 2013

“The vice of the poor reader is to say the words to himself without actively making judgments as to what they reveal.”

E. L. Thorndike, 1917

Metacognition and comprehension monitoring: Definition and assessment

My early empirical research on comprehension monitoring

A new pursuit: Neuroscience connections Work in progress:

◦ Pilot results for fMRI cognitive monitoring tasks◦ Research plan

Overview

My research on metacognition had its roots in the 1970s work of cognitive developmental psychologists John Flavell and Ann Brown, whose interests in memory development led them to ask what children know about their memory and how they learn to control it.

Research on metacognition in relation to student achievement continues to be a significant area of inquiry around the world.

Evidence of popular appeal: A Google search for “metacognition and achievement” yielded 586,000 hits in April of 2013.

Origins and impact

Knowledge: Knowledge about the skills, strategies, and resources that are needed to perform a task effectively

Control: the use of self-regulatory strategies to ensure successful task completion ◦ Planning◦ Evaluating◦ Monitoring◦ Checking◦ Revising

Baker & Brown,1984

Two Components of Metacognition

Conceptions of good readers/good reading◦ What makes someone a really good reader? ◦ If you know someone was having trouble reading, how would you

help that person?

Strategies for dealing with comprehension difficulties◦ What do you do when you come to a word that you don’t know? ◦ What do you do when you don’t understand something that you

have read?

Strategies for studying/remembering◦ What do you do to help remember what you’ve read?◦ If you needed to study a chapter in your history book for a test,

how would you do it?

Meyers & Paris, 1978

Interview Questions for Assessing Metacognitive Knowledge of Reading

Directions: Listed below are statements about what people do when they read academic or school related materials such as textbooks or library books. After reading the statements circle the number that applies to you. Please note there are no right or wrong answers.

  I think about what I know to help me understand what I

read. (global reading strategy)

I summarize what I read to reflect on important information in the text. (support reading strategy)

I adjust my reading speed according to what I’m reading. (problem solving strategy)

(Each item is rated on a 5-point scale with respect to how often the reader does the described activity.)

 (Mokhtari & Reichard, 2002)

METACOGNITIVE AWARENESS OF READING STRATEGIES INVENTORY

Evaluation◦Keeping track of the success or failure of one’s own ongoing efforts to understand

Regulation◦Taking appropriate steps to deal with whatever difficulties arise

Comprehension Monitoring

Detection of errors in passages

Ratings of felt understanding

Self-corrections during oral reading

Real-time measures of processing◦ Eye movements, reading times, NEUROIMAGING???

Verbal reports◦ Retrospective◦ Concurrent (thinking aloud)

Approaches to Assessing Comprehension Monitoring

The planet Jupiter is far from the sun. It is also the largest planet in our solar system. Jupiter circles the sun once every 12 years. Jupiter is a thousand times smaller than our planet. Jupiter has a giant red spot. The spot is the planet’s most famous feature.

Clouds form when warm and cold air meet. Clouds can bring rain, hail, sleet or snow to earth. Some people are paid to study the clouds. The clouds tell them what moultin we will have. There are many types of clouds. Each type means different weather is coming.

Long ago North America was covered with woodlands. Many early settlers lived in the woodlands. They collected nuts, berries, and fruits to eat. They used sand from the trees to make many things. They built houses, boats, and weapons. They even made medicine from the tree bark.

Please read:

Issues to Consider with Error-Detection Approaches

• Readers may use fix-up strategies for resolving comprehension difficulties and so they think they understand.

•Readers may identify problems other than those intended for them to find.

•Whether or not a problem will be reported depends on several factors:

•the readers’ goal for reading•the criteria they adopt for evaluating their understanding•their threshold for deciding when a problem is serious enough to report•personality characteristics (e.g., tolerance for ambiguity)

Baker & Cerro, 2000

Fourth grade students ◦ More skilled readers◦ Less skilled readers

Three instructional conditions◦ Monitor understanding at a local level ◦ Monitor understanding at a global level◦ No special instructions

Individual instruction on using local or global standards to evaluate comprehension

Assessed on use of standards while reading a longer passage immediately and two weeks later

Baker & Zimlin, 1989

Early Evidence that Comprehension Monitoring Can be Taught

◦ Lexical Are there any words I don’t understand?

◦ External consistency Is there any information that doesn’t agree with

what I already know? ◦ Propositional cohesiveness

Are there any ideas that don’t fit together because I can’t tell who or what is being talked about?

Standards for Evaluating Comprehension: Microstructure (Local )Level

◦ Structural cohesiveness Are there any ideas that don’t fit together

because I can’t tell how the ideas are related◦ Internal consistency

Are there any ideas that don’t fit together because I think the ideas are contradictory?

◦ Informational completeness Is there any information missing or not clearly

explained?

Standards for Evaluating Comprehension: Macrostructure (Global) Level

Instruction

Local (micro) Global (macro) No Instruction

IMMEDIATE

Level

Microstructure 4.06 2.38 1.28Macrostructure

2.38 4.09 1.11

DELAYED

Level

Microstructure 3.15 2.27 0.94Macrostructure

2.61 4.10 1.33

Mean Number of Problems Identified at Two Levels of Evaluation (maximum = 6)

“Throughout the reading of the text, good readers do much monitoring, are very aware of characteristics of the text (e.g., its difficulty; relevance to their reading goals; when the text is ambiguous; when the author is attempting to bias the reader; and, how the ideas in text relate to prior knowledge, for example, recognizing the ideas in the text as familiar to ones encountered before). The readers also monitor when they are having problems reading, for example, losing concentration or failing to understand terms in the text or get the meaning of the text. Such awareness of difficulties (or lack of them) can cause the reader to adjust reading, either speeding up or slowing down, or perhaps even seek other text to provide some background. “

Pressley & Gaskins, 2006

Metacognitively competent reading …

Research to date has not examined neural correlates of comprehension monitoring, but related research on discourse processing and executive function suggests that neuroimaging studies will yield valuable information.

Executive function includes processes typically regarded as metacognitive in nature, such as planning, monitoring, and error correction and detection.

Fernandez-Duque, Baird, and Posner (2000) were among the first to suggest that activation in the brain in response to conflict monitoring (in the anterior cingulate) and conflict resolution (in the lateral prefrontal cortex) may reflect the metacognitive processes of cognitive monitoring and cognitive control.

Neuroscience Connections

fMRI and ERP are now being used to study neural correlates of higher-level processes related to comprehension, such as:

◦generating inferences while reading short narrative passages differing in complexity (Kuperberg et al., 2006; Prat, Mason, & Just, 2011)

◦detecting subtle and blatant violations of world knowledge in simple passages (Sanford et al., 2011)

◦ resolving referential ambiguities (Nieuwland & Van Berkum, 2008)

Relevant neuroimaging studies

Discourse processing research has shown changes in brain functioning as readers attempt to make sense of the texts that they read, with involvement of both left and right hemispheres. For example, Kuperberg et al. found greater activation in bilateral anterior inferior prefrontal gyri, the left middle frontal gyrus and bilateral medial prefrontal gyri for less coherent passages.

A few correlational studies have indicated that the brains of skilled readers respond more efficiently to tasks that challenge comprehension (e.g., Prat & Just).

Much of this research has been undertaken by cognitive psychologists who do not frame their work in an educational or metacognitive context and who do not consider the implications for intervention.

Those neuroimaging studies that do have an explicit focus on metacognition tend to address memory of discrete items rather than text comprehension (Shimamura, 2008).

Basic Neuroimaging Findings

Purpose:

◦ to determine whether systematic differences in brain functioning are apparent when adult readers encounter texts that demand monitoring of their ongoing comprehension processes

◦ to examine individual differences in brain activation patterns in relation to reading skills and self-regulated learning strategies (including metacognitive strategies and motivation).

Collaborators: Nicole Else-Quest, UMBCMalle Tagamets, MPRC

Cognitive Monitoring and Reading: An fMRI Study of Individual Differences

Participants: ◦ 22 UMBC undergraduates

Each completed two tasks:◦ Think-aloud while rating the comprehensibility of one

set of materials ◦ Computer presentation: response times to evaluate

comprehensibility of a second set of materials Instructions:

◦ Specified that materials had been deliberately modified by the research team to contain content that should make the passages harder to understand

Materials Task A Task B

Pilot Study of the Cognitive Monitoring Tasks to be Used in the Scanner

Zack took a test in school. He did not know the material. He scored the highest in the class.

Zack took a test in school. He did not take good notes. He scored the highest in the class.

Zack took a test in school. He studied hard for the exam. He scored the highest in the class.

Task A

The class took a field trip to the fair. The teacher bought tickets for the rides.

 The class took a field trip to the fair. The teacher bought food for the rides.

  The class took a field trip to the fair. The

teacher bought drugs for the rides.  The class took a field trip to the fair. The

teacher bought whibbles for the rides.

Task B

◦ Task A: Internal consistency 60 3-sentence passages, some of which contain

inconsistencies that are easily resolvable or not Three separate lists, counterbalanced across

problem types◦ Task B: External consistency

80 2-sentence passages, some of which contain subtle or blatant violations of prior knowledge or lexical knowledge

Four separate lists, counterbalanced across problem types

Pilot Study Materials

Zack took a test in school. He did not know the material.

He scored the highest in the class.

?

Computer Presentation Format for Internal Consistency task

The class took a field trip to the fair. The teacher bought ______ for the rides.

drugs

Computer Presentation Format for External Consistency Task

Passage Type

Accuracy(%“correct

”)

Response Time(msec)

ResponseTime

(“correct”)

Rating of Clarity

(1-3 scale , 3 = clear)

Consistent 90 593 551 2.54

Resolvable 52 749 695 1.86

Inconsistent 85 806 768 1.09

Pilot Data for Internal Consistency Task

Completion Type

Accuracy(%“correct

”)

Response Time(msec)

ResponseTime

(“correct”)

Rating of Completion Quality

(1-3 scale , 1 = good)

Good 94 1003 905 1.17

Subtle violation 67 1346 1228 2.29

Blatant violation 91 1031 978 2.70

Non-word 97 1077 1072 2.62

Pilot Data for External Consistency Task

Three phases:1. experience the fMRI simulator2. complete a series of laboratory-based

tasks and assessments3. complete two cognitive monitoring

tasks in the fMRI scanner along with control tasks for baseline information

Cognitive Monitoring and Reading: An fMRI Study of Individual Differences

36 UMBC undergraduate students◦ 18 men and 18 women

Selection criteria: ◦ Inclusion: right-handed, native mono-lingual

speaker of English. ◦ Exclusion: no first degree family member with a

psychiatric disorder, no contraindications for fMRI

Participants

Individual Difference Assessments (lab)◦ Reading Ability (comprehension and vocabulary)◦ Working Memory◦ Fluency◦ Self-Regulated Learning

metacognitive and cognitive strategies, motivation◦ Epistemic Beliefs

Cognitive Monitoring Assessments (fMRI)◦ Internal Consistency◦ External Consistency

Measures

The Nelson-Denny Reading Test is a standardized norm-referenced test of reading comprehension and vocabulary knowledge

The reading comprehension test consists of 7 informational passages with 38 multiple-choice comprehension questions.

The vocabulary test consists of 80 items; the examinee selects which of 6 words is the best definition of the target item.

Brown, Fishco, & Hanna, 1993

Measures: Reading Ability

The Reading Span test assesses working memory by requiring participants to read sentences, verify their truth value, and remember a word presented immediately after reading and responding to the sentence.

An example sentence is “Mr. Jones left the lawnmower in the lemon,” to which the participant should respond “incorrect”; the word EAGLE is then presented, followed by the next sentence in the set.

After all of the sentences in a given set have been presented, the participant is asked to recall the target words in order. Set sizes range from three to seven.

(Bailey, 2012)

Measures: Working Memory

The Reading Fluency subtest of the norm-referenced Woodcock Johnson III Tests of Achievement (WJ III) gives examinees 3 minutes to read and verify the plausibility of as many sentences as possible

The Math Fluency subtest of the WJ III gives examinees 3 minutes to solve as many single-digit math calculation problems as possible

Woodcock, McGrew, & Mather, 2001

Measures: Fluency

The Learning and Study Strategies Inventory (LASSI) is a norm-referenced assessment of students' awareness about and use of learning and study strategies.

10 scales: attitude, motivation, time management, anxiety, concentration, information processing, selecting main ideas, study aids, self-testing, and test strategies.

Participants indicate on a 5-point scale the extent to which each described thought or activity is typical of themselves. An example Item on the self-testing (metacognitive) scale is, “I stop periodically while reading and mentally go over or review what was said.”

Weinstein et al., 2001

Measures: Self-Regulated Learning

The Epistemological Beliefs Inventory taps students’ beliefs about the nature of knowledge and the nature of learning.

Five subscales: ◦ Certain knowledge, innate ability, quick learning, simple knowledge,

and omniscient authority

Participants rate on a 6-point scale the degree to which they agree with each assertion. An example statement on the certain knowledge scale is, “There are certain truths in life that won’t ever change.”

Bendixen, Schraw, & Dunkel, 1998

Measures: Epistemic Beliefs

Internal Consistency Task ◦ Participants evaluate whether each passage is clear

and consistent

External Consistency Task◦ Participants verify whether each target word is a good

sentence completion

Order of tasks is counterbalanced, organized into blocks and runs in the scanner, with periods of rest and a continuous performance task.

Measures: Cognitive Monitoring

SPM software will be used to analyze the imaging data. Patterns of activation will be contrasted across the different conditions within each task.

Changes in brain activity will be analyzed in conjunction with the behavioral data (response times and accuracy).

Voxelwise analyses will reveal the distribution of activation during cognitive monitoring in relation to reading skills and self-reported use of cognitive and metacognitive learning strategies.

Analyses

Both left and right hemispheres will be involved in evaluating coherence, consistent with other studies of discourse processing.

The right temporal cortex and the right inferior prefrontal cortex in particular may show stronger hemodynamic responses for the inconsistent passages.

Better readers will show less brain activation while monitoring their comprehension than poorer readers, with evidence for some compensatory processing taking place in the right hemispheres of those whose reading processes are less efficient (in terms of reading fluency, reading memory span).

Expected Findings

To examine developmental changes in brain activity among adolescents and young adults as they engage in tasks requiring cognitive monitoring. Age-related differences in comprehension monitoring are apparent at the high school level. Increases in executive functioning that are seen at this time are often attributed to maturation of the pre-frontal cortex.

To determine whether training in comprehension monitoring and self-regulated learning strategies increases efficiency of neural functioning. A recent study involving mathematical problem solving provides reason to expect such an effect (Shroeder, 2012).

Longer-Term Goals

My UMBC undergraduate research assistants on the fMRI project:

Iffath Farzano Narcisse Fon Allyson Nedzbala Alicia Serrato Jonathan Shonk Keerthi Yarlagadda

With thanks to …