Embed Size (px)
Citation preview
Advanced Data Analysis Student Engagement
CARE February 25th
Facilitator: Geneviève Légaré
Last time
• Day 1: Analysis of provincial exam results – Common table of results – Common writing
• Day 2: Disaggregated results—item analysis
– Strength and weakness analysis of the History exam
• Day 3: Other types of data-TTFM- – Student engagement
Learning intention
• To develop a deeper understanding of student engagement by exploring various indicators and by carrying out an advanced data analysis of TTFM results
Today AM Analytical • Understanding engagement • Analysis of a case
PM Practical implications • Carrying out a perception survey in schools
– Research methods – Carrying out a survey
• Use of data – TTFM and students – Connections to strategic planning : MESA and annual report
Approaches
Build understanding together: As a group—a collective concept of student engagement in schools by
1. Brainstorming 2. Working together in teams 3. Synthesis: group thought
Outcomes 1. An understanding of the three measures of student engagement:
intellectual, social and institutional as presented in TTFM 2. A data analysis: An in-depth analysis of student engagement in a
school using TTFM results by applying the reflective and iterative collaborative inquiry process connecting to Wellman and Lipton
3. A critical understanding about using perception surveys in schools : An overview of the research methods and information gathering implied in the development of a questionnaire to gather information about student engagement
4. Making connections to school’s strategic planning and the use of TTFM (linking research with practice)
Main references
Data/Collaborative Inquiry • Victoria Bernhardt:
Continuous school improvement
• Nancy Love: Data Coach Collaborative Inquiry
• Lipton and Wellman: --Collaborative inquiry
Engagement • Willms : Tell Them From Me • Marzano and Pickering: The
highly engaged classroom • Rumberger : Drop out
Collaborative learning cycle Three steps: Lipton and Wellman, p.26
Structure the
dialogue
Activating and
engaging
Exploring and
discovering
Organizing and
integrating
Outcomes (Part 1)
1. An understanding of the three measures of student engagement: intellectual, social and institutional as presented in TTFM
– What are the constructs and what does the research says about engagement and the connection with student success?
Part 1: Engagement Driving questions What do we know about engagement? Mode of inquiry: How do we know about engagement? Why is engagement important?
Part 1: Engagement
• Brainstorm individually all that you know about engagement
• Compare and contrast definition • Pick three to share in group
Engagement: Willms
• Engagement tends to decline steadily over the years (trend patterns)
• What are early signs of disengagement? At elementary level – Low sense of belonging – Poor social behavior – Lack of interest in school work
TTFM: Engagement model
Engagement
Social
Institutional Intellectual
TTFM: Engagement model
Engagement
Social •Belonging •Participation: Sports & clubs •Relationships
Institutional •Value of school outcomes •Positive school behavior •Homework
Intellectual •Quality of instruction •Effort •Intention and motivation
Intellectual engagement
Willms: Intellectually engaged students • Students feel more confident • Girls more engaged than boys • See natives and immigrants populations • Less engaged: more than 1.5 x more anxious
– Anxiety: level of literacy in math and ELA – Reading difficulty early years – Failure
Measures of engagement
Engagement
Social •Belonging •Participation: Sports
& clubs •Relationships
Institutional •Value of school
outcomes •Positive school
behavior •Homework
Intellectual •Quality of
instruction •Effort • Intention and
motivation
Institutional engagement
• Research says: • More variation between schools even after
controlling for family background and attendance
• What are the practical implications from an analytical point of view?
Institutional engagement
Practical implications-- school based: • Quality of instruction • Teacher-student relationship • Learning climate • Expectations • Student advocacy
Part 2: Data analysis
Outcomes
2. A data analysis: An in-depth analysis of student engagement in a school using TTFM results by applying the reflective and iterative collaborative inquiry process connecting to Wellman and Lipton
– Referring to V.B’s model, what type of data is collected in TTFM?
– How can we improve, tweak our collaborative inquiry process?
Reports to look at • Board 2014-2015: Student engagement • A primary school engagement report • A secondary school –engagement report • For school of choice (2-3 years of one-shot
reports, including 2014-2015) • MESA of the selected school
Collaborative learning cycle Activating and engaging:
– What are the assumptions that we bring to the question?
– What questions are we asking? – What are some of the
possibilities for learning that this experience bring to us (data analysis)?
Structure the
dialogue
Activating andf
engagging
Exploring and
discovering
Organizing and
integrating
Collaborative learning cycle Exploring and discovering • What important points seem
to pop up? • What patterns, categories or
trends are emerging • What seems to be surprising
or unexpected • What are some way not yet
explored with the data
Structure the
dialogue
Activating andf
engagging
Exploring and
discovering
Organizing and
integrating
Collaborative learning cycle
Organizing and integrating • Generating theory • What inferences, explanations or
conclusions might we draw (causation) • What additional data sources might we
explore to verify our explanation (confirmation)
• What solutions might we explore as results of our conclusions (actions
• What data will we need to collect to guide implementation
Structure the
dialogue
Activating andf
engagging
Exploring and
discovering
Organizing and
integrating
Marzano and Pickering
Model of attention and engagement • Attention:
– Emotion: How do I feel? – Interest: Am I interested?
• Motivation
– Perceived importance? – Self-efficacy: Can I do this?
Practical implications: Teaching strategies
• Interest? – Curiosity – Cognitive dissonance
• Relevance? – From abstract to concrete – Self
• Self-efficacy? – Goal setting – Feedback on progress
Rumberger
Dropping out : Why students drop out of high school and can be done about it. Harvard University Press • Comprehensive text about different concepts
pertaining to ‘droppout’ • Several models of engagement reviewed
Rumberger
• Dropout: Engagement is one aspect in the process of dropping out of school:
‘Dropping out itself might be better viewed as a process of disengagement from school, perhaps for either academic or social reasons that culminates in the final act of leaving’. P. 151
Wehlage et al.
• Model of student engagement in academic work ‘the student’s psychological investment and effort directed toward learning, understanding, or mastering the knowledge, skills, or crafts that academic work is intended to promote’. P. 151
Wehlage et al.
…because engagement is an inner quality of concentration and effort, is it not readily observed, so it must be inferred from indirect indicators such as the amount of participation in academic work (attendance, time spent, enthusiasm, interest). P. 151
National Research Council (1 of 2)
Student engagement and effect of context Engagement involves : • observable behaviors ( class participation, work
completion, taking challenging classes) • unobservable behaviors (effort, attention,
problem solving, and the use of cognitive strategies)
• emotions (interest, motivation, pride in success). Rumberger, p.151
National Research Council (2 of 2)
Effect of educational context • Context = instruction, school climate, school
organisation, school composition and school size • Context is mediated by three psychological variables:
– Student beliefs about their competence and control (I can) – Their values and goals (I want to) – Their sense of social connectedness or belonging (I belong)
In Rumberger, p.151
Part 3: Tools, data type, etc.
Outcomes
3. A critical understanding about using perception surveys in schools : An overview of the research methods and information gathering implied in the development of a questionnaire to gather information about student engagement
– What are the objections that schools might have with regards to the use of a questionnaire such as TTFM ? How do you build a questionnaire –Validity and reliability, sampling etc.
Part 3: Implications
• Brainstorming question: • Using TTFM in schools: What are the
objections that you hear about the use of TTFM?
• Making connections to issues pertaining to building a questionnaire—validity and reliability, etc
Implications
• Using TTFM in schools: Data types • Surveying all the question and items in TTFM:
What type of data is collected? • Issues pertaining to perception questionnaire • See V.B.
Part 4: Using data
Outcomes
4. Making connections to school’s strategic planning and the use of TTFM (linking research with practice)
– Are the results about student engagement included in in the MESA or the annual report of the school?
– How are the results used in the schools
Group synthesis
What are two or three enduring concepts About engagement? TTFM as instrument? Type of data? Using data in schools?
Exit cards
One thing I learned On thing I will apply Next time, I would like to…
Further Readings • Bernhardt, V. (1998). Data Analysis for Continuous School Improvement.
Larchmont, NY: Eye on Education.
• Boudett Parker, K., City, E. A., and Murnane, R. J., eds. (2005). Data Wise: A step-by-step-guide to using assessment. Results to improve teaching and learning (Cambridge, MA, Harward Education Press).
• Lipton, L. & Wellman, B. (2012). Got Data? Now what? Creating and leading cultures of inquiry. Bloomington, IN: Solution Tree Press.
• Love, N. (2009). Using data to improve learning for all: A collaborative inquiry approach. (Cambridge, Massachusetts, Corwin Press).
• Love, N., Stiles, K. E., Mundry, S., and DiRanna, K. (2008). The Data Coach's guide to improving learning for All Students, Library of Congress Cataloging-in-Publication Data edn (Thousand Oaks, California, Corwin Press).
• Marzano, R. J., and Waters, T. (2001). Classroom Instruction that Works: Research-based Strategies for Increasing Student Achievement.(Alexandria, VA: ASCD Press).
Thank you
Evidence-based Practice Project for the Anglophone Community www.evidencebasedquebec.qc.ca Twitter: @ebppractice
Background
Data Driven Dialogue
• What is the portrait of the CST results at your board? • What are the strengths and weaknesses?
Adapted from Wellman, B., & Lipton, L., 2004. Data-Driven Dialogue: A Facilitator’s Guide to Collaborative Inquiry. Sherman, CT: MiraVia, LLC. Used with permission.
Revised Student Learning Problem
Verify Causes Tree
A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved.
Adapted from Paul G. Preuss, Root Cause Analysis: School Leader’s Guide to Using Data to Dissolve Problems. 2003. Larchmont, NY. Eye on Education. Used with permission.
Types of Causes
• Curriculum: See next slide • Instruction: Research-based, quality, differentiation • Assessment: Formative evaluation, feedback to
students • Equity: Attitudes and perceptions toward certain
groups • Critical supports: engagement, leadership,
collaboration, help for students, high quality PD for teachers
Curriculum
Standards (QEP)
Teaching Evaluation
Types of Data
Victoria Bernhardt
DEMOGRAPHICS
PER
CE
PTIO
NS
STUDENT LEARNING
SCH
OO
L PR
OC
ESSE
S
Types of Data
Tell Them From Me: What type of data do we refer to when refeerring to engagement?
DEMOGRAPHICS
PER
CE
PTIO
NS
STUDENT LEARNING
SCH
OO
L PR
OC
ESSE
S
Theories of Causation: Lipton and Wellman, p.46
Causation
Curriculum
Instruction
Infrastucture Teachers
Students
VC Decision making
A Data Coach’s Guide to Improving Learning for All Students: Unleashing the Power of Collaborative Inquiry © 2008 by Corwin Press. All rights reserved.
Adapted from Paul G. Preuss, Root Cause Analysis: School Leader’s Guide to Using Data to Dissolve Problems. 2003. Larchmont, NY. Eye on Education. Used with permission.
Planning PD model
WANT
NE
ED
CAN
FUN
—(D
RIV
E)
Collaborative Inquiry Process
Data Wise Model (Boudett et. al)
Collaborative Inquiry Process
Instructional Data Team Model (Leadership and Learning)
Data collection
Analysis of strengths
and weaknesses
Set SMART Goals
Select teaching practices
Determine indicators
