1

Measuring Academic Behaviors to Improve Student Performance

GEAR UP Tennessee Kick-Off Nashville,TN, August 26, 2013

2

Dr. Glenn Beer

Director of Learning Solutions DeliveryACT Learning Insights Team

33

O U R M I S S I O NHelping people achieve education and workplace success

O U R V A L U E SExcellence

Diversity

Leadership

Empowerment

Learning

Sustainability

44

Agenda

ACT’s College and Career Readiness System ACT’s Engage (Grades 6-9)

Research Administering Engage Reports

Using your ACT Engage results Connecting results to interventions Case Studies

55

Learning Objectives

ACT’s Definition of College and Career Readiness ACT’s College and Career Readiness System and how

ACT Engage fits in that system The role of academic behaviors on college and career

readiness and success How to access and administer the ACT Engage ACT Engage Reports How to use ACT Engage Results

66

ACT’s Definition of College and Career Readiness:

The level of preparation a student needs to be ready to enroll and succeed in—without

remediation–a first-year, credit-bearing course at two or four-year institutions or in trade and

technical schools.

Adopted by the Common Core State Standards Initiative

77

How do we measure College and Career Readiness?

Empirically derived scores needed on an ACT subject-area test to indicate a 50% chance of obtaining a B or higher or a 75% chance of obtaining a C or higher in the corresponding first-year credit-bearing college course.

Test College Course

ACT ExploreGrade 8

ACT Explore Grade 9

ACT PlanGrade 10

ACT

English English Composition

13 14 15 18

Math College Algebra

17 18 19 22

Reading Social Science

15 16 17 22

Science Biology 20 20 21 23

8

And statements thatprovide suggestions to

progress to a higher levelof achievement

Statements that describe what students are likely to know and be able to do...

http://act.org/standard

99

ACT College and Career Readiness System

MEASURING STUDENT PROGRESS TOWARD READINESS IMPROVING COURSE RIGOR

SUPPORTING SOLUTIONS

PLANNING SCHOOL IMPROVEMENT

EXPLORE

8th and 9th grade

curriculum-based educational and career planning

program

PLAN

10th grade curriculum-based educational and career planning

program

The ACT

11th and 12 grade

curriculum-based assessment for

learning outcomes

ENGAGE

Middle and high school

assessments that measure

behavioral factors of academic success

QualityCore

Research-driven solutions for

strengthening curriculum

CoreWork Diagnostics

Online service to diagnose and improve content and practice

areas

Core Practice Audit

Framework for evaluating current practices

1010

ACT Learning Insights TeamWhat We Do

LIT-designed

Professional

Learning Experienc

es

Insights fromACT Research

Insights from ACT Data

Insights from Professional

Practice

Raise Academic Standards and Increase Achievement to Ensure All Students Are

College and Career Ready (CCR)

1111

Practical and flexible

Administer online at any time

Reports available online 24/7

Identify areas of challenge for students

Create a crosswalk of ACT Engage scales to existing services

ACT EngageOverview

1212

Grounded in research Developed based on meta-analysis of predictors of

academic performance and persistence (Robbins et al., 2004)

Validated across many studies (e.g., Allen et al., 2010; Casillas et al., in press; Robbins et al., 2006, 2008)

Helps educators to consider the whole student Provides profile of relative strengths and needs

ACT EngageResearch Base

1313

ACT Engage Research Base

Students’ personal characteristics and psychosocial development influence their ability to stay in school and be successful

ACT research shows that psychosocial/behavioral development can be structured into 3 broad domains: Motivation Social Engagement Self-Regulation

ACT’s Engage academic behavior assessments measure these domains

© 2012 by ACT, Inc. All rights reserved.

14

ACT Engage

Domain Engage Grades 6-9 Engage Grades 10-12

Motivation (Getting work done)

Academic Discipline Commitment to School Optimism

Academic Discipline Commitment to College Goal Striving General Determination Study Skills Communication Skills

Social Engagement (Getting along)

Family Attitude toward Education

Family Involvement Relationships w/ School

Personnel School Safety Climate

Social Activity Social Connection

Self-Regulation (Keeping your cool)

Managing Feelings Thinking Before Acting Orderly Conduct

Academic Self-Confidence

Steadiness

1515

ACT Engage Grades 6-9

Measures students’ perceptions of themselves, their families’ commitment to education, school-related factors, and important behavioral data

Developed to predict academic success and high school graduation in middle school and early high school

Low stakes, self-report instrument, 4th-grade reading level

Items form 10 scales Online administration

1616

Administering ACT EngageGetting Started

Fall semester administration Review User’s Guide before administering

Engage: ACT Engage Grades 6–9 User’s Guide Available online at

http://www.act.org/engage/6-9_materials.html

1717

Administering ACT EngageGetting Started

Recommend administering the test in the computer lab during class

Need a teacher/staff member available to help students access the test site

Students can also take ACT Engage outside of class, 24/7

Compatible browsers: Internet Explorer (v. 8 & up), Chrome, Firefox, Safari

Notify IT to adjust spam filters prior to administration.

1818

Administering ACT EngageSet-Up Page

19

Administering ACT EngageAdministration Page

20

Unique School URL

Administering ACT EngageStudent Access

2121

ACT Engage ReportsOverview

• Academic Success and Graduation/Retention Indices in the Advisor Report allow early identification of students at risk

• Student profiles can be used to:– Identify relative strengths and needs– Match students’ needs to resources– Inform student advising and identify appropriate interventions

• Aggregate reports can help identify school-level needs and allocate district resources more effectively

• Roster reports can be used for sorting and merging Engage data with other information

• All reports available online 24/7

22

ACT Engage ReportsRetrieving Reports

23

ACT Engage ReportsGenerating Reports

24

ACT Engage ReportsGenerating Reports

25

ACT Engage ReportsViewing Reports

26

ACT Engage Student Report

Profile of scores

Interpretive feedback, sorted from strengths to needs

27

ACT Engage Student ReportInterpretive Feedback

28

ACT Engage Student Report Interpretive Feedback

2929

ACT Engage Advisor Report

Profile of scores

Success Indices(only on Advisor Report)

Interpretive feedback, sorted from strengths to needs

30

ACT Engage Advisor ReportProfile

31

Roster Report

32

School Aggregate Report

33

Using Engage ResultsStudent Crosswalk of School Resources

Scales Percentile Resources

Academic Discipline High/Medium/Low

Commitment to School High/Medium/Low

Family Attitude Toward Education

High/Medium/Low

Family Involvement High/Medium/Low

Managing Feelings High/Medium/Low

Optimism High/Medium/Low

Orderly Conduct High/Medium/Low

Relationships with School Personnel

High/Medium/Low

School Safety Climate High/Medium/Low

Thinking Before Acting High/Medium/Low

34

Using Engage ResultsCrosswalk Activity

Is this service offered at your school? (Y/N)c

GEAR UP Student Services

Engage 6-9 ScalesAcademic Discipline 

Commitment to School 

Family Attitude Toward Education 

Family Involvement 

Managing Feelings 

Optimism

Orderly Conduct

Relationships with School Personnel

School Safety Climate

Thinking Before Acting

Tutoring/Homework Assistance

Comprehensive Mentoring

Financial Aid Counseling/Advising Counseling/advising/academic planning/career counseling College visit/college student shadowing Job site visit/job shadowing Summer Programs

Educational Field Trips

Student Workshops Parent/family workshops of college prep/financial aid Parent/family counseling/advising Parent/family college visit Family events

3535

Case StudiesNorah

Student who may be at risk: Norah

3636

Case StudiesActivity: Norah

• What interventions would you select for Norah?

3737

Case StudiesIntervention Plan: Norah

3838

Case StudiesJuan

• On-track student: Juan

3939

Case StudiesActivity: Juan

• What interventions would you select for Juan?

4040

Case StudiesIntervention Plan: Juan

4141

Case StudiesMcPherson USD

McPherson USD (KS) administers Engage to all students: Grade 6 (baseline) Grade 9 (on-track for high school success)

Engage results used to target interventions to three groups of students: Low Academic Success Index Low scores in key Engage Scales (e.g. Academic Discipline) Pattern of low academic achievement & low Engage scores

Engage results used in: PLC meetings Student advisory meetings Parent meetings

42

Questions?