Pattern Analysis and Data Visualization of Student Data ... · Pattern Analysis and Data Visualization of Student Data for Decision Making ... EBSCO, Educational Full Text Wilson

Pattern Analysis and Data Visualization of

Student Data for Decision Making

Alex J. Bowers, Ph.D.

Associate Professor of Education Leadership

Teachers College, Columbia University

February 14, 2013

A research seminar for the Teachers College

Learning Analytics Program

Teacher Assigned Grades as Useful Data to

Inform Decision Making in Schools

• What are the most accurate predictors of who

will dropout or graduate? Do we know?

– Grade Trajectories (Growth Mixture Models)

– Grade Patterns (Cluster Analysis Heatmaps)

• What do grades represent as useful data for

decision making?

Bowers, A.J. (2013)








decision making?

Bowers, A.J. (2013)

Issues of Accuracy in Predicting High School Dropout

• Dropping out of high school in the U.S. is associated with a multitude of negative outcomes.

• However, other than a select number of demographic and background variables, we know little about the accuracy of current dropout predictors that are school-related.

• Current predictions accurately predict only about 50-60% of students who actually dropout.

• According to Gleason & Dynarski (2002), accurate prediction of who will dropout is a resource and efficiency issue:

– A large percentage of students are mis-identified as at-risk.

– A large percentage of students who are at-risk are never identified.

• Current reporting of dropout “flags” across the literature is haphazard. – Almost none report accuracy

– Many report specificity or sensitivity, but rarely both

• A dropout flag may be highly precise, in that almost all of the students with the flag dropout, but may not be accurate since the flag may identify only a small proportion of the dropouts.

Balfanz et al. (2007); Bowers (2010); Gleason & Dynarski (2002); Pallas (2003); Rumberger (2004, 2011) Bowers, A.J. (2013)

Re-analyzing Past Dropout Flags for Accuracy,

Precision, Sensitivity and Specificity

• Literature search. • Queried multiple databases:

– JSTOR, Google Scholar, EBSCO, Educational Full Text Wilson Web

• Studies were included that: – Were published since 1979 – Examined High School dropout – Examined school-wide characteristics and included all students – A focus on the student level – Reported frequencies for re-analysis

• Initially yield 6,434 overlapping studies – 301 studies were read in full – 140 provided school-wide samples and quantifiable data – 36 articles provided enough data for accuracy re-calculations – Yield 110 separate dropout flags

• Relative Operating Characteristic (ROC) – Hits versus False-Alarms

Bowers, Sprott & Taff (2013) Do we Know Who Will Drop Out? A Review of the Predictors of Dropping out of

High School: Precision, Sensitivity and Specificity. The High School Journal. 96(2), 77-100.

Bowers, A.J. (2013)

Dropout Graduate

Dropout a

True-positive

(TP)

Correct

b

False-positive

(FP)

Type I Error

a+b

Graduate c

False-negative

(FN)

Type II Error

d

True-negative

(TN)

Correct

c+d

a+c b+d a+b+c+d=N

Event

Pre

dic

tor

Precision = a/(a + b) Positive Predictive Value

True-Positive Proportion = a/(a + c) Sensitivity

True-Negative Proportion = d/(b + d) Specificity

False-Positive Proportion = b/(b + d) 1-Specificity

Event table for calculating dropout contingency proportions

Fawcett (2004); Hanley & McNeil (1982); Swets (1988); Zwieg & Campbell (1993)

“Hits”

“False Alarms”

Bowers, A.J. (2013)

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0

False-positive proportion (1-Specificity)

Perfect prediction

Better prediction

Worse prediction

Low attendance

An example of the true-positive proportion plotted against the false-positive

proportion for Balfanz et al. (2007) comparing the relative operating characteristics

(ROC) of each dropout flag.

Bowers, A.J. (2013)

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0


Relative operating characteristics (ROC) of all dropout flags reviewed, plotted

as the true-positive proportion against the false-positive proportion. Numbers

refer to dropout indicator IDs.

Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The High School Journal. 96(2), 77-100. http://muse.jhu.edu/journals/high_school_journal/v096/96.2.bowers.html

http://muse.jhu.edu/journals/high_school_journal/v096/96.2.bowers.html

0

0.2

0.4

0.6

0.8

1.0

Tru

e-p

ositiv

e p

roport

ion (

Sensitiv

ity)

0 0.2 0.4 0.6 0.8 1.0


Relative operating characteristics (ROC) of all dropout flags reviewed, plotted

as the true-positive proportion against the false-positive proportion. Numbers

refer to dropout indicator IDs.

Bowers, A.J., Sprott, R., Taff, S.A. (2013) Do we Know Who Will Drop Out? A Review of the Predictors of Dropping out of High School: Precision, Sensitivity and Specificity. The High School Journal. 96(2), 77-100. http://muse.jhu.edu/journals/high_school_journal/v096/96.2.bowers.html

http://muse.jhu.edu/journals/high_school_journal/v096/96.2.bowers.html

Growth mixture model for the simultaneous estimation of latent trajectory classes

using non-cumulative GPA from the first three semesters of high school.

GPA 9S1 GPA 9S2 GPA 10S1

Intercepts Slopes

High School

Dropout

Behavior & Structure:

Student:

Extracurricular

Retained

Negative Behavior

School:

Student-Teacher Ratio

Academic Press

Small School

Large School

Extra-Large School

Demographics:

Student:

Female

African American

Asian

Hispanic

Non-Traditional Family

SES

School:

Urban

Rural

% Students Free Lunch

Latent

Trajectory

Classes

C

Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational Research, 105(3), 176-195.doi:10.1080/00220671.2011.552075

Bowers, A.J. (2013)

http://dx.doi.org/10.1080/00220671.2011.552075

A:

B:

0

1

2

3

4

9S1 9S2 10S1Semester

No

n-C

um

ula

tive

GP

A

Mid-DecreasingLow-IncreasingMid-AchievingHigh-Achieving

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

0

1

2

3

4

0

1

2

3

4

9S1 9S2 10S1

Semester

0

Non

-cum

ula

tive

GP

A

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

Semester

0

1

2

3

4

9S1 9S2 10S1

0

1

2

3

4

9S1 9S2 10S1

Semester

Low-Increasing: 13.8%Mid-Decreasing: 10.8% Mid-Achieving: 56.5% High-Achieving: 18.9%

Dropout: 39.7% 52.1% 7.0% 1.2%

Longitudinal Non-Cumulative GPA Trajectories in the First

Three Semesters of High School

n=5,400

Mid-decreasing & Low-increasing accounted for:

24.6% of the sample

91.% of the dropouts

Bowers, A.J., Sprott, R. (2012) Examining the Multiple Trajectories Associated with Dropping Out of High School: A Growth Mixture Model Analysis. The Journal of Educational Research, 105(3), 176-195.doi:10.1080/00220671.2011.552075

Bowers, A.J. (2013)

http://dx.doi.org/10.1080/00220671.2011.552075








decision making?

Bowers, A.J. (2013)

Grades as Useful Information for

Data Driven Decision Making (3DM)

Bowers, A.J. (2013)

Grades as Useful Information for

Data Driven Decision Making (3DM)

• In the U.S. we have a dual assessment system

– Standardized test scores are reported to administrators, policymakers and

the state

– Teacher assigned grades that are reported to students and parents

– Bowers (2010,2011); Farr (2000)

• However little criterion validity exists for standardized tests and their link

to overall schooling outcomes, while grades are well known to be

predictive of high school graduation and college success.

• Can we leverage grades, as data we already collect in schools, to help school leaders predict future student performance and on-time graduation in an effort to improve educational practice by providing fine-grained data to help direct the limited resources of a school district to specific student needs?

Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18. http://pareonline.net/pdf/v15n7.pdf

Bowers, A.J. (2013)

http://pareonline.net/pdf/v15n7.pdf

Research Questions

1. To what extent do past student grade patterns predict future student grade patterns?

2. To what extent do student grade patterns predict on-time graduation or student dropout?

Sample • Two student cohorts: The entire 2006 graduating cohorts for two small

Midwestern first-ring suburban school districts, enrollment < 3000 for both districts.

• 188 students included in the sample

• Overall, as defined by the state, these two districts serve students who are about 60% economically disadvantaged, 60% European American,13% Hispanic, 6% African American, 2% Asian, 2% multi-ethnic, with 17% non-reporting

Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18. http://pareonline.net/pdf/v15n7.pdf Bowers, A.J. (2013)



Method

• Recorded all assessment records for each student in each of the two cohorts, K-12. About 650 variables per student.

• The dataset mirrors typical district assessment datasets that often go unexamined by district and school leaders

• Converted all grades to a 4 point scale

• Hierarchical Cluster Analysis to identify patterns predictive of high school dropout




Cluster Analysis

• A descriptive statistic

• Fewer assumption violation issues than with regression statistics

• Uses all of the available data

• Rarely used in education




Student 1

Student 2

Student 3

Student 4

Student 5

Student 6

Student 7

Student 8

A

B

C

D

F

K 1 2 3 4 5 6 7 8 9 10 11 12

Grade Year

Gra

de M

ark

ing

Hypothetical Hierarchical Clustering Data




K 1 2 3 4 5 6 7 8 9 10 1112

Grade Year

A B C D F

Hypothetical Hierarchical Clustering Data: Clustergram

Student 3

Student 7

Student 5

Student 4

Student 8

Student 6

Student 1

Student 2




K Elementary MS 9th 10th 11th 12th

Hierarchical Clustering of Teacher Assigned Subject-Specific Grades

Clu

ste

r T

ree

Su

bje

cts

Students

+3 -3 0 No Data Bowers, A.J. (2013)

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th

Bowers, A.J. (2013)

Hierarchical Clustering of Grades

K Elementary MS 9th 10th 11th 12th High-High

Low-Low

Low-High

High-Low

Bowers, A.J. (2013)

0

0.5

1

1.5

2

2.5

3

3.5

4

K 1 2 3 4 5 6 7 8 9S1 9S2 10S1 10S2 11S1 11S2 12S1 12S2

Grade-Level

Mean N

on-c

um

ula

tive G

PA

Mean non-cumulative GPA trends for clusters high-high,

low-low, low-high and high-low, K-12

High-High Low-Low High-Low Low-High

Bowers, A.J. (2013)

• Examine why teacher assigned grades are predictive of dropout

• Further examine different types of high school dropouts

• Replicate and extend cluster & pattern analysis – Collaborating with local districts in Texas

– Longitudinal patterns of 10,000 student cohorts

– Test additional clustering algorithms • Self Organizing Maps

• Multi-level GMM

• Survival models

– Testing interventions for 3DM

Future Work K12 Schooling Outcomes











decision making?

Bowers, A.J. (2013)

Grades and Standardized Tests

• In the U.S. we have a dual assessment system – Standardized test scores that are reported to administrators,

policymakers and the state – Teacher assigned grades that are reported to students and

parents

• However little criterion validity exists for standardized tests and their link to overall schooling outcomes, while grades are well known to be highly predictive of high school graduation and college success

• The purpose of the study is to describe the multi-

dimensional nature of grades as an assessment of both academic knowledge and an “other” dimension

Bowers, 2009; Farr, 2000

Bowers, A.J. (2013)

Hodge-podge & Kitchen Sink Grading

• Grades are historically seen as “hodge-podge” and “kitchen sink” subjective measures of student performance that include student: – Academic knowledge – Participation – Discipline – Attendance – Homework – Engagement

• Researchers encourage teachers to move to more standardized grading

Brookhart, 1991; Cizek, 2000; Cross & Frary, 1999; Kuncel, 2005; Trumbull 2000

Bowers, A.J. (2013)

What is it that Grades Assess?

We must ask, "if hodgepodge grading is so deplorable, why haven't students, parents and administrators, or the general public called for reform?" It may well be that they share a common understanding that grades often do, in fact, represent a hodgepodge of attitude, effort, conduct, growth, and achievement and that is what they expect and endorse. (Cross & Frary, 1999, p.70)

• “Conative” skills - Engagement – Willingham, Pollack & Lewis, 2002

• Common grade dimension – Klapp Lekholm & Cliffordson, 2008, 2009, Thorsen & Cliffordson, 2012

• Success at school factor (SSF) – Bowers, 2009, 2011

Bowers, A.J. (2013)

Standardized Tests

Grades

25% Shared Variance

The Difference in What Standardized Tests and Grades May Assess

Bowers, A.J. (2011) What's in a Grade? The Multidimensional Nature of What Teacher Assigned Grades Assess in High School. Educational Research & Evaluation, 17(3), 141-159. doi: 10.1080/13803611.2011.597112

Bowers, A.J. (2013)

http://dx.doi.org/10.1080/13803611.2011.597112

http://dx.doi.org/10.1080/13803611.2011.597112

Core Grades

Non-Core Grades

Standardized Tests Academic Knowledge

???

The Difference in What Standardized Tests and Grades May Assess

Bowers, A.J. (2013)

Research Questions

• To what extent are teacher assigned grades and standardized test scores related?

• To what extent is the relationship multi-dimensional across core and non-core grades versus standardized tests.

• To what extent does the relationship between test scores and grades vary across different high school course subjects?


Bowers, A.J. (2013)

http://dx.doi.org/10.1080/13803611.2011.597112

http://dx.doi.org/10.1080/13803611.2011.597112

Sample & Method

• Education Longitudinal Study 2002 (ELS:2002) Transcript Survey Component – NCES collected all grades in all subjects 9-12 – Tested students in reading and mathematics grade 10

• Standardized test equated to NAEP and PISA.

– n=5,230 students with complete data

• Teacher Assigned Grades – Extracted 638,000 individual course grades across 1,600 different course

subject titles – Letter grades scaled from 4.0 (A) to 0 (F) – Classification of Secondary School Courses (CSSC) to aggregate grades into

seven main course topics: • Core courses: mathematics, English, science, social studies • Non-core courses: foreign/Non-English Lang, physical education (PE), art

– Average grades calculated for each semester of grades 9-12, and for each core and non-core subject for high school

• Method – Multi-Dimensional Scaling (MDS) – Grades and standardized test scores z-scored – Ordinal MDS with a Euclidean distance measure – 2-dimensional MDS across time, 3-dimensional MDS across subjects

Bowers, A.J. (2013)

MDS of standardized tests and core and non-core GPA – Grades 9-12

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

12S1

12S2

12S1

12S2

9S1

9S2

11S2

11S1 10S210S1

11S2

11S1 10S210S1

9S19S2

Standardized Test Core GPA Non-Core GPA

Legend:

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

12S1

12S2

12S1

12S2

9S1

9S2

11S2

11S1 10S210S1

11S2

11S1 10S210S1

9S19S2

-3 -2 -1 0 1 2 3

Dimension 1

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

12S1

12S2

12S1

12S2

9S1

9S2

11S2

11S1 10S210S1

11S2

11S1 10S210S1

9S19S2


Legend:


Legend:

Bowers, A.J. (2011) What's in a Grade? The Multidimensional Nature of What Teacher Assigned Grades Assess in High School. Educational Research & Evaluation, 17(3), 141-159. doi: 10.1080/13803611.2011.597112 Bowers, A.J. (2013)

http://dx.doi.org/10.1080/13803611.2011.597112

http://dx.doi.org/10.1080/13803611.2011.597112

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

Dimension 1 Dimension 3

Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English

Standardized Test Core Subject GPA Non-Core Subject GPA

Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

Three-dimensional MDS of standardized tests, grades in core subjects, and grades in non-core subjects, n=5230

Bowers, A.J. (2013)

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.


Bowers, A.J. (2013)

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 2

Reading

Mathematics

ArtPE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies

-3 -2 -1 0 1 2 3

Dimension 1

0

-1

1.0

1.5

0.5

-0.5

-1.5

Dim

ensio

n 3 Reading

Mathematics

Art

PE

Foreign Languages

Mathematics

Science

English

Social Studies


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Dim

ensio

n 2


Dim

ensio

n 2


Dim

ensio

n 2

Reading

Math

PE

MathScience

Social Studies

Foreign LangArt

English


Legend:

A.

B.

C.


Bowers, A.J. (2013)

Conclusions 1. Grades appear to be a multi-dimensional assessment of student

academic achievement and an “other” grades dimension

2. The “other” grades dimension appears to be consistent across core and non-core subjects

3. The “other” grades dimension appears to conform to the “engagement” or “success at school factor” hypothesis as a component of grades.

4. Recommendation is to consider grades as a useful multi-dimensional assessment of both academic achievement and a student’s ability to negotiate the social aspects of the schooling process. – Low grades may indicate that a student needs either academic tutoring or

tutoring in how to negotiate the social and engagement aspects of school.

• Future work: – Replication using a larger dataset – Replication in multiple contexts – Addition of more grade levels and standardized tests


Bowers, A.J. (2013)

http://dx.doi.org/10.1080/13803611.2011.597112

http://dx.doi.org/10.1080/13803611.2011.597112

Summary

• Grades and grade trajectories appear to be some of the best predictors of dropping out or graduating.

• Pattern analysis through GMM or HCA combined with data visualization can help schools make data informed decisions.

• Grades may represent a multidimensional assessment of behaviors and outcomes we want assessed in school.

Bowers, A.J. (2013)

Thank you!

Graduate Student Research Team:

Jared Boyce

Youmna Dbouk

Jooyoung Lee

Ryan Sprott

Sherry Taff

Angela Urick

Bowers, A.J. (2013)

Documents

Pattern Analysis and Data Visualization of Student Data ... · Pattern Analysis and Data Visualization of Student Data for Decision Making ... EBSCO, Educational Full Text Wilson