1

MEET PATThe Performance Assessment Template

Emily ThomasDirector of Planning and Institutional Research

emily.thomas@stonybrook.edu

2

QUESTION:How did a group of students perform?

• Learning Communities freshmen• Scholarship recipients• Athletes• Honors society students• Transfers

3

PerformanceAssessmentTemplate

Create a standard procedure/write a program to

• Assemble variables that characterize students and their performance

• Display those variables to assess the performance of a particular group of students

4

Two Kinds of Assessment

Assessment of how students performStandard data and outcomesOften comparative analysis vsAssessment of learningNo standard measuresOften intracranial variables

5

Pat’s Tasks

1. Identify variables that describe students and their performance

2. Locate those variables

3. Extract those variables to a reporting table

4. Design standard summary reports

5. Design a report to display student detail

6

Describing Students

• Name• Gender• Ethnicity• Citizenship• SATs: math/verbal/combined• High school• High school location• High school average• Freshman/transfer entrance• Resident/commuter• Credits transferred at entry• Math placement score grade• Writing placement score grade

• Transfer institution• Special admit: EOP, athlete, other

at risk, Honors College, WISE

Split variables• Athlete• Learning Community student• Scholarship recipient

Possible additions• Age• Income• English as 2nd language• Home county/state

7

Describing Performance

Term data

• Registered?

• Credits completed

• Term GPA

• Cumulative GPA

• Credits completed in term

• Cumulative credits

• Credits transferred

• Class standing

• Academic standing

• Dean’s list?

• Major(s)• Minor(s)

Course detail• Courses taken• Grades

Degree• Degree• Degree field

8

Pat’s Data

1. Student-level data to describe students

2. Term-level data to summarize performance

3. Course-level detail

4. Degree data

Linked by student id and term id

9

Pat’s Data SourcesSnapshotextract

Additional student characteristicse.g. transfer institution

Database/warehouse Additional academic characteristicstables e.g. special admit types

Degrees

Term characteristicsEnrolled?e.g. credits, GPA,major dean's list

Courses and grades

Course-level detail reporting table

Summary reporting table

Cohort selectionBasic demographics

Term level; use first termfor population stats

10

Pat’s Programming Lessons

• SAS and SPSS access databases like Oracle very easily.

• SQL (structured query language) is a powerful IR tool, especially for multitable data.

• Ideally, institutional researchers should be able to use warehouse tables structured to support reporting.

• The data design for a performance assessment project provides specs for warehouse tables.

• Many/most IR projects can be done with SAS or SPSS tables.

11

SAS Database Read/SQL Code

LIBNAME PS ORACLE PATH=HE80PROD SCHEMA=SYSADM USER=ETHOMAS DBPROMPT=YES ;

PROC SQL ;

CREATE TABLE PAT.SEMESTER AS

SELECT EMPLID, STRM, REGISTERED, ACAD_LEVEL_BOT, UNT_TEST_CREDIT, UNT_TRNSFR, UNT_PASSD_PRGRSS,

TOT_CUMULATIVE, CUR_GPA, CUM_GPA

FROM PS.PS_STDNT_CAR_TERM

WHERE (STRM BETWEEN '0988' AND '1048') AND ACAD_CAREER = 'UGRD' ;

DATA PAT.SEMESTER ;

SET PS.PS_STDNT_CAR_TERM ;

RUN ;

12

 

NumberAll

freshmen

PercentAll

freshmenLearning Communities

Other freshmen

Learning Communities

Other freshmen

Gender Female 1 4 5 25.0 66.7 50.0

Male 3 2 5 75.0 33.3 50.0

*TOTAL 4 6 10 100.0 100.0 100.0

Ethnicity Asian American 1 2 3 25.0 33.3 30.0

African American   1 1   16.7 10.0

Hispanic American 1 1 2 25.0 16.7 20.0

White 2 2 4 50.0 33.3 40.0

*TOTAL 4 6 10 100.0 100.0 100.0

Citizenship Citizen 3 4 7 75.0 66.7 70.0

Permanent resident 1 2 3 25.0 33.3 30.0

*TOTAL 4 6 10 100.0 100.0 100.0

SAS Table

13

SAS Table CodeODS RTF FILE = "X:\OIR\SAS Programs\PAT\Learning Communities\LC 1998

summary.rtf" STYLE=SIMPLE10 ;

PROC TABULATE MISSING ;

CLASS GROUP VALUE SPLIT ;

FORMAT SPLIT S. GROUP GG. VALUE $V. ;

KEYLABEL ALL = '*TOTAL' ;

TABLE GROUP=' '*(VALUE=' ' ALL) , (SPLIT='Number' ALL='All freshmen') * N=' '*F=COMMA11.0(SPLIT='Percent' ALL='All freshmen') * PCTN<VALUE ALL>=' '*F=COMMA10.1 / RTS=40 MISSTEXT = ' ' ;

TITLE1 'LEARNING COMMUNITY STUDENTS' ;

TITLE2 'AND OTHER FULL-TIME FRESHMAN ENTERING FALL 1998' ;

TITLE3 'CHARACTERISTICS' ;

ODS CLOSE ;

14

SPSS TableLEARNING COMMUNITY STUDENTS

AND OTHER FULL-TIME FRESHMEN ENTERING FALL 1998CHARACTERISTICS

1 25.0% 4 66.7% 5 50.0%

3 75.0% 2 33.3% 5 50.0%

4 100.0% 6 100.0% 10 100.0%

0 .0% 0 .0% 0 .0%

0 .0% 0 .0% 0 .0%

1 25.0% 2 33.3% 3 30.0%

0 .0% 1 16.7% 1 10.0%

1 25.0% 1 16.7% 2 20.0%

0 .0% 0 .0% 0 .0%

2 50.0% 2 33.3% 4 40.0%

4 100.0% 6 100.0% 10 100.0%

0 .0% 0 .0% 0 .0%

3 75.0% 4 66.7% 7 70.0%

1 25.0% 2 33.3% 3 30.0%

0 .0% 0 .0% 0 .0%

4 100.0% 6 100.0% 10 100.0%

Female

Male

Total

Gender

Unknown

International

Asian American

African American

Hispanic American

Native American

White

Total

Ethnicity

Unknown

Citizen

Permanent Resident

International

Total

Citizenship

Count Column N %

Learning Communities

Count Column N %

Other freshmen

Count Column N %

Total

15

SPSS Table CodeCTABLES

/VLABELS VARIABLES=SPLIT display=none

/vlabels variables= XSEX XETHNIC XCITCODE DISPLAY=DEFAULT

/TABLE XSEX [COUNT F40.0, COLPCT.COUNT PCT40.1 ]

+ XETHNIC [COUNT F40.0, COLPCT.COUNT PCT40.1 ]

+ XCITCODE [COUNT F40.0, COLPCT.COUNT PCT40.1 ] BY SPLIT [C]

/CATEGORIES VARIABLES=SPLIT ORDER=A KEY=VALUE EMPTY=INCLUDE TOTAL=YES POSITION=AFTER

/CATEGORIES VARIABLES=XSEX XETHNIC XCITCODE ORDER=A KEY=VALUE EMPTY=INCLUDE

TOTAL=YES POSITION=AFTER

/TITLES

TITLE= 'LEARNING COMMUNITY STUDENTS'

'AND OTHER FULL-TIME FRESHMEN ENTERING FALL 1998'

'CHARACTERISTICS'.

16

SPSS Table Wizard

17

Pat’s Reports:Student Characteristics

1. Demographics: number and percent by gender, ethnicity, citizenship

2. SATs and high school average: mean, median, minimum, maximum, number

3. Admission categories: number and percent who are in EOP, other at-risk groups, Honors College, WISE, academic scholarship recipients

18

Pat’s Reports:Performance by Term (1)

4. Term enrollment: Number and percent enrolled, graduated, percent living on campus

5. Term statistics: average semester GPA, cumulative GPA, semester credits, total credits, test credits, transfer credits

6. Class standing: Number and percent by term (freshman, sophomore, junior senior)

7. Dean’s list: number and percent

8. Good academic standing: number and percent

9. Academic standing: number in each category

19

Pat’s Reports:Performance by Term (2)

10. Majors by term: first major, number by term

11. Major in entry term: first major, number and percent in entry term

12. Second majors and minors: number by term

13. Degrees: number by major by term

20

Pat’s Reports:Details

14. Grade distribution: course or percent grade distribution for target group

15. Grades in key courses: number and percent for selected courses

16. Student-level detail: transcript data plus entry characteristics

* With filter to select students who dropped below good academic standing or left/missed a semester

21

 

NumberAll

freshmen

PercentAll

freshmenLearning Communities

Other freshmen

Learning Communities

Other freshmen

Gender Female 1 4 5 25.0 66.7 50.0

Male 3 2 5 75.0 33.3 50.0

*TOTAL 4 6 10 100.0 100.0 100.0

Ethnicity Asian American 1 2 3 25.0 33.3 30.0

African American   1 1   16.7 10.0

Hispanic American 1 1 2 25.0 16.7 20.0

White 2 2 4 50.0 33.3 40.0

*TOTAL 4 6 10 100.0 100.0 100.0

Citizenship Citizen 3 4 7 75.0 66.7 70.0

Permanent resident 1 2 3 25.0 33.3 30.0

*TOTAL 4 6 10 100.0 100.0 100.0

(1) Demographics

22

(2) SAT Scores and HS Average

Learning Communities

Mean Median Minimum MaximumStudents with data

SAT math 592 575 490 750 4

SAT verbal 533 495 400 730 4

SAT combined 1125 1075 890 1480 4

HS Average 92 93 86 96 4

Other freshmen Mean Median Minimum MaximumStudents with data

SAT math 580 595 520 620 6

SAT verbal 673 670 620 730 6

SAT combined 1253 1075 1245 1320 6

HS Average 91 91 84 95 6

23

 Learning Communities Other freshmen TOTAL

Number Percent Number Percent Number Percent

WISE

    1 16.7 1 10.0Yes

No 4 100.0 5 83.3 9 90.0

TOTAL 4 100.0 6 100.0 10 100.0

ACADEMIC SCHOLARSHIP

1 25.0 1 16.7 2 20.0Yes

No 3 75.0 5 83.3 8 80.0

TOTAL 4 100.0 6 100.0 10 100.0

AT RISK ADMISSION

    2 33.3 2 20.0Yes

No 4 100.0 4 66.7 8 80.0

TOTAL 4 100.0 6 100.0 10 100.0

(3) Admissions Categories

24

Learning Communities

Students enrolled

Percent retained

Number already graduated

Percent graduated

Percent living on campus

Fall 1998 4 100% 0 0% 100%

Spring 1999 4 100% 0 0% 100%

Summer 1999 1 25% 0 0% 0%

Fall 1999 4 100% 0 0% 75%

Spring 2000 3 75% 0 0% 50%

Summer 2000 2 50% 0 0% 0%

Fall 2000 3 75% 0 0% 50%

Spring 2001 3 75% 0 0% 50%

Summer 2001 4 100% 0 0% 0%

Fall 2001 4 100% 0 0% 50%

Spring 2002 4 100% 0 0% 50%

Summer 2002 2 50% 2 50% 0%

Fall 2002 2 75% 2 50% 25%

Spring 2003 2 50% 2 50% 0%

(4) Retention, Graduation and Residence

25

Learning Communities

Average semester

GPA

Average cumulative

GPA

Average semester

credits

Average total credits

Average test credits

Average transfer credits

Fall 1998 3.53 3.53 13.25 19.75 0.75 5.75

Spring 1999 3.12 3.29 13.75 33.50 0.00 0.00

Summer 1999 3.67 3.84 3.00 36.00 0.00 0.00

Fall 1999 2.77 3.11 9.75 44.00 0.00 0.00

Spring 2000 3.75 3.62 12.67 62.67 0.00 0.00

Summer 2000 3.84 3.61 4.50 62.00 0.00 0.00

Fall 2000 3.92 3.69 13.33 79.00 0.00 0.00

Spring 2001 2.06 3.63 8.00 88.00 0.00 1.00

(5) Term Statistics

26

(6) Class Standing

Learning Communities

Total students

Freshman Sophomore Junior Senior Graduated

Number Percent Number Percent Number Percent Number Percent Number Percent

Fall 1998 4 4 100.0                

Spring 1999 4 3 75.0 1 25.0            

Fall 1999 4 1 25.0 3 75.0            

Spring 2000 3     2 66.7 1 33.3        

Fall 2000 3         3 100.0        

Spring 2001 3         2 66.7 1 33.3    

Fall 2001 4         2 50.0 2 50.0    

Spring 2002 4         2 50.0 2 50.0    

Fall 2002 4         2 50.0 2 50.0

Spring 2003 4             2 50.0 2 50.0

27

Learning Communities

Total students

On the Dean's List

Yes No

Number Percent Number Percent

Fall 1998 4 1 25.0 3 75.0

Spring 1999 4 0 0 4 100.0

Fall 1999 4 2 50.0 2 50.0

Spring 2000 3 1 33.3 2  66.7 

Fall 2000 3 1 33.3  2 66.7 

Spring 2001 3 2 66.7 1 33.3

(7) Dean’s List

28

(8) Good Academic Standing

Learning Communities

Total students

Good academic standing

Yes No

Number Percent Number Percent

Fall 1998 4     4 100.0

Spring 1999 4     4 100.0

Fall 1999 4 3 75.0 1 25.0

Spring 2000 3 2 66.7 1  33.3

Fall 2000 3 3 100.0    

Spring 2001 3 3 100.0    

Fall 2001 4 4 100.0

Spring 2002 4 4 100.0

Fall 2002 2 2 100.0

Spring 2003 2 2 100.0

29

Learning Communities

Total students

Academic Standing

Good Warning 1 Probation

Fall 1998 4 4

Spring 1999 4 4    

Fall 1999 4 3 1  

Spring 2000 3 2   1

Fall 2000 3 3    

Spring 2001 3 3    

Fall 2001 4 4    

Spring 2002 4 4    

Fall 2002 2 2

Spring 2003 2  2    

(9) Academic Standing

30

(10) Majors by Term

Learning Communities Fall 1998 Spring 1999 Fall 1999 Spring 2000

*General/Undeclared 2 2 1  

AOI in Pre-Physician Assistant 1 1 1 1

Computer Science 1 1    

Multidisciplinary Studies        

Philosophy     1 1

Political Science     1 1

TOTAL 4 4 4 3

31

(11) Entry Major

Fall 1998 Major

Number of Students

Total Students

Percent of Students

Learning Communities

Other freshmen

Learning Communities

Other freshmen

*General/Undeclared 2 4 6 50.0 66.3

AOI Pre-Health Profs   1 1   16.7

AOI in Pre-Physician Asst 1   1 25.0  

Computer Science  1 1   16.7

Political Science  1 1 25.0  

TOTAL 4 6 10 100.0 100.0

32

(12) Second Majors and Minors

Other freshmenSpring 2000

Fall 2000

Spring 2001

Fall 2001

Spring 2002

SECOND MAJORS Women's Studies 1 1 1 1

MINORS Biology 1 1 1  1  1 

33

(13) Degrees

Learning CommunitiesSummer

2002Fall 2003 Spring 2004 TOTAL

Physician Assistant 1 1

Political Science 1 1

TOTAL 2 2

34

(14) Grade Distribution

 Fall 1998

Total students

Grade (number or percent)

A A- B B+ B- C C+ C- D D+ F I/F NC NR W

CHE131

MAT123

PHI100

PSY103

35

Fall 1998 CHE131

Number of StudentsTotal

Students

Percent of Students

Learning Communities

Other freshmen

Learning Communities

Other freshmen

A 1 1 2 50.0 50.0

B 1 1 50.0

C 1 1 50.0

TOTAL 2 2 4 100.0 100.0

(15) Grades in Selected Courses

36

Entry data ** Learning Communities

* Academic scholarship

  High school average 88.2

  High school: Ward Melville  

  Location: East Setauket  

  SAT Math Score 720

  SAT Verbal Score 740

Score: Math: Placement 4

Score: Writing Placement 4

Fall 1998 **Academic standing: Good

**Dean’s List

*Major: General/Undeclared  

  *Residence: Commuter  

  *Term GPA 3.11

  *Term credits 13.0

  *Total GPA 3.41

  *Total credits 34.0

  CHE133: GENERAL CHEMISTRY LB A

  CHE141: HONORS CHEMISTRY C

  CSE106: INTRO MODULA 3 PROG S

  MAT142: HONORS CALCULUS II NC

  PHI100: CONCPTS OF TH PERSON A

  SPN111: ELEM SPANISH I B+

(16) Detail

37

Conclusions

• Performance assessment projects have enough commonality to make it efficient to create a template.

• SPSS and SAS can create the template.

• The programming is complicated, and each project has its own complications.

• Ideally, the data source should be data warehouse tables designed to support reporting.

• A performance assessment template can contribute to the design of warehouse tables.

• Standard reports are often only the first phase of an assessment project, but assembling the data for standard reports lays the foundation for additional analyses.