A Qualitative Study Using Quantitative Data

Colleen ByeKeith White

Ian Sorensen

Outcomes Assessment—Phase 1

• Common Departmental Final• Part multiple-choice/part free response• Very subjective grading

• Large database from IRI• Received periodically• Analysis in Excel• One/few people involved

Outcomes Assessment—Phase 2

• Too dependent on IRI• Developed department measure

• Submitted by each faculty member for every course every semester

• Single page• < 10 minutes to fill out

• Primary objectives:• Own our assessment• Encourage reflection

Sample Form

percent numberEnrolled 28

Pass 86% 24Retain 100% 28

Success 86%

TypeMedian

Grade numberA 10A- 2B+ 4B 1B- 3C+C 1C- 3D+D 2D- 1E 1W

UW

median number % tookMidterm 89% 28 100%

Final 85% 27 96%Chapter 84%

# students with final > 70 22% students with final > 70 81%% students endured 96%% students persisted 96%

Comments:

94%

MAT 0990-020

Outstanding class. Probably the best class I have ever taught. All those A grades are legitimate. They did their homework, performed well on exams, and were very fun to teach.

Exams

Pass/Success/Retention

Grades

Keith White Fall 2011

HomeworkOnline

0

2

4

6

8

10

12

A A- B+ B B- C+ C C- D+ D D- E W UW

Grade Frequency Statistics

0%20%40%60%80%

100%

Pass Retain Success

Course Statistics

0%20%40%60%80%

100%

Midterm Final Chapter

Exam Statistics

median

% took

Department Data

• Semester-by-semester snapshots• Pass/Success/Retention rates• Compare

• Semesters• Courses• Other

• Departmental Final• Last 3 years moved to multiple choice for

standardization

Sample Department Data

Next Steps

• Data incomplete• Longitudinal data• Sequence data• Instructor data

• Scary!!!• IRB approved• Professionally

Outcomes Assessment—Phase 3

• Created by colleague Ian Sorensen• PCP

• passed original class and continued and passed next class in first attempt

• Department average = 29%• PC-P

• only considers students that passed original class and continued onto next class and passed

• Department average = 61%

Definitions Example

• 30 enrolled in original class (e.g. Pre-Algebra)

• 24 passed original class (e.g. Pre-Algebra)• 20 continued to next class (e.g. Beginning

Algebra)• 18 passed next class (e.g. Beginning

Algebra)• 18/30 = 60% PCP• 18/20 = 90% PC-P (change in denominator)

Selection Process

• ~26 full-time faculty• ~80 adjunct faculty• Subjective

• Large n (> ~250)• Above average PCP• Above average PC-P

Department Instructors

-100 100 300 500 700 900 1100 1300 1500 17000%

10%

20%

30%

40%

50%

60%

70%

PCP (PC-P < -2σ)PCP (-2σ > PC-P > -.5σ)PCP (-.5σ > PC-P > +.5σ)PCP (+.5σ > PC-P > +2σ)PCP ( PC-P > +2σ)

Instructor StatsDept Avg 62% 67% 29% 61%

Inst ID#Teaching Events Pass Continued PCP PC-P Color

1330 1326 66% 74% 36% 65% Green1294 1126 65% 74% 36% 66% Green

11680 1089 60% 82% 38% 70% Purple6295 683 82% 71% 39% 62% Green

14337 495 61% 72% 36% 71% Purple14768 371 78% 71% 38% 64% Green

347618 371 76% 71% 38% 64% Green 3503 369 65% 73% 36% 68% Purple

18276 267 70% 62% 36% 68% Purple7272 263 60% 64% 35% 73% Purple4524 250 65% 78% 39% 69% Purple

302118 246 92% 80% 64% 81% Purple

Qualitative Study Design

• Process1. Interviews2. Identifying Themes3. Observations

• Preliminary/Ongoing

Biographical Data

• What is your educational background?

• How long have you been teaching? Developmental math?

• Did/do you teach K – 12?

• Who was most influential in your development as a math teacher?

General Philosophy

• What is your philosophy of teaching?

• What are your objectives in teaching developmental math?

Philosophical Specifics• From data collected, a good percentage of your

students have been successful in subsequent math courses. Why do you think that is the case?

• What personal strengths do you find especially helpful in your teaching?

• What do you think are the most important factors for a student to be successful working through the developmental math series?

Practical Specifics

• How do you structure a class?

• Which element/activities would you say contribute most to the success of your students?

Paul Axelrod’s Characteristics of Effective Teaching

1. Accessibility and Approachability2. Fairness3. Open-mindedness4. Subject mastery and delivery5. Enthusiasm6. Humor7. Knowledge and Inspiration Imparted

Axelrod, Paul (2008). Student Perspective on Good Teaching: What History Reveals. Academic Matters: The Journal of Higher Education, February 2008, 24 – 27.

Methods

Literature-Recommended Strategies

Flipping Classroom

Active Learning

Groups/Team Based

Employ different Learning Styles

Methods

Interview Sample Data

WHY SUCCESSFUL Build confidence More visual encourage get over Build confidence Structure of class for Makes connectionsWrote up how to pass

Encourage Stop make sure anxiety Change mind set taking responsibili Is able to spend more You tube vieos

Taught 1050, so can students understand SRI: Mr. hart madeHave positive experience More like 1050 time on more Tell them to get

get them prepared for before moving on me feel like I c/b Successful experience Don't rush problems challenging material help early on and connect next Clear methodologies successful Practice exams for review class. Cover homework ? never go on w/o Used skills learned w.langStructure class keep Many examples class undertsand Adopt useful stratgies in mind struggles

PERSONAL STRENGT empathy for struggle Very Patient photo memory-learn Never belittle#1 Patience Ability to break down anxiety Love math show teach like coatch Visual and Artsy names quickly Build trust Non threatening Aresenal of weapons to teach passionfavorite math teach easy going Encourage Relaxed person Learned tech while teach sp Enthusiasmmotivator ability to simplify Patience Observe teachers/attend Humorencourage relaxed personality Kindness conference keep learning Not rigidpatience Give Praise Caring Encourage questions

Musical abailitySTRUCTURE each unit hand out Start w/questions 30 minutes q&a start w/demo prob Q&A homework 5 - 15 min hw quest Daily quiz beginning

mini syllabus Lecture material lecture broken up work individual stand lecture Intro new material Go thru text examplCome class questions Let students gravitat w/finger stretches q&a technique Don't skip steps by concept on boadGroup work towards group that stories, you tube Walk around room stop Don't rush problems Sample problem on Encourage questionsHave them come to want to. videos at each desk Let students direct board If time at end do board/rotate Many times student discussion End turn in few quest Okay w/room being Other problems at whatever studentTeach each other/ will ask other worked as groups quite will wait desk walk around w/mentor Traditional lecture switch roles questions during Breaks things up Volunteer to come to boardEach day different review

Connections

• All had minimum 10 years teaching• All mentioned the importance of creating a safe

atmosphere where students could ask questions • Almost all emphasized need to build student

confidence• Most characterized themselves as patient• 6 out of 7 started class with homework questions• None used power point to deliver lecture

Incongruities

• Homework – 4 paper and book / 3 online homework

• 2 stopped at Bachelor Degrees (both in math) • Only 2 held Master’s Degree in Mathematics• Group Work – 3 had groups / 4 did not• Structure – 3 stated they were a traditional

lecturer• Attendance – 3 required / 4 did not

Observations

• Had teachers select time to come in and observe class

• Looked for themes identified in interview

• Found specific techniques

Knowledge is Power27tested Chapter 1-2 23tested Chapter 3 22 tested Chapter 4

Ques Missed Focus Ques Missed Focus Ques Missed Focus4 64 Evaluate 15 84 Vertical 27 54 Liquid mixture

26 57 Solve linear equation 16 56 Vertical 1 42 d=rt system27 55 Complementary angles 25 50 Perpendicular slopes 16 40 System solve25 53 Solve formula 14 47 Eqt'n of slope, point 20 36 Construct system15 49 Create & Solve equation 1 42 Equation, 2 pts 14 33 System addition24 49 Solve inequality 4 36 Calc/compare slopes 26 26 System solve19 38 Percent 13 36 Interpret intercept 24 25 Cost system21 33 Solve inequality 18 32 Vertical 8 23 System substitution14 32 Create & Solve equation 24 32 Equation, 2 pts 3 22 System addition6 28 x divided by zero 21 30 Calc/compare slopes 2 21 System addition

10 26 Solve linear equation 26 29 Find slope of 2 pts 18 20 Construct system23 23 Solve inequality 11 26 Parallel slope 25 20 System substitution9 20 Solve linear equation 22 26 Perpendicular slopes 4 18 System addition

11 20 English vs Algebra 19 24 Vertical 9 17 System solve13 19 Solve linear equation 27 23 Equation, 2 pts 15 16 Cost system5 16 Zero divided by x 10 22 Sketch a graph 23 14 Evaluate inequality

16 16 Simplify 23 22 Parallel slope 19 13 System addition22 16 Solve inequality 20 19 Rise/run 6 11 System solve12 14 Solve linear equation 7 18 Horizontal 12 11 System substitution1 12 Substitute & Solve 5 16 Vertical 17 11 System solve7 12 Clear parentheses 2 12 Horizontal 5 8 System addition

20 10 Solve inequality 17 12 Solve eqt'n for m, b 10 7 System addition3 8 Evaluate 9 9 Evaluate 21 7 System addition8 6 Solve linear equation 3 8 Horizontal 7 6 System addition

18 4 Solve linear equation 12 8 Eqt'n of slope, intrcpt 13 4 System graph2 0 English vs Algebra 8 6 Sketch a graph 11 0 System evaluate

17 0 No question 6 4 Evaluate 22 0 Joke

Knowledge is Power27tested Chapter 1-2

Ques Missed Focus4 64 Evaluate

26 57 Solve linear equation27 55 Complementary angles25 53 Solve formula15 49 Create & Solve equation24 49 Solve inequality19 38 Percent21 33 Solve inequality14 32 Create & Solve equation6 28 x divided by zero

10 26 Solve linear equation23 23 Solve inequality9 20 Solve linear equation

Knowledge is Power• Qualification verified• Gradebook

• Homework Average• Daily Quiz Average• Chapter Test Average• Final Exam Score• Score needed to pass• Attendance percentage• Grade with Test Corrections• Grade w/o Test Corrections

• Did they sign up for next class?• Did they pass next class?

Board Work Innovation

• Have groups of three to four come to the board.• One person rotates to next group for next

problem.• Group leaders come to the board and race.

• Quadratic Equation – One uses formula other completing square.

Hard at Work

Music and Math

Future Work

• Additional instructors• Course-by-course comparisons• Course delivery comparisons• Student surveys?