Analyzing and Improving Analyzing and Improving College Teaching: Here’s College Teaching: Here’s an IDEAan IDEA

Alan C. Lacy, Associate DeanCollege of Applied Science and TechnologyIllinois State University

OverviewOverviewAssessment of Teaching

◦Components of Teaching◦Sources of Evaluation

The IDEA Student Rating System◦Mission of IDEA◦Administration of IDEA◦Using the results from IDEA

ASSESSMENT OF ASSESSMENT OF TEACHINGTEACHING

How well did I teach it?How well did I teach it?What are the components of effective teaching?

Who should assess these components?

Components of TeachingComponents of TeachingContent ExpertiseCourse DesignInstructional DeliveryCourse Management

Sources of AssessmentSources of AssessmentStudentsPeersAdministratorsSelf

1 = not important 1 = not important 2 = of minor importance2 = of minor importance3 = somewhat important 3 = somewhat important 4 = very important 4 = very important CJ- Cannot judgeCJ- Cannot judge The appropriateness of an

instructors course objectives Faculty member’s contributions to

course/curriculum development Severity, laxness, and consistency

of grading standards

1 = not important 1 = not important 2 = of minor importance2 = of minor importance3 = somewhat important 3 = somewhat important 4 = very important 4 = very important CJ- Cannot judgeCJ- Cannot judge The degree to which subject

matter presented is balanced and current

Relevance and quality of assessment and testing methods

Faculty member’s disciplinary knowledge/skills

Students are not qualified to Students are not qualified to assess:assess:Faculty expertiseAppropriateness of goals,

content, and organization of course

Materials used in deliveryHow student work is evaluated,

including grading practices

Nor are students qualified to Nor are students qualified to assess:assess:Support for departmental effortsAssistance to colleaguesContributing to a positive

atmosphere

Student are qualified to Student are qualified to assess:assess:Instructional DeliveryCourse DesignInstructor AvailabilityProgress To Course Objectives

SOURCES

COMPONENTS

Students Peers Chair Self

Content Expertise

No Yes/No Yes Yes

CourseDesign

Yes Yes No Yes

Instructional Delivery

Yes No No Yes

Course Management

No No Yes Yes

Assessment MatrixAssessment Matrix

THE IDEA STUDENT THE IDEA STUDENT RATING SYSTEM RATING SYSTEM

What is IDEA?What is IDEA?Individual Development &

Educational Assessment (IDEA) Student Rating System

The IDEA CenterKansas State UniversityManhattan, Kansas http://www.idea.ksu.edu/

Mission of the IDEA CenterMission of the IDEA CenterTo serve colleges and universities

committed to improving learning, teaching, and leadership performance

IDEA Formal Student IDEA Formal Student EvaluationEvaluationDeveloped at Kansas State

UniversityBased on research on teachingHas national normsIs tailored to instructor and courseVery detailed and helpful reports

(Formative/Summative)Currently utilized at 275 institutions

◦(8 years ago – 110)

IDEA – Improvement of IDEA – Improvement of Student LearningStudent Learning

Diagnostic tool to guide improvement

Evaluative tool to assess teaching effectiveness

Improvement of Student Improvement of Student LearningLearningStudent ratings have positive

impact if….◦The instrument:

Is learning focused Provides a diagnostic

◦Emphasis on summative evaluation is appropriate 30-50% of overall evaluation of teaching 6-8 representative classes (more if n<10)

◦Faculty trust the process

Improvement of Student Improvement of Student LearningLearningCampus and college culture has

great impact.◦Is teaching excellence a top priority?◦Are student ratings given an

appropriate weight in overall evaluation of teaching?

◦Does the institution provide resources to aid those seeking to improve instruction?

◦Is credible support for improvement available?

Other Teaching ActivitiesOther Teaching ActivitiesTeaching Lab ClassesCurriculum DevelopmentMentoring StudentsSupervising Field Experiences

Student Learning Model Student Learning Model AssumptionsAssumptionsTypes of learning must reflect

instructor’s purposeEffectiveness determined by

student progress on objectives stressed by the instructor

Specific teaching behaviors influence certain types of student progress under certain circumstances

IDEA – Assessment of IDEA – Assessment of TeachingTeaching

Pilot tested at ISU in 2001-200240 Tenure/Tenure track Faculty975 studentsFully implemented now

Administration of the IDEA Administration of the IDEA SystemSystemComplete Faculty Information

Forms (FIF)Administering Student Course

EvaluationsShip Evaluations to IDEADistribute IDEA Results to

Departments

Costs for IDEACosts for IDEAStaff timeMonetary

◦$0.13 per course evaluation form◦Shipping costs◦$6 per class◦Small charges for reports

Fall 2008 AdministrationFall 2008 Administration

Assessed 400 classes in 7 departments

11,500 student response forms$3,224 total cost

◦Purchase of forms ◦Processing fees

Rationale for UseRationale for UseShows commitment to teachingProvides validated course

evaluation instrument based on research

Provides consistency across the College

Summative aspects aid personnel decisions

Formative aspects aid professional development

Using the IDEA Results Using the IDEA Results Fall 2008 College Data Compared

to the IDEA Database

December 2008

Adjusted ScoresAdjusted ScoresAccounts for factors beyond

instructor’s controlUses regression equation

◦Student work habits (#43)◦Student motivation (#39)◦Class size (enrollment, FIF)◦Student effort (multiple items)◦Course difficulty (multiple items)Nothing like this in homegrown instruments

Using the ResultsUsing the ResultsOutcomes

◦Student LearningProcess

◦Teaching Methods◦Instructional Approach◦Course and Curriculum Emphasis

Program Assessment Inputs◦Student Background◦Student Characteristics

Guiding Principles to Formal Guiding Principles to Formal Evaluation DecisionsEvaluation DecisionsComprehensive and systematicValid, summative informationFormative information for

improvementConsider teaching componentsMultiple sources of assessmentConsider scholarship on best

practicesFair and equitable

Thank you for your Thank you for your attention!attention!

Any Questions?