Improving the Online Evaluation Process and Response Rates

SAIR 2011

WCU• 9,352 students• Master’s Comprehensive• Mountain location• Resident and Distance

History of Evaluations at WCU• No university-wide policy on administration,

instruments, courses covered, and use of results• Scanned, paper forms administered during class time• Long wait for results• No flexibility in types of reports• Poor security– esp. comments

Implementation Timeline2002 through 2006

• Faculty Senate establish ‘teaching evaluation committee’

• Provost mandated online evaluations (PACE initiative)• Developed SAI instruments based on course types• Developed guidelines for results usage• Researched available options for administering evals

and selected ConnectEdu’s CoursEval

Faculty “own” the evaluation process!

Timeline cont.: Course Evaluation Pilot – Spring 2007

• 5 departments• 328 courses (5689 surveys)• 45% response rate• Problems: log-in and email issues

lack of experience

Timeline cont.: Establishing Procedures (2007-2008)

• Scheduling guidelines• Low-enrolled courses• Student incentives• Refined SAI instruments• Planned validity study

• Advertising/incentives• Faculty training• Summer parts-of-term• Instrument defaults• Student email policy• Login issues

Faculty Senate Implementation Committee

Lessons Learned• Lesson 1: Involve everyone with a stake in the

evaluation procedures and results; everyone with experience in processing & administering evaluations

• Lesson 2: Consider all issues before implementing

• Lesson 3: Build a strong communication system between faculty and survey administrators

IR Responsibilities• Set-up & administer each semester’s evaluations• “Help desk” for faculty and students• Learn about new tools and related software, reporting,

and response rates• Communicate to Faculty Senate any issues arising

affecting senate policies and survey administration

Overview of Current Policies• All course sections, all terms• Open period set by course length• University-wide incentives allowed• Low-enrolled courses• Crosslisted courses• Periodic response rates & student reminders• Faculty evaluation (not course content)• Results availability• Use in faculty AFE/TPR process• IR office responsible for administration• Department heads have final say in instrument used

Response Rates over time

Spr 07 Fall 07 Spr 08 Fall 08 Spr 09 Fall 09 Spr 10 Fall 10 Spr 110%

10%

20%

30%

40%

50%

60%

70%

80% Term RateSpr 07 44.7%

Fall 07 43.5%

Spr 08 32.8%

Fall 08 46.5%

Spr 09 47.3%

Fall 09 43.4%

Spr 10 49.6%

Fall 10 73.4%

Spr 11 73.4%

MarketingStrategy Did it work?Large Banners Yes

Postcards Maybe

Emails Yes

Flyers around campus Maybe

Radio announcement Maybe

Giveaways No

“Early” grade release YES!

“Early” Grade Release• Students are encouraged to complete evaluations in

order to receive their final grades “early”• A list of all students who complete evaluations is sent

to the Registrar• Emails go out to students as faculty members submit

grades• Those who do not complete evaluations get final

grades after exam week is over

Overall, what works for us• Faculty Senate owns the process• Changes are made slowly, with much deliberation• Small, centralized survey administration• SaaS – few technical issues on our end, backed up

data, no questions about data integrity

Features of CoursEval that we like• Students and Faculty – Find all associated courses

and surveys with one click• Consolidation of people without deleting anything• Active Directory login• Automated emails, open/close, results release• Response rates – real time, easy to locate and email• Easy to use administrative tools• Customizable emails and log-in site

Features we would like• More flexibility with reports• Advance notice of outages and problems• Ability to access individual student responses (for

research purposes)• Faculty member attached to course cannot be

changed• More stream-lined set-up

Contact InformationWCU’s Course Evaluation website:

http://www.wcu.edu/8356.asp“Guidelines & Procedures for Administration & Oversight of Student Assessment of Instruction”

Institutional Planning & Effectiveness: http://www.wcu.edu/12829.asp 828-227-7239Elizabeth Snyder ecaveny@email.wcu.edu 828-227-2570Alison Joseph ajoseph@email.wcu.edu 828-227-3042

ConnectEdu website:http://www.connectedu.com/