Evaluation of Online and Blended

Evaluating the Effectiveness of Faculty Development Programs for Blended and Online

ProgramsAugust 12th, 2014

Distance Teaching and Learning Conference

Tanya Joosten, @tjoosten, Dylan Barth, @dylanbarth, Nicole Weber, @nwebsUniversity of Wisconsin – Milwaukeetjoosten@uwm.edu | djbarth@uwm.edu | nicolea5@uwm.edu

Overview

Introduction to UWM

(3) Breakout activities, questions to consider in planning to determine faculty development effectiveness

Sharing of UWM’s pedagogical model, faculty development, evaluation strategies, and evaluation tools as well as other institutions

Conclusions

University of Wisconsin –Milwaukee

Tanya Joosten, tjoosten@uwm.edu, @tjoosten

Dylan Barth, djbarth@uwm.edu , @dylanbarth

Nicole Weber, nicolea5@uwm.edu, @nwebs

Introductions

Tech

Enh

ance

d

Blen

ded

Onl

ine

TraditionalSelf-paced

MOOCsFlex

Faculty development programs and pedagogical consultation

Technology training and support

Evaluation and research

About us

About UWM Online

What is blended and online?

A Backwards Design Approach to Evaluation

Evaluation planning

Input

• Medium (F2F, Tech)

• Instructor demographics

• Discipline• Course level• Instructor

past experiences

• Social information

Process

• Defining good blended and online course

• Defining pedagogical model

• Designing faculty development program

Output

• Course level• Program

level• Institutional

level

F2F is not the gold standard

Focus on the process…the

pedagogy

Inputs

Define a good blended and online course

Define pedagogical

model

Design a faculty

development program

Process

What are we doing today?

Discuss: Steps to evaluating effectiveness

1.) Break into discussion groups using Adobe Connect. You will be able to use audio or text chat.

2.) Access the Google Docs to document your activities at:

DTL2014.wikispaces.com

3.) Discuss, share, and brainstorm responses to the following series of questions on the 3 steps to planning evaluation.

Don’t forget to identify the member of your group and document your group’s response in the Google Doc.

Step 1: Defining effective

1. What is a “good” blended or online course? What pedagogical model facilitates a “good” blended or online course?

Good course

Higher retention

Better grades on assessment• Quizzes and exams• Projects and papers• Overall

Satisfied students• Future enrollments

Learning

Engaging

Enrollments

Pedagogical model

Content• Text• Images• Audio• Video

Interactivity• Discussions• Groups• Feedback

Assessment• Written and oral examination• Discursive• Portfolio

Content

Interactivity

Assessment

Cognitive

AffectivePsychomotor or

Behavioral

Individual Group

Projects

Discussions

Writing

Quizzes

Step 2: Facilitating success

2. What elements and format should be considered in designing and developing a faculty development program?

What opportunities and experiences should be available to help instructors learn effective practices in design, delivery, and teaching blended and online courses?

UWM’s faculty development

• Ten questions • Designing learning modules• Online vs. F2F - Integration

• Decision rubric for content choices

• Learning objects

Content

• Progressive/summative• Before, during, and after• Self evaluation• Peer evaluation• Student evaluation

Course Evaluation

• Rubrics• CATs• Templates • Traditional formats

Assessment

• Synchronous/asynchronous• Establishing voice• Discussion forums• Small groups

Interactivity

• Managing expectations• Time management• Technology support

Helping Your Students

• Staying organized• Managing workload• Avoiding course and a half

Course Management

Course Redesign

Transitioning to online and

blended teaching

Pedagogy and more

Experience

F2F 13 hrs

Online 1F2F 23 hrs

Online 2 F2F 33 hrs

Showcase

3 hrs

Post-Program

Blended

Backwards design

Active learning

Practice-based

Participant Self-

Reflection

Progressive

Participant Evaluation

Facilitator Self-

Reflection

Summative

Participant Evaluation

Program evaluation

Step 3: Documenting success

3. How will you know when faculty are providing quality blended and online courses?

What tools or services could be provided for evaluating the effectiveness of blended and online courses?

How will quality be communicated to others?

Did it work?

CC Flickr katherine.a

Course evaluation

Course evaluation

Student Evaluation Data

Course evaluation

Course evaluation

Who is your audience?What variables will be

examined?How will the data be collected?

analyzed?How (and where) will the data

be presented?Who will be involved in the

evaluation process?When will the evaluation be

completed?

Evaluation plan

Audience

Input Process OutputStudents Interactivity Students/FacultyDemographics Communication Learning Age Engagement Satisfaction Gender Social Presence Performance Ethnicity Content Student Status Assessment

Course/Institutional Data

Full time/Part time RetentionEmployment Status Drop/Withdrawal RateZip Code Grade Course/Instructor

EvaluationCourse Discipline Course Level Instructor Mode of Delivery

Variables

Collecting

Analyzing

Dissemination

CC Flickr bengray

Human ResourcesCC Flickr Vandy CFT

Timeline

CC Flickr zamboni.andrea

ChallengesCC Flickr EverExplore

Questions

Thank you!

Tanya Joosten, tjoosten@uwm.edu, @tjoosten

Dylan Barth, djbarth@uwm.edu, @dylanbarth

Nicole Weber, nicolea5@uwm.edu, @nwebs