Evaluating Distance Education

Focus on Online Course Evaluation

Fall 2008

Standards

• For K12, NACOL (http://nacol.org/ ) leads the way but SREB was the stepping stone

• For Higher Education, the Quality Matters Rubric with its peer evaluation process is prominent

Quality Matters Rubric“Quality Matters (QM) is a faculty-centered, peer review process designed to certify the quality of online courses and online components. Sponsored by MarylandOnline, Inc, Quality Matters has generated widespread interest and received national recognition for its peer-based approach to quality assurance and continuous improvement in online education. Originating from a FIPSE grant (2003), Quality Matters is now a self-supporting organization offering institutional subscriptions and a range of fee-based services including Quality Matters-managed course reviews and an array of trainings.”

http://www.qualitymatters.org/

QM Rubric StandardsQuality Matters created a set of forty specific elements, distributed across eight broad standards, by which to evaluate the design of online and hybrid courses.

1. Course Overview and Introduction 2. Learning Objectives 3. Assessment and Measurement 4. Resources and Materials 5. Learner Engagement 6. Course Technology 7. Learner Support 8. Accessibility

http://www.qualitymatters.org/Rubric.htm

QM Rubric

Handouts• Annotated version

– Supports a thorough understanding of the rubric, cool version at:http://www.cvtc.edu/pages/272.asp

• Snapshot version– Great checklist, new version, can’t officially use but

it’s available for download on the website:– http://qminstitute.org/home/Public%20Library/About

%20QM/RubricStandards2008-2010.pdf

QM Rubric

• To meet official Quality Matters review expectations a course must: Answer “yes” to all 3-point Essential Standards AND Earn 72 or more points.

• We took a SLOAN-C workshop, reviewed with QM Rubric, didn’t pass.

QM Status

Subscription handout

The current rubric, as well as the Quality Matters name and logo, cannot be used without the explicit written permission of MarylandOnline, Inc. (Courses that were reviewed and that received QM Recognition to date may continue to use the logo they received).

Additional Ways We Evaluate

1. Surveys are administered using WebCT survey tool including end of course surveys as well as various surveys related to effectiveness of program communication methods, effectiveness of program/course orientation activities, etc. (Formative and Summative)

2. The instructors engage in debriefing activities after orientations, at the end of semesters, and at the end of program cycles. (Summative)

3. The asynchronous discussion forum tool is utilized in WebCT wherein students are provided with the opportunity to provide anonymous feedback throughout the course as well as the opportunity to discuss what they would do to improve activities and instruction. (Formative and Summative)

4. The OTLGC instructors utilize the QM rubric as a checklist and to evaluate the CEL courses. (Formative and Summative)

Use of Student Final Product

Students engage in a program practicum where each learner creates/has created online materials and uses them to teach each other in small groups. Their materials and teaching are evaluated using the Quality Matters Rubric by their peers and the instructors. Based on the results, we know the next steps for course/program improvement and revision. (Summative)

Dealing with the Tech

• Know your learners– We do a tech pre-survey– Phases of engagement with technology and

collaborative activities, 5 Stages Model

• If you are going to teach online, you really should take an online course.

• eLearning Ecosystem Handout

Shall we take a peek?

• CEL 560: Fostering Online Learning Communities

• CEL 570: Designing Online Learning Environments

Our Fave Books

http://astore.amazon.com/otlgc-20

Slideshare

http://www.slideshare.net/desertjul