Quality Matters Quality Matters OverviewOverview

Deborah Adair, Ph.D. Director, Quality MattersOctober 14,

2008

“Quality Matters: Inter-Institutional Quality Assurance in

Online Learning”

Quality Matters is a not-for-profit subscription service providing tools and training for quality assurance of online courses

Colleges and universities, both public and private, throughout the country are joining our community of practitioners

www.qualitymatters.org

QM Process Provides:

Institutional toolset and process to meet quality expectations: Online course design Student learning Improved instruction Assessment and feedback loops Professional development

QM as a National Standard

270 + current subscribers(http://www.qualitymatters.org/Documents/Subscriber%20List%20for%20Publication.pdf

)

42 states represented QM has trained 4000+ faculty and

instructional design staff Recognized by

- Sloan C Excellence in Online Teaching and Learning Award

-2008 USDLA Outstanding Leadership in the field of Distance Learning

MONTANA

WYOMING

IDAHO

WASHINGTON

OREGON

NEVADA

UTAH

CALIFORNIA

ARIZONA

NORTH

DAKOTA

SOUTH DAKOTA

NEBRASKA

COLORADO

NEW MEXICO

TEXAS

OKLAHOMA

KANSAS

ARKANSAS

LOUISIANA

MISSOURI

IOWA

MINNESOTA

WISCONSIN

ILLINOISINDIANA

KENTUCKY

TENNESSEE

MISS

ALABAMA GEORGIA

FLORIDA

SOUTHCAROLINA

NORTH CAROLINA

VIRGINIA

WV

OHIO

MICHIGANNEW YORK

PENN

MARYLAND

DELAWARE

NEWJERSEY

CONNRI

MASS

MAINE

VTNH

ALASKA

HAWAII

PUERTO RICO

VIRGIN ISLANDS

Current Subscribers

Statewide Subscribers

QM Subscribers

Principles of QM

The QM toolset and process A faculty-driven, peer review process

that is…CollaborativeCollegialContinuous Centered - in academic foundation - around student learning

Courses do not have to be “perfect” but QM aims at better than just “good enough”

What Quality Matters is NOT

Not about an individual instructor (it’s about the course design)

Not about faculty evaluation (it’s about course quality)

Not a win/lose or pass/fail test (it’s a diagnostic tool to facilitate continuous improvement of online/hybrid courses)

Goal: Make online instruction as good as it can be

Better than average; more than “good enough”

An attempt to capture what’s expected in an effective online or hybrid course at about an 85% level

85 %

HistoryHistory of Quality of Quality Matters Matters

Winter 2008

Quality Matters Was Launched by MarylandOnline

Maryland consortium for distance learning received 3-year FIPSE grant in 2003

Motivation: Need to provide inter-institutional quality assurance for courses in consortium seat bank

Approach: Research-based, collaborative, peer-centered

Outcome: Sustainable quality assurance process embraced by institutions beyond MOL and Maryland

Success of the QM Grant

Early presentations generated widespread interest

MarylandOnline began to receive recognition for QM WCET Outstanding Work (WOW) Award, 2005 USDLA 21st Century Best Practice Award, 2005 Maryland Distance Learning Association (MDLA) Best

Program Award, 2005.

The Sloan Consortium online workshops introduced hundreds of faculty members and staff to QM.

Peer reviewer training spread far beyond Maryland: 700+ faculty trained to review online courses using

the rubric individuals from 158 different institutions in 28 states

QM Today

Self-supporting non-profit program subscriptions, trainings, and course reviews

Wide-spread adoption Independent subscribers (75), consortium

subscribers (67), and statewide system affiliates (136)

Program development through collaboration User community directs program

improvement

Continuous improvement of tools and services

QM Basics: QM Basics: More than the Sum of its (3) More than the Sum of its (3)

Parts Parts

Winter 2008

The Rubric

The Rubric

Eight standards: Course Overview and

Introduction Learning Objectives Assessment and

Measurement Resources and Materials Learner Interaction Course Technology Learner Support Accessibility

Key components must align.

Rubric Scoring

Points are awarded for 40 specific standards based on: the team majority, AND the pre-assigned weighting of each standard

Specific standards have a point value of 1, 2, or 3; the total points possible in a review is 85

If 2-3 Reviewers believe that a standard is: met, then the full pre-assigned points are

awarded not met, then zero points are awarded

To Meet Expectations…

A course must achieve:

“Yes” on all 17 of the 3-point “essential” standards

A minimum of 72 out of 85 points

72/85 = 85%

The Peer Review Process

Peer Course Peer Course ReviewReview

FeedbackFeedback

CourseCourse

Instructional Designers

InstitutionsFaculty Course Developers National Standards &

Research Literature

Rubric

Course Meets Course Meets Quality ExpectationsQuality Expectations

Course Course RevisionRevision

Quality Matters:Quality Matters: Peer Course Review Peer Course Review

ProcessProcess

TrainingFaculty ReviewersCourse

continues to be offered

Characteristics:

Experienced online/hybrid instructor Trained in applying the Quality Matters

Standards and certified by QM

Training updated with each new edition of the rubric

Receives a small stipend for each review

The Peer Reviewer

Consists of:

3 QM Certified peer reviewers

The chair is a Master Reviewer (having received additional training)

One reviewer must be a subject matter expert

At least one reviewer must external to the institution sponsoring the course

The Peer Review Team

What to Expect from Course Reviews

Consistency and Rigor

Professionalism and Commitment

Useful and Constructive Feedback

Positive Outcomes for Faculty

Survey of faculty whose course wasreviewed indicates that … 91% of respondents made changes in the

course as a result of the review

89% of respondents felt that the quality of course design improved as a result of the review

Sample comments:“I was too close to see what could be improved.”

“Provides a great way to get an objective view of your course.”

“It made all of my online courses better.”

“It provides a view from a more student oriented perspective.”

“Many elements that might contribute to a student withdrawing can be eliminated.”

Quality Matters Training

QM Training

QM trainings support the other program components and are integral to a quality assurance effort

QM principles and the rubric standards are at the heart of all the trainings

There are two general categories of training: Faculty Development and Implementing Quality Matters

Faculty Development Implementing Quality matters

Types of Trainings

Applying the QM Rubric Building Your Online

Course Using the QM Rubric

Improving Your Online Course Using the QM Rubric

Design that Welcomes Your Students

Measurable Learning Objectives at the Course and Module Level

Choosing and Using Media Effectively

Peer Reviewer Certification Master Reviewer Train-the-Trainer Institution Representative/

Course Review Manager

Why Quality Matters? Why Quality Matters?

Winter 2008

Multiple Uses of QM

Reported Uses of QM Tools: Guidelines for initial online course

development Quality assurance of existing courses Ongoing faculty professional

development Institutional re-accreditation packages Focus attention on distance learning

policies & steering committees

Alignment with Accrediting Best Practices

Best Practices Principles* Quality Matters PrinciplesThat education is best experienced within a community of learning where competent professionals are actively and cooperatively involved with creating, providing, and improving the instructional program;

QM is a peer review process involving faculty, instructional designers and other support staff in a cooperative effort to continuously improve online instruction.

That learning is dynamic and interactive, regardless of the setting in which it occurs;

QM treats interactivity and active learning a critical component of every online course.

That instructional programs leading to degrees having integrity are organized around substantive and coherent curricula which define expected learning outcomes;

QM treats the alignment of expected learning outcomes with the contents, activities and assessments as a critical element in every online course.

That institutions accept the obligation to address student needs related to, and to provide the resources necessary for, their academic success;

QM expects every online course to address student access to the academic, technical, and student support services essential to student success.

hat institutions are responsible for the education provided in their name;

Adoption of QM standards reflects institutional commitment to online instructional quality, wherever an institution has endorsed the rubric standards.

hat institutions undertake the assessment and improvement of their quality, giving particular emphasis to student learning;

The QM standards are based on research and best practices to enhance student learning in online environments. Adoption of the QM review process is a clear demonstration of institutional or programmatic commitment to assessment and continuous improvement.

hat institutions voluntarily subject themselves to peer review. QM is essentially a peer review process involving both internal and external peers in the evaluation of courses.

*“Best Practices for Electronically Offered Degree and Certificate Programs” adopted in 2001 by CHEA and 8 regional accreditation bodies.

Collaboration and Community in Distance Education

QM provides platforms to help users collaborate... With QM in building tools that work for Distance

Education With their own institutional users to adapt QM tools to

their specific institutional needs Across the community of QM subscribers to share best

practices, find new resources, and support collegial efforts to improve student learning outcomes

www.qminstitute.org

What’s In It For Faculty?

Improvement of online courses

External quality assurance

Review other courses and gain new ideas for own course; expand professional community

Participation useful for professional development plan and portfolio

Faculty development trainings

Winter 2008

Quality Matters Quality Matters Implementation Implementation

QM’s Role in Quality Assurance

QM looks at course designThe harnessing of technology to deliver

instruction and promote student learning

QM provides a process for peer-to-peer feedback for faculty in the continuous improvement of their course

Quality Matters is not the complete answer to quality assurance for online education, but it can be a critical component

Campus Decisions for Quality

Governance Who will lead a QM project and where will it report?

Engagement How will you gain faculty commitment? Communication, training opportunities, incentives…?

Rubric Use For course assessment, development, maintenance?

Course Reviews Mandatory or optional, official or informal?

Selection Criteria Which courses in what order?

Types of Course Reviews

QM-Managed Course Reviews QM staff assigns review team and

manages process Available with a Supplemental Package,

otherwise on an as-available basis

Official, Subscriber-Managed Course Reviews Trained institution representative

manages according to QM criteria

Independent Course Reviews Schools set up own informal process

Scalability in Implementing QM

Managing Your Reviews Delivering Your Training (APPQMR)

Implementing QM – Roles and Criteria

Preparing for Course Reviews

Course is mature

Learning outcomes (course & unit) are specified

Review team has access to all elements students do

Use of QM rubric in course development

Faculty familiar with QM rubric

Helpful Links

Current Subscriber List http://www.qualitymatters.org/Documents/Subscriber List for Publication.p

df

Courses Recognized in 2008 http://www.qualitymatters.org/Courses 2008.htm

Training Calendar http://www.qualitymatters.org/Training.htm

Fee Schedule http://qminstitute.org/home/Public%20Library/Fee%20Schedule/Quality%2

0Matters%20Fee%20Schedule%20effective%208_1_2008.pdf

QM Staff http://www.qualitymatters.org/Contact.htm

Winter 2008

Thanks to YOU…Thanks to YOU…Quality Matters! Quality Matters!

2008 Post Review Interviews: The peer reviewers fulfilled their responsibilities on the review team.

2008 Post Review Interviews: The review followed the official QM course review process:

2008 Post Review Interviews: The feedback provided by the peer reviewers in the final report was constructive and useful: