Effectiveness of Blended and Online Programs

  • View

  • Download

Embed Size (px)

Text of Effectiveness of Blended and Online Programs

PowerPoint Presentation

National Research Center for Distance Education and Technological Advancements (DETA)Tanya JoostenCo-Director and PINational Research Center for Distance Education and Technological Advancement

Director, eLearning Research and DevelopmentUniversity of Wisconsin-Milwaukee

@tjoosten, tjoosten@uwm.edu


DETA research to measure effective of blended and online programs (Tanya) (15m preso)Activity: 20m QA-Disc with questions/promptshow our work contributes to student access and quality research tool kits we can provide to help other programs measure effectivenesscurrently available:research questionsframework of inquiryto come shared measuresmethodology guide, including data collection (instruments, data mining techniques) and analyses (statistical syntax)data repositoryeffective use cases based on researchcourse and program rubric based on researchRFP later this year - join us!

1How do we ensure all students have access to a quality higher education? 2Access through distance educationStudent are allowed to push time

Courses from k-12 and post secondary/higher ed are based on a system of time and credit hours. Yet, as we live in a time that is much more hectic, with greater demands on our time due to changes in our society (two income households, more students, if not all, are working students, more undergraduates coming with families and jobs), we see that technology can help us become more efficient and overcome some of these barriers. Mediated learning opportunities, such as CBE, Online, and Blended, offer new pathways to degree providing students an access to an education that provides them more flexibility to manage their time and their lives.

However, these alternate forms of mediated learning have been questions through the decades, which has often raised the question of quality.

3Ensuring quality

Flickr vaxzineF2F to onlineMode comparison

research indicates that there was little difference in student satisfaction (Allen et al., 2002; Castle & McGuide, 2010; Lim, Morris, & Kupritz, 2006) and learning (Allen et al., 2004; Park & Gemino, 2001).

However, as Dziuban and Picciano (2015) discuss the no significant difference phenomenon refering to Roberts (2007) where they allude to the idea that research in online learning as a kind of collective amnesia surrounds changes that happened over a more distant time frame. We tend to trust what we have seen for ourselves and thus dismiss events that occurred in the more distant past (p. 13). Some researchers in disciplines newer to online learning tend to replicate the same studies with very little new to contribute to the understanding of the phenomenon.

Moore and Kearsley (2011) mention one of the major threats to good practice as well as to good scholarship in distance education is the common failure of the newcomers to the field to understand what a depth of knowledge there is (p. xvi). More recently, some practitioners and researchers realized to better understand online learning they need to look more at process variables and build off of the previous decades of research.4F2F gold standard

Flickr xenonbpicsEnsuring quality isnt about comparing f2f and onlineensure quality is more. We know that F2F is not the gold standard either. There is good f2f and bad f2f, good and bad onlinenow, lets ensure all education is good, quality education.

The online medium brings many new questions to an instructor. It is no longer about just how do I put my lectures and exams online. Well, those probably should have never been the questions.

The transformation becomes just as much, if not more, about the pedagogy and social processes in the classroom as it does about the technology.

5Ensuring quality?

F2FOnlineWhich brings us back to the questionhow do we ensure quality. When we think about how we might communicate in our f2f and then consider how do we transform our course for the online environment, we have new questions. Or, we should.

What documentation and evidence can students provide for us to assess them?Is this rigorous?Does it provide opportunity for frequent or feedback?

What activities can we develop that utilize technology that will assist them in producing this assessment?What technologies do we use for which activities?What is the best way to delivery content? What type of media due I use and when?Text, Text and Images, Video

How do I organize this stuff online?

How do I best support my students to ensure they have a quality experience and do well in the course?

What is this peer instruction and team-based learning? Why is it so important in distance education?

UWM actually developed a document called the 10 questions for blended, then online, then MOOCs

These questions and opportunities provided when we think about our instruction and learning in online courses are then in return impacting our f2f courses.


Identify practices (instructional and institutional) that impact those outcomes


What is going on inside the black box?

Mean study on content deliveryPicciano, Shea, Swan, etc., examined social processes, such as communication or social presenceDziubin Moskal effectively communication with your students and showing you care

7Conduct rigorous, interdisciplinary, and standardized research

Flicker cc astronomyblogto identify outcomes and influences on all students, including those with underrepresented -- disabilities, minorities, socioeconomically disadvantages, adult learners.

8Desired Outcomeshttp://uwm.edu/deta/desired-outcomes/What do we want to accomplish? What is the goal of higher ed?

Desired Outcomes


All learners who wish to learn online can access learning in a wide array of programs and courses,1 particularly underrepresented, those with disabilities and minorities.2 An essential component in distance education is a comprehensive infrastructure for learning that provides all individuals with the resources they need when and where they are needed. The underlying principle is that infrastructure includes people, instructional resources, processes, learning resources, policies, broadband, hardware, and software. It brings state-of-the art technology into learning to enable, motivate, and inspire all students, regardless of background, languages, or disabilities, to achieve.4

Data can be collected by examining administrative and technical infrastructure, which provides access to all prospective and enrolled learners. Access quality metrics are used for information dissemination, learning resource delivery, and tutoring services.1 Other possibilities include data gathered from student information systems, from student perception surveys, or objective accessibility ratings of online courses and programs.

Learning effectiveness

Learning effectiveness indicates a demonstration that learning outcomes were met or exceeded standards.1 This includes areas of study with research outcomes focusing on student success in achieving learning outcomes2 and other potential indicators of achievement (success, failure, achievement gains, academic achievement, improvement).3 Moreover, learning effectiveness could also include topics of retention (of content) and retention in a course (sometimes called attrition) or program (degree completion).

Typically data are gathered through direct assessment of student learning (e.g. overall grades, exam grades, or other assessments), faculty perception surveys, faculty interviews comparing learning effectiveness in delivery modes, and student focus groups or interviews measuring learning gains.1 Additionally, requests for new and better ways to measure what matters include concurrent data collection. Here, focusing on diagnosing strengths and weakness during the course of learning provides the opportunity for more immediate improved student performance. Furthermore, these technology-based assessments provide the opportunity to allow data to drive decisions on the basis of what is best for each and every student based on their unique attributes and interactivity in class.4 Other possibilities include data gathered from student information systems or from student perception surveys.


Faculty are pleased with teaching online, citing appreciation and happiness. Students are pleased with their experiences in learning online, including interaction with instructors and peers, learning outcomes that match expectations, services, and orientation.1

Faculty and student surveys can indicate equal or growing satisfaction to traditional forms of learning. Other metrics can include repeat teaching of online courses by individual faculty and increase in percentage of faculty teaching online showing growing endorsement. Qualitative methods can include interviews, focus groups, testimonials with faculty, staff (including advisors and tutors), and/or students.1

Instructional effectiveness

Instructional effectiveness indicates the quality of education meets program, institutional, and national standards.1 The focus is on what and how we teach to match what people need to know, how they learn, where and when they will learn, and who needs to learn.4 The areas of study might include instructional improvement, program effectiveness, administrator effectiveness, curriculum evaluation, educational quality, outcomes of education programs, and instructional media.3 Additionally, instructional effectiveness is not limited to instruction provided inside the classroom, but extends itself to instructional support or supplemental instruction and guidance provided through institutional