Upload
catherine-gray
View
215
Download
1
Embed Size (px)
Citation preview
Virtual School Symposium 2008
1
Pre-Conference Program
Evaluation in Online Learning
2
Pre-Conference on Evaluation in Online Learning
Presenters Talbot Bielefeldt, ISTE Martha Donaldson, Alabama
ACCESS Mickey Revenaugh,
Connections Academy Donna E. Scribner, VHS Inc. Moderator: Tom Clark, TA
Consulting Special Thanks to Brian
Lekander,OII
3
Pre-Conference on Evaluation in Online Learning
What are your interests? What’s your interest in the topic
of evaluating online learning? What's your role in online
learning? What do YOU hope to learn
about evaluating online learning today?Please write down 1-2
questions that you hope will be answered.
4
Pre-Conference on Evaluation in Online Learning
Agenda1:00 Overview & Introductions
1:10 Preparing to Evaluate
2:20 Structured Discussion
3:00 Break
3:15 Lessons Learned & Next Steps
4:10 Structured Discussion
4:50 Summing Up
5
Pre-Conference on Evaluation in Online Learning
Office of Innovation and ImprovementU.S. Department of Education
.
Evaluating Online
Learning: Challenges
and Strategies for
Success
The latest guide in the Innovations in Education series
Office of Innovation & Improvement, U. S. Department of
Education
6
Pre-Conference on Evaluation in Online LearningWhy Did OII Prepare This Guide? Continued skepticism in some
quarters about quality of online learning
Desire for greater accountability in Gov’t funding programs
Concern that innovation often outpaces what we know about educational technology
Evaluators of federally-funded projects are sometimes unfamiliar with online technologies
7
Pre-Conference on Evaluation in Online LearningGuide is Based on Case Studies Alabama ACCESS Algebra I Online (Louisiana) Appleton eSchool (Wisconsin) Arizona Virtual Academy Chicago Public Schools Virtual
High School Digital Learning Commons
(Washington) Maryland Public Television’s
Thinkport
8
Pre-Conference on Evaluation in Online Learning
Who is this Guide For? Online program administrators
who need to think strategically about evaluation – and how they will use it as their program evolves
Evaluators who are not very experienced with online learning and the challenges it presents
9
Pre-Conference on Evaluation in Online LearningWhat is the Guide’s Focus? Evaluation Challenges Commonly
Encountered in Online Evaluations
Instructive Examples of Responses to these Challenges
10
Pre-Conference on Evaluation in Online LearningEvaluation Challenges Featured Meeting the Needs of Multiple
Stakeholders Building on the Existing Base of
Knowledge Evaluating Multifaceted Online
Resources Finding Appropriate Comparison Groups Interpreting the Impact of Program
Maturity Translating Evaluation Findings into
Action
11
Pre-Conference on Evaluation in Online LearningSome Things OII Has Learned Evaluation of online learning is
inherently political Programs will always be pressed for
information/results before they would like to be
It’s best to be proactive by anticipating the needs of your stakeholders
You won’t always be able to control the spin on your results
It’s helpful to think of evaluation as an ongoing process
12
Pre-Conference on Evaluation in Online Learning
Where Can I Get the OII Guide?
To view online: http://www.ed.gov/about/pubs/intro/innovations.html
From Ed Pubs:Order online at http://www.edpubs.gov or call 1-877-433-7827 (request order number ED004344P)
13
Pre-Conference on Evaluation in Online Learning
Office of Innovation and ImprovementU.S. Department of Education
.
Quality and Effectiveness in Online Learning
Issues Brief
Research Committee
North American Council for Online
Learning
14
Pre-Conference on Evaluation in Online LearningNACOL Research Committee Quality and Effectiveness in K –
12 Online Teaching Research Based Practices Practices currently adopted by
Online course providersPractices and Policies for K-12
Online Teaching and LearningOnline Professional Development
Standards Across North AmericaFeatures of Teaching in Virtual
Schools
15
Pre-Conference on Evaluation in Online LearningNACOL Research Committee Online Teacher Support Programs:
Mentoring and Coaching Models Description of the Mentoring/Coaching
relationships in vignettes from perspectives of several virtual high schools
Alabama ACCESS Distance Learning Colorado Online Learning Florida Virtual School Idaho Digital Learning Academy Mississippi Virtual School Missouri Virtual Instructional Program Tennessee: e4TN Virtual High School Global Consortium
16
Pre-Conference on Evaluation in Online Learning
Introduction Why evaluate online learning? To demonstrate the value or
worth of your program To improve your program over
time To document participant
outcomes related to program goals
To meet stakeholder interests/accountability needs
17
Pre-Conference on Evaluation in Online Learning
Introduction What happens when evaluation is
neglected? Program set up without clear goals Focus on activities and simple outputs Desired change in participant outcomes
undefined Data essential to studying success
remains undefined, ungathered Focus on anecdotal evidence,
testimonials Stakeholder information needs
neglected Program unable to demonstrate value
or worth
18
Pre-Conference on Evaluation in Online Learning
Preparing to Evaluate How can you set up your program to
study quality/effectiveness over time? Plan for evaluation early Think about program goals and how to
show you’ve achieved them Thinking at program inception about:
What outcomes are desired?What data need to be collected?What program elements/policies
would support success?
19
Pre-Conference on Evaluation in Online Learning
Preparing to Evaluate Question posed to panelists
(program managers and evaluators):
Please think back to the start of the online learning program. What did you do to prepare to evaluate quality, effectiveness, and impact?
20
Pre-Conference on Evaluation in Online Learning
Alabama ACCESS(Alabama Connecting Classrooms, Educators,
and Students Statewide)
Evaluation
Results and IssuesMartha Donaldson ([email protected])Alabama Department of EducationTalbot Bielefeldt ([email protected])International Society for Technology in Education
Preparing to Evaluate
21
Pre-Conference on Evaluation in Online Learning
ACCESS—Focus on Vision/Mission/ Goal
Vision
The State of Alabama will provide equal access to high quality instruction to improve student achievement through distance learning opportunities.
Mission
To provide an infrastructure that delivers quality learning opportunities for all citizens
Goal
To provide additional educational offerings for all Alabama public high school students
Preparing to Evaluate
22
Pre-Conference on Evaluation in Online Learning
Look at Specific Objectives to be Met
To Provide:
Equal Access to High Quality Instruction An Infrastructure That Delivers Quality
Learning Opportunities Greater Equity for all Alabama Public High
School Students Through Cutting-Edge Technology
Wide Range of Courses Available to Relatively few Alabama Students Today (“Advanced Diploma” Courses, Advanced Placement Courses, Additional Course Offerings, Remediation and Supplemental Resources)
Preparing to Evaluate
23
Pre-Conference on Evaluation in Online LearningACCESS—Don’t Forget the Big
Picture and its Impact on the Evaluation
ACCESS Delivers:Not Just Virtual Online (Asynchronous) CoursesNot Just IVC (Synchronous) Courses
But Utilizes: A unique blended approach to teaching and
learning utilizing 21st century equipment that breaches the disconnect between the traditional and online learning environments by using different delivery models.
Impact for Evaluation:Possibility of Comparing the Effectiveness of Synchronous and Asynchronous Course Delivery
Preparing to Evaluate
24
Pre-Conference on Evaluation in Online Learning
ACCESS—Look at Program Growth/Expansion
Enrollments Sites
Summer 2004 - Spring 2005 300 0 Summer 2005 - Spring 2006 2,064 24 Summer 2006 - Spring 2007 7,289 70 Summer 2007 - Spring 2008 18,955 170
As Reported in SREB State Virtual School Reports
(Is the program growing at an acceptable rate?)
Preparing to Evaluate
25
Pre-Conference on Evaluation in Online Learning
ACCESS—Established Need for Evaluation—Quality, Effectiveness, Impact
Built Into Program Design Purpose
Provide Evidence that Goals of Initiative are Met
Gauge Satisfaction with Courses Recommend Changes in Procedures and
Resources to Improve and Strengthen Program and Increase its Positive Impact
Determine Effectiveness of Regional Support Centers
Preparing to Evaluate
26
Pre-Conference on Evaluation in Online Learning
ACCESS—Preparation for Evaluation
Development of RFP at Beginning of Initiative
Development of Evaluation Plan with ISTE Identification of:
Questions to be AnsweredData that will be NeededData that will not be AvailableEvaluation Methods to be Used
Development of Data Collection Instruments
Preparing to Evaluate
27
Pre-Conference on Evaluation in Online Learning
ACCESS—Methods
Interviews: What are the issues? Surveys: How do people feel about
the issues? “State-istics”: How has the system
grown? Outcomes: How well does the system
work to meet its goal of access to quality courses for all students?
Preparing to Evaluate
28
Pre-Conference on Evaluation in Online Learning
ACCESS—What Worked
Financial support from the state. Rapid expansion of the infrastructure. Highly rated Regional Support
Centers. Thousands of students take classes
that would be otherwise unavailable. Teaching practices change with
increased technology integration and student-centered pedagogy.
Preparing to Evaluate
29
Pre-Conference on Evaluation in Online Learning
ACCESS—Strategy-in-Progress
Seek out teachers with virtual and F2F students.
Compare outcomes for students in similar classes with the same teacher.
Preparing to Evaluate
30
Pre-Conference on Evaluation in Online Learning
ACCESS—Areas for Improvement
A percentage of students dissatisfied with various aspects of the system.
Requests for better communication within the system from students, teachers, counselors, and facilitators.
Concerns with student readiness for distance learning.
Technical issues with evaluation.
Preparing to Evaluate
31
Pre-Conference on Evaluation in Online Learning
ACCESS—Evaluation Issues
Availability of data (e.g., lack of common end-of-course tests).
Access to data (e.g., Advanced Placement records held by testing vendor).
Technological barriers (e.g., firewalls prevent submission of survey responses).
Research design challenges.
Preparing to Evaluate
32
Pre-Conference on Evaluation in Online Learning
ACCESS—Issue: What to Compare?
IVC send/receive sites? (Where sending sites are typically advantaged schools?)
Outcomes for face-to-face/virtual courses? (Where teachers and schools are different?)
Preparing to Evaluate
33
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
Evaluating for Continuous Improvement
Dr. Donna E ScribnerChief Learning [email protected]
Preparing to Evaluate
34
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
Mission To develop and deliver standards
based student centered online courses to expand students’ educational opportunities and 21st century skills and to offer professional development to teachers to expand the scope and depth of their instructional skills.
Preparing to Evaluate
35
Pre-Conference on Evaluation in Online Learning
VHS Believes that:• Student-centered online courses can
be designed and delivered to students to promote a high quality collaborative learning environment.• in which student exchange and interaction is
a valued component of the instructional process.
• Educational opportunity need not be limited by barriers of time/place/lack of qualified faculty.
• Rather, we believe that high-quality education is possible-today-for all students in all locations.
• Online education offers any school with Internet connectivity a wealth of trained, experienced faculty members qualified in numerous disciplines, for teaching a wide array of courses designed to meet the needs of all students. An innovative, standards-based curriculum delivered online offers diverse, exciting learning choices for students, and the opportunity and skills to participate in a national and global community.
Preparing to Evaluate
36
Pre-Conference on Evaluation in Online Learning
• Online teaching should augment rather than replace traditional classroom teaching. • The Virtual High School's online courses are a
proven, flexible solution for schools needing an expanded curriculum, teachers seeking new horizons, parents wanting more involvement with their children's education, and a society grappling with ways to offer opportunity to all its citizens.
• The goals of education are advanced best by putting value and service first. • When schools work together in a collaborative
network such as VHS, they become part of an abundant and generous educational community that promotes the affordable sharing of professional resources
VHS Believes that:
37
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium 1996 – Technology Innovation
Grant 5-year, $7.4 million US Department of Education
Non-profit; non-degree granting Consortium of schools
575+ member schools 30 states; 39 countries 11,000+ students
Preparing to Evaluate
38
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium Program Evaluation
Outsider reviewer Quality, Growth, Program Goals Surveys Superintendents, Principals,
VHS Teachers, VHS Students, VHS Site Coordinators
Published Annually Available to Public via Website
(www.goVHS.org) Quality Benchmark Indicators
(QBIs)
Preparing to Evaluate
39
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
QBIs Growth Indicators measured
against Growth Goals Quality Indicators – tie back to
Mission & Beliefs Quality of Courses Quality of Professional Development Quality of Services & Program
Preparing to Evaluate
40
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
# of VHS Schools
28 30
87
232183 174
207
278
345
461
544
0
100
200
300
400
500
600
97/98 98/99 99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08
Preparing to Evaluate
41
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
# of VHS Courses
30 37
94
156134
121
154
184
216241
293
0
50
100
150
200
250
300
350
97/98 98/99 99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08
Preparing to Evaluate
42
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
# of VHS Students
710 943
2516
3675 3736 4063
5069
6114
7604
911110111
0
2000
4000
6000
8000
10000
12000
97/98 98/99 99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08
Preparing to Evaluate
43
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
Course Quality Benchmark Indicators
0%
20%
40%
60%
80%
100%
00/01 01/02 02/03 03/04 04/05 05/06 06/07
AP exam pass rate
AP exam take Rate
Course completion rate
Credit recovery rate
Preparing to Evaluate
44
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
Course Completion Rate over Enrollments
0%10%20%30%40%50%60%70%80%90%
100%
00/01 01/02 02/03 03/04 04/05 05/06 06/07
0
2000
4000
6000
8000
10000
Course completion rate
VHS Students
Preparing to Evaluate
45
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
PD Quality Indicators - First-year Teachers
0%
20%
40%
60%
80%
100%
02/03 03/04 04/05 05/06 06/07 07/08
PD Graduation Rate
PD EffectivenessRate
Preparing to Evaluate
46
Pre-Conference on Evaluation in Online Learning
VHS Global Consortium
Program Services Quality Benchmark Indicators
65%
70%
75%
80%
85%
90%
95%
02/03 03/04 04/05 05/06 06/07
Membership RetentionRate
Course Seat UtilizationRate
Preparing to Evaluate
47
Pre-Conference on Evaluation in Online Learning
Connections Academy Connections Academy programs
are mostly full-time and include K-8 Unique research challenges:
Seeking data from younger children and parents; no additional “program ally” such as site facilitator
Unique research benefits: Address whole learner, gather all demographics, include state test results/NCLB data
Preparing to Evaluate
48
Pre-Conference on Evaluation in Online Learning
Connections Academy Built into Connections
Academy program: SIS data within our LMS Data analysis: Data views Log: Teacher communication, action Parent Satisfaction Survey StarTracker: Embedded feedback
on every lesson plus school as a whole
Measurable school and company goals
Preparing to Evaluate
49
Pre-Conference on Evaluation in Online LearningStructured Discussion: Topic 1 Preparing to Evaluate: How can
you set up your online learning program to study quality/effectiveness over time?
5’ —Choose group topics/join a group
25’—small group discussion 15’—report group findings
Preparing to Evaluate
50
Pre-Conference on Evaluation in Online LearningStructured Discussion: Topic 1Select a Topic, and Join a Discussion Group a. Program goals: The art and science of
setting measurable goals. b. Outcomes evaluation and logic models:
Will your virtual school achieve the outcomes (changes in knowledge, skills and status) you desire? How will you know you're getting there?
c. Demographic data: Why you must collect demographics, which ones matter, and how to do collect these reliably and unobtrusively
d. User feedback tools: Embedding them in your virtual school.
e. Data analysis: Balancing system-generated and user-generated data in analyzing virtual school performance.
Preparing to Evaluate
51
Pre-Conference on Evaluation in Online Learning
Break
See you in about 15 minutes!
52
Pre-Conference on Evaluation in Online Learning
Lessons Learned/Next Steps What are some lessons learned
about effective practices in evaluating online learning?
What can we learn from the results of an evaluation?
How can we use results to improve the program? To inform stake-holders & decision-makers?
53
Pre-Conference on Evaluation in Online Learning
Lessons Learned/Next Steps Seven evaluation challenges for
online programs (OII, 2008) Meeting needs of multiple stakeholders Building on limited base of knowledge Evaluating programs that lack uniformity Finding appropriate comparison groups Solving data collection problems Interpreting impact of program maturity Translating evaluation findings into action
54
Pre-Conference on Evaluation in Online Learning
Lessons Learned/Next Steps Some Lessons Learned (OII,
2008) – Evaluations should: Effectively inform stakeholder groups Share tools and research methods Focus on outcomes, not activities Recruit willing research populations
early Obtain data access, or plan to gather it Move from formative to summative Disseminate timely information to
internal & external decision-makers
55
Pre-Conference on Evaluation in Online Learning
Lessons Learned/Next Steps What can we learn from an
evaluation? Satisfaction measures Quality & effectiveness measures Changes in knowledge & skills via
participation (participant outcomes) How can we use results to
improve the program? To inform stakeholders & decision-makers?
56
Pre-Conference on Evaluation in Online Learning
Lessons Learned/Next Steps Question posed to panelists
(program managers and evaluators): Based on your experience using or
conducting evaluations of online learning programs, please share:
1) What are some lessons you’ve learned about effective evaluation practices?
2) How has evaluation helped you improve an online learning program or demonstrate its worth?
57
Pre-Conference on Evaluation in Online Learning
Connections Academy When Evaluation Pays Off
Early MoVIP K-5 Results: User satisfaction high, teachers make the difference – validation of model
Mississippi K-8 Pilot: Even a small, short pilot can be positively revealing if designed with evaluation in mind
Ongoing Parent Satisfaction Surveys: Overall high rates (90%+) persist, and improvements in “iffy” areas absolutely trackable
Lessons &Next Steps
58
Pre-Conference on Evaluation in Online Learning
Connections Academy Lessons Learned
No substitute for familiarity: Evaluators need to be equipped to really dig into curriculum and platform
Data transmission is an art: Foster friendships between the evaluators and the program data wonks
Positive results are no guarantee: As in Mississippi example – can’t make up for lack of support
Lessons &Next Steps
Lessons &Next Steps
59
Pre-Conference on Evaluation in Online Learning
Connections Academy Lessons Learned
Transparency takes some getting used to: Educators are not accustomed to having practice so visible
Evaluation is only half the battle: True continuous improvement takes precision and persistence
Patience is a virtue: User satisfaction and academic results may diverge in beginning, but will converge if students stay
Lessons &Next Steps
60
Pre-Conference on Evaluation in Online Learning
VHS—Lessons Learned Don’t make decisions on a snap-
shot in time… use longitudinal data and look for trends
Evaluation criteria derives from Objectives; Objectives derive from Mission
It may be interesting but does it inform?
Lessons &Next Steps
61
Pre-Conference on Evaluation in Online Learning
VHS—Lessons Learned 2Continuous Course
ImprovementStudentProfessional Development
Need for Teacher SupportProgressElluminate SessionsFaculty Advising on graduated
scale
Lessons &Next Steps
62
Pre-Conference on Evaluation in Online Learning
How has the evaluation helped to improve the ACCESS program?
Year I Findings
Course Changes Needed
Implications/Improvements
Course Revisions Made Increased Use of Voice
Tools, Addition of Improved Speaking Assignments and Examples, and Use of Headphones in Foreign Language Courses
More Detailed Alignment and Gap Analysis Process
Addition of Course Development/Revision Component
Lessons &Next Steps
63
Pre-Conference on Evaluation in Online Learning
Year I Findings
Need for Better/Increased Communication and Interaction Among Teachers, Students, Facilitators, and Support Center Staff
Implications/Improvements Introduction of Regular
Faculty Meetings via Web Conferencing
Assignment of SDE Liaison to Each Support Center Region
Expansion of Contract for Web Conferencing Capability
Additions to Training Agenda Review of Teacher Pay Issues
How has the evaluation helped to improve the ACCESS program?
Lessons &Next Steps
64
Pre-Conference on Evaluation in Online Learning
Year I Findings
Need for Additional Professional Development and Training Modules
Implications/Improvements Development of Additional
Training Modules Modification of
Professional Development Plan
Addition of SDE Staff Member to Coordinate Professional Development
Establishment of Teacher Mentoring Plan
Development of C.A.S.T. Site for Teachers
How has the evaluation helped to improve the ACCESS program?
Lessons &Next Steps
65
Pre-Conference on Evaluation in Online Learning
Year I Findings Need for Assistance
With Scheduling, Registration, and Enrollment Issues
Increased Number of Students Not Prepared/Ready for Assigned Class
Implications/Improvements
Development of Training Module for Counselors on the Registration Process
Expansion of Meetings with Counselors (Regional and State Meetings)
Decision to Develop a New Student Registration Site
Onsite Visits and Individualized Telephone Calls to Assist With Process
Further Look at Course Prerequisites
How has the evaluation helped to improve the ACCESS program?
Lessons &Next Steps
66
Pre-Conference on Evaluation in Online Learning
Year I Findings
Need for Assistance With Technical Issues
Implications/Improvements
Use of SDE and Regional Support Center Helpdesks
Additional School Visits Addition of Staff at SDE Equipment/Connectivity
Checks by SDE, Support Centers, and Alabama SuperComputer Authority
Increased Communication With School Staff
Identification of Key Areas of Concern
How has the evaluation helped to improve the ACCESS program?
Lessons &Next Steps
67
Pre-Conference on Evaluation in Online Learning
Year I Findings
Need for Increased Number of Responses on Interviews and Surveys
Implications/Improvements
Identification of Reasons for Poor Response Rate Timing Surveys Used Filters/Blocks at
School Level Content/Clarity
Issues
How has the evaluation helped to improve the ACCESS program?
Lessons &Next Steps
68
Pre-Conference on Evaluation in Online Learning
Access—Focus of Year 2 Report
Essentials for Student Success in Distance Learning Courses
Ability to Meet Stated Objectives Providing Equitable Access to High School
Courses Leveraging Existing Resources Providing Multimedia and Technology
Tools to Enhance Instruction Planning and Implementing a Successful
Statewide Distance Learning Plan
Lessons &Next Steps
69
Pre-Conference on Evaluation in Online Learning
Access—Focus of Year 2 Report (Cont.)
Program Growth and Quality of Support to Teachers by Regional Support Centers
Satisfaction Level of Stakeholders Recommendations for Revising
and Enhancing Program Outcomes
Lessons &Next Steps
70
Pre-Conference on Evaluation in Online Learning
Access—Support Centers Training
Ratings High (Mean Between 3.46 and 3.74 on Scale of 4)
Suggestions for Improvement—Inc. into Training• More Focus on Job-Specific Applications• More Time and Hands-on Practice With Appns
Results From Teacher Surveys 90% Agreed or Strongly Agreed With Positive
Statements About Support High Ratings for Training Content, Scheduling,
and Delivery Open-Ended Items All Positive (Availability,
Helpfulness)
Lessons &Next Steps
71
Pre-Conference on Evaluation in Online Learning
Access—Satisfaction Surveys • Ratings Generally High (Agree/Strongly Agree all Items)
• Concerns• Lack of Prerequisite Skills by Students• Enrollment Process Cumbersome• Need for More Science Courses• School Technology Inadequate• Need for Hands-on Instruction Specific to Counselors’ Jobs
• Implications/Results• Additional Training/Assistance for Counselors• Continuation of Development of New Student Registration System
• Updating of 2 Science Courses, Addition of 1 More
• Awarding of Additional 21st Century Site Grants
Lessons &Next Steps
72
Pre-Conference on Evaluation in Online Learning
Access—Recommendations for Future
• Increase Teacher Training, Including Additional Hands-on Time With ACCESS Technologies
• Further Develop Ways to Ensure High Interaction Between Teachers and Students (Such as Moving up of Timeline for Blended Model)
• Increase Role of Support Centers in Providing Training (More Targeted Training for Counselors and Facilitators)
• Continue Annual Audits of Support Centers• Continue emphasis on What Data is Available and
How it Can be Obtained• Ensure That New Student Registration & Data
System is Designed to Collect Needed Data and to Integrate Effectively with LMS for Data Collection and Reporting
Lessons &Next Steps
73
Pre-Conference on Evaluation in Online LearningStructured Discussion: Topic 2 Lessons Learned/Next Steps:
What are some lessons learned about effective evaluation practices, and using evaluation results effectively?
5’ —Choose group topics/join a group
25’—small group discussion 15’—report group findings
Lessons &Next Steps
74
Pre-Conference on Evaluation in Online LearningStructured Discussion: Topic 2Select a Topic, and Join a Discussion Group a. Effective evaluation practices: adjusting
evaluation methods to fit program realities. b. Changing course: Early warnings/interim
evaluations/mid-course corrections – when not to wait til the year end evaluation report.
c. Follow-up: How to build an action plan that addresses your evaluation's findings
d. Program design: Re-designing the program to support ongoing evaluation efforts.
e. Dissemination: Effectively communicating evaluation results to different stakeholders
Lessons &Next Steps
75
Pre-Conference on Evaluation in Online Learning
Summing Up
Summing up by Moderator Summary thoughts of our
panelists on evaluating online learning
76
Pre-Conference on Evaluation in Online Learning
Thank You! Thanks for your participation in
our pre-conference today!
Tom Clark (moderator)Talbot Bielefeldt
Martha DonaldsonMickey RevenaughDonna E. Scribner
Thanks to OII & NACOL