View
20
Download
0
Category
Preview:
Citation preview
DEPLOYING MISSION CRITICAL
LMS USING OPEN SOURCE
SOFTWARE
Leigh Jin
San Francisco State University
OMG Standards for FOSS Governance Workshop
Santa Clara, California
December 11, 2013
OUTLINE
Introduction
Research Questions
Methodology
Research Findings
Discussions and Conclusions
INTRODUCTION
Open Source Learning Management System – an
attractive alternative: as institutions of higher
education confronting budget cuts and financial
crisis
Low cost (+): no licensing fee
Flexibility (+): open source code means
customizability
Deploying and maintenance cost (-)
Hardware
People with Technical Skills
INTRODUCTION: NACUBA SURVEY
NACUBA survey (2004) finds members perceive
open source software as a viable alternative to
commercial software due to:
Open standards and interoperability with other
application systems (61%)
Freedom to modify the code (58%)
Software designed by and for the industry (58%)
Lower cost of ownership (55%)
NA
CU
BA
refe
rs to N
atio
nal A
ssocia
tion
of
Colle
ge a
nd
Un
iversity
Bu
siness O
fficers
OSS DEVELOPMENT VS. IMPLEMENTATION
OSS Development: two assumptions:
Developers are users with “itch to scratch”
OSS movement is unique because of the way software is being
developed
OSS Implementation: Fitzgerald and Kenny
(2003) study OSS implementation in an Ireland hospital leads to cost
savings of €13 million over five years
Free access to source code is not the primary motivation for
adoption
INTRODUCTION OSS IMPLEMENTATION:
BENEFITS VS. BARRIERS:
OSS Implementation
Benefits • Low licensing cost
• Open standard and interoperability, avoid vender
“lock-in”
• Freedom to modify the code
• Greater choice and control for end-users
• Innovation
Barriers • Lack of documentation and support service from the
OSS provider
• Technical skills required to implement, customize,
and maintain the software
• Steep learning curve and end user training and
support
CASE STUDY RESEARCH QUESTIONS
How OSS LMS is implemented at an University
in California
The motivation, benefits, and challenges
associated with adopting open source LMS in the
public education sector
How its impacts are perceived by the
stakeholders ?
INTRODUCTION: IMOODLE
IMPLEMENTATION FRAMEWORK
OSS LMS Community
•Frequent releases on weekly/monthly basis
•Global audience: 40079 sites from 203 countries
•186 Moodle developers (with write access) from all over the world
University Deployment
•New release per semester
•Designed specifically to suit the local needs for a University in California
•Developed and supported by Academic Technology Group within the university
A Course Using
iMoodle
•Contents are updated throughout the semester
•Features of iMoodle are selectively chosen to meet the pedagogy needs for a specific class
•Designed and managed by the instructor of the class
OSS IMPLEMENTATION: STAKEHOLDER
VIEWS
Technologists
Ideology towards Common Good
Software Development Methodology
Security & Risk Management
Adoption of Innovation
Total Cost of Ownership
Educators
Constructivist Learning Environments
Online Pedagogy
Commercial LMS Limitations
Easy to Use
Administration
Cost Saving
Strategic Benefits
User Satisfaction
Policy
METHODOLOGY
Interpretive Case Study
Participant Observation
Longitudinal Study 2008-2013
Interviews with
Administrators
IT Staffs
Faculty
Focus Group study with 40 students (college of
business)
IMOODLE IMPLEMENTATION MOTIVATION:
Limitations of Commercial LMS
10-12 years experience with Blackboard
Earliest adopters of version 6 on 2002
We had a class that went over 1000
students…Blackboard forced us to break a class into
two sections
We ended up having blackouts, we had 13 hours
outage during the finals week…
we find that they are not on our timeline… they said
we would fix it in six months, when we release that
new version. But for us, we need the answer now…
Officially move to Moodle on 2007, currently having 26,000
(out of 30,000) students uses iMoodle (customized Moodle) in
their classes.
IMOODLE INITIAL IMPLEMENTATION:
SCALABILITY AND STABILITY ISSUES
“We have some major slow time, where people would login
and it would take a minute for the page to load…”
“We found that there are a number of issues that cause this,
part of it was some of the ways that people were using the
tool, part of it was how it was configured, part of it was our
hardware…”
“but when you have 1,500 students in one class, with the
same deadline of midnight on Wednesday, and they are all
taking a quiz, we would see 16,000 quiz attempts in a 12
hours period, nobody else in the whole [Moodle] community
has seen this.”
ADMINISTRATOR’S TAKE ON OSS
“So I didn’t have anything to do with that decision. I came from a
really structured project environment for a proprietary software…”
“There wasn’t a project plan that I’m used to seeing, there wasn’t a
scope of the project…”
“What is this leap of faith into this open source technology
without this documentation?”
“That scared me, I’m still scared. And I don’t know if I would have
made the decision to go open source given the situation that I came
into.”
UPGRADING IS AN ISSUE
“The tipping point I think was 1.8… based on the manpower that
we have and the resources, we can't test this out thoroughly.
But the [manager] really wanted to upgrade to 1.8, and it was just
a nightmare”
“The automated process would not complete, we were manually
downloading the files for each course, manually takes a 1.5
courses and make it become 1.8 compatible and uploaded into 1.8
and restore it…There were thousands of courses there. And our
team stayed until 11pm.”
WHEN TO UPGRADE ?
“First of all there were no procedures in place to define what should be the
process of maintaining and upgrading these systems…”
“Everybody said, well, this is open source and when an open-source bug is
fixed we implement it and we keep going…”
“But you have to follow the natural timeline for your business
processes, which in this case is the academic process. The academic
process has a beginning of the academic year and an ending of the
academic year...”
“So every academic year we would do a major change. Minor changes
like bug fixes we would allow if it’s something critical, but we wouldn’t go
from a Moodle, let’s say, 1.84 to 1.92 or something. Only on an annual
basis, where there would be major changes.”
ISSUE OF TESTING
“They had no testing, but for ongoing work you have to decide
when you are going to implement changes, test them and then put
them into production. So you have to distinguish between
development, testing and production…”
So our second task was to instill a process for upgrades and
testing and we hired student assistants to write test case
scenarios.”
ISSUE OF VIRTUALIZATION
“You’ve got to remember, we’ve got Linux upgrades, we’ve got Apache
upgrades, PHP upgrade, and on top of that you’ve got the MySQL. So all
four components plus Moodle are changing. It's a lot of different pieces
that have to be synchronized…”
“So what we would do is to create a virtual system which consisted of the
Linux operating system, that was the platform, Apache, PHP and MySQL
of a particular version, married to a particular Moodle version with its
content database. And we would create a new virtual system for every
new semester. So we would do this on an annual basis…”
Once the concept of virtual machine is accepted, then it becomes a much
more stable environment and also a scalable environment…”
PROJECT MANAGEMENT SKILLS IS
NECESSARY
“It doesn’t matter whether it’s open-source or private or in an academic
institution, it has to be treated like any other piece of software that
goes into production…”
“And these people don’t know the ABCs of project management. That’s the
problem, they grow up as developers, from developers they became sort of
supervisors and they don’t have any skills in project management…”
“Open source gets hurt more because they don’t have the discipline
of project management…”
“I took what we taught in project management class and implemented it
step-by-step. This is what had to be done…”
THE MOODLE COMMUNITY
“The Moodle core team members, it’s a very interesting mix of younger
developers who always want to just try to implement new features
without thinking about usability, without thinking about
scalability or how easy it can be maintained…”
“Also, it is very different from Linux kernel, where most of the developers
really know the code and also they are the users of the code. The Moodle
community is different in the way that most of the users might be
teachers and students and they might not know all the technical
details…”
“So whoever will be paying the Moodle core team a lot of money and say,
‘hey, you do this,’ they would just do it. And after they do it, most likely the
newly written code will be added into the core code and got released despite
that it might not be really helpful for most of the university…”
CONTRIBUTING BACK TO MOODLE
COMMUNITY
“The way that I have kind of established my own voice is to volunteer a lot
in the community. Either help out with testing or with documentation or
with reports and things like that…”
“We have taken ownership of certain modules or blocks as they are called.
So now we maintain the particular QuickMail and Gradebook Plus for
the whole community. So when an upgrade happens, we test them to make
sure they works with the upgrade and make any changes necessary and
then release them back to the community…”
ESTABLISHING RELATIONSHIP WITH THE
MOODLE CORE TEAM
“Because of our size, right now I think we have 26,000 out of the 30,000
students use it for one or more classes, some of the things that we’ve been
experiencing that other campuses have not…”
“We’ve been in communication with the core team: here are some issues
with scalability that, when [an university] gets big, they are going to face
these things too so it’s to your advantage to kind of learn from us and make
the fixes… “
“They already recognized us as [a national leader]. They’ve had me
be the keynote speaker last year. We co-organized the Moodle conference
this year. And then people call me from around the country to ask
questions. And some from Canada. I’ve had one from Israel. Santa
Barbara. All over the place.”
“In 2008 Moodle conference, the core team actually came to our campus to
visit every one of us, and that’s because they recognized us.”
CONTRIBUTIONS TO EDUCATION
COMMUNITY
“When I consult with different campuses, I give them strategies for finding
answers: find a small group of local campuses that are either a little bit
ahead of you or at the same place [in Moodle implementation] and get their
phone numbers. So I said, I can be one of those phone numbers…”
“We have the Quick Guides project which is how-to documents for people,
to learn how to use iMoodle. And because of this consortium with other
Moodle campuses, we modified the quick guides tool to allow multiple
campuses to use the same information…”
“We worked with [some community colleges], so we gave them all of our
training materials and our Moodle course for instructors which has lots of
pedagogical information and technical how-tos… And then a month later
they said we made podcasts from all of your documents would you like
those?”
BENEFITS TO CAMPUS
Revenue Generation: “We could be hired by other campuses with zero
programmers to make a widget. We can provide help desk for a campus
that has no help desk. We can bring money in through support,
through development or even through consulting. We can start
charging for me to go to campuses or something.”
Curriculum Innovation: “we have been able to encourage many more
faculty to move beyond just storing their documents for students to
download. Now they are engaging students in different types of activities
like discussion forums or self-assessment quizzes or chats or wikis…
Administrative Use: “we have over 250 organizations and committees
using iMoodle for non-coursework. We have HR who wants to use it for all
their training sessions. I’m trying to push the defensive driver training
because I don’t want to go to that room for three hours...”
Other OSS Projects on Campus: I think the campus has gained a whole lot
more in terms of those commonalities that we have across projects than
just paying a fee to some service provider to provide us with a service.
TECHNOLOGIST: WHY WE NEED TO
CONTRIBUTE BACK
“Because the more you customize, the next time you want to upgrade, when
you’re into a new version of Moodle, you can’t just upgrade to what the new
Moodle community says is Moodle. You’ve got all these dependencies and
customizations that you have to make sure they work. Now you have to re-
customize all of these things.”
ADMINISTRATOR’S TAKE ON
CONTRIBUTION OUTSIDE OF CAMPUS
“In the early days, I participated in Moodle forum for outreach reasons, we
encountered a lot of stability problems and we really need help. So it
started for work purposes, not for personal gain. But later on, people
noticed that I gained too much visibility in Moodle community, so now we
had a policy that doesn’t allow forum participation during office hours, so
I have to use my spare time to do it instead.”
“Speaking of revenue, one of our staff members is extremely generous and
was making a lot of visits to other campuses to spread the word of Moodle.
And you can put a price on that. At $300 a day it looks like you just gave
away $4000 worth of work. And some of our needs on our own campus are
not being met. So I said, we have to stop that, and if they want you to come
then we will set up a costing structure for what we will charge internally
for consultations and what we charge externally.”
STAFF RETENTION ISSUE
“We lost seven people in seven weeks this summer. And the other
manager and I, who oversees this group, we talked about having
posttraumatic stress disorder because when anybody would ask to meet
with us privately, we were afraid that they were going to resign…”
“I mean, one person that left was making $27,000 a year when I came in.
And I got him up to $50,000 but then he still left for a $72,000 a year job
which is what his market rate is. He deserves to make $70,000.”
“I think that a lot of the people who left had a certain loyalty to [the
manager who left]. And I think that is interesting to see the wave that is
still hanging on that I'm really fighting to keep are a wave that came in
with [the new manager]. So I think there is a certain loyalty there.”
OPEN SOURCE LMS: MOTIVATIONS AND
BENEFITS
Flexibility and ability to customize our own
implementation is important”
Usability and Accessibility
Technical innovation and rich features
Pedagogical innovation and sharing
“I have seen just as many contributions to the moodle
community pedagogically derived, so people say “this
is how I am teaching math with this tool”, nobody
asks them to supply that, they just happy to share.”
Ideology and Value
Leadership role and Reputation
CHALLENGES
Technical skills and know-how
Technical support related issues
Upgrades
Scalability
Testing
Staff Retention
User Training and Education
Unrealistic expectations from users
“Concerns people have are not only cost related
issues, but the safety net, if something goes wrong,
they want to yell at somebody else, now when
something goes wrong, they yell at me…”
COST OF OWNERSHIP
Cost-: Free licensing
Cost+: Pay for hosting and other costs associated
with customization and support
Cost-: Engaging other local campus who are
interested in adopting Moodle, establishing
collaborative relationships
Cost-, Revenue+: developing expertise
surrounding Moodle support, potential consulting
and revenue generating opportunities
ACTIVELY PARTICIPATING IN MOODLE
COMMUNITY
At developer level
Contributing and maintaining the “quickmail” block
Maintaining tools like “gradebook+” in the previous
release
At system administrator level
Communicating with core teams regarding scalability
issues
Training, support and documentation level
Making contributions to forum discussions and
documentation
FROM MOODLE TO IMOODLE
Initially, upgrades following Moodle release cycle caused stability and scalability issues
Measures taken to solve the problem: Carefully evaluate features of Moodle and customize it to
meet needs of a specific campus
Establish testing environment, thoroughly test new build before releasing to the campus
Adjust the upgrade cycle of iMoodle to match the academic semester
Virtualization, instead of attempting to upgrade all courses across all semesters, isolating upgrades within each semester
More effective project management, focusing resources to stabilize and support the system
As the result, the current iMoodle implementation has been quite stable, no major outages and disruptions
FROM IMOODLE TO THE CLASSROOM –
THE FACULTY PERSPECTIVE
iMoodle is very effective and easy to set up comparing to
Blackboard
Faculty computer skills vary across the campus
Faculty don’t have time to attend training sessions
System stability is really important
Do we really need new upgrade every semester ?
Online course is a whole different beast to support:
“my department benefit a lot when I get 1300 students in my online
class, but AT people are not paid for supporting me. I think online
course should be treated differently because in this case, the
technology is the classroom”
FROM IMOODLE TO CLASSROOM:
STUDENTS PERSPECTIVE
iMoodle is easy and intuitive to learn and use
How effective iMoodle is really depends on how instructor engage and set up the course Biggest complaint: instructors do not access students work on
time
The instructor uses iMoodle actively tend to be better instructors, students tend to perform better in the class
Students do expect instructor to be the problem solver when they encounter technical problems
If a feature is available in iMoodle, students do expect instructors to use them
Feature Requests: Deadline for instructor update grades?
Automatic notification and reminding emails before homework due
Who is logged in now feature, dynamically forming work groups
POLICIES AND GUIDELINES
Service Level Agreements
Let campus know this is what we can provide now with our
limited staff and what we can do , and then this is what we ask
you to do to help us out.
Incentives for faculty to make effort in curriculum
innovation: Tenure and Promotion policy
Providing venues for faculty to learn and share best
practices of using technology effectively in the classroom
Providing effective and fair funding mechanism to support
technology innovation
Recommended