1

Developing with Open Source Developing with Open Source SoftwareSoftware

March 27, 2006March 27, 2006

CIS6515 Yuko YamamotoCIS6515 Yuko Yamamoto

2

What does “open source” mean?What does “open source” mean?

What is “open source”?What is “open source”?

Common characteristicsCommon characteristics1.1. They adhere to the Open Source Definition. They adhere to the Open Source Definition. 2.2. Developers are always users. Developers are always users.

• Open Source Definition (Open Source Definition (OSDOSD) was composed by ) was composed by Open Source Initiative (OSI), a non-profit Open Source Initiative (OSI), a non-profit corporation dedicated to managing and promoting corporation dedicated to managing and promoting the OSD for the good of the community.the OSD for the good of the community.

3

OSD: Three main criteriaOSD: Three main criteria

1.1. Ability to distribute the software freelyAbility to distribute the software freely

2.2. Source code’s availabilitySource code’s availability

3.3. Right to create derived works through Right to create derived works through modificationmodification

4

OSD: criteriaOSD: criteria

• Six more criteria deal with licensing issues and spell Six more criteria deal with licensing issues and spell out the requisite “no discrimination” stance, that is, out the requisite “no discrimination” stance, that is, anyone can use this software. anyone can use this software.

4.4. Integrity of The Author's Source CodeIntegrity of The Author's Source Code

5.5. No Discrimination Against Persons or GroupsNo Discrimination Against Persons or Groups

6.6. No Discrimination Against Fields of EndeavorNo Discrimination Against Fields of Endeavor

7.7. Distribution of LicenseDistribution of License

8.8. License Must Not Be Specific to a ProductLicense Must Not Be Specific to a Product

9.9. License Must Not Restrict Other SoftwareLicense Must Not Restrict Other Software

5

Variable characteristics Variable characteristics

Project starting pointsProject starting points• MotivationMotivation• CommunityCommunity• Software development supportSoftware development support• LicensingLicensing• Size Size

Project starting pointsProject starting points• Start from scratchStart from scratch• Convert closed-source software to open source Convert closed-source software to open source

softwaresoftware

6

Variable characteristics - Variable characteristics - MotivationMotivation

Individuals / CorporationsIndividuals / Corporations• To satisfy a perceived needTo satisfy a perceived need

IndividualsIndividuals• For personal satisfactionFor personal satisfaction• Strong philosophical beliefs about the resulting software’s Strong philosophical beliefs about the resulting software’s

opennessopenness• Peer recognitionPeer recognition

CorporationsCorporations• To gain market shareTo gain market share• To undermine their competitorsTo undermine their competitors• No need to build an equivalent product from scratchNo need to build an equivalent product from scratch

7

Variable characteristics - Variable characteristics - CommunityCommunity

CommunityCommunity• Active open source projects have a well-defined Active open source projects have a well-defined

community with common interests, evolving related community with common interests, evolving related products or using results.products or using results.

• Many open source projects have no clear community Many open source projects have no clear community structure or involve just one person.structure or involve just one person.

This characteristic involves two issues:This characteristic involves two issues:• Balance of centralization and decentralizationBalance of centralization and decentralization• Meritocratic cultureMeritocratic culture

8

Variable characteristics – Variable characteristics – Community (2)Community (2)

Balance of centralization and decentralizationBalance of centralization and decentralization• Some communities have a strict hierarchy differentiating Some communities have a strict hierarchy differentiating

various levels of developersvarious levels of developers

CentralizationCentralization• Others have a much looser structureOthers have a much looser structure

DecentralizationDecentralization

Meritocratic cultureMeritocratic culture• Knowledge shown through contributions increases the Knowledge shown through contributions increases the

contributor’s perceived merit, which in turn leads to power.contributor’s perceived merit, which in turn leads to power.

9

A classification of open source users and developersA classification of open source users and developers

10

Variable characteristics - Variable characteristics - Software development supportSoftware development support

Software development supportSoftware development support• ModularityModularity• Visibility of software architectureVisibility of software architecture• Documentation and testingDocumentation and testing• Accepting submissionsAccepting submissions• Tool and operational supportTool and operational support

11

Variable characteristics - Variable characteristics - Software development support(2)Software development support(2)

ModularityModularity• In most cases, well-defined interfaces and In most cases, well-defined interfaces and

modularized source code are prerequisites because modularized source code are prerequisites because open source development is usually distributed. open source development is usually distributed.

Visibility of software architectureVisibility of software architecture• An open source software system’s architecture might An open source software system’s architecture might

be available or not.be available or not.

12

Variable characteristics - Variable characteristics - Software development support(3)Software development support(3)

Documentation and testingDocumentation and testing• These two areas are often overlooked or vary widely These two areas are often overlooked or vary widely

during open source development.during open source development.• Open source tends to replace the formal testing process Open source tends to replace the formal testing process

with the “many eyeballs” approach to eliminating bugs.with the “many eyeballs” approach to eliminating bugs.

Accepting submissionsAccepting submissions• Open source projects often have processes for Open source projects often have processes for

accepting various types of submissions, while clearly accepting various types of submissions, while clearly specifying how to handle multiple concurrent specifying how to handle multiple concurrent submissions.submissions.

13

Variable characteristics (2)Variable characteristics (2)

Tool and operational supportTool and operational support• To facilitate concurrent software development, most To facilitate concurrent software development, most

open source projects implement some form of open source projects implement some form of configuration management.configuration management.

e.g.) CVS (Concurrent Versions System)e.g.) CVS (Concurrent Versions System)

• Communities communicate almost exclusively by Communities communicate almost exclusively by electronic means.electronic means.

e.g.) mailing lists, newsgroups, web sitese.g.) mailing lists, newsgroups, web sites

SizeSize• Size varies widely from project to project.Size varies widely from project to project.

14

Variable characteristics (3)Variable characteristics (3)

LicensingLicensing• Some ensure that if any of the software code is used in other Some ensure that if any of the software code is used in other

software development, all the software will come under the software development, all the software will come under the terms of that original license.terms of that original license.

• Another aspect of these licenses concerns whether they restrict Another aspect of these licenses concerns whether they restrict distribution of any of the original source code to binary form in distribution of any of the original source code to binary form in future derived software products.future derived software products.

15

Case study 1Case study 1

• Beaumont HospitalBeaumont Hospital, Ireland, Ireland

Reason for moving to Open Source Software (OSS)Reason for moving to Open Source Software (OSS)CostCost Beaumont’s IT budget has significantly decreased Beaumont’s IT budget has significantly decreased

since 2000 in the wake of Y2K preparation costs. (In since 2000 in the wake of Y2K preparation costs. (In 2003 alone, 2003 alone, €17 million)€17 million)

Initial cost savingInitial cost saving Cost savings over yearsCost savings over years To get the best possible return for the taxpayers’ To get the best possible return for the taxpayers’

moneymoney

16

Phase 1: Desktop and desktop-productivity toolsPhase 1: Desktop and desktop-productivity tools

Phase 2: Application-support solutionsPhase 2: Application-support solutions

17

Desktop applicationsDesktop applications

Star Office 5.2 desktop suiteStar Office 5.2 desktop suite Product of Sun MicrosystemsProduct of Sun Microsystems Runs on multiple operating systems including Runs on multiple operating systems including

Windows, Solaris, and LinuxWindows, Solaris, and Linux

Thin-client strategy whereby applications are Thin-client strategy whereby applications are downloaded from the network where practicaldownloaded from the network where practical

Users unpredictably lost network connectionsUsers unpredictably lost network connections

18

Desktop applications (2)Desktop applications (2)

Star Office 6Star Office 6 Installed on the desktop insteadInstalled on the desktop instead Problem of network connections solvedProblem of network connections solved Star Office’s ability to exploit its built-in XML Star Office’s ability to exploit its built-in XML

capabilitiescapabilities

let users’ structured documents incorporate let users’ structured documents incorporate processing logicprocessing logic

e.g.) After creating an online human resources form, that is then e.g.) After creating an online human resources form, that is then automatically routed to the HR department for processingautomatically routed to the HR department for processing

19

Content Management System (CMS)Content Management System (CMS)

Digital Creation’s ZopeDigital Creation’s Zope• Downloadable for freeDownloadable for free• Implementation in Beaumont Implementation in Beaumont cost cost €20,000 in terms €20,000 in terms

of supportof support from a small local software company from a small local software company• Zope application server lets the IT department Zope application server lets the IT department

automatically manage documents by using the automatically manage documents by using the metatags associated with each document typemetatags associated with each document type

implement rules about how to display information, who is implement rules about how to display information, who is authorized to see it, who can change it etc.authorized to see it, who can change it etc.

20

X-ray imagingX-ray imaging

Until relatively recently, most hospitals had Until relatively recently, most hospitals had printedprinted x-ray x-ray images.images.

Now they use Now they use digitaldigital images. images.

Beaumont will need to spend an estimated Beaumont will need to spend an estimated €250,000 to upgrade €250,000 to upgrade its network’s quality to sustain rapid data retrieval.its network’s quality to sustain rapid data retrieval.

It will also need to spend approximately €400,000 on additional It will also need to spend approximately €400,000 on additional high-resolution workstations to ensure that the radiologists can high-resolution workstations to ensure that the radiologists can make safe and consistent clinical diagnosis.make safe and consistent clinical diagnosis.

Beaumont have spent approximately €480,000 for x-ray film Beaumont have spent approximately €480,000 for x-ray film annually.annually.

21

E-mailE-mail

BeforeBefore Lotus Lotus DominoDomino 800 user license800 user license Demand to expand email coverage to all 3,000 staff Demand to expand email coverage to all 3,000 staff

arosearose

AfterAfter Open source Open source SkyrixSkyrix e-mail package e-mail package all the basic email functions + administrative all the basic email functions + administrative

functionsfunctions E-mail access to all 3,000 staffE-mail access to all 3,000 staff

22

Phase 2Phase 2

• Phase 1 largely relies on selecting and Phase 1 largely relies on selecting and implementing generic products.implementing generic products.

• Phase 2 has focused on application-Phase 2 has focused on application-support solutions.support solutions.

23

Vista hospital systemVista hospital system

• Open source VISTA (Veterans Health Information Open source VISTA (Veterans Health Information Systems and Technology Architecture)Systems and Technology Architecture)

• An overall hospital information systemAn overall hospital information system• Developed by the Veterans Administration of the US Developed by the Veterans Administration of the US

Department of DefenseDepartment of Defense• It has been thoroughly field-tested over 25 years in It has been thoroughly field-tested over 25 years in

the U.S. the U.S.

24

Compiere financial systemCompiere financial system

• A fully functional open source financial management A fully functional open source financial management systemsystem

• The developers made it available as open source The developers made it available as open source because they recognized that the marketing because they recognized that the marketing investment necessary to go head-to-head against investment necessary to go head-to-head against the more established financial solutions was so the more established financial solutions was so significant.significant.

25

Payroll systemPayroll system

• Beaumont developed a staff rostering system.Beaumont developed a staff rostering system.

• This area is characterized by a great variation in rules This area is characterized by a great variation in rules and work practices.and work practices.

• In addition, it is necessary to ensure that the requisite In addition, it is necessary to ensure that the requisite skills, from a medical nursing point of view, are skills, from a medical nursing point of view, are available on each shift.available on each shift.

26

Payroll system (2)Payroll system (2)

• The system incorporates rules-based logic – using The system incorporates rules-based logic – using the Extensible Business Rules Language (XBRL) to the Extensible Business Rules Language (XBRL) to express it as a set of business rules.express it as a set of business rules.

• Beaumont intends to incorporate a payroll capability Beaumont intends to incorporate a payroll capability in the rostering system and to develop a full payroll in the rostering system and to develop a full payroll system in XBRL, thus saving the system in XBRL, thus saving the €95,000 annual fee €95,000 annual fee being paid to a vendor for this service.being paid to a vendor for this service.

27

Cost comparisonCost comparison

Initial saving: €4.7 mil. over 5 yrs: €8.2 mil.Initial saving: €4.7 mil. over 5 yrs: €8.2 mil.

Initial saving: €6.5 mil. over 5 yrs: €12 mil.Initial saving: €6.5 mil. over 5 yrs: €12 mil.

28

Lessons learnedLessons learned

• Beaumont perceived a risk in deploying OSS-based Beaumont perceived a risk in deploying OSS-based solutions:solutions:

Reliance on a standard maintenance contract is Reliance on a standard maintenance contract is not an option, and bulletin boards might be the not an option, and bulletin boards might be the main source of support.main source of support.

Beaumont’s users have become involved in Beaumont’s users have become involved in

identifying and acquiring OSS solutions.identifying and acquiring OSS solutions. The traditional line between internal IT staff and The traditional line between internal IT staff and

users became blurred.users became blurred.

29

Lessons learned (2)Lessons learned (2)

• Because OSS is more or less free, there is often the Because OSS is more or less free, there is often the misperception that service and support should misperception that service and support should also be correspondingly pricedalso be correspondingly priced..

• Beaumont spent Beaumont spent €20,000 for support of Content €20,000 for support of Content Management System.Management System.

30

Lessons learned (3)Lessons learned (3)

““Free rider” issueFree rider” issue• Whereby Individuals or organizations merely receive Whereby Individuals or organizations merely receive

all the benefits of the OSS development work of all the benefits of the OSS development work of others and never contribute anything themselvesothers and never contribute anything themselves

• Beaumont has made several applications they Beaumont has made several applications they developed (a staff rostering system, a tissue-developed (a staff rostering system, a tissue-matching system, and a casualty system) available matching system, and a casualty system) available as OSS.as OSS.

31

Lessons learned (4)Lessons learned (4)

• There was a resistance from staff who feared being There was a resistance from staff who feared being deskilled by losing experience with popular deskilled by losing experience with popular commercial software packages.commercial software packages.

• Some users – who either already had current Some users – who either already had current alternative products or the money to purchase them – alternative products or the money to purchase them – opted not to install Star Office opted not to install Star Office

• Approximately 8% of usersApproximately 8% of users

32

Case study 2Case study 2

• Macadamian TechnologiesMacadamian Technologies, a software , a software development firmdevelopment firm

• A development team at Macadamian Technologies A development team at Macadamian Technologies was asked to become a major contributor to the was asked to become a major contributor to the WineWine project.project.

• Wine Wine is an open source implementation of the is an open source implementation of the Windows API, a compatibility layer that lets native Windows API, a compatibility layer that lets native Windows programs run on X-Windows and Unix.Windows programs run on X-Windows and Unix.

33

What the Macadamian What the Macadamian Technologies team expectedTechnologies team expected

Chaos in the organization, Chaos in the organization, development, and code! development, and code!

The anticipated chaos just did not exist.The anticipated chaos just did not exist.• There was a team organization.There was a team organization.

34

Team organizationTeam organization

• DevelopersDevelopers• ReviewersReviewers• The committerThe committer

DevelopersDevelopers• Unlike commercial software teams, Wine developers Unlike commercial software teams, Wine developers

choose their own assignments.choose their own assignments.• Most often, they implement features from the Wine Most often, they implement features from the Wine

To Do list (To Do list (http://wiki.winehq.org/TodoListhttp://wiki.winehq.org/TodoList))

35

Team organization (2)Team organization (2)

ReviewersReviewers Any developer on Wine can be a reviewer.Any developer on Wine can be a reviewer. When a developer submits code to the mailing list, When a developer submits code to the mailing list,

anyone can critique it, finding errors and making anyone can critique it, finding errors and making suggestions.suggestions.

The committerThe committer• The top of the leaders, a person responsible for The top of the leaders, a person responsible for

committing changes to the source tree.committing changes to the source tree.

36

Submitting codeSubmitting code

A developer submits code to the Wine mailing list.A developer submits code to the Wine mailing list.

Anyone on the mailing list can review the code and Anyone on the mailing list can review the code and submit comments through the mailing list.submit comments through the mailing list.

The committer makes the final decision to accept the The committer makes the final decision to accept the code or request changes.code or request changes.

37

Concurrent Versions SystemConcurrent Versions System

CVS (Concurrent Versions System)CVS (Concurrent Versions System)

• CVS implements a version control system: it keeps CVS implements a version control system: it keeps track of all work and all changes in a set of files. track of all work and all changes in a set of files.

• CVS utilizes a client-server architecture: a server CVS utilizes a client-server architecture: a server stores the current version(s) of the project and its stores the current version(s) of the project and its history, and clients connect to the server in order to history, and clients connect to the server in order to check-out a complete copy of the project, work on check-out a complete copy of the project, work on

this copy and then later check-in their changes.this copy and then later check-in their changes.

38

What happened when the Macadamian What happened when the Macadamian team submitted their code?team submitted their code?

• When the Macadamian Technologies team submitted When the Macadamian Technologies team submitted their first patches, they were sent back.their first patches, they were sent back.

• It was not just the committer who had things to say It was not just the committer who had things to say about their code.about their code.

• For Wine developers, code review happens For Wine developers, code review happens automatically.automatically.

• No one wants to see bugs into the source tree.No one wants to see bugs into the source tree.

39

Lessons learnedLessons learned

• Bugs were caught earlierBugs were caught earlier in the cycle, before they in the cycle, before they were introduced into the source tree.were introduced into the source tree.

• Through the mailing list, developers Through the mailing list, developers quickly learned quickly learned what their common errors were and how to avoid what their common errors were and how to avoid themthem..

• When developers took pride in creating patches that When developers took pride in creating patches that were committed on first pass, they were committed on first pass, they challenged challenged themselves to produce their best workthemselves to produce their best work..

40

Lessons learned (2)Lessons learned (2)

• Junior developers Junior developers trained quicklytrained quickly, having access to their more , having access to their more experienced peers. experienced peers.

• The The training effort was dividedtraining effort was divided, so one person did not lose a , so one person did not lose a significant portion of development time to train others.significant portion of development time to train others.

• Despite the team’s worldwide distribution, the project had a Despite the team’s worldwide distribution, the project had a high level of communicationhigh level of communication through the mailing list. through the mailing list.

• Even though the project was not even close to finished, Even though the project was not even close to finished, the the code was constantly ready for releasecode was constantly ready for release..

(The committer adds code only when it is in a working state, not (The committer adds code only when it is in a working state, not before.)before.)

41

Application to non-open source Application to non-open source projectsprojects

• The Macadamian Technologies team decided to The Macadamian Technologies team decided to apply the procedure to one of their apply the procedure to one of their non-open sourcenon-open source projects.projects.

Improvements of the procedureImprovements of the procedure In their adapted method, the submitting developer In their adapted method, the submitting developer

assigns responsibility for review to a team member.assigns responsibility for review to a team member. When a reviewer reports a bug, he/she indicates the When a reviewer reports a bug, he/she indicates the

bug’s exact location and detailed description of the bug’s exact location and detailed description of the problem. problem.

42

ResultsResults

Benefits the Macadamian Technologies team had Benefits the Macadamian Technologies team had encountered on Wine crossed over to commercial encountered on Wine crossed over to commercial developmentdevelopment

• Time savingsTime savings The single-committer method cuts the time spent The single-committer method cuts the time spent

reviewing code significantly.reviewing code significantly.

• Consistent reviewConsistent review The single–committer method builds code review into The single–committer method builds code review into

daily work as a consistent part of the development daily work as a consistent part of the development process.process.

43

Results (2)Results (2)

• Sharpened skillsSharpened skills Reviewing code often, and in small pieces, keeps Reviewing code often, and in small pieces, keeps

reviewing skills sharp.reviewing skills sharp.

• Errors kept out of code baseErrors kept out of code base The only thing better than getting errors out of your The only thing better than getting errors out of your

source code is not putting them there in the first source code is not putting them there in the first place.place.

• Release-ready codeRelease-ready code The committer adds code only when it is in a working The committer adds code only when it is in a working

state.state.

44

Results (3)Results (3)

• Teaching toolTeaching tool Developers discover common mistakes and stop Developers discover common mistakes and stop

making them.making them. Having senior developers review new developers’ Having senior developers review new developers’

work gets juniors up to speed quickly.work gets juniors up to speed quickly.

• Iterative natureIterative nature With the single-committer model, a developer gets With the single-committer model, a developer gets

the comments and makes changes, then resubmits the comments and makes changes, then resubmits the code. The reviewer ensures that the developer the code. The reviewer ensures that the developer has made the changes without adding any errors has made the changes without adding any errors during the fixes.during the fixes.

45

ConclusionConclusion

• Although thousands of projects are classified as open Although thousands of projects are classified as open source, they all share only two main characteristics: source, they all share only two main characteristics: adherence to the Open Source Definition; developers adherence to the Open Source Definition; developers are always users.are always users.

Some of the good procedures of open source Some of the good procedures of open source projects can be applied to commercial projects. We projects can be applied to commercial projects. We should try to adopt them without being constrained by should try to adopt them without being constrained by

convention.convention.

46

ReferencesReferences

[1] C. Gacek and B. Arief, “The Many Meanings of Open Source,” [1] C. Gacek and B. Arief, “The Many Meanings of Open Source,” IEEE SoftwareIEEE Software, Vol. 21, , Vol. 21, No. 1, Jan./Feb. 2004, pp. 34-40. No. 1, Jan./Feb. 2004, pp. 34-40.

[2] B. Fitzgerald and T. Kenny, “Developing an Information Systems Infrastructure with [2] B. Fitzgerald and T. Kenny, “Developing an Information Systems Infrastructure with Open Source Software,” Open Source Software,” IEEE SoftwareIEEE Software, Vol. 21, No. 1, Jan./Feb. 2004, pp. 50-55., Vol. 21, No. 1, Jan./Feb. 2004, pp. 50-55.

[3] S. Lussier, “New Tricks: How Open Source Changed the Way My Team Works,” [3] S. Lussier, “New Tricks: How Open Source Changed the Way My Team Works,” IEEE IEEE SoftwareSoftware, Vol. 21, No. 1, Jan./Feb. 2004, pp. 68-72., Vol. 21, No. 1, Jan./Feb. 2004, pp. 68-72.

[4] Open Source Initiative. The Open Source Definition [Internet]. c2006. [4] Open Source Initiative. The Open Source Definition [Internet]. c2006. http://www.opensource.org/docs/definition http://www.opensource.org/docs/definition

[5] Open Source Initiative. Open Source Initiative [Internet]. c2006. [5] Open Source Initiative. Open Source Initiative [Internet]. c2006. http://www.opensource.org/http://www.opensource.org/

[6] J. Green, Wine Wiki. Wine TODO List [Internet]. [last modified 2006 Mar[6] J. Green, Wine Wiki. Wine TODO List [Internet]. [last modified 2006 Mar 9]. 9]. http://wiki.winehq.org/TodoList http://wiki.winehq.org/TodoList

[7] Wikipedia. Concurrent Versions System [Internet]. [last modified 2006 Mar 24]. [7] Wikipedia. Concurrent Versions System [Internet]. [last modified 2006 Mar 24]. http://en.wikipedia.org/wiki/Concurrent%5FVersions%5FSystem http://en.wikipedia.org/wiki/Concurrent%5FVersions%5FSystem

47