11

Glassfish QualityCommunity

Pascal PrévotSoftware Quality ManagerSUN Microsystems

2

Agenda

What is GlassFish? Test Challenge in Open Source FishCAT Program Student Program Main outcomes

3

● Application Server (Middleware)● Java EE 5, 6 Reference Implementation● Enterprise functionality● Open Source product

● A strong community (more than 7,000 members)● Users, partners, testers, developers● Started in 2005

What is GlassFish ?

4

J'08

A'0

8

S'0

8

O'0

8

N'0

8

D'0

8

J'09

F'0

9

M'0

9

A'0

9

M'0

9

J'09

100

200

300

400

500

Goal - 250K Total Unique GlassFish Registered UsersTotal GlassFish RegistrationsTarget Unique Registered Users - FY'09Total Unique Registered Users

•Over 14 million downloads in less than 2 years

•Over 7,000 members

•1200 Registrations per / day (over 300K in 18 months)

•Over 2500 mail messages / per month

FY 06 FY 07 FY 08 FY 09

0

2000000

4000000

6000000

8000000

10000000

12000000

Y/Y Dow nload Grow th

Add. Forcast

YTD

GlassFish / Java EE adoption

5

Test challenges in Open source

Active Open source products change very rapidly with new features added by external contributors• How to ensure QA teams test new bits on a timely fashion?• How to make sure new deliveries are of good quality?

Open source products can be configured in many ways:• How to validate all the relevant combinations?• How to determine real life configurations?

Quality coverage increases over time to take into account new features, new supported platforms, ...• … while having to maintain an existing base of tests• … at constant QA resource staffing

6

Testers / ISV's

Communities Students

• Enhance overall GlassFish quality> More test cases for higher coverage> Testing engagement during projects

• Inject “Voice of the customer” into GlassFish Quality> Tested End-user configurations> Concrete customers scenarios

• Increase GlassFish adoption in the IT industry> Early adopters in the development cycle> Awareness in the universities

GlassFish QA Community goals

7

FishCAT program

Early access program• Field testing of the GlassFish project• Window of opportunity to provide direct feedbacks to

engineers while the project is still under development• Performed while GlassFish QA continues testing

> Ability to anticipate issues on real life scenarios

Process• The GlassFish QA team drives the community:

> Weekly meetings> Recommended tasks based on areas of expertise

• Transparent email discussion in the community• Shared bug tracking system

> Leadership, responsiveness and transparency

A Win-Win program to improve GlassFish quality

8

Student program

An opportunity to learn● Mentor Master level projects leading to developments in

GlassFish● Quality courses based on GlassFish

> Strong links with universities to communicate onSUN portfolio and our Quality approach

An engagement to contribute• Bug filing in the GlassFish project • Students develop automated tests in dedicated areas• Opportunity to interact with Sun Engineers on real projects

and be an active GlassFish contributer after graduation. > Win-win deal where GlassFish quality improves

while students increase their experience

A proactive approach to increase GlassFish awareness and adoption in the future developer community

9

FishCAT

Students

• 32 professionals from 18 different countries

• 125 defects submitted, 94% resolved. 5 manuals reviewed

• More than 700 emails exchanged

• Influenced universities to teach QA based on GlassFish. QA class students filed more than 250 bugs

• Mentored university students in developing cool technologies based on GlassFish. 50+ Students are working on more than 25 projects.

Main outcomes

10

FishCAT - Success Factors

• Constantly engage the community in testing> Identify key contributors and reward them> Setup win-win situations where feedbacks / issues are

promptly taken into account

• Create a transparent environment for sharing> State goals of the required efforts> Share your defect tracking system, defect resolution

stages, test cycle information> Pro-actively communicate with the community on any

relevant topic. Be clear and unambiguous !

11

Students - Success Factors

• Effective link between students and SUN engineers> Keep up interest so that students don't feel isolated> Solid reference in student resumes

• Concrete projects> Students are saving time without building the sample

applications/projects during testing. > They can make practical use of the GlassFish popular

open source components during their testing courses.

12

References

• GlassFish projecthttp://glassfish.org

• GlassFish Aquariumhttp://blogs.sun.com/theaquarium

• GlassFish Quality Communityhttp://glassfish.org/quality

• FishCAT programhttp://wiki.glassfish.java.net/Wiki.jsp?page=FishCAT

13