Upload
aleesha-briggs
View
218
Download
1
Embed Size (px)
Citation preview
Sakai 2.1 Update
Charles Severance
December 7, 2005
Sakai Babies
• Rachel/Stanford (1.0 pre-alpha)• Quingru/Stanford (1.0),• Rashmi/Tanush/Foothill (1.5)• Mallika/Saahil/Foothill (1.5.1)• Joanne/UM (2.0)• Seth/Zoe/Columbia (2.0.1) • Rob Lowden/TBD/IU (2.1)• Anthony Whyte/TBD/Sakai (2.1.1)• John Ellis/TBD/rSmart (2.1.2)
Rob LowdenQuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Rob LowdenQuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Carol Dippel
We love you Carol,
Oh yes we do
We love you Carol,
And we’ll be true
If you don’t do QA,
We’re blue!
Oh Carol, we love you.
We thank you Carol,
Oh yes we do
We thank you Carol,
And we’ll be true
Your work on Sakai,
Has been cool!
Oh Carol, we thank you.
August 2005
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
August 11, 2005
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
August 12, 2005, 12:46 PMFrom: Charles Hedrick <[email protected]>To: Bill Crosbie <[email protected]>Cc: Glenn Golden <[email protected]>, ZE Joseph <[email protected]>, Charles Severance <[email protected]>, Jon Andersen <[email protected]>, lance D Speelmon <[email protected]>, Andrew J Poland <[email protected]>, John Leasia <[email protected]>, Robert J Lowden <[email protected]>, David Haines <[email protected]>, "Bradley C. Wheeler" <[email protected]>Date: Aug 12, 2005 12:46 PMSubject: Re: Urgent: Memory Leak in Sakai
I agree with Bill. Some comments.
1) I'll tell you what we're seeing, but it's possible not all sites areseeing the same thing.
2) Your first priority needs to be setting up monitoring, so you cansee what you are actually running out of. Our current theory is thatthe problem is in permanent space. This would never be visible fromnormal log files. The only way we see what's happening is because we'rerunning Sun's memory monitoring tools, particularly VisualGC.
3) My current theory is that our problem is with dynamically loadedclasses. When the system starts, we have about 10000 classes loaded,with permanent space taking about 34 MB. After 14 or so hours today, wehave 15490 class loads, and permanent memory use of 60.8 MB. Thedefault is 64 MB maximum, so we think it's likely that by the end ofthe day we will have exhausted the default permanent memory, unless aGC gets lots of space back. Yesterday it did not. It unloaded somethinglike a dozen classes and reclaimed very little memory.
I don't know what's causing the continuing loading of new classes. Idon't know whether that is intended or not, and if it's intended, whenthe classes are supposed to be released. I've now get the maximum inpermanent expanded to 128MB. That will let us see whether spaceeventually stabilizes, and if so at what level.
My theory is that this problem has been intermittent because usagequickly expands to 60 out of 64. That's so close to the limit thatdepending upon details of the load sometimes it goes over and sometimes it doesn't. Whether there's an actual bug (i.e. memory usage expands for ever) I don't feel I can tell yet. I think by sometime next weekthe pattern will be clear enough. In the short run, we will try to addenough permanent memory to avoid the problem. If memory use growsforever, we'll try to add enough that we can survive by doing a nightlyrestart.
Note on setting up VisualGC: Most of the tools come with Java 5.0.Visual GC itself is a separate download. You will need Java 5.0installed, because the tools require it, but it doesn't need to be (andat Rutgers, isn't) your default version of Java. Setting it up is quiteeasy. I've used VisualGC running locally (displaying remotely usingX11), and also the remote version (which needs a copy of jstatd runningon the server).
The commands for remote use are not entirely obvious from the man page, so here they are:
On the server: ./jstatd -p 25209 -J-Djava.security.policy=<FILE>where 25209 is a random TCP port number and <FILE> is a file containinga Java policy (which is given in the man page)
On the client: bin/visualgc rmi://[email protected]:25209where 26429 is the PID of the JVM on the server, and 25209 is therandom TCP port number
In this case the server is Solaris 9 SPARC and the client OS X Tiger,but that shouldn't matter. The visualgc shell script requiressignificant repair on OS X, though the program runs fine.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Sakai 2.1 - Dates
• 2.0 - 06/15/2005
• 2.1 Integration Week - 10/24/2005
• 2.1.0_001 - 10/28/2005– Release Engineering and QA begins
• 2.1.0_004 - 11/17/2005– Maintenance branch established
• 2.1 Release - 12/01/05
Code Trends
Code Size over Sakai Release
0
20
40
60
80
100
120
1.0 1.5 1.5.1 2.0 2.1
Sakai Release
Megaytes
Provisional
JSF Tools
Samigo
Framework II
Framework
Legacy
Sakai 2.1.0 is just under one million lines of code
Sakai 2.1 Release Mgt
• Carol Dippel• Glenn Golden• Clay Fenalson• Peter Knoop• David Haines• Lydia Li• Josh Holzman• Ray Davis• Margaret Petit• Crystal Hancock• Zhen Qian
• Charles Severance• Lance Speelmon• Rob Lowden• Mike Elledge• Oliver Heyer• Ed Smiley• Daisy Flemming• Gonzalo Silverio• Jim Eng• Seth Therauilt• Stephen Marquad
Sakai 2.1 - Numbers
• JIRA– 1752 Issues Updated– 684 Bugs Resolved– 233 Tasks Completed
• SVN– Sakai 2.0 - 550 svn commits – Sakai 2.1 - 3793 svn commits (through
r4344)
Sakai 2.1: Process in Transition
• Sakai 1.0, 1.5, and 2.0 were managed “Sakai Project Style”– Central control– Controlled communication– Kind of like a company
• Sakai 2.1 needed to be managed “Sakai Foundation Style” to the extent possible– Expand commit lists (started in March 2005)– Manage the 2.1 release in the open– Kind of like a community
Process Changes for 2.1
• My Goal: Just be more open on all fronts– Send regular status reports including good, bad, and the
ugly straight to the dev list– Publish all tags, and target dates openly– Discuss bugs, JIRA, problems, and progress openly– When plans had to change - just admit and change the plans
in public– When tradeoffs had to be made - discuss rationale in public– No “internal” mailing list - no internal announcements (i.e.
tags, release candidates, etc)– Break down communication barriers between QA and
Development teams– Invite more folks to the release management calls
Sakai [email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@caret.cam.ac.uk
[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]@[email protected]
[email protected]@[email protected]@[email protected]@[email protected] ManchóDavid [email protected]@[email protected] [email protected] [email protected] [email protected]@pagels.dkKun [email protected][email protected]@[email protected] [email protected]@[email protected]
Contributors
Becoming A Committer
• Figure out some chunk of Sakai• Become a contributor working through a
current committer in that area• At some point you will be contributing so
much that the committers will – Realize that you are a great asset to the group
and quite sharp and – get tired of putting in your changes for you and
make you a committer
Sakai Project Leaders• Framework - Glenn Golden• Resources Tool - Jim Eng• OSP - John Ellis• Samigo - Lydia Li / Marc
Brierley• Gradebook - Ray Davis• I18n - Beth Kirshner• Web Services - Steven
Githens• Providers - Charles Severance• Site Info - Zhen Qian• Schedule - Joanne Sui• WSRP - Vishal Goenka
• Legacy Tools - John Leasia• Sections Tool - Josh
Holzman• Wiki - Ian Boston• Roster - Lance Speelmon• Syllabus - Chen Wen• Profile - Lance Speelmon• Admin Tools - Glenn Golden• Skin/CSS - Gonzalo Silverio• Portal Integration - Charles
Severance / Andrew Petro• TwinPeaks - Jeff Kahn• lancs.uk - Adrian Fish
How 2.0 Branched
2.1
2.0.x
Indiana
2.0
2.1
UM2.0.1
Fea
ture
s
Jun 05 Aug 05 Oct 05 Dec 05
How 2.0 Branched
Jun 05 Aug 05 Oct 05
2.1
2.0.xIndiana
2.0
2.1UM2.0.1
Dan
ger
->
Dec 05
Everyone Else in 2.0
Jun 05 Aug 05 Oct 05
2.1
2.0.xIndiana
2.0
2.1UM2.0.1
Dan
ger
->
Dec 05
2.1 Branch Plan
2.2
2.1.x2.1
2.22.1.1
Dan
ger
->
Dec 05
2.1.2
2.2_nnn
QA
QAQA
QA
User Visible Features in 2.1
• Improved Resources Tool with Meta-Objects• Section Tool• Group Support in Site Info• Announcements Tool Section-Aware• Samigo Section-Aware• Gradebook Section-Aware• Course Site Template Out of the Box• Localizations - completed and partial
New Resources Tool
Meta Objects
Section Tool
Group Support
Gradebook Section-Aware
Course Site Template
Localization
Localization Status
• Substantially complete– Chinese (China) - Tianhua Ding, Kun Lv– Korean - Il-hwan Kim– Dutch - Jim Doherty
• Partial– Japanese - Tatsuki Sugura / Shoji Kajita
Localiztions in Progress
• Danish - Kasper Pagels
• Hebrew - Dov Winer • Brazilian Portugese
- Alceu Fernandes Filho
• Portugese - Feliz Gouveia
• Slovokian - Michal Mosovic
• Catalan - Alex Balleste
• Chinese (Taiwan)• French - Yves
Epelboin• Spanish - Cynthia
Gonzalez
Sakai 2.1 Framework
• AuthzGroups - Groups within Sites
• SecurityAdvisor - Multi-context Authz
• Provider Cleanup – CourseProvider getTab– UserDirectory providerOrder, alwaysCreate
• Rutgers MySql Performance Changes
• Entity Producer - Refactor
Pre-AuthZGroup Structures
Site: CS1
Roster: CS1
doglezhencsevlancemarc
InTATAStSt
In: rwd TA: rwSt: r
ID: F05 CS1 A
doglecsevlance
InTASt
ID: F05 CS1 B
doglezhenmarc
InTASt
Course/RealmProviders
AuthZGroups in 2.1
Site: CS1
Roster: CS1 B
doglecsevlance
InTASt
Roster: CS1 A
doglezhenmarc
InTASt
In: rwd TA: rwdSt: r
In: rwd TA: rwdSt: r
Roster: CS1
doglezhencsevlancemarc
InTATAStSt
In: rwd TA: rSt: r
ID: F05 CS1 A
doglecsevlance
InTASt
ID: F05 CS1 B
doglezhenmarc
InTASt
SectionTool
Site: CS1
Roster: CS1 B
doglecsevlance
InTASt
Roster: CS1 A
doglezhenmarc
InTASt
In: rwd TA: rwdSt: r
In: rwd TA: rwdSt: r
Roster: CS1
doglezhencsevlancemarc
InTATAStSt
In: rwd TA: rSt: r
ID: F05 CS1
SectionTool
Protecting a Resource
/content/eecs280 content.read
AccessServlet
ResourcesService
Attaching a Resource Across Sites
/content/eecs280 content.read
01/02/05blah
/site/sakai-dev schedule.read
AccessServlet
ScheduleService
Access is allowed for eecs280 content.read users but not for sakai-dev users so this fails in the sakai-dev schedule attachment case.
ResourcesService
Attaching a Resource (compromise)
/content/eecs280 content.read
/attachments/FF987DEF
01/02/05blah
/site/sakai-dev schedule.read
AccessServlet
ResourcesService
ScheduleService
Copy
Using SecurityAdvisor
/content/eecs280 content.read
01/02/05blah
/site/sakai-dev schedule.read
AccessServlet
ResourcesService
ScheduleService
When schedule tool sends the user to Access, Schedule adds a SecurityAdvisor. This allows the bypass of the normal security protecting the content.
EntityProducer
• Re-factored the 2.0 Resources Interface and made much more rich
• Allows cleaner “drop-in” of new capabilities by distributing brittle code in central locations into each of the Components– Function creation and management (melete.author)– AuthZGroup resolution (Reference.java)
• Ends up adding constraints to Components which produce “entities”
• Going forward will enable other capabilities like improved import/export, hierarchy, search, fine-grained entity reuse, etc..
Provisional Tools
• Came from the “bundle” debate we have had throughout the project– We needed something between “yes” and “no”
• Re-thought the inclusion/exclusion decision from a “community or peers” perspective rather than top-down central approach
• Came up with a set of criteria• Provisional tools are “almost in” Sakai
– Part of the release– Available and tested during QA period– Hidden in the release with a simple way to unhide
Provisional Criteria
• Community Support– Must have commit list and be in SVN– Must run in production at >=2 sites– Must have proper license– Must be willing to answer questions– Need test plans and specifications– Needs to be tracked in JIRA
Provisional Criteria (cont)
• Technical– Support HSQL, MySql, Oracle– Use AutoDDL properly– Use sakai.properties– Do AUTHZ functions like the rest of Sakai– No patches to other elements– Must cluster– Use proper versions of Spring, Hibernate, etc.
Provisional Criteria (cont.)
• Interaction and Visual Design– Inherit skins properly– Look “like” the rest of Sakai tools (fit in)– Follow interaction designs in style guide– Use JSF UI Components (if applicable)– Include help - properly added to the Sakai
Help system
Provisional Tools (cont)
• Desirable elements (required for full inclusion)– Internationalized– Accessible (including a review)– Separation of persistence and business logic into a properly
factored Sakai Component – Event generation as appropriate– Fit logically scope-wise with the rest of Sakai
• Process– Notify community 30 days before code freeze to allow
comment– If problems arise during QA that are not easily fixed,
provisional tools are dropped
*Sample* Attribute Matrix
Announce Melete Jforum Rwiki Profile
AutoDDL Yes Yes Yes Yes Yes
Properties Yes Yes No Yes Yes
MySql Yes Yes Yes Yes Yes
Oracle Yes Yes No Yes Yes
Skinnable Yes Yes No Yes Yes
Cluster Yes No Yes Yes Yes
Resource Yes No No Yes No
I18N Yes Yes Yes ?? Yes
Events Yes No No No No
Provisional Tools For 2.1
• SakaiScript - Web Services– Steven Githens - Lead
• WSRP - Web Services For Remote Portals– Vishal Goenka - Lead
• Become User Tool– Zach Thomas - Lead
• Roster Tool– Lance Speelmon - Lead
• Wiki - Based on Radeox– Ian Boston - Lead
• TwinPeaks - DR OSID– Jeff Kahn - Lead
SU Tool
TwinPeaks
Wiki
Roster
WSRP Side-by-Side
Portlet = Placement
Kernel Tool Registry
Sakai WSRP Provider
Tool A Tool B Tool C
Site Placements
Request Filter
Apache WSRP4J
WSRP Consumer(uPortal)
Web Services
MercuryPlacements
WSRPArchitecture
List Portlets Tool ID
Placement ID
Get Markup
URL Rewriting
Sakai 2.2 *Potentials*
• The Hierarchy “Grail”• Pluggable WSYWIG• Re-factor Resources into
Helpers• Co-Release with OSP• CourseManagement• Internal DR OSID• Accessibility
Improvements• JSR-168/WSRP CSS• Unit Test Framework
• Section Aware Tools• WSRP Producer Portal• Provisional Tools
– MailTool– IU Message Center– Melete– JSR-168 Portlet– IMS Tool Portability– Blog (lancs.uk)– WSRP Consumer
• RSS• Tool ID Based URLs
Way Long Term
• Sakai / uPortal tightly integrated– JSR-168, WSRP Consumer, WSRP Producer
• JSR-170 - Java Content Repository• IMS Content Packaging Import and Export• IMS Common Cartridge• Lucene• SCORM• Enhanced LAMS integration• All domain objects fully modeled with published data
models and RDF/OWL support
Sakai Community Practices
• Board has initiated a group to examine and document process practices with the “Duffy document” and the Requirements DG work
• While that work goes on, existing efforts will also– Keep opening up - evolve project behavior patterns into
community behavior patterns– Begin to record existing de-facto processes and then review
and see how the existing processes can be improved
• Looking forward, we need to keep a mindset of “community of peers”
Sakai Foundation Board Principles• There’s acceptance that contribution to the code and contribution to the project will
be provided in a variety of forms. Contributions are encouraged and made as easy as possible.
• The best way to positively effect the outcome of Sakai is to contribute resources to the project.
• The focus of centralized activities will be to maximize the effectiveness of the community, making the large number of resources as effective as they can be.
• The Sakai Foundation central staff will be light weight and depend on the community for major development efforts
• All the processes and activities in the Sakai community should strive for as much transparency and syndication of its activities as possible in recipient friendly format.
• The process needs to be clear and well-known and well-publicized so that anybody in the community knows how to participate to make something happen.
• To ensure the guiding principles of the foundation vision is clear to the community.• Sakai is on a trajectory and will continue to evolve in various area, with some
things being more or less mature.
These are *DRAFT* from a brainstorming session and will evolve - likely discussed in the strategy and advocacy or some other DG. Did I say that these
are *DRAFT*??
Questions