Upload
garry-morrison
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
The 3The 3rd rd Pan-GalacticPan-Galactic
WorkshopWorkshop
Schedule9:00 – 10:00 BOINC status report10:00 – 10:15 Break10:15 – 11:30 BOINC plans11:30 – 12:30 Discussion12:30 – 1:30 Lunch1:30 – 2:15 Breakout session 12:30 – 3:15 Breakout session 23:30 – 4:00 reports4:00 – 5:00 Wish list, discussion, wrap-up
Outline
The state of volunteer computing The state of BOINC What’s happened since the last workshop
and what hasn’t happened What’s in the works What’s in the future
The state of volunteer computing Folding@home leads the way
200 TFLOPS from PCs, 50 TFLOPS from GPUs, 600 TFLOPS from PS3s
Storage@home BOINC-based activity
~40 projects 500 TFLOPS from PCs
No big discoveries so far Grid.org shuts down April 07 IBM WCG transitions to BOINC Volunteer population: stagnant No major (> 10K hosts) projects in last year
The state of BOINC Über-goals
use all computers in the world for important work enable new science involve the public in science
BOINC’s role provide software technology catalyze/enable communities
Partners: projects, BAM!, GridRepublic, translators, alpha testers, volunteers, ...
Staff: me, Rom Walton, Charlie Fenton 50% Contributors
Kevin Reed, John McLeod, Rytis Slatkevicius, Nicolas Alvarez, David Barnard, Frank Weiler, many others
Funding: new NSF grant, 9/07-8/10
Requests for BOINC projects
If you need a new feature: talk to me about it first develop it yourself but stay in synch
Upgrade server software frequently Participate in email lists Include some BOINC funding in your grant
proposals Tell me about publications
What’s happened since the last workshop?
BOINC now uses trac
Trac bug-tracking system replaces BOINCzilla Subversion replaces RCS
sandboxes Milestones Documentation is Wiki-fied
please participate!
Facilitating project creation
VMware BOINC server run on Windows/Mac/Linux all software pre-installed, compiled
Wrapper application handles sequence of “legacy” apps, which use
stdin/stdout or named files checkpointing at “sub-job” level
make_project improvements reasonable config.xml defaults
Client software releases 5.6 (Sept 06)
new CPU scheduler Mac sandbox
5.8 (Dec 06) Simple view Snooze button CPU throttling memory usage prefs
5.10 (May 07) Project selection menu Advanced prefs dialog Grid-based views
Multi-platform support
client scheduler
supported platforms:win64win32
applicationversion number
platform
Some hosts can run >1 type of binary
Homogeneous redundancy
Multiple HR types each classifies hosts based on numerical properties per-application HR types
Feeder reserves shared-memory slots per HR class in proportion to RAC
“Census” program computes ratios Feeder DB queries scans all unsent results
feeder
win32
scheduler
linux/x86 mac new
shared mem
scheduler. . .
Load-aware scheduler
client scheduler
job deadline CPU time remaining1 17 42 35 12...
For each candidate job J:if run J with existing jobs,are any new deadlines missed(or missed by more)?If not, accept J
Client simulator
Simulates 1 client and N projects
Uses actual client code Web interface Uses:
study new scheduling policies debug client optimize preferences for a particular client
project description
host description
cc_config.xml
global_prefs.xml simulator
optionsgraphical timeline
message log
figures of merit:idle CPUwasted CPUshare violationmonotony
Recruitment and retention
Newsletter support Reminder email support
dropouts initial failures
Grass-roots PC magazine publicity campaign http://boinc.berkeley.edu/trac/wiki/BoincPr
BOINC-wide teams
http://boinc.berkeley.edu/teams create BOINC-wide teams here offers list of teams/founders by Web RPC
team_update.php (periodic task) fetches team list creates users and teams as needed
BOINC-wide teams are automatically created on new projects
Skype-based “customer support”
Volunteer registration languages projects
Server polls Skype status Users can call or email Users can rate volunteers
Scheduler and daemons
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
use reliablehosts for retries
homogeneousredundancy
changesmulti-platform
supportabort,abort-if-unstarted
EDF simulationbased on workload
Client
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
memory-awarescheduling
“suspend if idle”preference
security-relatedchanges
auto-update
log availability info
improvednetwork BW estimates
improved slot dirmanagement
Manager
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
per-applicationslide show
project selectionmenugrid-based
viewscompleteskin mechanism
preferences dialog,day-of-week prefs
API and runtime system
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
supportmemory-aware
apps
kill unresponsive
apps
message-passingmore robust
multi-jobwrapper
Web features
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
certificateof computation
private messages
forum moderationfeatures
team foundershiptransfer
show all projectson user page
Paypal donationsystem
mass host merge
Web RPCs and AMS support
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
AMS can specifyprefs, venue,
resource share
team historydelta
BOINC-wide teams
Google gadget
Simulators
Nov
20072006
Oct AugJulJunMayAprMarFebJanDec
client simulator
SIMBA(global simulator)
What did we not get around to?
Document app development using Cygwin, MinGW, FORTRAN
Disk space management Revised credit system
better FLOPS accounting generalized credit system
Community features buddy lists cross-project profiles, message boards
BOINC protocols as Internet standards
What’s in the works?
Version 6 application graphics Old: graphics in separate thread New: graphics in separate program
takes –fullscreen option has logical name “v6graphics”
Library support: supply render function, input-handling functions
Support for shared memory between app and graphics
BOINC client
graphics appmain app
BOINC manager,screensaver
shmem
Other V6 changes
Windows Sandboxing: create unprivileged account for BOINC Move data directory out of C:Program Files Wix-based installer
Mac/Linux Use memory-mapped files instead of shmat()
V5/V6 transition
V6 client is compatible only with V6 apps Scheduler change:
API version is attribute of app version Can have both V5 and V6 app versions Scheduler will send only V5 apps to pre-V6 clients
Projects can maintain backwards compatibility as long as they want
What’s in the future?
Berkeley@home Campus-level “meta-project” Applications
6 pilot apps: climate, geophysics, nanotechnology, genetics, phylogenomics, scientific decision systems
Volunteers 1,000 instructional PCs 5,000 faculty/staff 30,000 students 400,000 alumni general public
NSF proposal submitted
Multi-threading support What’s in a $1000 PC?
2007: dual-core CPU, 4 GFLOPS, 1 GB RAM 2010: 80-core CPU, 100 GFLOPS, 8 GB RAM Volunteer computing provides a use for all those cores,
but you may run out of RAM BOINC support for multi-thread apps
Languages/libraries for parallel programming Open MP Titanium, Cilk, ...
core client app
Try to use N cores
OK, I’m using M cores
Non-PC platforms
Sony Playstation 3 Support from Sony: on hold PS3Grid.org
Other game consoles TV set-top boxes Hand-held devices
not clear if feasible
Energy and volunteer computing
Computing == energy usage [== carbon emission ] Energy-aware computing
dynamic clock rate shut-off when idle disk spin-down
BOINC must be energy-aware friendly “compute while in use” by default allow auto-shutoff don’t assume constant clock rate
Make sure volunteers are aware of energy cost Computer as space heater
How to validate results and credit? Redundancy
homogeneous redundancy: messy heterogeneous: must write fuzzy validator wastes computing power
Non-redundancy (CPDN, Rosetta) result consistency checking (application-specific) granted credit: constant, or based on project-wide
distribution Variable redundancy?
hosts start with 2X redundancy, asymptote to 1X redundancy as trust increases
(doesn’t solve comparison problem)
Skill aggregation (human computing) Web-based vision tasks
Stardust@home, Clickworkers, galaxy classification Amazon “Mechanical Turk” Validation Formulation as multi-person game
Louis von Ahn: image tagging Motivational axes:
competition
community
Rosetta@home plan
Protein structure prediction low-res: combinatorial, spatial, intuitive; humans do
better than computers high-res: computers do better
Interactive “protein manipulation” program Teams as management structures
tasks are given to (possibly multiple) teams managers organize and schedule sub-groups with
particular skills or resources communication paths between sub-groups
Berkeley Open System for Skill Aggregation (BOSSA)
DB-driven web-based system Tools for community, competition, collaboration,
and organization Teams, groups, sub-groups Communication mechanisms Tasks, validation, credit
Integration with BOINC for tasks that involve large computation or storage
How to verifiably train volunteers? Interspersed lessons and exercises
Optimal refresh for long-term retention
Analytics for content assessment/refinement
Adaptive teaching Volunteer computing and skill aggregation
provide an ideal venue for education research
lesson exercise lesson exerciseexercise
lesson exercise (3 weeks)(1 week) exerciseexercise
lesson
lesson
lesson exercise
Berkeley Open Learning Technology (BOLT)
DB-driven CMS and analytics engine for web-based teaching
content(lessons, exercises)
course structure(XML)
teaching engine (PHP)Sequencing, navigation
student info,interaction DB
Students
analytical tools Educators
DB and web integration
Accounts, teams and groupsCommunication
Credit and competition
BOINChosts
applications
jobs
BOLTlessonscourses
BOSSAtasks
Background utility compatibility
Background utilities disk defrag disk indexing virus scanning web pre-fetch disk backup
Most run only when computer is idle volunteer computing ==> they never run
A) ignore zero-priority CPU activity B) Background manager
intelligent decision about when to run activities