globus online
Globus Online for Research Data Management
Rachana Ananthakrishnan Great Plains Network Annual Meeting 2013
We started with technology proven in many large-scale grids
GridFTP GRAM
MyProxy GSI-OpenSSH
…
1.2 PB of climate data delivered to 23,000 users
Typical of large, well funded research projects using GT
1.2 PB of climate data delivered to 23,000 users
GT provides robust infrastructure for the 1%
What about the 99%?
GT provides robust infrastructure for the 1%
What about the 99%?
BIG SCIENCE. Small labs
GT provides robust infrastructure for the 1%
globus online
Managing data should be easy …
Registry
Staging Store
Ingest Store
Analysis Store
Community Store
Archive Mirror
Ingest Store
Analysis Store
Community Store
Archive Mirror
Registry
… but it’s hard and frustrating!
Registry
Staging Store
Ingest Store
Analysis Store
Community Store
Archive Mirror
Ingest Store
Analysis Store
Community Store
Archive Mirror
Registry
Quota exceeded
!
Expired credentials
!
Network failed. Retry.
!
Permission denied
!
What is Globus Online?
Transfer and sharing of large data sets…
…with dropbox-like characteristics…
…directly from your own storage systems
We adopted SaaS approaches to transform the user experience
… for both researchers and resource owners/system
administrators
We started with reliable, secure, high-performance file transfer …
Data Source
Data Destination
User initiates transfer request
1
Globus Online moves and syncs files
2
Globus Online notifies user
3
… and then made it simple to share big data off existing storage systems
Data Source
User A selects file(s) to share, selects user or group, and sets permissions
1
Globus Online tracks shared files; no need to move files to cloud storage!
2
User B logs in to Globus Online and accesses
shared file
3
Log into Globus Online
Alternate Logins
Login using InCommon
InCommon Login
Source endpoint
Destination endpoint
Activation
Transfer data
Share data
Set permissions
Manage Groups
Interactive login to command line interface:
Running commands remotely:
Using CLI with gsissh:
Globus Online CLI
$ ssh [email protected]
$ ssh [email protected] <command>
$ gsissh [email protected] <command>
$ ssh [email protected] scp –r –s 3 -D \ nersc#dtn:~/myfile* mylaptop:~/projects/p1 Task ID: 4a3c471e-edef-11df-aa30-1231350018b1 $ _
Usage is accelerating
Early Adopters
• What is GCMU? – Globus Connect version for easily creating (sharable) endpoints
on multi-user storage servers – Packages a GridFTP server and MyProxy CA authentication
server, pre-configured for use with Globus Online
• Why GCMU? – Create transfer endpoints in minutes – Avoid complex GridFTP install
• To download: www.globusonline.org/gcmu
Globus Connect Multiuser (GCMU)
29
“We used GCMU to form a campus-‐wide GSI authenAcaAon service spanning mulAple servers. Now my users have a fast, easy way to get their data wherever it needs to go, and the setup process was trivial." -‐-‐University of Michigan
“As a resource admin, I've found GCMU an exceedingly useful tool.... With GCMU, seHng up a GridFTP server and handling authenAcaAon for mulAple users is easy." -‐-‐Oak Ridge Na8onal Lab
We are a non-profit service provider to the non-profit
research community
Our challenge:
Sustainability
We are a non-profit service provider to the non-profit
research community
Globus Online Provider Plans
Support ongoing operations
Offer value-added capabilities Engage more closely with users
• Endpoint operations management • Branded web sites • Alternate identity provider • Usage reporting • MSS optimizations • Multiple GridFTP servers per endpoint
Provider Plans offer…
Starting at $20k per year
End User Plans
• Basic: Free – File transfer and synchronization to/from
servers – Personal endpoints with Globus Connect – Access to shared endpoints created by others
• Plus: $7/month (or $70/year) – Create and manage shared endpoints – Peer-to-peer transfer and sharing
Globus Platform-as-a-Service
Globus Nexus (Identity, Group, Profile)
…
Sharing Service
Transfer Service
Dataset Services
Globus Toolkit
Glo
bus
Onl
ine
API
s
Glo
bus
Con
nect
Our research is supported by:
U.S. DEPARTMENT OF
ENERGY
Questions?
Contact: [email protected]
Providers: globusonline.org/provider-plans
www.globusonline.org
Recommended
