Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
CSAL: A CLOUD STORAGE ABSTRACTION LAYER TO ENABLE PORTABLE CLOUD APPLICATIONS Zach Hill & Marty HumphreyDept. of Computer Science, University of Virginia
A Cloud Application
Worker
2Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Object/Blob
ServiceTable
ServiceQueueService
Front-EndUser Requests
WorkerWorker
Worker
Many clouds, many code versions
Worker
3Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Blobs
Tables
Queues
Front-End
Worker
Storage Services
WorkerStorage Services
Front-End
Single code version, many clouds
Worker
4Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Blobs
Tables
Queues
Front-End
Worker
Storage Services
WorkerStorage Services
Front-End
CSAL
CSAL
CSAL
CSAL
CSAL
CSAL Overview
5Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Application
CSALBlobStorecreateContainer
listContainersdeleteContainer
getBlobputBlob
deleteBlob
QueueStorecreateQueuedeleteQueuegetMessageputMessage
peekMessagedeleteMessage
TableStorecreateTabledeleteTable
InsertUpdateDeleteQuery
getItem
Metadata Manager
Service Manager
S3 PluginSimpleDB
PluginAzure Queue
Plugin…
Metadata Store table service(s)
Blob Namespace Table Namespace Q Namespace
CSAL Namespaces
One namespace for each abstraction type Metadata only for containers Service endpoint, identifier, user credentials
Each abstraction has an independent metadata store
Metadata caching Container ops are not very common If data is stale, simply re-fetch and retry
Dec. 2, 2010Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia 6
CSAL Namespaces
Dec. 2, 2010Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia 7
CSAL
X
Y
X
foo
(“X”, http://service1.com, testusr, testKey,…)
Service1
Service2
Service3
Metadata Manager
BlobStore
Cache
Plugin
2. Lookup “X” metadata – cache first
1. Call getBlob(“X”, “foo”);
3. Retrieve from table if not in cache
5. Access “foo” in “X” in Service1
4. Use metadata to determine plugin to use
CSAL Implementation
Client-side java library i.e. BlobStore.getBlob(“Container”,”foo”);
Metadata backing store in the cloud
Currently supports Azure & AWS storage Both SOAP and REST
8Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Performance of CSAL
Adding software layers isn’t free Compare CSAL to Azure’s and Amazon’s SDK
APIs Set of micro-benchmarks to test operation
latency Container Ops and Data Ops Expect a slowdown for container ops due to
metadata
9Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Performance – Container Ops
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Create Container Create Table Queue Create
Med
ian
Ope
rati
on T
ime
in S
econ
ds
CSAL in AWS
CSAL Total Time Native API
Core Op Time Metadata Op Time
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Create Container Create Table Queue Create
Med
ian
Ope
rati
on T
ime
in S
econ
ds
CSAL in Windows Azure
CSAL Total Time Native API
Core Op Time Metadata Op Time
Note: Error Bars indicate 1 Standard Deviation
10Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Performance – Data Ops
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Med
ian
Ope
rati
on T
ime
in S
econ
ds
CSAL in AWS
CSAL Total Time Native API
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Med
ian
Ope
rati
on T
ime
in S
econ
ds
CSAL in Windows Azure
CSAL Total Time Native API
Note: Error Bars indicate 1 Standard Deviation
11Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
What about Standards?
Standards Efforts OCCI OVF
Standards take time to develop and are resisted by vendors
Multi-cloud APIs SimpleCloud, jClouds, DeltaCloud, LibCloud SAGA
12Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Future work
What if Cloud X doesn’t have tables/blobs/queues? Map one abstraction to other (i.e. filesystem) 3rd party services: Hbase, HyperTable, Cassandra…
Placing, replicating, and migrating data in real-time for performance and/or cost
Real-world applications such as multi-cloud MR
13Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Summary
Application lock-in and portability are problems in clouds
Standards are great, but don’t hold your breath just yet
CSAL provides storage abstractions to make the application code itself portable with little performance impact for common data operations
14Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia Dec. 2, 2010
Questions?
Dec. 2, 2010Z. Hill and M. Humphrey, Dept. of Computer Science, University of Virginia 15