Upload
vuonghanh
View
224
Download
4
Embed Size (px)
Citation preview
Archipelago:New Cloud Storage Backend of GRNET
Filippos [email protected]
Chrysostomos [email protected]
Greek Research and Technology Network (GRNET) S.A.
16th TF-Storage MeetingFebruary 13, 2015
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 1 / 21
Background & Motivation Background
Background
Synnefo
OpenStack compatible cloud platform
Powers:
~okeanoshttps://okeanos.grnet.gr
~okeanos-globalhttps://okeanos-global.grnet.gr
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 2 / 21
Background & Motivation Motivation
Motivation
Pithos (Storage Sevice)
Swift APIUploaded Files / Images
Plankton (Image Service)
Glance APIRegistered Images
Cyclades (Compute/Network/Volume Service)
Nova/Neutron/Cinder APIVirtual Disks / Volumes
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 3 / 21
Background & Motivation Motivation
Unified View of Storage Resources
FilesUser files, with Dropbox-like syncing.
ImagesTemplates for VM creation.
VolumesLive disks, as seen from VMs.
SnapshotsPoint in time snapshots of volumes.
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 4 / 21
Archipelago Features Storage Features
Archipelago
Storage Virtualization System
Powering storage in Synnefo
Decouples storage resources from storage backends
Files / Images / Volumes / Snapshots
Unified way to provision, handle, and present resources
Decouples logic from actual physical storage
Software-Defined Storage
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 5 / 21
Archipelago Features Storage Features
Backend Agnostic
Advantages:
No vendor lock-inMigration from one storage backend to anotherPossibility of combining backends
~okeanos started with NFS and migrated to RADOS
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 6 / 21
Archipelago Features Storage Features
Storage Backend for the Cloud
With Archipelago we offer:
DeduplicationSyncable File/Image upload with partial uploadImage RegistrationThin clonesDownloadable thin snapshots
with zero data movement.
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 7 / 21
Archipelago Features Services Overview
Services Overview
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 8 / 21
Under the Hood Everything is a Resource
How it is Done
Everything is a resource on Archipelago
The same resource is exposed as:
A file through the API of the Storage Service (Pithos)An image through the API of the Image ServiceA live disk / VM volume through the API of the Volume ServiceA snapshot through the API of the Volume Service
All data remain in one place
No copying of data around
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 9 / 21
Under the Hood Mapfiles
Mapfiles
Mapfile for each resource
Keeps track of the mappings from resource offset to objectsKeeps metadata information for each resource
Operates on mapfiles
not on the actual data
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 10 / 21
Under the Hood Mapfiles
From Mapfiles to Data
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 11 / 21
Under the Hood Architecture
High Level Architecture
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 12 / 21
Under the Hood Architecture
Low level Architecture
Same Archipelago software stack on each node
Modular/Pluggable architecture
Distinct processes
Communicate via requests on a shared memory segment
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 13 / 21
Under the Hood Architecture
Low level Architecture
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 14 / 21
Under the Hood Architecture
Core
Volume composer
Composes resource I/O from individual objects
Mapper
Keeps and updates the mappings from resource offsets to individualobjects which actually hold the data
Flexible I/O pipeline
Can be extended with other components offering common functionality
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 15 / 21
Under the Hood Interfaces
Client Endpoints
Upstream Native QEMU virtio driver
VM interface
Blktap2 driver
Admin/VM interface
Pithos
Web interface / REST API
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 16 / 21
Under the Hood Interfaces
Backend Drivers
Currently supported backends:
Files (Any FS with POSIX semantics)RADOS (Using native librados interface)GlusterFS (Using native libgfapi interface)S3/SWIFT (Experimental)
General: Anything that can support object semantics
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 17 / 21
Advantages over other Solutions
Advantages over other Solutions
Allows for resource composition from already existing objects
Standalone resources
Unified view of all cloud resources
Backend agnostic
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 18 / 21
Experience
Experience from Production
3000 VM volumes
≈2.8M user uploaded files
≈20TB deduplicated data
≈80TB total data before replication
22M actual objects on the storage backend
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 19 / 21
The Future
Upcoming Features
New mapfile format
More compactAllows for larger volumes with minimal metadata overhead(1PB → 2GB)
Garbage collection
Deferred reference countingAutomatic deletion of unused objects
SSD Read-only caching
Lower latency
ISCSI target
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 20 / 21
Thanks
Thanks
Software lives at:
https://www.synnefo.org
Repository:
https://github.com/grnet/archipelago
Documentation:
https://www.synnefo.org/docs/archipelago/latest
Mailing list:
F. Giannakos, C. Nanakos Archipelago 16th TF-Storage Meeting 21 / 21