Red Hat Openstack and Ceph Meetup, Pune | 28th NOV 2015 | vumrao@redhat.com

Vimal A.Rvimal@redhat.com

RADOS GatewayCeph's ReST Object Store interface

Agenda

Ceph overview/architecture

Understand Rados Gateway

RGW data flow

RGW Federated Gateways

1) Ceph overview

Ceph Overview

Massively scalable storage system

Reliable No single point of failure

Autonomous Self-governing and intelligent components

Distributed Consists of monitor and OSD nodes

Object StorageA popular storage method for unstructured data

Flat hierarchy compared to traditional filesystems

Rich custom metadata

Ceph Architecture

Rados Gateway

Object storage interface emulating both Amazon S3 and Openstack Swift.

Accessible through a ReST-ful HTTP interface

ReST APIs for Amazon S3 and OpenStack Swift protocolshttp://docs.ceph.com/docs/master/radosgw/s3/

http://docs.ceph.com/docs/master/radosgw/swift/

Two frontends:Ceph Firefly (0.80.x) - Apache with FastCGI

Ceph Hammer (0.94.x)- CivetWebhttps://github.com/civetweb/civetweb

Light-weight Webserver, can be embedded in C/C++ apps, or act standalone.

Works in Linux, Windows, Mac-OS, Unix, Android, iOS etc..

Supports Regions, Zones, Users, ACLs, Quotas etc.. similar to S3/Swift

Flickr's RGW Object store case study (Not RHCS) http://goo.gl/uS5V3I

Rados Gateway (Continued)

M

M

M

APPLICATIONAPPLICATIONRADOSGWRADOSGW

ReST HTTP(S)

socket

RADOS CLUSTER

Rados Gateway (Continued)

Rados Gateway installation

RGW node installation is done using 'ceph-deploy'.# ceph-deploy install --rgw

Installs 'ceph-radosgw' starts the service.

RGW listens on the port 7480

If successful, the following XML should be printed# curl http://:7480

anonymous

Rados Gateway pools

RGW stores data in various Ceph pools due to S3/Swift semantics

# radosgw-admin zone get.rgw- Domain root

.rgw.root- Region/Zone information for the RGW

.rgw.gc- Garbage collection pool. Stores information on objects marked to be deleted

.users- Access keys for S3 users

.users.uid- User names

.rgw.buckets- Buckets

.rgw.buckets.index- An index on buckets and objects in it

.users.swift- Swift subuser info

.log- Bucket creation info

.intent-log - Stores a copy of the object before an update in case a undo is required.

.usage - Per user usage log

http://fpaste.org/294240/

Rados Gateway

RGW in Ceph Hammer (0.94.x) does not support SSL.

A proxy service is needed to terminate SSL requests before the RGW node.https://bugzilla.redhat.com/show_bug.cgi?id=1258961

RGW requires separate user accounts for S3 and Swift access

Data in RBD pools cannot be accessed from RGW, and vice versa.

Rados Gateway (S3 user)

Rados Gateway (Swift user)

Rados Gateway (Data flow)

RADOSGWFRONTENDREST DIALECTEXECUTIONRGW-RADOSlibrados

RGW OBJCLASSESRADOS

APPLICATION

Rados Gateway (Data flow)

RGW nodesFrontend (Apache/FastCGI, CivetWeb)

ReST dialect (Pluggable architecture for other dialects)

RGW core execution layer

RGW-RADOS object mapping layer

Librados

RADOS Object Classes for RGW

RADOS backend (OSDs)

Federated Gateways

RGW nodes spanning multiple geographical locations

Region: A logical geographical area

Zone : A grouping of either one or more RGW entry pointsOne master RGW, multiple slaves if needed

A RGW entry point per zone or per region.

A sync-agent syncs data between zones

Inter-zone data replication

Inter-region metadata replication

Ceph OpenStack

Ceph is a preferrable storage solution for OpenStackBlock : RBDKrbd : Kernel RBD module

Librbd : RBD client-side library

Filesystem : CephFS

Object Store : RGWNFS support coming in an upcoming release

RADOS ObjClasses makes it easy to add in another interface.

Strong and mature integration with the various OSP interfaceshttp://docs.openstack.org/openstack-ops/content/storage_decision.html

http://ceph.com/community/more-than-an-object-store/

Ceph OpenStack

Ceph - OpenStack

Cinder (Block storage) : RBD

Glance (Image registry) : RBD and RGW

Nova (Compute service): RBDhttp://docs.ceph.com/docs/master/rbd/rbd-openstack/

Keystone (Authentication) : RGWhttp://docs.ceph.com/docs/master/radosgw/keystone/

Ceilometer (Telemetry): RGWTo track Ceph object storage usage

http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html#ceph-object-storage

Thank you!