IT Press Tour #17 - OpenIO & Technology

88% (2015)

Unstructured dataworldwide ratio

Each year

Facts:

ESG Digital Archive Market Forecast IDGSource:

+62%

Scalability + PerformanceChallenge:

Object storageSolution: ?

Object storage ?

Object storage ?

Object storage !

Object storage

Advantages But not optimized for

Scalability 100s PBs, low TCO

Ease of access Simple API GET/PUT/DELETE

Resiliency Replication, erasure coding,geodistribution

Heavily updated data

No POSIX-like API

• VMs • SQL DBs

10+ yearsMarket: 1.2 B$ today 4.6 B$ > 2020

Maturity Fast growing

IDCSource:

Gartner, Critical Capabilities for Object Storage

Scalability Efficiency Interoperability Ease of management Performance Resiliency Security

Very high capacity Optimized TCO* Unstructured data Price per GB low

Rather low capacity Very high performanceVMs, SQL DBs Price per GB high

Object store Flash Arrays

Vision: 88% 12%

No future for traditional vendors?

*Total Cost of Ownership

Performance

Tightly integrated application connectors

Quality of Service

< Vision:

Very high capacity Optimized TCO Unstructured data Price per GB low

88%

Object store

Object storage is not only storage. It is a new way to build backend services for applications. As Docker has disrupted sysadmin, we want to disrupt storage management.

WHO

AntiSpam - AntiPhishing - eMail classification 235M MailBoxes in 76 countries

IaaS - PaaS - DaaSNetwork & Security for legacy, SaaS and Web Apps

Lille San Francisco Montréal Hong-Kong Tokyo

Innovative technologiesfor IT infrastructure

100 people

7 8 50fellow co-founders

years experience

million end users

16people

2015launch (June)

2006

Idea & 1st concept

2007

Designdev starts

2009

1st massiveproductionabove 1PB

2006

Idea & 1st concept

2007

Designdev starts

2009

1st massiveproduction above 1PB

2015

OpenIOfork

2012

Opensourced

2014

10+ PBmanaged

Lille

@OpenIO

OpenIO

github.com/open-io

San Francisco

FRANCE

USA

Tokyo

JAPAN

Member of Partners Customers

WE LOVE OPEN SOURCE

• AGPL V3 for core services

• LGPL V3 for libraries andclient APIs

LICENSES

github.com/open-io

WHAT

Grid for apps

ConscienceDirectory with indirections

1. 2. 4.

Grid of nodes

3.

Unique design

…

……

Namespace

Account

Container

Object

Multi tenancy

Flat structure

Track containers, not objects

Directory with indirections1.UNIQUE DESIGN

containers objects

Distributed Hash Tables Consistent Hashing

Single name node

• Good for trillion of objects

• Bad because of rebalancing data when scaling

• Good for few large files

• Bad for numerous small objects

Directory with indirections1.

• Good for trillion of objects

• Good, no need to rebalance data when scaling

Indirection table

Existing technologies OpenIO

UNIQUE DESIGN

Grid of nodes2.UNIQUE DESIGN

• Share nothing architecture

• Scale-out design with no contention

• New nodes are auto discovered

• No gateway in native object mode

• No SPOF, resilient to node failures

1. Collects metrics from the services of each node

2. Computes a score for each service

3. Distributes scores to every nodes and clients

4. On the fly best match making for each request

Conscience3.UNIQUE DESIGN

Realtime load balancing for optimal data placement

The score is computed with a configurable formula usually based on: capacity, io performance, CPU

Storage

CPU/RAMusage

Over the years

> Don’t waste servers Use their free CPU & RAM!

Grid for Apps4.UNIQUE DESIGN

Scality Ceph OpenIO

LICENSE Proprietary Open Source Open Source

ARCHITECTURE RING Crush maps 3-level distributed directory

CONTAINERS External DB RADOS GW only Native with strong consistency

SCALING rebalance rebalanceNo rebalance,immediate availability

LOAD BALANCING

Static, consistent hashing based

Static, consistent hashing based Real time, automatic

Competitors

Hardwareagnostic

0TB > 1000+PB

1. 2.

Never Rebalance

3.

Different

• Simple to use

• Low-latency and linear performance

• Never rebalance when scaling

• Immediate benefits in terms of capacity & performance

• Flexible increments

“True scalability”

0TB > 1000+PB1.DIFFERENT

Consistent hashing

Each node owns a part of the key space

New nodes get parts of the key space from the older ones

Never rebalance2.DIFFERENT

Recalculate key space

Scaling

OpenIO

New nodes automatically discovered and immediately available

Scaling

Immediate benefits in terms of performance from the new nodes

Never rebalance2.DIFFERENT

• Work with heterogeneous hardware

• No vendor lockin

Hardware agnostic3.DIFFERENT

• Hardware is used according to its capacity and performance

• Scores are also based on I/O figures and CPU, not only capacity

Conscience

Interoperability

1. 2. 3.

ConnectorsOpen Source Native APIs

>_Amazon S3® OpenStack

Swift® Optimized native APIs

• Python • Go • C • Java

Command line interface

Native APIs2.INTEROPERABILITY

Storage Email Video

Connectors3.INTEROPERABILITY

• File System (NFS)

• OpenStack SWIFT

• Amazon S3

• Cyrus 3.0

• Zimbra

• Mail Object

• Dovecot

• HTTP connector

• Adaptive streaming connector

• Event based transcoding

Hardware agnostic, True scalability 0 > 1000+PBs

Storage-tier, Conscience auto-optimization, Grid for Apps

Open Source, S3 and SWIFT APIs, Specific App connectors

Human free management, Simple to deploy

Low-latency design, Specific App connectors

Grid of nodes, Replication, Erasure Coding, Geodistribution

Multitenancy, silent corruption checks, WORM for compliance and encryption

Scalability Efficiency Interoperability Ease of management Performance Resiliency Security

Gartner, Critical Capabilities for Object Storage

HOW

Time >

PB >

Deployed Used ISP 40+ M mailboxes

Real life use caseREFERENCES

OIO Core

Standard Support 24/7

Premium Support 24/7

ProfessionalServices

MailConnectors

VideoConnectors

FileConnectors

WebUI

Open SourceSubscriptionService

New release scheme & new release 15.12

• One release for our projects

• Fuse & NFS file connector

• First LTS (long term support) release

• Multiple OpenIO core optimizations

• New python-based CLI

#1

File connector (Fuse & NFS)

• To ease adoption by the market

• Ability to « mount » an account as a filesystem

• Targets performance - close to our core solution

• Files will also appear as read-only objects under the traditional object APIs

#2

Philippe Nicolas

#3

As advisor

Welcome aboard! :-)

OpenIO to quickly become a major player

info@openio.io