55

IT Press Tour #17 - OpenIO & Technology

  • Upload
    openio

  • View
    522

  • Download
    0

Embed Size (px)

Citation preview

Page 1: IT Press Tour #17 - OpenIO & Technology
Page 2: IT Press Tour #17 - OpenIO & Technology
Page 3: IT Press Tour #17 - OpenIO & Technology
Page 4: IT Press Tour #17 - OpenIO & Technology
Page 5: IT Press Tour #17 - OpenIO & Technology
Page 6: IT Press Tour #17 - OpenIO & Technology
Page 7: IT Press Tour #17 - OpenIO & Technology

88% (2015)

Unstructured dataworldwide ratio

Each year

Facts:

ESG Digital Archive Market Forecast IDGSource:

+62%

Page 8: IT Press Tour #17 - OpenIO & Technology

Scalability + PerformanceChallenge:

Page 9: IT Press Tour #17 - OpenIO & Technology

Object storageSolution: ?

Page 10: IT Press Tour #17 - OpenIO & Technology

Object storage ?

Page 11: IT Press Tour #17 - OpenIO & Technology

Object storage ?

Page 12: IT Press Tour #17 - OpenIO & Technology

Object storage !

Page 13: IT Press Tour #17 - OpenIO & Technology

Object storage

Advantages But not optimized for

Scalability 100s PBs, low TCO

Ease of access Simple API GET/PUT/DELETE

Resiliency Replication, erasure coding,geodistribution

Heavily updated data

No POSIX-like API

• VMs • SQL DBs

Page 14: IT Press Tour #17 - OpenIO & Technology

10+ yearsMarket: 1.2 B$ today 4.6 B$ > 2020

Maturity Fast growing

IDCSource:

Page 15: IT Press Tour #17 - OpenIO & Technology

Gartner, Critical Capabilities for Object Storage

Scalability Efficiency Interoperability Ease of management Performance Resiliency Security

Page 16: IT Press Tour #17 - OpenIO & Technology

Very high capacity Optimized TCO* Unstructured data Price per GB low

Rather low capacity Very high performanceVMs, SQL DBs Price per GB high

Object store Flash Arrays

Vision: 88% 12%

No future for traditional vendors?

*Total Cost of Ownership

Page 17: IT Press Tour #17 - OpenIO & Technology

Performance

Tightly integrated application connectors

Quality of Service

< Vision:

Very high capacity Optimized TCO Unstructured data Price per GB low

88%

Object store

Page 18: IT Press Tour #17 - OpenIO & Technology

Object storage is not only storage. It is a new way to build backend services for applications. As Docker has disrupted sysadmin, we want to disrupt storage management.

Page 19: IT Press Tour #17 - OpenIO & Technology
Page 20: IT Press Tour #17 - OpenIO & Technology

WHO

Page 21: IT Press Tour #17 - OpenIO & Technology

AntiSpam - AntiPhishing - eMail classification 235M MailBoxes in 76 countries

IaaS - PaaS - DaaSNetwork & Security for legacy, SaaS and Web Apps

Lille San Francisco Montréal Hong-Kong Tokyo

Innovative technologiesfor IT infrastructure

100 people

Page 22: IT Press Tour #17 - OpenIO & Technology

7 8 50fellow co-founders

years experience

million end users

16people

2015launch (June)

Page 23: IT Press Tour #17 - OpenIO & Technology

2006

Idea & 1st concept

2007

Designdev starts

2009

1st massiveproductionabove 1PB

Page 24: IT Press Tour #17 - OpenIO & Technology

2006

Idea & 1st concept

2007

Designdev starts

2009

1st massiveproduction above 1PB

2015

OpenIOfork

2012

Opensourced

2014

10+ PBmanaged

Page 25: IT Press Tour #17 - OpenIO & Technology

Lille

@OpenIO

OpenIO

github.com/open-io

San Francisco

FRANCE

USA

Tokyo

JAPAN

Page 26: IT Press Tour #17 - OpenIO & Technology

Member of Partners Customers

Page 27: IT Press Tour #17 - OpenIO & Technology

WE LOVE OPEN SOURCE

Page 28: IT Press Tour #17 - OpenIO & Technology

• AGPL V3 for core services

• LGPL V3 for libraries andclient APIs

LICENSES

github.com/open-io

Page 29: IT Press Tour #17 - OpenIO & Technology

WHAT

Page 30: IT Press Tour #17 - OpenIO & Technology

Grid for apps

ConscienceDirectory with indirections

1. 2. 4.

Grid of nodes

3.

Unique design

Page 31: IT Press Tour #17 - OpenIO & Technology

……

Namespace

Account

Container

Object

Multi tenancy

Flat structure

Track containers, not objects

Directory with indirections1.UNIQUE DESIGN

containers objects

Page 32: IT Press Tour #17 - OpenIO & Technology

Distributed Hash Tables Consistent Hashing

Single name node

• Good for trillion of objects

• Bad because of rebalancing data when scaling

• Good for few large files

• Bad for numerous small objects

Directory with indirections1.

• Good for trillion of objects

• Good, no need to rebalance data when scaling

Indirection table

Existing technologies OpenIO

UNIQUE DESIGN

Page 33: IT Press Tour #17 - OpenIO & Technology

Grid of nodes2.UNIQUE DESIGN

• Share nothing architecture

• Scale-out design with no contention

• New nodes are auto discovered

• No gateway in native object mode

• No SPOF, resilient to node failures

Page 34: IT Press Tour #17 - OpenIO & Technology

1. Collects metrics from the services of each node

2. Computes a score for each service

3. Distributes scores to every nodes and clients

4. On the fly best match making for each request

Conscience3.UNIQUE DESIGN

Realtime load balancing for optimal data placement

The score is computed with a configurable formula usually based on: capacity, io performance, CPU

Page 35: IT Press Tour #17 - OpenIO & Technology

Storage

CPU/RAMusage

Over the years

> Don’t waste servers Use their free CPU & RAM!

Grid for Apps4.UNIQUE DESIGN

Page 36: IT Press Tour #17 - OpenIO & Technology

Scality Ceph OpenIO

LICENSE Proprietary Open Source Open Source

ARCHITECTURE RING Crush maps 3-level distributed directory

CONTAINERS External DB RADOS GW only Native with strong consistency

SCALING rebalance rebalanceNo rebalance,immediate availability

LOAD BALANCING

Static, consistent hashing based

Static, consistent hashing based Real time, automatic

Competitors

Page 37: IT Press Tour #17 - OpenIO & Technology

Hardwareagnostic

0TB > 1000+PB

1. 2.

Never Rebalance

3.

Different

Page 38: IT Press Tour #17 - OpenIO & Technology

• Simple to use

• Low-latency and linear performance

• Never rebalance when scaling

• Immediate benefits in terms of capacity & performance

• Flexible increments

“True scalability”

0TB > 1000+PB1.DIFFERENT

Page 39: IT Press Tour #17 - OpenIO & Technology

Consistent hashing

Each node owns a part of the key space

New nodes get parts of the key space from the older ones

Never rebalance2.DIFFERENT

Recalculate key space

Scaling

Page 40: IT Press Tour #17 - OpenIO & Technology

OpenIO

New nodes automatically discovered and immediately available

Scaling

Immediate benefits in terms of performance from the new nodes

Never rebalance2.DIFFERENT

Page 41: IT Press Tour #17 - OpenIO & Technology

• Work with heterogeneous hardware

• No vendor lockin

Hardware agnostic3.DIFFERENT

• Hardware is used according to its capacity and performance

• Scores are also based on I/O figures and CPU, not only capacity

Conscience

Page 42: IT Press Tour #17 - OpenIO & Technology

Interoperability

1. 2. 3.

ConnectorsOpen Source Native APIs

Page 43: IT Press Tour #17 - OpenIO & Technology

>_Amazon S3® OpenStack

Swift® Optimized native APIs

• Python • Go • C • Java

Command line interface

Native APIs2.INTEROPERABILITY

Page 44: IT Press Tour #17 - OpenIO & Technology

Storage Email Video

Connectors3.INTEROPERABILITY

• File System (NFS)

• OpenStack SWIFT

• Amazon S3

• Cyrus 3.0

• Zimbra

• Mail Object

• Dovecot

• HTTP connector

• Adaptive streaming connector

• Event based transcoding

Page 45: IT Press Tour #17 - OpenIO & Technology

Hardware agnostic, True scalability 0 > 1000+PBs

Storage-tier, Conscience auto-optimization, Grid for Apps

Open Source, S3 and SWIFT APIs, Specific App connectors

Human free management, Simple to deploy

Low-latency design, Specific App connectors

Grid of nodes, Replication, Erasure Coding, Geodistribution

Multitenancy, silent corruption checks, WORM for compliance and encryption

Scalability Efficiency Interoperability Ease of management Performance Resiliency Security

Gartner, Critical Capabilities for Object Storage

Page 46: IT Press Tour #17 - OpenIO & Technology

HOW

Page 47: IT Press Tour #17 - OpenIO & Technology

Time >

PB >

Deployed Used ISP 40+ M mailboxes

Real life use caseREFERENCES

Page 48: IT Press Tour #17 - OpenIO & Technology
Page 49: IT Press Tour #17 - OpenIO & Technology

OIO Core

Standard Support 24/7

Premium Support 24/7

ProfessionalServices

MailConnectors

VideoConnectors

FileConnectors

WebUI

Open SourceSubscriptionService

Page 50: IT Press Tour #17 - OpenIO & Technology
Page 51: IT Press Tour #17 - OpenIO & Technology

New release scheme & new release 15.12

• One release for our projects

• Fuse & NFS file connector

• First LTS (long term support) release

• Multiple OpenIO core optimizations

• New python-based CLI

#1

Page 52: IT Press Tour #17 - OpenIO & Technology

File connector (Fuse & NFS)

• To ease adoption by the market

• Ability to « mount » an account as a filesystem

• Targets performance - close to our core solution

• Files will also appear as read-only objects under the traditional object APIs

#2

Page 53: IT Press Tour #17 - OpenIO & Technology

Philippe Nicolas

#3

As advisor

Welcome aboard! :-)

Page 54: IT Press Tour #17 - OpenIO & Technology

OpenIO to quickly become a major player