Data storages in Microsoft Azure

Andriy DerenCEO, Onlizer/Dreams ITa.deren@onlizer.com

#dfua

Agenda

• Why cloud storages? • Cheap multipurpose NoSQL cloud storage (Azure Cloud Storage)• Cloud-based relational storage (MS SQL Azure)• Document-oriented storage in cloud (Azure DocumentDb)• Fast and furious in cloud (Azure Redis Cache)

Pros/Cons, Usage CasesWhy cloud storages?

#dfua

Solution search

• When engineer has hummer in hands all problems looks like nails• When engineer has screwdriver in hands all problems looks like screws

Use wrong tool

Use tool wrong way

We are trying to topple a tree with a

sledgehammerWe are trying to

topple a tree with a saw

#dfua

First conclusions

• Universal solution - a solution that is equally bad for all tasks.• Specialized solution - well-suited to address only specific tasks.• Best the enemy of the good.

#dfua

So when cloud storages are useful?

• On and off (short-time and periodical load)• Fast grow• Unpredictable load• Predictable load• Geo-redundant access• Real-time sync and near 100% transactions success

#dfua

General Pros & Cons

Pros• SaaS & PaaS models – lower developer responsibility• Easier management and scale• Transparent work & API models• Predictable cost

Cons• Lower control• Public cloud• Cost

#dfua

Storages in Microsoft AzureSQL VM SQL Azure Document DB Table Storage Blob Storage

Relational DBMS

Reach query syntax

Transactions support

Service model

Elastic scale

Schema less

Access via HTTP(S)

Any data formats

Cheap multipurpose NoSQL storageAzure Storage

#dfua

Azure Storage• Exposed via RESTful web services• Scalable, durable, available globally• Only pay for what the service uses

#dfua

Storage SDKs on many languages

C#/.NET Python Ruby

Perl Node.JS Java

PHP Erlang Common LISP

Objective-C

#dfua

Storage security

• HTTP or HTTPS• Two independent 512bit symmetric master keys• Shared access signatures for more granular access

#dfua

Storage artifacts

Blobs Drives Queues

Tables Files

#dfua

Blob storage concepthttps://contoso.blob.core.windows.net/image/pic01.jpg

Block/Page

Blob

Container

Account contoso

image

pic01.jpg

block01 block02

pic02.jpg

video

mov01.avi

page01 page02

#dfua

Blob storage operations

• Get• Put• Delete• Copy• Snapshot• Lease

#dfua

Block blobsblobName = “TheBlob.wmv”;PutBlock(blobName, blockId1, block1Bits);PutBlock(blobName, blockId2, block2Bits);…………PutBlock(blobName, blockIdN, blockNBits);PutBlockList(blobName,

blockId1,…,blockIdN);

TheBlob.wmv

Bloc

k Id

1

Bloc

k Id

2

Bloc

k Id

3

Bloc

k Id

N

BenefitEfficient continuation and retryParallel and out of order upload of blocks

#dfua

Page blobs0

10 GB

512

1024

1536

2048

2560

10 GB Address Space

#dfua

Table storage concept

Entity

Table

Account contoso

users

FName=Bob

LName=Smith

FName=Inga

Phone=123

groups

Name=Admin

#dfua

Entity details• Entity can have up to 255 properties–Up to 1MB per entity

• Mandatory Properties for every entity–PartitionKey & RowKey (only indexed properties)•Uniquely identifies an entity•Defines the sort order

–Timestamp •Optimistic Concurrency•Exposed as an HTTP Etag

• No fixed schema for other properties–Each property is stored as a <name, typed value> pair–No schema stored for a table–Properties can be the standard .NET types: String, binary, bool, DateTime, GUID, int and double

#dfua

Entity details• Entity Locality–Entities in the same partition will be stored together–Efficient querying and cache locality–Endeavour to include partition key in all queries

• Entity Group Transactions–Atomic multiple Insert/Update/Delete in the same partition in a single transaction

• Table Scalability–Target throughput – 2 000 tps/partition, 20 000 tps/account–Azure monitors usage patterns of partitions–Automatically load balance partitions–Each partition can be served by a different storage node–Scale to meet traffic needs of your table

#dfua

Data operations

• Create• Query• Delete

Table operations Entity operations• Insert• Update• Merge• Replace• Upsert• Query• Delete

Cloud-based relational storageMicrosoft SQL Azure

#dfua

Relational DBMS as a ServiceService

Resources management

Routing

Platform

Infrastructure

Billing & monitoring

SQL Server

Database Management

General Management

#dfua

SQL Azure Scale

Document-oriented storage in cloudAzure DocumentDb

#dfua

Azure DocumentDb

NoSQL database solution designed “from scratch” for better JavaScript and

JSON support on data-engine level.

#dfua

Azure DocumentDb

Transactions support Service model Schema less

Elastic scale Access via HTTP(S) Rich query syntax

#dfua

Azure DocumentDb Conception

#dfua

Azure DocumentDb access via HTTP

POST https://<REST URI>/docs HTTP/1.1...x-ms-documentdb-isquery: TrueContent-Type: application/query+json

{ "query": "SELECT * FROM Families f WHERE f.id = @familyId", "parameters": [ {"name": "@familyId", "value": "AndersenFamily"} ]}

Fast and furious in cloudAzure Redis Cache Service

#dfua

Azure Redis Cache

• Simple but powerful Key/Value storage• Service model• Easy scale to cluster• Monitoring and metrics• Web-based CLI• Persistent save (Premium)

Thank you!Questions?

@dralligierihttps://www.facebook.com/dalligieriwww.onlizer.com