How to scale to 100k users using Windows Azure

Ionut Antiu

www.get-jukebox.com

@johnnyantiu

1. What are we talking about

2. What is Style Jukebox

3. What were the challenges

4. Wrap up

Style Jukebox is a High Fidelity cloud-music streaming service that helps you keep all your music – be it bought, self-produced,

or imported from CDs or other media, always backed up and synced across all your devices.

TRACTION

125,000+ registered users

52,000+ MAU

Peak of 13,000 users DAU

4,5+ million songs stored in the Cloud

50,000 – 100,000 songs listened daily

Low latency

Scalability

High Availability

OBJECTIVES

What is the basic Cloud architecture?

“LE SERVER”

Database SQL Storage

API

LE CLIENT APP LE CLIENT APP LE CLIENT APP

“LE SERVER”

API

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

Scale Up Scale Out

Database, API, Content bandwidth, Website

Scale Up

One operation that cannot be divided requires more compute power

(Audio/Video encoding, Your own DB, etc.)

Basic A11 Core AMD CPU1,75 GB RAMEUR 41/month*

G5, 32 coresIntel Xeon448 GB RAM6 TB EUR 4865/month*

Scale Out

Used for API, Website, etc.

Ex: Each machine can have a limited number of connected users

Basic A11 Core AMD CPU1,75 GB RAMEUR 41/month*

4xBasic A11 Core AMD CPU1,75 GB RAMEUR 41/month*

SYNC SERVICE

API (GetSongURL, Login)

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

BLOCK SERVICE

API (DownloadSong, UploadSong)

SYNC SERVICE

API (GetSongURL, Login)

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

BLOCK SERVICE

API (DownloadSong, UploadSong)

MEMCACHE

Redis

Memcache Worker

The Memcache Worker predicts and prepares the results

Memcache Hit Ratio 86-91%

Reduced time of response with 80% to an average of 250ms

SYNC SERVICE

API (GetSongURL, Login)

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

BLOCK SERVICE

API (DownloadSong, UploadSong)

MEMCACHE

Redis

Memcache Worker

CDN

Windows Azure CDN

CDN Worker

SYNC SERVICE

API (GetSongURL, Login)

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

BLOCK SERVICE

API (DownloadSong, UploadSong)

MEMCACHE

Redis

Memcache Worker

CDN

Windows Azure CDN

CDN Worker

STORAGE

Upload Buffer

Upload Processing

SYNC SERVICE

API (GetSongURL, Login)

LOAD BALANCER

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

BLOCK SERVICE

API (DownloadSong, UploadSong)

LOAD BALANCER

MEMCACHE

Redis

Memcache Worker

CDN

Windows Azure CDN

CDN Worker

STORAGE

Upload Buffer

Upload Processing

Download Processing

Load Balancer

Integrated in the Azure Cloud Services

Makes sure all instances have the same loadAuto-Scale creates new instances or removes automatically

SYNC SERVICE

API (GetSongURL, Login)

LOAD BALANCER

LE CLIENT APP LE CLIENT APP LE CLIENT APP

STORAGE

Windows Azure Storage Blobs

DATABASE

SQL Azure

BLOCK SERVICE

API (DownloadSong, UploadSong)

LOAD BALANCER

MEMCACHE

Redis

Memcache Worker

CDN

Windows Azure CDN

CDN Worker

STORAGE

Upload Buffer

Upload Processing

Download Processing

TRAFFIC MANAGER

SYNC SERVICE – WEST US

API (GetSongURL, Login)

DATABASE - AMSTERDAM

SQL Azure

SYNC SERVICE – WEST US

API (GetSongURL, Login)

DATABASE - AMSTERDAM

SQL Azure

~ 9,000 km * 3 calls = ~ 30,000 km

30,000 km : 300,000km/s = 100 ms

LE CLIENT APP LE CLIENT APP LE CLIENT APP

TRAFFIC MANAGER

SYNC – WEST US

API x VMs

LOAD BALANCER

BLOCK – WEST US

API x VMs

LOAD BALANCER

SYNC – WEST EU

API x VMs

LOAD BALANCER

BLOCK – WEST EU

API x VMs

LOAD BALANCER

CDN

Windows Azure CDN

STORAGE

Windows Azure Storage Blobs

MAIN DATABASE – WEST EU

SQL Azure

MEMCACHE – WEST EU

Redis

SLAVE DATABASE – WEST US

SQL Azure

MEMCACHE 2 – WEST US

Redis

STORAGE

Upload Buffer

Traffic Manager

stylejukebox.trafficmanager.net

4 VM machinesin West Europe

4 VM machinesin West US

Closest location to the userLoad balancing

Fail-overWorks for Websites, Cloud Services

Azure Datacenters

How to test?

Test 1

1,000 concurrent users1 core machine

Test 2

1,000 concurrent users4 core machine

$ 0,4 more per hour

Autoscale applies to Cloud Services and Websites

1 core machine can handle 200 connections simultaneous

4 core machine can handle 1,000 connections simultaneous

Monitoring

NewRelic

Always prepare for things to go wrong

Ex: Routers on the Internet

Low latency

Scalability

High Availability

OBJECTIVES

Ionut Antiu

www.get-jukebox.com

@johnnyantiu