Heroku Apps Scale-out, performance monitoring・visualization・ management

Herokuアプリスケールアウト構成、

性能測定・可視化、性能改善法

Marat Zhanikeev [email protected]

maratishe.github.io

2015/04/11

Cloudy＠tenjin

Heroku Apps Scale-out, performance

monitoring・visualization・management

.

maratishe.github.io

Today we...

1. review what scale -up -down -OUT are

2. review heroku design and features3. discuss cloud and specifically heroku populations4. run a measurement experiment and find out that heroku lives in theVirginia DC

5. formulate the heroku bottle model to explain experimental results

M.Zhanikeev -- [email protected] Herokuアプリ: スケールアウト構成、性能測定・可視化、性能改善法 2/37...

2/37

.

maratishe.github.io

Scale-?


3/37

.

maratishe.github.io

Scalability, scale -up -down -OUT

.Scalability..

.

...ability of a system to increase or decrease scale drasticallywithout much change in complexity

• (poor but still an) example: multi-thread programming• scale-up: interprocess communication over the network• scale-down: multicore parallel processing• ... heroku needed a new term -- the scale-out concept


4/37

.

maratishe.github.io

Scale-OUT

.scale-out feature..

.

... represents a design in which the same item (source code) is replicatedand forms a flat (1-level) population

• basically, zero or very low complexity

• complexity does not change because population is flat• several active replicas of the same code is a kind of parallel programming


5/37

.

maratishe.github.io

heroku basics


6/37

.

maratishe.github.io

heroku : your code is a webapp

• feature 1: heroku apps are already webapps

• written, built, and ran as web apps

• so, heroku environment is very web-like◦ even for non-web code in the background


7/37

.

maratishe.github.io

heroku : scaling• feature 2: scaling is easy• both from web dashboard and heroku CLI


8/37

.

maratishe.github.io

heroku : scaling (2)• for example, setting scale to 15 literaly means that there are 15 replicas of mycode running in parallel

• this is an academic account, one would normally get a huge amount in the bill


9/37

.

maratishe.github.io

heroku : scale is money


10/37

.

maratishe.github.io

heroku : scale-out basics

• when you increase the scale, you getparallel replicas of your code

• your code can be web or workerdynos◦ dyno = process = container

• which one gets requests? heroku usescedar stack, which selects webdynos randomly


11/37

.

maratishe.github.io

heroku : front and back

• feature 3 (or a bug?): webs andworkers cannot communicate directly

• webs get an incoming port forrequests, workers have no ports

• the only solution is to use 3rd partyspace to communicate

• again, parallel processing withlocking, etc., is applicable


12/37

.

maratishe.github.io

heroku deploy is by git (feature 4)


13/37

.

maratishe.github.io

heroku performance monitoring


14/37

.

maratishe.github.io

heroku evolution

• not much monitoring until the late 2014 version• in 2014 version, some 3rd party tools became part of core functionality =part of dashboard


15/37

.

maratishe.github.io

heroku performance metrics


16/37

.

maratishe.github.io

heroku performance metrics (2)


17/37

.

maratishe.github.io

heroku performance metrics (3)


18/37

.

maratishe.github.io

what's missing?

• nothing really, basically all metrics are covered

• ... maybe no visualization for your entire populations (of dynos)


19/37

.

maratishe.github.io

heroku populations


20/37

.

maratishe.github.io

heroku populations

• heroku requests are routed by heroku nodes (bottlenecks?)• your app to addons: inner? in-DC? depends on add-on?• your app to users: outer!• your app to 3rd party services: outer!


21/37

.

maratishe.github.io

heroku : outer vs inner


22/37

.

maratishe.github.io

populations : VM version, streaming)

VMpopulation

CurrentSources

ServiceProvider

(SP)

Tracker

ServiceProvider

(SP)

Parentpeers

P2Pstreaming

Cloudstreaming

ClientClient


23/37

.

maratishe.github.io

populations : docker exampel

• Docker is a DiY platform, youhave to do everything by yourself

• proper population managementtechnology is in demand

• can include geographicaldistribution, topology control,etc.


24/37

.

maratishe.github.io

are heroku dynos populations?

• any difference in performance across dynos?• where are the bottlenecks?• can one measure? visualize for entire dyno populations?


25/37

.

maratishe.github.io

heroku probing experiment


26/37

.

maratishe.github.io

heroku probing experiment

• need outside of heroku : EC2◦ might be poor choice because heroku itself is based entirely on EC2

• send probes from various EC2 regions to visualize regional diversity• need to try out various scales and gradual scaling

◦ gradual scaling is so that dynos would be mapped to different VMs

• the probing is when EC2 request and download some bulk from a heroku app


27/37

.

maratishe.github.io

heroku probing : EC2 + heroku• we can only hope that dynos are mapped to different VMs/PMs


28/37

.

maratishe.github.io

heroku probing (2) 1 run


29/37

.

maratishe.github.io

results

0

10

20

30

40

Thru

1k50

0k 1M 2M 3M

california.1b/1 california.1b/2 california.1b/3 california.1b/4 california.1b/5 california.1b/6 california.1b/7 california.1b/8 california.1b/9 california.1b/10 california.1b/11 california.1b/12 california.1b/13 california.1b/14 california.1b/15

frankfurt.1a/1 frankfurt.1a/2 frankfurt.1a/3 frankfurt.1a/4 frankfurt.1a/5 frankfurt.1a/6 frankfurt.1a/7 frankfurt.1a/8 frankfurt.1a/9 frankfurt.1a/10 frankfurt.1a/11 frankfurt.1a/12 frankfurt.1a/13 frankfurt.1a/14 frankfurt.1a/15

ireland.1a/1 ireland.1a/2 ireland.1a/3 ireland.1a/4 ireland.1a/5 ireland.1a/6 ireland.1a/7 ireland.1a/8 ireland.1a/9 ireland.1a/10 ireland.1a/11 ireland.1a/12 ireland.1a/13 ireland.1a/14 ireland.1a/15

oregon.2b/1 oregon.2b/2 oregon.2b/3 oregon.2b/4 oregon.2b/5 oregon.2b/6 oregon.2b/7 oregon.2b/8 oregon.2b/9 oregon.2b/10 oregon.2b/11 oregon.2b/12 oregon.2b/13 oregon.2b/14 oregon.2b/15

saopaulo.1a/1 saopaulo.1a/2 saopaulo.1a/3 saopaulo.1a/4 saopaulo.1a/5 saopaulo.1a/6 saopaulo.1a/7 saopaulo.1a/8 saopaulo.1a/9 saopaulo.1a/10 saopaulo.1a/11 saopaulo.1a/12 saopaulo.1a/13 saopaulo.1a/14 saopaulo.1a/15

singapore.1a/1 singapore.1a/2 singapore.1a/3 singapore.1a/4 singapore.1a/5 singapore.1a/6 singapore.1a/7 singapore.1a/8 singapore.1a/9 singapore.1a/10 singapore.1a/11 singapore.1a/12 singapore.1a/13 singapore.1a/14 singapore.1a/15

sydney.2b/1 sydney.2b/2 sydney.2b/3 sydney.2b/4 sydney.2b/5 sydney.2b/6 sydney.2b/7 sydney.2b/8 sydney.2b/9 sydney.2b/10 sydney.2b/11 sydney.2b/12 sydney.2b/13 sydney.2b/14 sydney.2b/15

tokyo.1c/1 tokyo.1c/2 tokyo.1c/3 tokyo.1c/4 tokyo.1c/5 tokyo.1c/6 tokyo.1c/7 tokyo.1c/8 tokyo.1c/9 tokyo.1c/10 tokyo.1c/11 tokyo.1c/12 tokyo.1c/13 tokyo.1c/14 tokyo.1c/15

virginia.1e/1 virginia.1e/2 virginia.1e/3 virginia.1e/4 virginia.1e/5 virginia.1e/6 virginia.1e/7 virginia.1e/8 virginia.1e/9 virginia.1e/10 virginia.1e/11 virginia.1e/12 virginia.1e/13 virginia.1e/14 virginia.1e/15

• figure is vector, so,zoom-in

• rows: DCs, columns:dynos

• x: bulk size, y:throughput

• does any DCstand out?


30/37

.

maratishe.github.io

results (without virginia)

0

4

8

12

Thru

1k50

0k 1M 2M 3Mcalifornia.1b/1 california.1b/2 california.1b/3 california.1b/4 california.1b/5 california.1b/6 california.1b/7 california.1b/8 california.1b/9 california.1b/10 california.1b/11 california.1b/12 california.1b/13 california.1b/14 california.1b/15

frankfurt.1a/1 frankfurt.1a/2 frankfurt.1a/3 frankfurt.1a/4 frankfurt.1a/5 frankfurt.1a/6 frankfurt.1a/7 frankfurt.1a/8 frankfurt.1a/9 frankfurt.1a/10 frankfurt.1a/11 frankfurt.1a/12 frankfurt.1a/13 frankfurt.1a/14 frankfurt.1a/15

ireland.1a/1 ireland.1a/2 ireland.1a/3 ireland.1a/4 ireland.1a/5 ireland.1a/6 ireland.1a/7 ireland.1a/8 ireland.1a/9 ireland.1a/10 ireland.1a/11 ireland.1a/12 ireland.1a/13 ireland.1a/14 ireland.1a/15

oregon.2b/1 oregon.2b/2 oregon.2b/3 oregon.2b/4 oregon.2b/5 oregon.2b/6 oregon.2b/7 oregon.2b/8 oregon.2b/9 oregon.2b/10 oregon.2b/11 oregon.2b/12 oregon.2b/13 oregon.2b/14 oregon.2b/15

saopaulo.1a/1 saopaulo.1a/2 saopaulo.1a/3 saopaulo.1a/4 saopaulo.1a/5 saopaulo.1a/6 saopaulo.1a/7 saopaulo.1a/8 saopaulo.1a/9 saopaulo.1a/10 saopaulo.1a/11 saopaulo.1a/12 saopaulo.1a/13 saopaulo.1a/14 saopaulo.1a/15

singapore.1a/1 singapore.1a/2 singapore.1a/3 singapore.1a/4 singapore.1a/5 singapore.1a/6 singapore.1a/7 singapore.1a/8 singapore.1a/9 singapore.1a/10 singapore.1a/11 singapore.1a/12 singapore.1a/13 singapore.1a/14 singapore.1a/15

sydney.2b/1 sydney.2b/2 sydney.2b/3 sydney.2b/4 sydney.2b/5 sydney.2b/6 sydney.2b/7 sydney.2b/8 sydney.2b/9 sydney.2b/10 sydney.2b/11 sydney.2b/12 sydney.2b/13 sydney.2b/14 sydney.2b/15

tokyo.1c/1 tokyo.1c/2 tokyo.1c/3 tokyo.1c/4 tokyo.1c/5 tokyo.1c/6 tokyo.1c/7 tokyo.1c/8 tokyo.1c/9 tokyo.1c/10 tokyo.1c/11 tokyo.1c/12 tokyo.1c/13 tokyo.1c/14 tokyo.1c/15

• 2 groups of DCs?

• absolutethroughput is 4times smaller thanwithin virginia


31/37

.

maratishe.github.io

heroku : the bottle model


32/37

.

maratishe.github.io

heroku : the bottle model


33/37

.

maratishe.github.io

intRA-DC throughput (pressure)

california

ireland

oregon

saopaulo

singapore

sydneytokyo virginia

key (probe)sizes (2000 5000)parties (aa)

• visualization from earlier Cloudy oncloud probing

• in-DC throughput wildly differentacross DCs

• but heroku (in US) leaves in Virginia, soirrelevant?

• ... still, traffic inside DCs is under somepressure


34/37

.

maratishe.github.io

intER-DC throughput (pressure)

california

ireland

oregon

saopaulo

singapore

sydney

tokyo

virginia

key (probe)sizes (2000 5000)parties (ab)

• average across DCs is mostly thesame for all DCs◦ all-to-all has smoothing effect

• pullees show the slowestremote party for a given DC, there issome difference here

• this is applicable to heroku because thistraffic goes through the bottleneck


35/37

.

maratishe.github.io

The Big Questions

1. How strong the effect of bottlenck compared to:◦ in-DC and inter-DC traffic flows◦ note that user traffic always goes through the bottleneck

2. If one uses heroku addons, is the related traffic in-DC? -- yet tomeasure

3. When using addons, what is the reduction factor so that final replywould fit through the bottleneck?◦ experiment showed factor of 4◦ the 30s reply deadline in heroku (feature/bug 5) is related to this

4. Note the huge latency = smallest throughput for small bulk --

interactivity apps on heroku?

5. ...


36/37

.

maratishe.github.io

That’s all, thank you ...


37/37

Technology

Heroku Apps Scale-out, performance monitoring・visualization・ management