EventWave: Programming Model and Runtime Support...

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

EventWave: Programming Model and Runtime Support

forTightly-Coupled Elastic Cloud Applications

Wei-Chiu Chuang, Bo Sang, Sunghwan Yoo, Rui Gu, Charles Killian, Milind Kulkarni

1

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Motivation

2

clients

server

world

building building

room

room

room

room

TimeResponse time

# clients

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Motivation

3

Scale up

TimeResponsetime

clients

server

# clients

Elasticity is hard

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Objectives

4

A programming model which supports:

Stateful computation

Simple sequential semantics

Transparent elasticity

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Related Work

5

Data Flow stateless

Live Migration

Live Migration of Virtual Machines[Clark et. al. NSDI ‘05]

Zephyr [Elmore et. al. SIGMOD ‘11]

Does not change scale:“split”/”merge” state

Scalable programming model

Orleans [Bykov et. al. SoCC ‘11]Transactional, reconcile conflicts

MapReduce [Dean et. al. OSDI ‘04]

Dryad [Isard et. al. EuroSys ‘07]

CIEL [Murray et. al. NSDI ‘11]

No Stateful Computation

No transparent elasticity

No simple sequential semantics

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Event Driven Systems

7

123

Event 1 commitsEvent 2

commitsEvent 3

commits

Typical event driven systems are not scalable.

Event queueclient

server

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Context

8

world

building building

room

room

room

room

Scalability comes from parallelism Partition program state into `contexts` An event accesses one or more contexts Events accessing disjoint contexts can run in parallel

world

building

room

hallwayContexts enable implicit

parallelism

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

EventWave

9

Event 1 commits

Event 2 commits

Event 3 commits

Event 2 finishes

Event 1 finishes

Event 3 finishes

Context 1

Context 2

Context 3

123

Enforce sequential orderingEvent 2 can not commit until Event

1 commits

Stateful

Sequential semantics

Parallelism

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Access Multiple Contexts

A player can move from one room to another

Remove it from source room

Insert it into destination room

10

world

building building

room

room

room

room

Room 1 Room 2

Alice BobBob

Player list

An event may access multiple contexts

Bob

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Access Multiple Contexts

Must ensure

Sequential semantics

parallelism

Event 1 finishes

Event 2 commits12

10

To be scalable, events can not access contexts arbitrarily

Event 2 can’t start before event 1 finishes

Context 1

Context 2

Context 3

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Hierarchical Contexts

12

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Contexts are not completely independent

The world has many buildings

A building has many rooms

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

13

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Must access contexts from top to bottom

The hierarchical access enables parallelism

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

14

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Move a player from room 1 to room 2

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

15

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Allow the next event to access Building<1>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

16

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Enter Room<1>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

17

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Remove playerRelease

exclusive access

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

18

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Enter Room<2>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

19

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Insert player

Event finishes, releasing all contexts

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

20

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Wave of Events

21

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Event commits, releasing snapshot

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Scale more by executing events across multiple nodes

Map contexts

Head node

22

Distributed Execution

Head nodeworld

Building<1> Building<2>

Room<1>

Room<2>

Room<3>

Room<4>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 23

Distributed Execution

Client Logical Node #1

Client Logical Node #2

Logical Node: a set of physical nodes

Server Logical Node

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 24

Distributed Execution

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 25

Elasticity

world

Building<1> Building<2>

Room<1>

Room<2>

Room<1>

Room<2>

Request nodes from cloud scheduler

Update context mapping

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 26

Elasticity

world

Building<1> Building<2>

Room<1>

Room<2>

Room<1>

Room<2>

Transfer contexts to the new node

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Evaluation

Does it scale?

Microbechmarks

Scalability

What is the cost of migration?

Microbechmarks

Migration latency

Case study

Multi-player game server

28

In the paper

Key-value store

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Microbenchmark-Scalability

Setup

One logical node, fixed context mapping

EC2 Small Instances

1 vCPU, 1.7GB RAM, 160 GB local disk

Distribute 160 contexts to physical nodes

Measures

Throughput

29

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Microbenchmark-Scalability

30

Takeaway:Throughput grows

w.r.t. # of nodes

P: workload

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Microbenchmark-Migration Latency

31

Setup

2 x 8-core 2.0 GHz Xeon, 8GB RAM

1Gb Ethernet connection

Scale does not change

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Microbenchmark-Migration Latency

Measure

Throughput of events

32

The migration event commits

Finished events must wait for migration event

Migrate a 100MB context

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Multi-player Game Server

33

Setup

Server logical node

1 x Extra Large Instance (head)

64 x Small Instances

Client logical nodes

128 clients on 16 EC2 Small Instances

Measure

Latency

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Multi-player Game Server

34

Server contexts spread to 64 physical nodes

Synthetic workload

Server contexts merge to 1 physical nodes

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Conclusion

Elasticity is crucial for cloud applications.

Our programming model enables transparent elasticity for tightly-coupled applications

Case studies show EventWave is efficient

35

http://www.macesystems.org

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Backups

36

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Language Construct

37

state_variables{Hallway hw;vector<Room> rooms;

}

context Hallway{int x;

}context Room<int>{

int y;}

Declare implicit

parallelism

Mace [Killian et. al. PLDI ‘07]

HallwayRoom[0]Room[1]…

Hallway

Room<0>

Room<1>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Event Handler

38

upcall deliver(Message m){

}

upcall [Room<m.roomID>] deliver(Message m){

}

AnnotationSpecify what context to access

Message(roomID = 2)

Context Room<2>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Key-value store

39

Setup

2 x 8-core 2.0 GHz Xeon, 8GB RAM

1Gb Ethernet connection

Measure

Latency

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Key-value store

40

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Microbenchmark-Migration Latency

41

Setup

2 x 8-core 2.0 GHz Xeon, 8GB RAM

1Gb Ethernet connection

Scale does not change

Context

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 42

Context Migration

Head

Old node

New node

Copy context state

Replicate context state

Event 1 goes to the old node

Event 3 goes to the new node

1 M 3

Update context-node mapping