Ciel universal distributed execution engine

. . . . . .

MotivationCIEL

SkywritingOptimizations & Fault Tolerance

Evaluation & Future work

.

.

. ..

.

.

CIEL universal distributed execution engine

Presenter: Emmanouil Dimogerontakis@{AdvDS}

EMDC KTH

November 6, 2012

Presenter: Emmanouil Dimogerontakis @{AdvDS} CIEL universal distributed execution engine 1/23

. . . . . .

MotivationCIEL



Distributed Execution Engines

.. .1 Motivation


.. .2 CIEL

Dynamic Task GraphsArchitecture

.. .3 Skywriting

.. .4 Optimizations & Fault Tolerance

.. .5 Evaluation & Future work

EvaluationFuture WorkConclusions


. . . . . .

MotivationCIEL




.. Purpose

Execute a Task Graph providing:

Task Scheduling

Data Distribution

Load Balancing

Transparent FaultTolerance

Figure: Task Graph


. . . . . .

MotivationCIEL




.. Purpose


Task Scheduling

Data Distribution

Load Balancing


Figure: Task Graph


. . . . . .

MotivationCIEL




.. Purpose


Task Scheduling

Data Distribution

Load Balancing


Figure: Task Graph


. . . . . .

MotivationCIEL




.. Purpose


Task Scheduling

Data Distribution

Load Balancing


Figure: Task Graph


. . . . . .

MotivationCIEL




.. Limitations

Task graphs used up to now:

Static

Acyclic

Limitations :

Limited Expressive Power

Poor Performance

Insufficient Fault Tolerance


. . . . . .

MotivationCIEL




.. Limitations

Task graphs used up to now:

Static

Acyclic

Limitations :

Limited Expressive Power

Poor Performance

Insufficient Fault Tolerance


. . . . . .

MotivationCIEL




.. Overview

Figure: Distributed Execution Engines comparison


. . . . . .

MotivationCIEL




.. .1 Motivation


.. .2 CIEL


.. .3 Skywriting





. . . . . .

MotivationCIEL




.. CIEL

WHY Universal?

Support same cluster of algorithms as a TM

HOW ?

using Dynamic Task Graphs


. . . . . .

MotivationCIEL




.. CIEL

WHY Universal?

Support same cluster of algorithms as a TM

HOW ?

using Dynamic Task Graphs


. . . . . .

MotivationCIEL




.. CIEL primitives

objects

references

tasks

Figure: A Task Grapha

aSource: http://www.cl.cam.ac.uk/~dgm36/CIEL-NSDI-slides.pdf


http://www.cl.cam.ac.uk/~dgm36/CIEL-NSDI-slides.pdf

. . . . . .

MotivationCIEL




.. Dynamic Task Graphs

Figure: A Dynamic Task Graph


. . . . . .

MotivationCIEL




.. Master & Worker

Figure: CIEL Master

Figure: CIEL Worker


. . . . . .

MotivationCIEL




.. Master & Worker

Figure: CIEL Master Figure: CIEL Worker


. . . . . .

MotivationCIEL




.. Architecture

Figure: CIEL Architecture


. . . . . .

MotivationCIEL



.. .1 Motivation


.. .2 CIEL


.. .3 Skywriting





. . . . . .

MotivationCIEL



.. Creating Tasks with Skywriting

Figure: Spawning a new task

Figure: Dereferencing futures


. . . . . .

MotivationCIEL



.. Creating Tasks with Skywriting

Figure: Spawning a new task Figure: Dereferencing futures


. . . . . .

MotivationCIEL



.. .1 Motivation


.. .2 CIEL


.. .3 Skywriting





. . . . . .

MotivationCIEL



.. Optimizations

Globally unique identifiers enable memoization

Streaming partially written objects between tasks


. . . . . .

MotivationCIEL



.. Fault Tolerance

Client (no driver program)

Worker (periodic heartbeat)

Master (persistent logging, secondary masters, object tablereconstruction)


. . . . . .

MotivationCIEL




.. .1 Motivation


.. .2 CIEL


.. .3 Skywriting





. . . . . .

MotivationCIEL




.. Performance Comparison with production system

Figure: DistrubutedGrep on Hadoop and Ciel


. . . . . .

MotivationCIEL




.. Perfomance of Iterative Algorithm

Figure: K-means on Hadoop and Ciel with 20 workers


. . . . . .

MotivationCIEL




.. Overheads

Figure: Speedup of Binomial Options Pricing Model on 47 workers


. . . . . .

MotivationCIEL




.. Future Work

Integrate CIEL with existing programming languages

Partition master state

Explore use of multiple cores (see [5])

Explore use of non-deterministic parallelism (see [3])


. . . . . .

MotivationCIEL




.. Conclusions

CIEL[4, 1] and Skywriting[2]

are not good for:

sharing large amounts ofdata

fine-grain parallelization

fully automatic parallelism

relation algebra environment

distributed operating system

are really good for :

writing iterative algorithms

data-dependent control flowusing dynamic task graphs

transparent fault toleranceand automatic distribution

scaling across hundreds ofmachines

Questions ?


. . . . . .

MotivationCIEL




.. Conclusions


are not good for:











Questions ?


. . . . . .

MotivationCIEL




.. Conclusions


are not good for:











Questions ?


. . . . . .

MotivationCIEL




.

.

[1] D.G. Murray.

A distributed execution engine supporting data-dependent control flow.

PhD thesis, PhD thesis, Univ. of Cambridge, 2011.

[2] D.G. Murray and S. Hand.

Scripting the cloud with skywriting.

In Proceedings of the 2nd USENIX conference on Hot topics in cloudcomputing, pages 12–12. USENIX Association, 2010.

[3] D.G. Murray and S. Hand.

Non-deterministic parallelism considered useful.

In HotOS XIII, 13th Workshop on Hot Topics in Operating Systems,2011.

[4] D.G. Murray, M. Schwarzkopf, C. Smowton, S. Smith, A. Madhavapeddy,and S. Hand.

Ciel: a universal execution engine for distributed data-flow computing.

In Proceedings of the 8th USENIX conference on Networked systemsdesign and implementation, page 9. USENIX Association, 2011.

[5] M. Schwarzkopf, D.G. Murray, and S. Hand.

Condensing the cloud: running ciel on many-core.

Proceedings of EuroSys SFMA, 2011.


. . . . . .

CIELSkywriting

Experiments

Part I.

.

. ..

.

.

Appendix


. . . . . .

CIELSkywriting

Experiments

.. .6 CIEL

.. .7 Skywriting

.. .8 Experiments


. . . . . .

CIELSkywriting

Experiments

.. Hidden slide 1

Figure: Task and Object table maintained in Master node


. . . . . .

CIELSkywriting

Experiments

.. .6 CIEL

.. .7 Skywriting

.. .8 Experiments


. . . . . .

CIELSkywriting

Experiments

.. Hidden slide 2

Figure: Spawning Tasks1

1Source: http://www.cl.cam.ac.uk/~dgm36/CIEL-NSDI-slides.pdf



. . . . . .

CIELSkywriting

Experiments

.. Hidden slide 3

Figure: Blocking on futures2

2Source: http://www.cl.cam.ac.uk/~dgm36/CIEL-NSDI-slides.pdf



. . . . . .

CIELSkywriting

Experiments

.. .6 CIEL

.. .7 Skywriting

.. .8 Experiments


. . . . . .

CIELSkywriting

Experiments

.. Hidden slide 4

Figure: Primary Master Failure


Technology

Ciel universal distributed execution engine