Simplifying concurrency: Actors In particular, AKKA
KRISTO IILA, ENGINEERING MANAGER
PLANET OS - MARCH 2016
The Many Ways of Concurrency
• Multithreading / semaphores / locking / mutexes
• Lock-free data structures
• Software Transactional Memory
• Message passing
• Futures / Async
• Actors
PLANET OS - MARCH 2016
What are Actors?
• Lightweight object
• No shared state
• Communication via message passing
• Supervision hierarchies
• Originally implemented in Erlang (high availability telephone switches)
• Don’t need to think in terms of threads and locks
• Distributable by design
PLANET OS - MARCH 2016
An Actor System Implementation - AKKA
• Implements actor model
• Makes remoting seamless
• Works with Java, Scala and .net
• Fits very well with scala
• Automatically scheduled to threads
• Can have multiple thread pools
Actor A Mailbox
Actor B
Message
PLANET OS - MARCH 2016
Messages
• Communication between actors
• Tell: Fire and Forget
• Ask: Tell + Future { reply }
• Guaranteed ordering for one sender and receiver pair
• Access to a mailbox
• Messages can be anything
// Tell exampleactorRef ! Notify("hi!")
// Ask exampleval future = qaActor ? Question(“What?”)
// Process resultfuture.onComplete { … }
// Send result to actorfuture pipeTo actorRef2
PLANET OS - MARCH 2016
Futures
• Future is a way to
Future { socket.read() }
retrieve results …
of concurrent operations …
asynchronously or synchronously.
• Examples
Eg code block:
Or actor ask: qaActor ? Question(“What is the meaning of life?”)
• Futures are composable
PLANET OS - MARCH 2016
Actor Creation
• Each actor
val qaActor = system.actorOf( Props[QuestionAnswerer], “Oracle”)
... is an instance of a class with parameters
… has a name
• To create an actor:
PLANET OS - MARCH 2016
Actor References
• You can not talk to actors directly
• When you create an actor, you get an ActorRef
• ActorRef is a reference to an actor… that can be passed around
… also in messages
… is serializable
… possibly remote
… hides local state
PLANET OS - MARCH 2016
QA Actor example
PLANET OS - MARCH 2016
But What if We Want an Already Existing ActorRef?
val selection = context.actorSelection("/user/AllKnowing/oracle")
• ActorSelection
val allAnswerers = context.actorSelection("/user/AllKnowing/*")
• The result can contain multiple actors:
allAnswerers ! Question(“Why are there petunias?”)
• But you can still send it messages:
• Transparently remote
PLANET OS - MARCH 2016
But how do I know the actor path?
• Actors are in a hierarchy (tree)
• Tree root is ActorSystem
• System guardian and user guardian
• All user-created actors live under /user
• It is a bit more complicated if the actor is in a remote machine
ActorSystem
/user /system
/user/Allknowing
/user/AllKnowing/Oracle
PLANET OS - MARCH 2016
What else can we do with the hierarchy? Supervision!
• Reliability
• Error kernel pattern
• Errors are escalated up the hierarchy
• Each level can decide how to respond
• Escalate error
• Restart child
• Stop child
• Resume child
• Top level guardian will shut down entire actor system
/c1
/b1
/c2
/a1
/b2
/user
/a2
/b3
/c4 /c5
/
/system
PLANET OS - MARCH 2016
Where do messages go? Mailboxes!
• Each actor has access to one
• Possibly shared
• Multiple types
• Bounded / Unbounded
• Priority
• Can be order-preserving, but not always
Actor A Mailbox
Actor B
Message
PLANET OS - MARCH 2016
Message Delivery Reliability
• Local message delivery is reliable Except for vm errors, and it’s better to always code for failure cases anyway.
• Mostly order preserving Unless you explicitly select non-order-preserving mailboxes of course
• Remote delivery is unreliable
• Should retry messages until acknowledged Or be ok with losing some, or most, or all, depending on the network.
• If message processing is idempotent, things get a lot easier to manage
PLANET OS - MARCH 2016
Who schedules actors to threads? Dispatchers!
• Messages arrive to actor mailboxes
• Dispatcher takes a message
• from actor mailbox …
• and schedules actor …
• to handle it on a thread
• Threads are taken from execution contexts
• … aka thread pools
• Dispatchers are assigned to actors via rules
Actor A Mailbox
Actor B
Message
Dispatcher
Examples
Planet OS Data Intelligence
PLANET OS - MARCH 2016
Websocket handling with actors
• Actor based web server
• One actor that receives connections
• Creates per-connection actors
• Supervises / terminates connection actors
• Connection actors handle data updates
• Better performance than thread-per-connection model
• Easier and scales better than … callbacks for data send/receive … event loops
ListeningSocket
Connection Manager Actor
ConnectionActor
ConnectionActor
ConnectionActor
PLANET OS - MARCH 2016
Data update publish
Kafka
ConnectionActor
ConnectionActor
ConnectionActor
Kafkareceiver actor
Socket Connection
Socket Connection
Socket Connection
PLANET OS - MARCH 2016
SCADA
SCADA
SCADA
WS Kafka Queue Process
Turbine 1
Turbine 2
Turbine 3
Power Plant
Dashboard
PLANET OS - MARCH 2016
Event processing with actors
• Actors represent domain entities
• wind turbine
• power plant
• alert
• Receive event updates from Kafka
• Produce new events sent to Kafka
• Some events are joins of multiple events (eg alerts)
• Some events go to Web Sockets
PLANET OS - MARCH 2016
Challenges
• Code must correctly handle idempotency
• Code must correctly handle crashes / restarts anywhere
• Data updates must not be lost
• Even cascaded data updates must not be lost
• Purely actor based model must mix consistency handling in business logic
Planet OS Data Hub
PLANET OS - MARCH 2016
DPipe - the magic behind the Data Hub
• Data pipeline:
• Acquisition
• Transformation
• Analysis
• Indexing
• For spatio-temporal data:
• Time series
• Forecasts
• Satellites
• Sensors
• Models
PLANET OS - MARCH 2016
DPipe - large scale data processing
• Multiple stages
• Each stage runs a cluster of actor systems
• Actor system has a scheduler that distributes work to worker actors
• Multiple pipelines also within a stage
• Can scale linearly with data sources
• Can do fork/join in data pipes with parallelisation
• Framework for exactly-once processing at scale
As if it wasn’t already quite obvious
We ARE in fact looking for new team members. There are many interesting challenges to work on.
Interested? Come talk to me or e-mail [email protected]