AWSSimple WorkFlow

by Nathan McCourtneyMay 2nd, 2012

@beaknit

"Workflow"?

A workflow management system is a computer system that manages and defines a series of tasks within an organization to produce a final outcome or outcomes. At each stage in the workflow, one individual or group is responsible for a specific task. Once the task is complete, the workflow software ensures that the individuals responsible for the next task are notified and receive the data they need to execute their stage of the process

http://en.wikipedia.org/wiki/Workflow

Amazon's Workflow Model

● A Workflow is the automation of a business process.● A Domain is a collection of related Workflows.● Actions are the individual tasks undertaken to carry out

a Workflow.● Activity Workers are the pieces of code that actually

implement the tasks. Each kind of Worker has its own Activity Type.

● A Decider implements a Workflow's coordination logic. - From Jeff Barr's post

Workers & Deciders = Actors

From wikipedia: In computer science, the Actor model is a mathematical model of concurrent computation that treats "actors" as the universal primitives of concurrent digital computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received.

Essentially, SWF is a framework for abstracting out actors and passing

messages between them via queues

Act!

Actor

Work Queue

Result Queue

Messages

1. Fetches its work2. Drops off its results

AWS Message Passing

● HTTPS● Long-poll interval of 60 seconds● Message format is JSON:

{"decisionType": "ScheduleActivityTask", "scheduleActivityTaskDecisionAttributes": {"activityType": {"name": "activityVerify", "version": "1.0"}, "activityId": "verification-27", "control": "digital music", "input": "5634-0056-4367-0923,12/12,437", "scheduleToCloseTimeout": "900", "taskList": {"name": "specialTaskList"}, "scheduleToStartTimeout": "300", "startToCloseTimeout": "600", "heartbeatTimeout": "120"}}

● One decision queue per Workflow● Many activity queues per Workflow

Decider to SWF Comms

Decider

DTL

SWF

1. Decider polls Decision Task List (DTL) for new decision tasks

2. Decision task returned with entire workflow execution history to provider context for decisions

3. Decider executes conditional logic and returns an activity task back to SimpleWorkFlow which then deposits in the Activity Task List (ATL)

Note: Multiple deciders can process the same DTL

ATL

From AWS Ruby SDK

# poll for decision tasks from 'my-task-list' domain.decision_tasks.poll( 'my-task-list') do |task|

# investigate new events and make decisions task.new_events.each do |event| case event.event_type when 'WorkflowExecutionStarted' task.schedule_activity_task 'do-something', :input => 'abc xyz' when 'ActivityTaskCompleted' task.complete_workflow_execution :result => event.attributes.result end end

end # decision task is completed here

Worker to SWF Comms

1. Worker polls ATL for appropriate tasks - different workers can consume different tasks

2. Task completion returned to SWF which makes the result available to the Decider via the DTL

Note: Multiple workers can process the same ATL

Worker

ATL

SWF

DTL

# poll 'my-task-list' for activities domain.activity_tasks.poll( 'my-task-list') do |activity_task|

case activity_task.activity_type.name when 'do-something' # ... else activity_task.fail! :reason => 'unknown activity task type' end end

From AWS Ruby SDK

AWS example order-processing SWF

Source: SWF Dev Guide

Practical Exercise

Nightly restore of a database to QA nodes combined with automated release migration.

Examples

Further Reading

● Official Announcement● Rightscale's blog on their use● Dev Guide - Basic Concepts● AWS Ruby SDK