IBM Research © 2008 IBM Corporation – All Rights Reserved System S – High-Performance Stream Computing Platform Olivier Verscheure IBM T.J. Watson Research

IBM Research

© 2008 IBM Corporation – All Rights Reserved

System S –System S –High-Performance Stream High-Performance Stream Computing PlatformComputing Platform

Olivier VerscheureOlivier VerscheureIBM T.J. Watson Research CenterIBM T.J. Watson Research Center

IBM Research

© 2008 IBM Corporation – All Rights Reserved 2

Outline

System S Overview

System S for Energy Trading

System S for Astronomy

IBM Research


Stream processing will be everywhere

IBM Research


What is Stream Processing?

Database/data warehouse


Data Sources

data

Stream Processing System

Process data as it is continuously generated

Extracting and organizing information and intelligence

Minimizing time to react

IBM Research


Without Stream Processing?



Data Sources

data




Transaction processing Batch

processing

IBM Research


What Makes a Stream Processing System?

operator

stream

data packet




Sensor Network

data




IBM Research


What Makes a Stream Processing System?


ToolingDeveloper UIDeveloper UI Composition

UIComposition

UI Analyst UIAnalyst UI

Hardware Platform Servers, networks, storage, operating system, file system

Runtime Environment

Job management, resource management, content routing, programming model, object store

Application Interconnection of operators

IBM Research


Continuous Event and Stream Processing

Analysis Complexity

Time Sensitivity

Event/Data Volume & Diversity

High Volume

Complex Analysis

Time Sensitive

Stream Processing enables…

– high message/data rates,

– low (msec-secs) latency,

– advanced analysis

Today’s Complex Event Processing (CEP) solutions target…

– 10K messages/sec,

– secs-minutes latency,

– rules-based analysis

IBM Research


System S Stream Processing

New stream computing paradigm

Pull information from anywhere in real time

Ultra-low latency, ultra-high throughput

Scalable

IBM Research


System S: A Closer Look

This notional System S application…• Calculates VWAP• Calculates P/E, based earnings from Edgar• Refines earnings based on encumbrances

identified in newsfeeds

System S continually adapts to new inputs, new modalities

System S continually adapts to new inputs, new modalities

Analytics may be a combination of provided and user-developed/legacy operators

Analytics may be a combination of provided and user-developed/legacy operators

System S applications can seamlessly process structured (event) and unstructured data

System S applications can seamlessly process structured (event) and unstructured data

IBM Research


SPADE Building BlocksClassifiers, Annotators, Correlators, Filters, Aggregators

Correlate Transform

Annotator

Segmenter

Classifier

Filter

Edge Adapters

IBM Research


Application Programming

Consumable

Reusable set of operators

Connectors to external static or streaming data sources and sinks

Source Adapters Sink AdaptersOperator Repository

SPADE: Stream processing dataflow scripting language

MARIO: Automated Application Composition

Platform Optimized Compilation

IBM Research


SPADE

SPADE (Stream Processing Application Declarative Engine) is an intermediate language for streaming applications.

– Simplifies design of applications used by System S

– Hides complexities of

• manipulating data streams (e.g., contains generic language support for data types and building block operations)

• fanning out applications to distributed heterogeneous nodes

• transporting data through diverse computer infrastructures (ingesting external data, routing intermediate results, looping in feedback, branching, outputing the results, ...)

IBM Research


Basic Promises of SPADE

SPADE is easy to use

– Programmers provide descriptions of stream-based data processing tasks using SPADE’s intermediate language

– SPADE’s query engine comes up with an execution plan, builds it, and hands it off to System S runtime for deployment

SPADE enables rapid application development

– Customizable operators – do not require low-level coding

– Support for user defined operators and legacy code

SPADE is high performance

– Optimized code generation

IBM Research


A simple example[Application]SourceSink trace

[Typedefs]typespace sourcesink

typedef id_t Integertypedef timestamp_t Long

[Program]// virtual schema declarationvstream Sensor (id : id_t, location : Double, light : Float, temperature : Float, timestamp : timestamp_t)

// a source stream is generated by a Source operator – in this case tuples come from an input filestream SenSource ( schemaFor(Sensor) ) := Source( ) [ “file:///SenSource.dat” ] {}

// this intermediate stream is produced by an Aggregate operator, using the SenSource stream as inputstream SenAggregator ( schemaFor(Sensor) ) := Aggregate( SenSource <count(100),count(1)> ) [ id . location ] { Any(id), Any(location), Max(light), Min(temperature), Avg(timestamp) }

// this intermediate stream is produced by a functor operatorstream SenFunctor ( id: Integer, location: Double, message: String ) := Functor( SenAggregator ) [ log(temperature,2.0)>6.0 ] { id, location, “Node ”+toString(id)+ “ at location ”+toString(location) }

// result management is done by a sink operator – in this case produced tuples are sent to a socketNull := Sink( SenFunctor ) [ “cudp://192.168.0.144:5500/” ] {}

SinkSource Aggregate Functor

IBM Research


Performance optimization and scalability

Split/Aggregate/Join architectural pattern– High-ingest rate input stream must be split– Aggregate: model creation– Join: correlation

Operator Fusion– Fine-granularity operators– From small parts, make a big one that fits

Code generation– Actual code must match the underlying

runtime environment• Number of cores• Interconnect characteristics• Architecture-specific instructions

Compiler-based optimization– Driven by automatic profiling– Driven by incremental learning of

application characteristics

Lo

gic

al a

pp

vie

w

Ph

ys

ica

l ap

p v

iew

IBM Research


Operator Fusion - Illustration

Fuse all except sources and sinksA truly random partitioning

One PE per Operator Spade compiler can generate optimized operator grouping schemes

IBM Research


X86 Box

X86 Blade

CellBlade

X86 Blade

FPGABlade

X86 Blade

X86 Blade

X86Blade

X86 Blade

X86Blade

Operating System

System S Runtime Services

Transport System S Data Fabric

Processing Element

Container

Processing Element

Container

Processing Element

Container

Processing Element

Container

Processing Element

Container

Optimizing scheduler assigns operators to processing nodes, and continually manages resource allocation


Runs on commodity hardware – from single node to blade centers to high performance multi-rack clusters


IBM Research


BG Node BG node BG nodeBG nodeBG nodeX86 Blade

FPGA Blade

X86Blade

X86 Blade

CellBlade

X86 Blade

X86 Blade

X86Blade

X86 Blade

X86Blade

Operating System

Transport System S Data Fabric

System S Runtime Services

Processing Element

Container

Processing Element

Container

Processing Element

Container

Processing Element

Container

Processing Element

Container


Optimizing scheduler assigns operators to processing nodes, and continually manages resource allocationAdapts to changes in resources,

workload, data ratesAdapts to changes in resources,

workload, data rates

Capable of exploiting specialized hardwareCapable of exploiting specialized hardware



IBM Research


Site C

Site B

Site A

Distributed operation

Site C

Site B

Site A

IBM Research


Advantages of Stream Processing as Parallelization Model

Automated, Optimized Composition (SPADE, MARIO)

Source Adapters

Sink Adapters

Operator Repository

Automated, Optimized Deploy and Management (Scheduler)

Operator and data source profiling for better resource management

Automated composition

– Query optimization over well-known operators

– Inquiry optimization using semantic tagging of operators and data sources

Reuse of operators across stored and live data

– MapReduce is similar programming model with storage as transport

Streams as first class entity

– Explicit task and data parallelism

– More intuitive approach to multi-core exploitation

IBM Research


Trading AdvantageTrading Advantage

GovernmentGovernment

ManufacturingManufacturing

AstrophysicsAstrophysics

Improving the quality semi-conductor wafers with dynamic manufacturing tools tuning

Identification of and response to opportunities in real-time market data

Detect & respond to phenomena based on large volumes of structured and unstructured information

World’s largest and first fully digital radio observatory for astrophysics, space and earth sciences, and radio research

5 Million events / secMillisecond latency5 Million events / secMillisecond latency

2.5K events / sec10 msec latency2.5K events / sec10 msec latency

1.5 Million events / sec1.5 Million events / sec

Semiconductor SolutionsSemiconductor Solutions

System S Pilots

IBM Research


Sneak Preview: IBM InfoSphere Streams

Business Process Management

Data Warehouse

Applications

Business Intelligence

IBM Research


Outline

System S Overview



IBM Research


The Energy Trading Scenario using Stream Computing Sample application showing power of Stream Computing

– Only one of many possible applications/services

Weather conditions and events drive pricing of energy futures

– natural events interfering with energy supply

– announcements, news stories, …

Energy traders today struggle to integrate info from multiple sources

– cannot get it in real time, to inform their trade decisions

– they see 8 screens, integrate manually via a spreadsheet

IBM Stream Computing assembles, deploys applications

– integrates diverse sources of data

– provides timely correlations, analyses

IBM Research


Illustrative Example: Fear and Opportunity in the Gulf

News Flash: Hurricane Dean Upgraded to

Category 5 Path Projected through Gulf Oil Stocks Uniformly Down

News Flash: Hurricane Dean Upgraded to

Category 5 Path Projected through Gulf Oil Stocks Uniformly Down

Others are up, showing recovery even beforethe storm hits

…the live tickers tell a different story

Properties with significant assets in path are still down

More affected companies have been identified(a bit late, no?)

Same story, viewed 2 hours later.The story’s the same, but…

• If you saw it coming, because you watched for more primal data…

• Like hurricane path predictions from NOAA• Or even weather satellite and/or sensor data• Real-time equities trade data

• If you’ve been accumulating intelligence on the location (and value) of company assets that could be in the path…

• If you could apply such analysis before the news cycle…

• You could take advantage – in both directions…Oil company stocks down, based on early fears.

IBM Research


Web Zero platform

Capture weather sensor data, analyses hurricane predicted path

Estimate impact on portfolios

Recommendations Based on Hurricane Forecast

Compute portfolio market indicators

(low latency) Make recommendations

and notify

Capture market data

(high volume)

System S platform

DHTML Result rendering

Real-time projections of hurricane path

Dynamically updated risk assessment

for assets in projected path

Correlate combined risk and trade VWAP to

determine buy/sell recommendations

IBM Research


Outline

System S Overview



– Past & current projects

IBM Research


Past Projects

Outlier detection from single tripole

Decomposing combined DOA’s from single tripole

– SPADE UDOP’s

– Linking against Lapack and Blas libraries

– About 50 non-trivial processing elements

– Being optimized by SPADE team now

Convolutional resampling (tConvolve) on System S

– Mostly built-in operators (soon built-in operators only)

– Fully parametrizable using Perl; e.g., # of w planes

– Does scale very well!

IBM Research


Outlier detection from single tripole

Receive 3D electric field

Demultiplex 3D electric field– Each UDP packet contains multiplexed electric fields

Compute intensity I(t)=|Ex(t)|2+|Ey(t)|2+|Ez(t)|2

Detect outliers Outlier detected if: mI(t-N:t-1) + T.I(t-N:t-1) I(t) mI(t-N:t-1) - T.I(t-N:t-1)

Visualize detected outliers in Matlab in real-time

IBM Research


|Ex|2

|Ey|2

|Ez|2

Barrier Seq, I

Field intensity

Barrier

Filter outSeq<10

UIL?I,{U,L}Filter out

empty lists

Outlier detection

SPADE Flow of Operators

UDPSource

DataDemux

UDPsink

Filesink

^2

AggregateAvg 10c,1c Avg


BarriermI

Sqrt(mI2-mI2)

mI

mI2

mI-T.I

mI+T.I

mI, I

Windowed statistics

U, L

IBM Research


|Ex|2

|Ey|2

|Ez|2

Barrier Seq, I

Field intensity

SourceData

Demux

Field Intensity

IBM Research


|E

x | 2

|Ey | 2

|Ez | 2

Barrier

Seq,

I

Field intensity

Source

Data

Dem

ux

IBM Research


|E

x | 2

|Ey | 2

|Ez | 2

Barrier

Seq,

I

Field intensity

Source

Data

Dem

ux

IBM Research


|E

x | 2

|Ey | 2

|Ez | 2

Barrier

Seq,

I

Field intensity

Source

Data

Dem

ux

IBM Research


|E

x | 2

|Ey | 2

|Ez | 2

Barrier

Seq,

I

Field intensity

Source

Data

Dem

ux

IBM Research


|E

x | 2

|Ey | 2

|Ez | 2

Barrier

Seq,

I

Field intensity

Source

Data

Dem

ux

IBM Research


|Ex|2

|Ey|2

|Ez|2

Barrier Seq, I

Field intensity

Barrier

Filter outSeq<10


empty lists

Outlier detection


UDPSource

DataDemux

UDPsink

Filesink

^2



BarriermI

Sqrt(mI2-mI2)

mI

mI2

mI-T.I

mI+T.I

mI, I

Windowed statistics

U, L

IBM Research


Windowed Statistics

^2



BarriermI

Sqrt(mI2-mI2)

mI

mI2

mI-T.I

mI+T.I

mI, I

Windowed statistics

U, L

IBM Research


^2

Aggregate

Avg 10c,1c

Avg

Aggregate

Avg 10c,1c

Avg

Barrier

mI

Sqrt(m

I 2-mI 2)

mI

mI 2

mI -T

.I

mI +

T.

I

mI ,

I

Window

ed statistics

U, L

IBM Research


^2

Aggregate

Avg 10c,1c

Avg

Aggregate

Avg 10c,1c

Avg

Barrier

mI

Sqrt(m

I 2-mI 2)

mI

mI 2

mI -T

.I

mI +

T.

I

mI ,

I

Window

ed statistics

U, L

IBM Research


^2

Aggregate

Avg 10c,1c

Avg

Aggregate

Avg 10c,1c

Avg

Barrier

mI

Sqrt(m

I 2-mI 2)

mI

mI 2

mI -T

.I

mI +

T.

I

mI ,

I

Window

ed statistics

U, L

IBM Research


|Ex|2

|Ey|2

|Ez|2

Barrier Seq, I

Field intensity

Barrier

Filter outSeq<10


empty lists

Outlier detection


UDPSource

DataDemux

UDPsink

Filesink

^2



BarriermI

Sqrt(mI2-mI2)

mI

mI2

mI-T.I

mI+T.I

mI, I

Windowed statistics

U, L

IBM Research


Outlier Detection

Barrier

Filter outSeq<10


empty lists

Outlier detection

UDPsink

Filesink

Intensity

U, L

IBM Research


Bar

rier

Filt

er o

utS

eq<

10

U

IL?

I,{U

,L}

Filt

er o

utem

pty

lists

Out

lier

dete

ctio

n

UD

Psi

nk

File

sinkInte

nsi

tyU

, L

IBM Research


Bar

rier

Filt

er o

utS

eq<

10

U

IL?

I,{U

,L}

Filt

er o

utem

pty

lists

Out

lier

dete

ctio

n

UD

Psi

nk

File

sinkInte

nsi

tyU

, L

IBM Research


Decomposing combined DOA’s from single tripole

IBM Research


Direction of Arrival (DOA)

Simple case

– Two orthogonal waves

–

IBM Research


Pseudo-Orbital MomentumTwo-wave case, same frequencies

IBM Research


Pseudo-Orbital MomentumTwo-wave case, same phases

IBM Research


Decomposing the combined DOA

Consider n = 1..N incident waves at time t

Can we possibly estimate DOA(n) for all N?

… from a single 3D sensor???

IBM Research


Signal Processing to the Rescue

Let’s get back to wave equations

x-component: (Ex)n(t) = (Ax)

n eixn ei

nt

Estimate n for all n = 1..N from x-component

– Matrix pencil method!

– Requires N timestamps only (minimum)

Plug estimates back in {x,y,z}-components

– System of 3xN linear equations

Retrieve estimates of (A{x,y,z})n ei

n{x,y,z}.

Estimate DOA(n) for all n = 1..N

• E.g., (Vx)n = (Ay)

n eiyn (Az

*)n e-izn - (Az)

n eizn (Ay

*)n e-iyn

IBM Research


SummarySimplified Processing Flow Graph

UDPSource

DataDemux

HankelConstruct

HankelConstruct

HankelConstruct

Pick TopSignals(SVD)



Frequencyestimates

Frequencyestimates

Frequencyestimates

Denoisefrequencyestimates

LeastSquareSolver

LeastSquareSolver

LeastSquareSolver

X

Y

Z

X

Y

Z

Separatedwaves

V vector

IBM Research


Sample Spade Code

IBM Research


Convolutional resampling (tConvolve)

45 antennas, 30 beamformers

ASKAP System: – 1% of SKA system

– Operational in 2012

– 8h observation produces 2.3TB!

Goal: Evaluate System S as the Central Processing Platform of the Australian SKA Pathfinder (ASKAP)

IBM Research


56

SKA Processing Graph66 PEs on 20 nodes

Heavy Convolutional PEsMain computation(1000-2200 MIPS,

20-42 Mbit/sec)

IBM Research


Scalability

10000 samples randomly generated

5+ times reduction in gridding time

57

IBM Research


Current Projects

Software imaging with cleaning

– Joint algorithmic/software/hardware optimization

– … in collaboration with Tim Cornwell et al.

Astronomical Signature Clustering

– … in collaboration with Bo Thide, Jan Bergman, Lars K Daldorff

Documents

IBM Research © 2008 IBM Corporation – All Rights Reserved System S – High-Performance Stream Computing Platform Olivier Verscheure IBM T.J. Watson Research