Tracking a Soccer Game With Big Data Presentation

8/10/2019 Tracking a Soccer Game With Big Data Presentation

1/31

Tracking a SoccGame with Big D

Srinath PereraDirector of Research, WSO2

Member, Apache Software Found

[email protected]

@srinath_perera
mailto:[email protected]:[email protected]


2/31

Outline Big D

CEP

Trac

Socc

Mak Conc

Photo by John Trainoron Flickrhttp://www.flickr.com/photos/trainor/2902023575/,


3/31

A day in your life

Think about a day in your life?

- What is the best road to take?- Would there be any bad weather?

- How to invest my money?

- How is my health?

There are many decisions that you can

do better if only you can access the data

and process them.

http://www.flic

51246
http://www.flickr.com/photos/kcolwell/5512461652/http://www.flickr.com/photos/kcolwell/5512461652/http://www.flickr.com/photos/kcolwell/5512461652/http://www.flickr.com/photos/kcolwell/5512461652/


4/31

Real Time Analytics

Most first usecases were of batch

processing, which take minutes if nothours to run

Many insights you rather have sooner

- Tomorrows weather

- Heart condition- Traffic congestion

- Natural disaster

- Stockmarket Crash


5/31

Realizing Real-time Analytic Processing Data on the fly, while

storing a minimal amount of

information and responding fast

(from


6/31

Internet of Things

Currently physical world and

software worlds are detached Internet of things promises to

bridge this

- It is about sensors and

actuators everywhere- In your fridge, in your blanket,

in your chair, in your carpet..

Yes even in your socks

- Google IO pressure mats


7/31

Vision of the Future

Sensors everywhere

Data collected from everywhere, analyzing,optimizing, and helping (and hopefully not

taking over)

Analytics and Internet of things .. Immersive

world Big data and real-time analytics will be

crucial. How far are we from realizing that?


8/31

What would take to

build such a world?

Sensors and actuators (Motes?) Fast interoperable event systems

(MQTT?)

Powerful query languages (CEP?)

Powerful control systems anddecision systems


9/31


10/31

Complex Event Processing


11/31

Complex Event Processing Ope

Filters or transformations (process a single event)- from Ball[v>10] select .. insert into ..

Windows + aggregation (track window of events: ti- from Ball#window.time(30s) select avg(v) ..

Joins (join two event streams to one)- from Ball#window.time(30s) as b join Players as p on

Patterns (state machine implementation)- from Ball[v>10], Ball[v10] select ..

Event tables (map a database as an event stream)- Define table HitV (v double) using .. db info ..


12/31

Sport (Soccer) Usecases?

Dashboard on game status

Alarms about critical events in the game Real-time game analysis and predictions about the

Updates/ stats etc., on mobile phone with customiz

Study of game and players effectiveness

Monitor players health and body functions


13/31

DEBS Challe

Soccer game, players

has sensors (DESB C

2013) sid, ts, x,y,z, v,a

Use cases: Running a

Ball Possession and S

Goal, Heatmap of Acti

WSO2 CEP (Siddhi) d

throughput


14/31

Usecase 1: Running Analys

Main idea: detect when speed thresholds have pas

define partition player by Players .id;

from s = Players [v 11] ,t = Players [v > 1 and v


15/31

Usecase 2: Ball possessio

Ball possession (you possess the ball from time yo

someone else hit it or ball leaves the ground)


16/31

Usecase 3: Heatmap of Activ Show where actions happened (via cells defined by

64X100 etc.), need updates once every second

Can solved via cell change boundaries, but does n

one player stays more than 1 sec in the same cell.

join with a timer.


17/31

Usecase 4: Detect kicks on the

Main Idea: Detect kicks on the ball, calculate direct

1m, and keep giving updates as long as it is in righ


18/31


19/31

Making it


20/31

CEP High Availability


21/31

Complex Event Processing Stream Proce

SQL like language Supports powerful temporal

operators (e.g. windows,

event patterns)

Focus on speed

Harder to scale e.g. WSO2 CEP,

Streambase, Esper

Operators connecnetwork, but you

write the logic

Distributed by des

Focus on reliabilit

loose messages),transactions

e.g. Storm, S4


22/31

Scaling CEP: Pipeline


23/31

Scaling Operators: e.g. Distribute


24/31

Siddhi Storm Bolt (next Major Re

We have written a Siddhi bolt that would let users r

distributed Siddhi Queries using Storm

SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. );

SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. );

TopologyBuilder builder = new TopologyBuilder();

builder.setSpout("source", new PlayStream(), 1);

builder.setBolt("node1", siddhiBolt1, 1)

.shuffleGrouping("source", "PlayStream1");

..builder.setBolt("LeafEacho", new EchoBolt(), 1)

.shuffleGrouping("node1", "LongAdvanceStream");

..

cluster.submitTopology("word-count", conf, builder.createTopolog


25/31

Conceptual Architecture

C


26/31

Data Collection Can receive

SOAP, HTT

WSO2 Even

fast (400K e

Default Age

can write cu

agents.

Agent agent = new Agent(agentConfiguration);

publisher = new AsyncDataPublisher("tcp://localhost:7612", .. );

StreamDefinition definition =new StreamDefinition(STREAM_NAME,

VERSION);definition.addPayloadData("sid", STRING);

...

publisher.addStreamDefinition(definition);

...

Event event = new Event();

event.setPayloadData(eventData);

publisher.publish(STREAM_NAME, VERSION, event);

B i A ti it M it


27/31

Business Activity Monitor


28/31

Think CEP AND Hive, not O

Real power comes when you can join the batch an

processing E.g. if velocity of the ball after a kick is different fro

average by 3 times of seasons standard deviation

event with player ID and speed

- Hard to detect velocity after the kick with Hive/ Mabut much easier with CEP

- Complicated to calculate season average with CE

end up with huge windows.


29/31

Oth A li ti


30/31

Other Applications

System/ Device Management

Fleet/ Logistic Management Fraud Detection

Targeted/ Location Sensitive Marketing

Smart Grid Control

Geo Fencing


31/31

Documents

Tracking a Soccer Game With Big Data Presentation