Upload
gigaom
View
107
Download
2
Embed Size (px)
DESCRIPTION
Presentation from Chris Haddad - VP Platform Evangelism, WSO2 #gigaomlive More at http://events.gigaom.com/structuredata-2014/
Citation preview
1
Last Updated: Jan. 2014
VP Pla&orm Evangelism @cobiacomm on Twi6er
h6p://blog.cobia.net/cobiacomm/
Chris Haddad
Tracking a Soccer Game with Big Data
Big Data Improves Team Performance
๏ Sports Teams
๏ Your Business Team 3
Improvement Requires A Feedback Loop
4
Internet of Things (IoT) Bridges Physical and Virtual Worlds
๏ Sensors, actuators everywhere
๏ Internet connecOvity
Improvement = IoT, Apps, and APIs
๏ VisualizaOon Apps
๏ Control APIs
DEBS Challenge
• Bridge game play to Internet of Things
• Sensors in shoes, ball, goalie hands
• Analyze Soccer Game
• SpaOal and Temporal Processing
• Visualize Game Play
• AcOvity Heat maps
• Recommend Performance Improvements
http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html
Bridging Game Play to IoT
7
Temporal Tracking
SpaOal Tracking
• Each event includes the locaOon (x,y,z), Ome stamp, velocity and acceleraOon
• Throughput 15,000 events per second
Analyze Game
8
๏ BALL POSSESSION STATE DIAGRAM
๏ SHOT ON GOAL STATE DIAGRAM
Analyze Game
9
๏ RUNNING LOCATION STATES
10
SoluOon Architecture ๏ Capture millions of events with
Common Events Collector
๏ Derive real-‐Ome decisions with temporal processing
๏ Further opOmize recommendaOons with spaOal Map-‐Reduce processing analyOcs
๏ View feedback dashboards
Batch AnalyOc FoundaOon
11
Business AcOvity Monitor
Realizing Real-‐Ome AnalyOcs ๏ Processing Data on the fly, while
storing a minimal amount of informaOon and responding fast (from <1 ms to few seconds)
๏ Idea of Event streams
๏ A series of events in Ome
๏ Enabling technologies
๏ Stream Processing (Storm)
๏ Complex Event processing (Siddhi)
Complex Event Processing
Complex Event Processing
Stream Processing
• SQL like language • Supports powerful temporal operators (e.g. windows, event paberns) • Focus on speed • Harder to scale • e.g. WSO2 CEP, Streambase, Esper
• Operators connected in a network, but you have to write the logic • Distributed by design • Focus on reliability (do not loose messages), has transacOons • e.g. Storm, S4
Complex Event Processing Operators ๏ Filters or transformaOons (process a single event)
๏ from Ball[v>10] select .. insert into ..
๏ Windows + aggregaOon (track window of events: Ome, length)
๏ from Ball#window.time(30s) select avg(v) ..
๏ Joins (join two event streams to one)
๏ from Ball#window.time(30s) as b join Players as p on p.v < b.v
๏ Paberns (state machine implementaOon)
๏ from Ball[v>10], Ball[v<10]*,Ball[v>10] select ..
๏ Event tables (map a database as an event stream)
๏ Define table HitV (v double) using .. db info ..
Usecase 1: Running Analysis ๏ Detect when speed crosses threshold limits
define partition player by Players .id; from s = Players [v <= 1 or v > 11] , t = Players [v > 1 and v <= 11]+ , e = Players [v <= 1 or v > 11] select s.ts as tsStart , e.ts as tsStop ,s.id as playerId , ‘‘trot" as intensity , t [0].v as instantSpeed , (e.ts - s.ts )/1000000000 as unitPeriod insert into RunningStats partition by player;
Usecase 2: Ball possession ๏ Ball possession
๏ Defined as possessing the ball from Ome you hit it unOl someone else hit it or ball leaves the ground)
Usecase 3: Heatmap of AcOvity
๏ Show where acOons happened (via cells defined by a grid of 64X100 etc.), need updates once every second
๏ Can solved via cell change boundaries, but does not work if one player stays more than 1 sec in the same cell. So need to join with a Omer.
Usecase 4: Detect kicks on goal
๏ Main Idea: Detect kicks on the ball, calculate direcOon aker 1m, and keep giving updates as long as it is in right direcOon
Results for DEBS Scenarios
Big Data and IoT Changes the Game ๏ Real-‐Ome Status Dashboard
๏ Player game play
๏ Momentum indicators
๏ PredicOons about the next move
๏ Improvement Dashboard
๏ Study of game and players effecOveness
๏ Monitor player health and body funcOons
Success Target
Photo by John Trainoron Flickr http://www.flickr.com/photos/trainor/2902023575/, Licensed under CC
SituaOonal Context: Traffic Example
{incidents: [ {impacting: false,
eventCode: 0, iconURL: "http://...",
lat: 38.743999, shortDesc: "Exit ramp closed on I 95...",
type: 1, severity: 0,
fullDesc: "In NEWINGTON exit ramp closed on I-95 ...",
startTime: "2010-02-21T00:14:07", lng: -77.188004, id: "368598263",
endTime: "2010-02-27T05:04:19" },
Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york-city/ Real-time traffic map: http://www.mapquestapi.com/traffic/
Accelerate interac6ons inside and outside the organiza6on
Reduce interac6on
fric6on and cost
Increase engagement and
enhance produc6vity
Sense business ac6vity
and automa6cally
adapt
Become a More Connected Business
http://wso2.com/landing/enabling-the-connected-business
Connected Business Reference Architecture
Conceptual Architecture
CEP High Availability
Siddhi Storm Bolt (next Major Release)
๏ We have wriben a Siddhi bolt that would let users run distributed Siddhi Queries using Storm
SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. ); SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. ); TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("source", new PlayStream(), 1); builder.setBolt("node1", siddhiBolt1, 1) .shuffleGrouping("source", "PlayStream1"); .. builder.setBolt("LeafEacho", new EchoBolt(), 1) .shuffleGrouping("node1", "LongAdvanceStream"); .. cluster.submitTopology("word-count", conf, builder.createTopology());
31
Contact us !
33
About the Presenter
๏ Chris Haddad
๏ VP Plaoorm Evangelism
๏ Learn more about me
๏ www.linkedin.com/in/cobiacomm/
๏ Follow me
๏ @cobiacomm on Twiber
๏ hbp://blog.cobia.net/cobiacomm
๏ On Google+ too