WHITE PA PE R

BY MIFAN CAREEM, DIRECTOR - SOLUTIONS ARCHITECTURE, WSO2

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL'S NEW 12TH MAN

TABLE OF CONTENTS

1. Introduction...................................................................................................................................................

2. To Analyze or not to Analyze – That is the Question...................................................................

3. Data Analytics Today................................................................................................................................

4. Technology Trends and Data Analytics.............................................................................................

4.1 Big data and in-memory complex event processing.....................................................

4.2 Internet of Things and Sensor networks...........................................................................

4.3 Computing power and cloud computing.........................................................................

5. Football Analytics Use Cases................................................................................................................

5.1 Live match analytics (Real-time streaming analytics)..................................................

5.2 Pre match analytics (Batch analytics)................................................................................

5.3 Post match analytics (Batch analytics)..............................................................................

6. WSO2 Data analytics for football........................................................................................................

7. Data Analytics in other Sports and Domains...................................................................................

8. References....................................................................................................................................................

03

04

05

05

06

06

06

07

07

07

08

08

11

12

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

02

1. INTRODUCTION

Goal! Lionel Messi lets loose a curling free kick from 20 metres out - over the Ajax wall and

into the top left corner of the netting. Jasper Cillessen has no chance. Alongside great skill,

relentless practice, individual presence of mind and the team effort that led to the free kick,

science also played a big role in that goal - knowing exactly how the defender would react,

how much spin is needed on the ball, the goal keeper’s strong side and who’d be taking the

left sided free kick was already part of the pre-match briefing. A great team [arguably the

best club today (disclaimer: the author of this white paper is an Arsenal fan so that means a

lot!)] turned to a super team with the science of data analytics.

This white paper will look at why data analytics is important in football, and for that matter

in any sport, along with the case for data analytics. It will also discuss the details of a

solution and explain the features such a solution or technology can provide. Moreover, the

paper will also take you through how this can be achieved with a technology stack that

supports big data analytics.

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

03

2. TO ANALYZE OR NOT TO ANALYZE – THAT IS THE QUESTION

Manchester City won the English Premier League 2013/2014 season last year – arguably the

most watched football (soccer, in some parts of the world) league around the world. In April

2013, BBC published an article on City’s usage of data for football analytics (BBC 2013). The

rest is history - you connect the dots.

Some would argue that football hasn’t reached a ‘Moneyball’ moment yet. Moneyball, the

2003 book (Amazon 2004) that described how Oakland Athletics built a strong team

regardless of the low payroll consisting of players that serve the team better than a bunch

of individual stars, shows how data and statistics were used to build the team consisting of

unique assets that led to a successful season. Football still follows traditional analytics -

match experts, performance experts and statisticians watch every move of the match and

the replays and analyze the positions, but that is changing fast.

Data analytics is still young in football; however, it is an emerging trend. With the increased

availability of technology, low-cost sensors, efficient and powerful hardware, and high

performance processing engines, clubs

are increasingly looking at technology

as the next competitive advantage

over their rivals. From

analyzing rival club plays

and analyzing one’s own

club, technology is fast

becoming the ideal SWOT

(strength, weakness,

opportunity, and threat)

tool for pre-match preparation.

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

Data analytics is coming to the fore in football

with more and more clubs looking to crunch

more data to provide them the analytical edge

over competing clubs. Post match discussions

are now focused on facts; ball possession,

pass percentage, distance covered, etc. A player is

increasingly judged from his work rate, average speed and

distance covered on and off the ball, number of successful

tackles, etc. rather than just the goals he scored. Post match

programs increasingly feature experts who use TV and camera technologies to analyze a

match, freeze frames, move players across the screen, and discuss strategy.

04

3. DATA ANALYTICS TODAY

It’s a given that a picture speaks a thousand words; however, in the Internet world, a link

would probably speak at least a 100. Google SEO philosophy aside, here’s a compilation of

links with self-explanatory titles that are good examples of the impact of data analytics in

football.

How computer analysts took over at Britain’s top football clubs - http://www.wired.co.uk/magazine/archive/2014/01/features/the-winning-formula

Arsenal’s ‘secret’ signing: Club buys £2m revolutionary data company - http://www.theguardian.com/football/2014/oct/17/arsenal-place-trust-arsene-wenger-army-

statdna-data-analysts

How the spreadsheet-wielding geeks are taking over football - http://www.newstatesman.com/culture/2013/06/how-spreadsheet-wielding-geeks-are-

taking-over-football

The numbers game: Why everything you know about football is wrong – review - http://www.theguardian.com/books/2013/may/24/numbers-game-everything-football-

wrong

Germany’s 12th man at the World Cup: Big data - http://blogs.wsj.com/cio/2014/07/10/germanys-12th-man-at-the-world-cup-big-data/

Man city hunts glory with data deal - http://www.decisionmarketing.co.uk/uncategorized/man-city-hunts-glory-with-data-deal

4. TECHNOLOGY TRENDS AND DATA ANALYTICS

Technology is slowly, but steadily finding its way into live matches today. From simple

technology, such as the disappearing white line used by the referee to mark free kick spots

to the more advanced goal line technology to determine whether the ball crossed the goal,

the reliance on technology has become more apparent in recent years.

Recent technology trends have made a mass amount of data and analytics of the same

readily available to clubs and teams. Some of the recent trends that contributed to sports

analytics are listed below:

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

05

4.1 BIG DATA AND IN-MEMORY COMPLEX EVENT PROCESSING

The focal point of sports analytics is of course data – this means obtaining, processing, and

analyzing large amounts of data – typically a big data problem. For real-time analytics as

such that is needed for sports, processing of complex events real time in memory is vital.

These real-time analytics engines should be able to process a large number of incoming

events if real-time stats are required.

Often the focus would be on batch analytics – the ability to push data to and store data in

scalable data storage solutions (e.g NoSQL storage), the ability to run analytics effectively

and efficiently on large amounts of data, and the capability to produce reports, summarized

and filtered information, visualizations and some level of forecasting or predictive analytics.

4.2 INTERNET OF THINGS AND SENSOR NETWORKS

Of course to analyze data, you

need data in the first place.

This means data about the game

itself being tracked and

transferred real time to a very

efficient backend system that

does processing real time.

Alternatively, data can be stored

and forwarded as batch data to the

backend system for later analysis.

There are many technologies out there used to

obtain data. Some shoe brands and wearables keep

track of movement, speed and direction information via in-built sensors. Some stadiums are

equipped with sensors or cameras that send data to a gateway solution.

4.3 COMPUTING POWER AND CLOUD COMPUTING

Big data processing requires powerful computing capacity. Fortunately, computing

resources have become a commodity with the advent of low-cost computing resources,

virtualization, and containerization of servers and the ability to scale up or down based on

usage via cloud computing service providers. This means anyone has access to on-demand

resources needed to process large amounts of data.

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

06

5. FOOTBALL ANALYTICS USE CASES

Technology has evolved to support big data analytics. What does football gain from this

though? Let’s look at a few business use cases and features of a potential football analytics

solution:

5.1 LIVE MATCH ANALYTICS (REAL-TIME STREAMING ANALYTICS)

Live analytics, as the game happens is useful to a number of people: managers and coaches

who analyze player performance real time, real-time streaming to users with smart devices

or possibly futuristic use cases where information is transmitted to the on-field referee or

the 4th official. Possible features are

• Real-time analytics and player performance

• Player performance vs fatigue

• Player marking efficiency

• Real-time SWOT analysis

• Midfield effectiveness

• Defensive performance: Defensive midfield, number of tackles

• Tackles and contact analysis – was that a dive or not?

• Offensive performance: Number of passes complete, shots on target, offside trap,

through balls to striker

• Offside detection, transmitted to referees (similar to how goal line technology works

today)

5.2 PRE MATCH ANALYTICS (BATCH ANALYTICS)

This refers to pre-match analysis in preparation of a match, available to managers, coaches

and the team

• Opposing team performance

• Opposing team player statistics, SWOT analysis

• Opposing team free kick and penalty performance, statistics

• Player marking vs speed vs physical presence

• Team selection vs opposition players

• Improvization during training; areas to work on

• Offside line and offside trap

• Formation analysis

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

07

5.3 POST MATCH ANALYTICS (BATCH ANALYTICS)

Stored data can be processed and analyzed post-match, providing various features for

managers, coaches, TV commentators and players themselves

• Post match analytics and statistics

• Goal build-up and open spaces

• Player performance analysis; player movement, player heatmap, efficiency

• Zonal marking performance of defenders

• Midfield battle

• Formation analysis vs effectiveness

6. WSO2 DATA ANALYTICS FOR FOOTBALL

An analysis of the above requirements indicates there’s a need for a data analytics solution

to monitor events in any game (football or otherwise). WSO2 provides a 100% open source

middleware platform with components ranging from integration, API management, security

to big data analytics. Here we focus on how the latter can be used to create a solution.

The WSO2 big data analytics platform provides a comprehensive platform for building

big data solutions, including support for real-time or streaming analytics, batch analytics,

predictive analytics (forecasting/machine learning), and interactive analytics. The football

analytics solution is built on the WSO2 data analytics platform, specifically WSO2 Complex

Event Processor (WSO2 CEP) as shown in Figure 2.

Figure 2: Dashboard view of WSO2 CEP football dashboard

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

08

The dashboard above provides a number of widgets showcasing real-time stats from a

football game. The football pitch graphic is reconstructed real time from a continuous

stream of player and ball location data, and visualizes the real game one to one (which

means a very efficient and high speed event processing engine is running in the

background). In addition to this visualization, it also calculates and differentiates between

passes, mis-passes, shots, shots on goal, goals, etc. It also has the ability to detect an

offside. A full video of this example is available here -

https://www.youtube.com/watch?v=nRI6buQ0NOM.

This football analytics solution is based on the dataset from the DEBS ACM 2013 challenge

(DEBS ACM 2013). The solution processes a steady stream of previously obtained real

data – the event stream based on the previous demo has location coordinates of players

on the pitch from shoe sensors: X, Y, Z coordinates, along with velocity, acceleration, and

timestamp info. The event stream also contains information from a sensor in the ball that

gives the exact location of the ball, the location of each goal keeper as well as his hand

movement (from sensors in the keeper’s gloves). Events were sent at a rate of 60Hz to the

backend system.

The event stream is passed through WSO2 Complex Event Processor, a highly efficient

event-processing engine. The logical flow of the events is illustrated in Figure 3.

Figure 3: Logical flow of event processing

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

09

CEP QueryAuthor

From Ball[team=’black’]#window.time(30s)select avg(v) insert into AvgBallSpeed

e.g.Thrift, HTTP,MQTT, SOAP, JMS, FIX 1. Filter

2. Windows+{JOIN or Aggregate}3. Event Patterns4. Event Tables

IncomingData Results

Queries

CEP Server

WSO2 CEP can handle continuous event streams and apply rules written in Siddhi (an

efficient event processing language) in memory to the incoming streams. Similar to how

database tables can be joined, the CEP allows multiple event streams to be joined for

queries and provides the ability to apply filters, windows, patterns, and aggregations to the

streams.

Individual events are broken down here, and passed through a set of predefined rules

written in Siddhi, e.g. to analyze whether a player is running or whether a goal has been

scored – this processing happens at a rate of around 200,000 transactions per second,

allowing for events to be processed real time or near real time.

Since the CEP is a framework for building very efficient event processing logic, the product

can be used to build solutions to varied use cases, which can be extended to various

domains.

Figure 4: Logical flow diagram of WSO2 CEP handling

ball possession statistics real time

The flow in Figure 4 shows a logical diagram of how ball possession is calculated in the CEP.

For this stat, an event stream on player information and an event stream on ball information

is joined, and the query is applied to the joint stream. Possession would be an accumulation

of the time the ball would remain with a player and can be assumed to be so when the ball

is within 1m of the player’s feet, for instance.

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

10

Categorize by player

Event Stream

Ball Stream

Po

ssessio

n S

tream

Player Stream

Publish Stats

Join when Distance (Ball Player)

<1m and Acc>55m/S2

Detect when ball possession

changes

And the set of features can go on. Since this is just a case of logic and simple to complex

math on a stream of events, this can be used for more adventurous stats as well. For

instance, the system can detect an offside violation by applying all offside rule conditions

to the events; for instance if the ball is passed to a player of the same team, who is furthest

up the field, and if all other offside rules comply, it can be deemed an offside. Imagine that

bit of information available to referees real time as it happens. And that’s just the beginning

– the next step for us is to detect physical contacts and ‘dives’ – now that would be very

interesting.

7. DATA ANALYTICS IN OTHER SPORTS AND DOMAINS

A generic analytics solution can be used for any kind of domain-specific analytics – all it

needs is the right set of rules, visualizations, and reports. As long as one has the ability to

configure these, the analytics platform can be targeted to various domains.

WSO2 Data Analytics Server is a platform that can be used to build custom analytics

solutions, e.g. rating the performance of a basketball game or evaluating the performance

of the offensive line in a rugby game is a few analytical steps away. Similarly, the WSO2

analytics solution has been used in many mission-critical domains as well, from live

transportation and fleet management to real-time financial fraud detection (WSO2 Webinar

2015).

Visit the WSO2 analytics page to learn more at http://wso2.com/

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

11

8. REFERENCES:

“How the spreadsheet wielding geeks are taking over football”, Newstatemen, 2013,

http://www.newstatesman.com/culture/2013/06/how-spreadsheet-wielding-geeks-are-

taking-over-football

“Football’s top teams tap into burgeoning data bonanza”, BBC, 2013,

http://www.bbc.com/news/technology-22299503

“DEBS ACM 2013 Grand Challenge”, ACM, 2013,

http://www.orgs.ttu.edu/debs2013/index.php?goto=cfchallengedetails

“WSO2 Analytics Landing Page”,

http://wso2.com/landing/big-data-analytics/

“Moneyball: The art of winning an unfair game”, Amazon, 2004,

http://www.amazon.com/Moneyball-The-Winning-Unfair-Game/dp/0393324818

“Catch them in the Act: Fraud Detection with WSO2 Complex Event Processor and WSO2

Business Activity Monitor”, WSO2, 2015,

http://wso2.com/library/webinars/2015/02/catch-them-in-the-act-fraud-detection-with-

wso2-cep-and-wso2-bam/

“Big data in the real world: Real time football analytics”: WSO2 Slideshare, 2014,

http://www.slideshare.net/wso2.org/football-analytics

“WSO2 Big data analytics platform”, WSO2, 2014,

http://wso2.com/landing/big-data-analytics/

“WSO2 CEP Football demo” WSO2 Youtube, 2014,

https://www.youtube.com/watch?v=nRI6buQ0NOM

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

12

ABOUT THE AUTHOR

ABOUT WSO2

Check out more WSO2 White Papers and WSO2 Case Studies.

For more information about WSO2 products and services,

please visit http://wso2.com or email bizdev@wso2.com

BIG DATA ANALYTICS IN SPORTS - INTRODUCING FOOTBALL’S NEW 12TH MAN ©2015 WSO2

Mifan Careem

Director - Solutions Architecture

Mifan Careem is a Director of Solutions Architecture at WSO2. In his role, Mifan provides

technology consulting on customer engagements across several industries and verticals,

and provides long term technology guidance to customers of WSO2. Prior to joining

WSO2, he co-founded Repere, a technology startup specialising in humanitarian ICT

solutions, and was also instrumental in the fomation and operation of the Sahana Software

Foundation, a global Disaster Management System of which he was also a board member.

Mifan is also WSO2’s resident domain expert on football, and following the practise before

preaching mantra, loves to play football and futsal. Mifan is an avid Arsenal fan, and at the

time of writing is awaiting the day when Arsenal will reclaim their former glory status!

WSO2 is the only company that provides a completely integrated enterprise application

platform for enabling a business to build and connect APIs, applications, web services,

iPaaS, PaaS, software as a service, and legacy connections without having to write code;

using big data and mobile; and fostering reuse through a social enterprise store. Only with

WSO2 can enterprises use a family of governed secure solutions built on the same code

base to extend their ecosystems across the cloud and on mobile devices to employees,

customers, and partners in anyway they like. Hundreds of leading enterprise customers

across every sector—health, financial, retail, logistics, manufacturing, travel, technology,

telecom, and more—in every region of the world rely on WSO2’s award-winning, 100% open

source platform for their mission-critical applications. To learn more, visit http://wso2.com

or check out the WSO2 community on the WSO2 Blog, Twitter, LinkedIn, and Facebook.