1
Big Data: IT does matter Dr. Sven Niedner, JettainereMail: [email protected], Twitter: S_N_I
London, November, 11th 2013
Still true?
Who we are and why IT matters for us
The leading ULD management companies Reliable partner for 13 airlines wordlwide 100% owned by Lufthansa Cargo
ULD fleet acqusition and lease back Savings potential enabled by intelligent asset tracking Over 60k air cargo ULDs – largest fleet in the world Active control concept (forecast-based) Zero offloads since …
Role of IT: IT products are a differentiator Jettware software for ULD management Used at 350+ stations worldwide Covers full ULD fleet lifecycle
Our customers
ULD management customers + JettLease short term lease
Our USP: 20% fleet size reduction due to our active control concept
Where are my ULDs?
Where do I have to reposition my ULDs?
How will the demand develop at an airport?
Will I survive the next holiday season without any
offloads?
Is my fleet to big?
Whom can I borrow ULDs from? (Smart Pooling!)
Business questions
EASY Where the hell is that overstock coming from? How many ULDs did not move in the last X days?
HARDER How did my mean time between repair develop? How did the demand develop in the last year before
christmas?
VERY HARD Has there been a similar situation as today occured
before and how was it resolved? I want to introduce this crazy new KPI, but what was
it‘s value last year?
Innovation is key for success – your Business questions will change
Access to real time data
Access to historic data
Link real-time data and historic
data
Jettware overview
TacticalStrategic Operational
Jettware: UCMs, LUCs and stock checks at stations
2005
UCD Cockpit: Stock monitoring & ULD positioning/move requests
2008
JettStart: Demand simulation and capacity planning
2011
Revised ULD Tracking Application & DWH reporting tools
2012
Management Cockpit: Overall performance monitoring
2012
JettApp: Digital ULD tracking for the ground handler
2013
<PARENTHESIS TOPIC=“mobile“>
Mobile devices collect more information than ever Not only the NSA, but also your company! Changing user beviour: Using application “on-the-go”, not only
“on premise”
People expect real time response & interactive UIs User expectations are set by Apple and Google Responses need to come within seconds You cannot satisfy your users with inferior quality
You think it’s different because your uses are your employees? Dream on! The pressure will come – from your users Think BYOD – consumer goods set the expectation level! These guys (Apple, Samsung, Sony, …) are fast and aggressive You will have to compete
Real world example: Mobile drives the hunger for realtime Big Data
JettApp device sends ULD messages from the ware house in real time
Increased data quality More information, more
events Tablet PC Running apps Build on Android
</PARENTHESIS >
How to do it?
How to do it
Keep all your data. Discard nothing.
Architectureis important
Challenges
l implement like:query = function ( all data )
http://www.flickr.com/photos/jblndl/3326092093/sizes/l/in/photostream/
Keep all your data. Forever.
30.000 ULD movements per day, 100.000 messages per day (mostly SITA)
Business is driven by data:
ULD inventories
stocks at stations
flight plans and changes
ULD histories back to 2005
20 TB+ (and growing)
>10.000.000.000 data sets
Input mostly SITA messages: varying data quality and compliance
… and our ULD control desk needs to act in real time
So where is “Big Data”?
Challenges for IT
It needs to be real big! (cargo ship) + 50 million USD
It needs to be real fast! (drag boat) + 20.000 USD
Conflicting goals high cost (aircraft carrier): 9 Billion USD
Foto by Bengt Nymann, http://www.flickr.com/photos/bnsd/6139750090
Really fast: 30.000 $
Challenges for IT
It needs to be real big! (cargo ship) + 50 million USD
It needs to be real fast! (drag boat) + 20.000 USD
Conflicting goals high cost (aircraft carrier): 9 Billion USD
Foto by cuxclipper, http://www.flickr.com/photos/36288065@N04/4949196610
Really big: 100.000.000 $
Challenges for IT
It needs to be real big! (cargo ship) + 50 million USD
It needs to be real fast! (drag boat) + 20.000 USD
Conflicting goals high cost (aircraft carrier): 9 Billion USD
Foto by US Navy, http://www.flickr.com/photos/usnavy/5781645295
Really fast and really big: 10.000.000.000 $
BatchLayer
ServingLayer
Really clever: The Lambda architecture pattern
Speed layerINPUTData
Views,Reports,
Apps
RealtimeView
BatchView
OUTPUT
OUTPUT
Lambda architecture proposed by Nathan Marz,Twitter
Combining real-time and batch procssing:
Really clever: The Lambda architecture pattern
Design principles: Human fault tolerance Data immutability Recomputation
BatchLayer
ServingLayer
Speed layerINPUT Data Views,Reports,
Apps
RealtimeView
BatchView
OUTPUT
OUTPUT
query = function(all data)
JettStart: Demand simulation and capacity planning
UCD Cockpit: Stock monitoring & ULD positioning/move requests
Jettware: UCMs, LUCs and stock checks at stations
Management Cockpit: Overall performance monitoring
ULD Tracking Application & DWH reporting tools
JettApp: Digital ULD tracking for the ground handler
Focus: historic data
Different applications have different focus
Tactical OperationalStrategic
Focus: real-time
Recommender System
Focus: historic data
http://www.flickr.com/photos/alreadytaken/2136780794/
Build a mental model of you business domain
And then map it on your existing data structures
Think beyond the obvious
Go for second order effects: Network effects Correlations non-linear phenomenons
Enhanced Business domain insight
Accept unreliable and inconsistent data as a fact
You cannot fix the quality of historic data
Statistics will help you to detect or cancel out errors
Have realistic expectations regarding the Data Scientist role
Nice idea in theory, but (almost) impossible to find in reality
Better kiss the frog now than wait for the prince forever
http://www.flickr.com/photos/fcstpauli/2067908929/sizes/l/in/photostream/
Don’t loose your business focus.
For us it’s all about…
1. Demand vs. Supply
2. Overstock vs. Understock (balance network)
3. Demand forecast and tactical ULD movements
4. Strategic movements
<BACKUP >
<PARENTHESIS TOPIC=“agile“>
The agile manifesto:
Individuals and interactions …… over processes and tools
Working software …… over comprehensive documentation
Customer collaboration …… over contract negotiation
Responding to change …… over following a plan
Implement lean and agile development tactics
</PARENTHESIS >
Recommended
