Upload
mfrancis
View
108
Download
2
Tags:
Embed Size (px)
DESCRIPTION
As M2M gateways and aggregator devices continue to expand the ever-widening array of sensors and sensor protocols they manage, the number of data events being generated is growing exponentially. Some events need to be acted upon locally; others need to be sent to the cloud for global analysis. Some events require immediate action; other events can wait for later processing. More importantly, the M2M ecosystem needs the ability to dynamically change the local analytics and business rules, depending on the global analytics in the cloud. Using a Raspberry Pi and a temperature monitor, this presentation will demonstrate how OSGi powers dynamic M2M event processing. Bio: Walt is currently the Chief Architect, Software Solutions with Hitachi Communications Technologies America. For the past five years Walt has focused on building an OSGi Ecosystem to deliver advanced services and applications to home networks and M2M devices. He has been involved with Telecommunications and Java since 1996 and has extensive experience in Java and OSGi development both as a developer and as a development manager. Walt has a BS in Mathematics and Computer Science from the University of Georgia and over 25 years of experience in software engineering.
Citation preview
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Powering Dynamic M2M Event Processing with OSGi Dynamic Complex Event Processing and OSGi
Hitachi Communication Technologies America, Inc.
6/12/2014
Walt Bowers
Chief Architect OSGi Solutions
Contents
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
1. The Vs of Big Data
2. Complex Event Processing
3. Dynamic Complex Event Processing
Dynamic Complex Event Processing and OSGi. Powering Dynamic M2M Event Processing with OSGi
1
4. Demo
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
1. The Vs of Big Data
Dynamic Complex Event Processing and OSGi.
Powering Dynamic M2M Event Processing with OSGi
2
3 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
The Vs of Big Data
4 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
The missing “V” of Big Data
Extracting VALUE from VIABLE Data
WHERE It Matters and WHEN It Matters
The missing “V” of Big Data
5 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Rise of The Intelligent Device
6 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Key Elements of Connected Intelligence
Intelligent
Devices
Always-on devices connected to variety
of sensors and running multiple
software applications
1
Real-Time Analytics
High-frequency
data analysis for instant decision
making and automation of
information flows
2
Big Data
Integration of data
from connected devices with enterprise
applications and historical data
3
7 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamic Predictive Analytics
• Local analytics and business rules are controlled by global analytics
– In-flight data analytics on the device
– Near real time response on the device
• Global Analytics for the Big Patterns
– Big Data post processing
– Discover Hidden Patterns/dependencies
• Dynamically Adjust the Rules
– Update new rules to the local device
– Enhances the devices local analytics
• Rinse and Repeat
8 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Transportation Example
Passenger
Counter
M2M Gateway)
1 Location tracking (LBS)
Camera (Normal operation,
local storage; emergencies such
as Amber alert, streaming to
emergency authorities)
Digital signage (Information
and advertising - Location
aware)
Vehicle diagnostics
(Sensors throughout
vehicle)
Route monitoring (Time of
day optimization)
Fraud detection
(Passenger counter,
camera, fare collection)
3 2
4
5 6
7
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
2. Complex Event Processing
Dynamic Complex Event Processing and OSGi.
9
Powering Dynamic M2M Event Processing with OSGi
10 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Intelligence Is Real-Time, Event-Based Analytics
Communication Events
Machine Events
Security Events
Environmental Events
Business Logic
Events
Complex Events Processing
enables real-time business insights from edge devices
11 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Complex Event Processing (CEP)
• Event-driven Architecture • A generic data management infrastructure for
processing in-flight data before data is potentially stored to deliver results in near real-time
• Programming language for defining rules
• It allows users to Aggregate/Correlate/Enrich/Detect Patterns in high speed streaming data
12 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Complex Event Processing (CEP)
Data/Msg. Feeds
Listener/SINK: User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
CEP Application Container
13 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Event Capture
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
CEP Application Container
• Events generated at sources
• Adaptor captures event and sends it into the Event Processing Network
14 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Event Processing
• Events processed in flight • Merging multiple event sources and types • Data enrichment by accessing external data sources
(e.g. databases)
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
CEP Application Container
15 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Event Dispatch
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
CEP Application Container
• Processing produces events • Adaptor receives event and sends it into the
downstream clients
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
3. Dynamic Complex Event Processing
Dynamic Complex Event Processing and OSGi.
16
Powering Dynamic M2M Event Processing with OSGi
17 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamic Environment
• Devices do not operate in a static environment
• Inputs change
• Knowledge is gained from analytics
• Additional systems want to receive the output
18 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamic Behavior
• Our Complex Event Processing engine needs
to be dynamic
• Ability to change behavior without stopping the
flow
• Allow a higher level system to change the
processing rules
19 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Enter OSGi
Powered by
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
CEP Application Container
OSGi Framework
Java
• Dynamic Modular System for Java
• Mature Lightweight Application
Framework
– Ideal for embedded environments
• Supports Module Lifecycle
– Install/start/stop/uninstall/upgrade
– Remotely manageable
– Versioning
• Services Model
– Advertise and discover services
– Modules are dependent on service
not implementation
20 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
OSGi Deployment Environment
OSGi Management System &
Repository
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi - CEP
OSGi Framework
Java
Remote Device
21 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamically Changing Behavior
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi CEP
OSGi Management System &
Repository
Remote Device
Deployed System. Happily processing…
22 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamically Changing Behavior
The Data Inputs Change
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi CEP
23 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamically Changing Behavior
Update the adaptor
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi CEP OSGi Management System &
Repository
Remote Device
24 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamically Changing Behavior
Process the Events Differently
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi CEP OSGi Management System
& Repository
Remote Device
25 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamically Changing Behavior
Forward to additional locations for processing
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi CEP OSGi Management System &
Repository
Remote Device
26 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamically Changing Behavior
Happily processing again…
OSGi Management System &
Repository
Remote Device
Data/Msg. Feeds Listener/SINK:
User Code (Plain Java)
Data Feed Adapters
Process Events (CEP)
OSGi CEP
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
4. Demo
How OSGi and Java enables smart data on M2M aggregators and gateways. Powering Dynamic M2M Event Processing with OSGi
27
28 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Example: JavaOne IoT In Motion
29 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
The Components
LCD Display
Phidget Temp Sensor
30 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
The Components
• OSGi based • Continuous Query Language (CQL) for defining rules • http://www.oracle.com/us/technologies/java/embedded/event-
processing/overview/index.html?ssSourceSiteId=opn
• Arm Based Linux platform
• http://www.raspberrypi.org/
• USB hardware devices
• Open and inexpensive
• http://www.phidgets.com/
• Hitachi’s OSGi Framework
31 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamic Behavior In Action
Start reporting temperature changes below ambient temperature
32 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamic Behavior In Action
Change the rules and redeploy remotely
33 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Dynamic Behavior In Action
Now reporting temperature changes above ambient temperature
© Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Hitachi Communication Technologies America, Inc.
Dynamic Complex Event Processing and OSGi.
Powering Dynamic M2M Event Processing with OSGi
6/12/2014
Walt Bowers
Chief Architect OSGi Solutions
END
34
Appendix
38 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Volume
3 006 477 107 200 GB added to the “digital universe” ~30% of it generated by machines
42 949 673 000 000 GB – 15 x increase 42 % will be generated by devices
Assuming 2GB/hr it will take 2.3 billion
years to download
According to IDC’s “Digital Universe in 2020“ study published in December 2012
39 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Velocity
40 © Hitachi Communication Technologies America, Inc. 2014. All rights reserved.
Variety
•Clickstream Data
•Twitter Feeds
•Facebook Postings
•Web Content
Web and Social Media
•Smart Meters Readings
•RFID Readings
•Oil Rig Sensors
•GP Signals
Machine-to-Machine
•Healthcare Claims
•Telecommunication Call Details Record
•Utility Billing Records
Big Transaction Data
•Facial Recognition
•Genetics
Biometrics
•Call Center Voice Recording
•Electronic Medical Records
Human Generated