View
2
Download
0
Category
Preview:
Citation preview
Internet of Things
Dilip Kumar Kari Dilip@hashmapinc.com
Agenda
• Problem Statement • IOT in Oil fields • Describing the use case
• Real-time operations monitoring • Seismic survey in deep oceans
• The Edge • Protocols
• Shipping the data • Data processing frameworks • Scalable Messaging Systems
• Processing the data • Real-time Distributed Processing Frameworks • Storage • Visualization
• Demo • Questions
Create New Requirements
• Businesses need trusted, real-time insights from full-fidelity data • Applications need access to both data in
motion and data at rest • IT needs to capture multi-directional IoT data
flows from point to point • Challenge: the IoT perimeter is jagged and
outside the data center
Collecting and Analyzing IOT Data
IOT + Predictive Analytics affects all industries • BI vs Data Science approach
• BI (traditional and doing for decades) • only structured data • Descriptive analysis • Standard Reporting • What happened?
• Data Science • Predictive analysis • prescriptive analysis (guidance of what to do on predicted
outcome) • Why? what will? what should id do?
Data Science Process
• Define Hypothesis to test or prediction to be made • Gather data DL (SQL + Hadoop) • Build Schema (schema on read/query) • Visualize the data • Build Analytical Model (SAS, MAtLab, MADLib,
Mahout, R…) • Evaluate the model results (probabilities, confidence
levels) - what is the output to different input params as they change
Today’s Architectural Limitations
• Data movement software was built for standardized data and one-way flows • Tools for unstructured data are customized,
disjointed and difficult to manage • Such tools cannot securely collect, conduct and
curate IoT data flows, which are multi-directional from point to point at the perimeter • Today’s architectures limit potential insight by
making it difficult and expensive to manage IoT data both in motion and at rest
The Use cases
• A large Oil & Gas services company is monitoring deep sea and onshore drilling jobs
across the world from Houston.
• A seismic survey company is generating raw data for oceanic surveys on behalf of BP,
Shells, Exxon’s of the world.
OILFIELD OPERATIONS
1000s of sensors per site
Temperature, Pressure, Flow,
Pumps, Actuators, Valves, etc.
REMOTE COMMS DEVICES
RTUs, PLCs, Instrumentation
Systems
DATA LINK
Satellite, Cellular, Radio
SCADA NETWORK
OPC Servers 15-20 for a large basin
DATA LINK
Satellite, Cellular, Radio
HISTORIAN/s
Time series
logs and history
DATA LINK
Corporate WAN
CORP SYSTEMS & HADOOP Systems of
Record & Analytics
Platform
ANALYTICS Real Time
Data +
All Other Datasets
Oilfield Data Collection
Data from the field, factory floors and plants
Technologies that enables IoT
• Bluetooth • zwave is the protocol used in most smart
home applications • LTE - communication specification for mobile
networks • Wifi-Direct - peer to peer communication
without need to have an access point • Open Standard Protocols like OPC UA/DA
Shipping the Data
AG.Imagesource"©SiemensAG2015,Allrightsreserved"
GATHER
DELIVER
PRIORITIZE
Trackfromtheedge(wellsite,equipment,etc.)
Throughthedatacenter(andback)
RECALIBRATE
Apache Nifi
It’s not just how quickly you move data… it’s about how quickly you can change behavior and seize new - Guarantee high fidelity data delivery
and eliminate data loss even with intermittent communications - make decisions from credible and reliable data
- Provide secure data collection and movement from data generation to data rest with 21 encryption algorithms
- Enable comprehensive provenance and lineage with a complete digital chain of custody for real time multidirectional data streams, flow specific quality of service, and visibility into valuation of your field data capture infrastructure
- Allow for real time prioritization of data and interactive changes to the data flow with visual command and control
RPC & OPC Specifications
• COM, DCOM, OLE • Classic OPC (Open Platform Communication) • OPC XML DA (Data Access) • OPC HDA (Historical Data Access) • OPC AE (Alarm & Events) • OPC Xi (Express Interface) • OPC UA
OPC UA
• OPC UA differs significantly from its predecessor ü International Standard IEC 62541 ü Multi-platform SOA based implementation, ü including portable ANSI C, Java and .NET implementations ü Scalability: from smart sensors and smart actuators to
mainframes (enterprise systems) ü Multi-threaded, as well as single-threaded/single-task
operation—necessary for porting the stack to embedded devices
ü Security, based on new standards ü Configurable time-outs for each service ü Chunking of big datagrams ü Model complex data structures for collaboration with other
standard organizations
Distributed Intelligence based on standards
• OPC UA standard for enterprise to connect DCS, ERP, MES, SCADA, PLC and Sensors
SCADA
• SCADA (Supervisory Control And Data Acquisition) is a system for remote monitoring and control that operates with coded signals over communication channels (using typically one communication channel per remote station). It is a type of industrial control system (ICS) • SCADA systems historically distinguish
themselves from other ICS systems by being large-scale processes that can include multiple sites, and large distances • These processes include industrial, infrastructure,
and facility-based processes
Choosing an Architecture
Do Lambda OR Kappa?
Lambda Architecture
Kappa Architecture
IOT Architecture
Producer Consumer+Spout RegionServer
HDFS+Hive
OPCServer
Broker
Push PushorPull
StormBolt(s)Bolt(s)
SCADA IoT Demo
Questions
Recommended