MMEA PlatformHarri Hytönen (Vaisala)September 23, 2015

Satellite

RadarIn-situ

Modelling Citizen science

Research

OverviewMMEA Platform = scalable real-time data processing with embedded notification, storage, routing, filtering and processing services.

Contributors

OSS Components• WSO2 Enterprise Service Bus (ESB)

– Integration solution• Service virtualization = proxy services• Message transformations• Transport (HTTP(S)/JMS/SMTP/FTP etc.)

switching– Authentication (WSO2 Identity Server) – Authorization– Monitoring and diagnostics

OSS Components• Apache ActiveMQ

– messaging backbone– quality of service

• PostgreSQL RDBMS• Apache Tomcat• Esper

– CEP engine• Wavellite

– framework for situation awareness• Octave

– High level language for computations• Puppet & Artifactory

– Deployment tools• CentOS

Commercial Components• Amazon AWS

– EC2 (for hosting)– RDS

• Profium Sense– RDF storage– Inference engine

• Confluence & JIRA– Documentation and issue tracking

MMEA Platform Deployment

Amazon EC2

binaries

config

buildssource code

Pilot - Indoor Air Quality (VTT, HiQ)deployment MMEA Testbed

Indoor Pilot MMEA platform

GetObservation

PutMeasurementComponents and

interfaces 2::Indoor observ ation data

source

GetObservation

PutMeasurement

«device»Measurement

dev ices

(from Components and interfaces 2)

GetObservationComponents and

interfaces 2::Measurement

Components and interfaces 2::

IndoorObserv ation

MMEA Framework::

SensorML

UI

GetObservationComponents and

interfaces 2::Indoor end-user system

UI

GetObservation

Components and interfaces 2::IndoorData

MMEAIndoorDataServ ice

PutMeasurement

MMEA Framework::MMEAObserv ationServ ice

GetObservation

MMEA Framework::MMEADataQualityServ ice

Other Use Cases• Helsinki area outdoor air quality• Weather stations data ingestion• Weather radar product generation• Hydrological data ingestion• Situational awareness applications

– Farmer’s situational awareness– Storm path detection– Semantic end user service

Complex Event Processing• Esper

– High speed event processing– Identifies meaningful events

• Targeted to real-time Event Driven Architectures• Designed for high-volume event correlation where millions of

events coming in would make it impossible to store them all to later query them using classical database architecture

• Some examples of applications in MMEA:– Detect patterns among events– Filter events and event aggregation– Generating notifications and alerts based on event patterns

Computations in the MMEA platform

ESB

Data source

EsperComplex event processing

Computation serviceOnline and offline computations

Mediator Mediator

Computation service

Computation service

Computation task1. Run preprocessor

2. Run Octave script

3. Run postprocessor

Workspace

Input data and parameters

Thank You – Questions?