Research Activities in Ambient Intelligence

Research activities in

Ambient Intelligence

Fulvio Corno – [email protected]

Politecnico di Torino Dipartimento di Automatica e Informatica

e-Lite Research Group

http://elite.polito.it

Home Automation System (HAS)

6/11/2012 Dario Bonino - Home and Building

Automation Systems

2

Home automation Automation of the home, housework or

household activity.

(remote) control of: lighting

HVAC (Heating, Ventilation and Air Conditioning)

appliances

and other systems

Home automation system An integrated system (computer-based)

offering home automation functionalities

Integrates electrical devices in a house Through a communication network

Possibly includes devices using different communication protocols

Home Automation System (HAS)

6/11/2012 Dario Bonino - Home and Building Automation Systems 3

Building Automation System (BAS)


Automation Systems

4

Building Automation System (BAS)


Automation Systems

5

Building automation

The advanced functionality provided

by the control system of a building

E.g., security & access control, fire

detection & alarms, HVAC, lighting

control, air quality, smoke detection,

intrusion detection, environmental

control, asset location/management

Building Automation System(BAS)

A computerized, intelligent network

of electronic devices designed to

monitor and control the mechanical,

electronic, and lighting systems in a

building

Home vs. Building Automation


Automation Systems

6

Building

Automation Home

Automation

Home Automation is almost a subset of

Building Automation

Most functionalities are shared

Different protocols and technologies

Main goals

26/03/2010 e-lite - AmI+DOG 7

Interoperability

Intelligence

Man-Environment

interaction

Accessibility

Technology Overview


Automation Systems

8

RS-485

Home

Automation

Building Automation

Pot-pourri di dispositivi

20/04/2012 e-lite - Smart Buildings 9

Network Technology


Automation Systems

10

Bus

MyOpen

KNX

Modbus

Echelon

Dali

CAN

MBus

Powerline

Echelon

X10

Insteon

Wireless

ZigBee

Z-Wave

EnOcean

Main goals

26/03/2010 e-lite - AmI+DOG 11

Interoperability

Intelligence

Man-Environment

interaction

Accessibility

Abstract high-level

(semantic) model

DogOnt

Intelligent Multi-protocol

Gateway

DOG (Domotic OSGi

Gateway)

Modular architecture and

open APIs

Open Source

Application Area


Automation Systems

12

Automation

Real-time Control

CAN

KNX, MyHome, Insteon, Echelon, Modbus, X10, ZigBee, Z-Wave, EnOcean

Lighting

Dali

(all Automation)

Metering

Mbus

RS-485

KNX, Modbus, Echelon, ZigBee, Z-Wave,…

Entertainment

UPnP

DLNA

General purpose

Bluetooth

WiFi

Challenges

6/11/2012 Dario Bonino - Home and Building Automation Systems 13

Integration

Different electrical requirements

Different interaction modalities

Different behaviors (temporal, etc.)

Interoperation

Different protocols

Different interaction modalities

Master/slave

Peer-to-peer

Etc.

Modeling

Different technologies & assumptions

A single shared, common description

Pot-pourri di applicazioni


Intelligent Domotic Environments

6/11/2012 Dario Bonino - Dog 2.3 15

System Architecture


D D D D D D D D Domotic devices (switches,

buttons, relays, sensors, meters,

…)

Domotic bus (wired, wireless)

GW Bus-to-IP gateway

Bus-to-serial gateway GW

Ethernet, Wi-Fi, USB

User

Interface

Mobile, Web, Home Display,

Multi Touch, Accessibility,

Natural language, …

Data

analysis

ERP, Web services, Stream

processors, Datawarehouse

Dog

Bundles

Device abstraction, Event

abstraction, State abstraction,

Rules engine, …

User

Interface User

Interface

Smart

Appliance

In Ontology We Trust


Devices and networks exposed by means of a formal,

unique, representation

DogOnt (Ontology)

Applications only see DogOnt-based device descriptions

Functionalities

Notifications

Commands

States

State values

Internal representations and drivers must be ontology-

aware, at different degrees

E allora?


D D D D D D D D D D D D

Applicazione Applicazione Applicazione Applicazione

Una lingua franca




Rappresentazione neutrale

Architetture orizzontali e aperte





Funzionalità di base

API

Driver di interfacciamento ai protocolli

Scambio dati

Elaborazione real time Regole e scenari Servizio

applicativo



API


Scambio dati


applicativo

Applicazioni consumano

dati e servizi






API


Scambio dati


applicativo

Applicazioni controllano




DogOnt

26/03/2010 e-lite - AmI+DOG 23

DogOnt [ISWC08]

Ontology-based formal modeling of domotic

environments (OWL)

Technology independent device modeling

Operation semantics for DOG [TCE08]

Interoperation semantics through relations [AMI09]

Automatic generation of inter-operation rules [TCE09]

Semantic Modeling (DogOnt)

26/03/2010 e-lite - AmI+DOG 24

RDF

OWL

XML

XSD

Different Aspects In the same Model

A Lamp is

A thing (BuildingThing)

It is controllable (Controllable)

(On/Off) (OnOffFunctionality)

It can be either Lit or not Lit (OnOffState)

It is located in a Room (isIn Room)

It is connected to a domotic plant (isA

DomoticNetworkComponent)

26/03/2010 e-lite - AmI+DOG 25

Different Aspects in the same Model

26/03/2010 e-lite - AmI+DOG 26

Lamp

House

Plant

Electric

System

Controllable

Building

Thing

Building

Environment

Building Apartment

Room

IsIn / contains

OnOff

Functionality

Control

Functionality

Functionality

hasFunctionality

Discrete

State

OnOff

State

State

hasState

In Practice (Protégé & OWL)

26/03/2010 e-lite - AmI+DOG 27

<owl:Class rdf:about="#SimpleLamp">

<rdfs:comment

rdf:datatype="http://www.w3.org/2001/XMLSchema#string"

>Simple lamp that can be just turn on or turn off</rdfs:comment>

<owl:disjointWith>

<owl:Class rdf:about="#DimmerLamp"/>

</owl:disjointWith>

<rdfs:label

rdf:datatype="http://www.w3.org/2001/XMLSchema#string"

>SimpleLamp</rdfs:label>

<rdfs:subClassOf>

<owl:Restriction>

<owl:someValuesFrom rdf:resource="#QueryFunctionality"/>

<owl:onProperty>

<owl:ObjectProperty rdf:about="#hasFunctionality"/>

</owl:onProperty>

</owl:Restriction>

</rdfs:subClassOf>

<rdfs:subClassOf rdf:resource="#Lamp"/>

</owl:Class>

Sample Room Model in DogOnt

26/03/2010 e-lite - AmI+DOG 28

Sample Room

Lamp

Switch

OnOffFunctionality

OnNotification

OffCommand

OnOffNotification

Functionality

OffNotification

OnOffState

OnOffState

hasState

OnCommand

hasFunctionality

hasCommand hasCommand

isIn isIn

hasState

hasFunctionality

hasNotification

hasNotification

generatesCmd

generatesCmd

DOG - Domotic OSGi Gateway

26/03/2010 e-lite - AmI+DOG 29

http://domoticdog.sourceforge.net/

System Architecture

26/03/2010 e-lite - AmI+DOG 30



…)





User

Interface




Data

analysis



Dog

Bundles



Rules engine, …

The KNX demo box

26/03/2010 e-lite - AmI+DOG 31

GW

D

The Z-Wave demo box

26/03/2010 e-lite - AmI+DOG 32

GW

D

Texas Intruments eZ430-Chronos

26/03/2010 e-lite - AmI+DOG 33

DogEye user interface

26/03/2010 e-lite - AmI+DOG 34

DogEye Video

26/03/2010 e-lite - AmI+DOG 35

DogEye with an eye tracker

26/03/2010 e-lite - AmI+DOG 36

Mobile House Control

26/03/2010 e-lite - AmI+DOG 37

System Architecture

26/03/2010 e-lite - AmI+DOG 38



…)





User

Interface




Data

analysis



Dog

Bundles



Rules engine, …

OSGi Alliance

Open Services Gateway Initiative

defines open specifications for

Creating and delivering distributed and

concurrent applications, called bundles

Bundle deployment and installation (core

specification)

Communication among bundles (core specification)

Devices and drivers management (service

compendium specification)

26/03/2010 e-lite - AmI+DOG 39

OSGi definitions

OSGi frameworks

Implementation in JAVA of the OSGi specifications

(equinox, oscar, ..)

Manage the life cycle of the bundles

Resolve the dependences among bundles

Manage Service Registry

Bundles

Jar file

contain java classes and resources

Provide and use services

Services

java objects registered in the framework

Defined by java interfaces

26/03/2010 e-lite - AmI+DOG 40

High-Level Architecture


OSGi - based

2 main layers

3 main bundle groups

Core

Drivers

Add-ons

DogXMLEndPoint DogRESTEndPoint

DogStateMonitor DogScheduler DogExecutor DogDeviceManager DogNotificationManager

DogLogger DogConfigurator

DogDeviceFactory

DogOntLibrary DogDeviceModel

DogSemanticHouseModel

DogSimpleHouseModel

Dog2Library DogJaxBLibrary DogSemanticLibrary MeasureLibrary org.rxtx

Do

gAuto

Sta

rt

Core


Command handling (1)


Application

Dog

Command


DogStateMonitor DogScheduler DogExecutor DogDeviceManager DogNotificationManager


DogDeviceFactory

DogOntLibrary DogDeviceModel


DogSimpleHouseModel

Dog2Library DogJaxBLibrary DogSemanticLibrary MeasureLibrary org.rxtx

Do

gAuto

Sta

rt

KNX Modbus Echelon OpenWebNet ZWave

Command handling (2)


Command

Network Message

Notification handling (1)


Application

Dog

Notification


DogState

Monitor

Dog

Scheduler

Dog

Executor

DogDevice

Manager

DogNotification

Manager


DogDeviceFactory

DogOnt

Library

DogDevice

Model


DogSimpleHouseModel

Dog2Library DogJaxBLibrary DogSemantic

Library MeasureLibrary org.rxtx

Do

gAuto

Sta

rt

KNX Modbus Echelon OpenWebNet ZWave

Rules

PowerModel

Bundle

PowerBundle

Notification handling


In DogOnt we trust (3)


Thing

Building thing

Controllable

Appliances

WhiteGoods

Brown

Goods

HousePlants

Electrical

System

HVAC

System Security

UnControllablle

Semantic

House

Model

Device

Representations

Auto

Generation

Configuration Lamp1

Lamp2

Button1

Controls

Controls

Power

Model

Power

Consumption

Eletric Power

Consumption

Configuration

Applicatio

ns

DogOnt

Instances

Lamp1Pw = 18W

Device Category (structure)

26/03/2010 e-lite - AmI+DOG 48

public interface Lamp extends Lighting

{

public static int MATCH_TYPE=100;

public static int MATCH_SUB_TYPE=50;

public static int MATCH_MANUFACTURER=0;

public static String ON_OFF_STATE =

"OnOffState";

public void off();

public void on();

}

Lamp

OnOffFunctionality

OnCommand

OnOffState

hasState

OffCommand

hasFunctionality

hasCommand

hasCommand

Lighting

Lamp

instance

isA

Device Model (Structure)

26/03/2010 e-lite - AmI+DOG 49

public class DogLamp extends DogDevice implements Lamp

{

public DogLamp(BundleContext context, Properties properties){

super(context);

if(properties==null)

properties=new Properties();

properties.put(Constants.DEVICE_CATEGORY, Lamp.class.getCanonicalName());

this.setDeviceProperties(properties);

this.states.put(Lamp.ON_OFF_STATE,

new DogStatusVariable(

properties.getProperty(DogDeviceCostants.DEVICEURI),

new StatusVariable(Lamp.ON_OFF_STATE, StatusVariable.CM_DER, "-"),

"current state of Lamp",true));

this.registerDevice(Lamp.class.getName());

this.registerStatusVariable();

}

@Override

public void off(){

((Lamp) this.driver).off();

}

@Override

public void on(){

((Lamp) this.driver).on();

}

} Lamp

OnOffFunctionality

OnCommand

OnOffState

hasState

OffCommand hasFunctionality

hasCommand

Lighting

Lamp

instance

isA

Device Category (Hi-Fi)

26/03/2010 e-lite - AmI+DOG 50

package it.polito.elite.domotics.model.devicecategory; /** * HiFiCategory - automatically generated by DogOnt2Dog */ public interface HiFi extends Entertainment { public static int MATCH_TYPE=100; public static int MATCH_SUB_TYPE=50; public static int MATCH_MANUFACTURER=0; public static String STAND_BY_ON_OFF_STATE = "StandByOnOffState"; public static String SOURCE_SELECTION_STATE = "SourceSelectionState"; public static String PLAY_STATE = "PlayState"; public static String TUNER_STATE = "TunerState"; public static String VOLUME_LEVEL_STATE = "VolumeLevelState"; public void stop(); public void stepUp(); public void rew(); public void set(Object value); public void setVolume(Integer volume); public void next(); … public void goToTrack(Integer trackNumber); }

Automatic Generation

Template based

Common structure of device categories and device models

Fill “placeholders” (possibly empty) with information

extracted from DogOnt

SPARQL

Works on the schema only

26/03/2010 e-lite - AmI+DOG 51

Automatic Generation (recipe)

Device Categories 1. GetAllDevicesWithAncestors

2. Foreach device

1. getDeviceCommands

2. getDeviceStates

3. Write the device category

code

3. Compile all

4. Jar all with manifest

(bundleize)

5. Use in Dog2.0

Device Models 1. GetAllDevicesWithAncestors

2. Foreach device

1. getDeviceCommands

2. getDeviceStates

3. Write the device model code

3. Compile all

4. Jar all with manifest

(bundleize)

5. Use in Dog2.0

26/03/2010 e-lite - AmI+DOG 52

Queries (1) - Devices

Get all devices with ancestors

SELECT DISTINCT ?x WHERE { ?x

rdfs:subClassOf dogont:Controllable FILTER

(?x!=owl:Nothing)} ORDER BY ?x

Recurse over ?x and call

OntClass deviceClass =

this.modelLoader.getPlainOntModel().getOntCla

ss(deviceURI);

ExtendedIterator iter =

deviceClass.listSuperClasses();

26/03/2010 e-lite - AmI+DOG 53

Queries (2) - Commands Non-Parametric Commands

SELECT DISTINCT ?commandValue WHERE {

dogont:"+deviceClass+" rdfs:subClassOf

[rdf:type owl:Restriction; owl:onProperty

dogont:hasFunctionality; owl:someValuesFrom

?functionality] . ?functionality

rdfs:subClassOf dogont:ControlFunctionality.

?functionality rdfs:subClassOf [rdf:type

owl:Restriction; owl:onProperty

dogont:hasCommand; owl:someValuesFrom

?command] . ?command rdfs:subClassOf

dogont:NonParametricCommand . ?command

rdfs:subClassOf [rdf:type owl:Restriction;

owl:onProperty dogont:realCommandName;

owl:hasValue ?commandValue]} ORDER BY

?commandValue

Parametric Commands

SELECT DISTINCT ?commandValue ?commandParamValue WHERE {

dogont:"+deviceClass+" rdfs:subClassOf [rdf:type

owl:Restriction; owl:onProperty dogont:hasFunctionality;

owl:someValuesFrom ?functionality] . ?functionality

rdfs:subClassOf dogont:ControlFunctionality . ?functionality

rdfs:subClassOf [rdf:type owl:Restriction; owl:onProperty

dogont:hasCommand; owl:someValuesFrom ?command] . ?command

rdfs:subClassOf dogont:ParametricCommand . ?command

rdfs:subClassOf [rdf:type owl:Restriction; owl:onProperty

dogont:realCommandName; owl:hasValue ?commandValue] .

?command rdfs:subClassOf [rdf:type owl:Restriction;

owl:onProperty dogont:commandParamName; owl:hasValue

?commandParamValue]} ORDER BY ?commandValue

26/03/2010 e-lite - AmI+DOG 54

Queries(3) - States

SELECT ?state WHERE{

dogont:"+deviceClass+" rdfs:subClassOf

[rdf:type owl:Restriction;

owl:onProperty dogont:hasState;

owl:someValuesFrom ?state] } ORDER BY

?state

26/03/2010 e-lite - AmI+DOG 55

Accessibility / Disability

26/03/2010 e-lite - AmI+DOG 56

Testing Dog in a Real Home

26/03/2010 e-lite - AmI+DOG 57

Stream processing engine

Stream Processing spChains


Stream

Processing

Block

Eve

nt D

rains

Eve

nt S

ource

s

spChains

Field-level sensors / data sources

Aggregate / Computed

Measures

Pattern Match / Alerts

Field

Data Streams

Data-centric

Application

(e.g., E.R.P.)

Industrial

Stakeholders

(e.g., Management)

Chain Definition

Stream

Processing

Chains

http://elite.polito.it/spchains

http://elite.polito.it/spchains

Alcuni esempi di spChains


Ogni spChain è definita in XML

Libreria di spBlock parametrici (13++)

Creazione automatica di query sul motore Esper (o altri)

Open Source


Visual rule builder

26/03/2010 e-lite - AmI+DOG 61

Visual Rules Builder: video

26/03/2010 e-lite - AmI+DOG 62

Simulation

26/03/2010 e-lite - AmI+DOG 63

StateCharts

Apache

Commons

SPARQL

Simulation

26/03/2010 e-lite - AmI+DOG 64

SampleLamp1

OnOffFunctionality

OffCommand

OnOffStateLamp1

hasState

OnCommand

hasFunctionality

hasCommand hasCommand

&id;OffState

&id;OnState

Lamp Machine Template

&id;lampOn

&id;lampOff

OffStateValue

Lamp1 OnStateValue

Lamp1

hasStateValue hasStateValue

Lamp isA

Simulation – Bank Door

26/03/2010 e-lite - AmI+DOG 65

Intelligence + Security

26/03/2010 e-lite - AmI+DOG 66

OWL

Drools

SWRL

SPARQL

Rules Generation (DOG1.0)

Query SELECT DISTINCT ?x ?y ?v ?c ?d

?class ?cv WHERE { ?x a

dogont:Controllable . ?y a

dogont:Controllable . ?x

dogont:controlledObject ?y . ?x

dogont:hasFunctionality ?f . ?f

dogont:hasNotification ?n . ?n

dogont:notificationValue ?v . ?n

dogont:generateCommand ?c . ?d

dogont:hasFunctionality ?f2 . ?f2

dogont:hasCommand ?c . ?c

rdf:type ?class . ?class

rdfs:subClassOf

dogont:DiscreteCommand . ?class

rdfs:subClassOf [rdf:type

owl:Restriction; owl:onProperty

dogont:realCommandName;

owl:hasValue ?cv]}

Rule when

dev:Device( id == "http://elite.polito.it/ontologies/simplehome.owl#"?x")

cmd:StateCommand(state =="?v")

status:DeviceStatus( status == cmd, device == dev)

rulesCore:RulesCore()

message: DogMessage(dataType == DogBundle.DataTypeEnum.DEVICE_STATUS, data == status)

then

Device newDev = new Device("http://elite.polito.it/ontologies/DOGHouseModel.owl#"+?y+");

DeviceStatus newStatus = new DeviceStatus(newDev, new StateCommand("+?cv+”,StatusTypeEnum.DISCRETE));

DogMessage myCmd = new DogMessage(MessageTypeEnum.CMD,new GregorianCalendar(), message.getDataType(), newStatus);

rulesCore.takeActions(myCmd);

26/03/2010 e-lite - AmI+DOG 67

Energy management

26/03/2010 e-lite - AmI+DOG 68

In-Home Display videos

DFP: Direct Feedback of

istantaneous Power GSE: Goal Setting for consumed Energy

26/03/2010 e-lite - AmI+DOG 69

Energy Awareness

26/03/2010 e-lite - AmI+DOG 70

Contacts

26/03/2010 e-lite - AmI+DOG 71

Web site:

http://elite.polito.it

Dog2.0 information, documentation and downloads

http://domoticdog.sourceforge.net

Publications:

http://elite.polito.it/publications-mainmenu-81

Contact e-mail:

[email protected]

http://elite.polito.it/

http://domoticdog.sourceforge.net/






mailto:[email protected]

Get involved!

26/03/2010 e-lite - AmI+DOG 72

User interfaces / User experience

Artificial intelligence / Semantic modeling

Wireless [sensor] networks

Ambient Assisted Living applications

Stream data processing

Extension to Buildings and Industrial settings

Energy savings / Energy management

Linked-data information exchange

Device modeling / Environment modeling

…

Licenza d’uso

26/03/2010 e-lite - AmI+DOG 73

Questa presentazione è rilasciate con la licenza Creative

Commons “Attribuzione-Non commerciale-Condividi allo

stesso modo 2.5 Italia”

Siete liberi di riprodurre e modificare quest’opera, per

scopi non commerciali, e citando la fonte. Eventuali

versioni modificate dovranno essere rilasciate con la

stessa licenza

Testo completo della licenza:

http://creativecommons.org/licenses/by-nc-sa/2.5/it/