Research Activities in Ambient Intelligence

Research activities in

Ambient Intelligence

Fulvio Corno – fulvio.corno@polito.it

Politecnico di Torino Dipartimento di Automatica e Informatica

e-Lite Research Group

http://elite.polito.it

Home Automation System (HAS)

6/11/2012 Dario Bonino - Home and Building

Automation Systems

Home automation Automation of the home, housework or

household activity.

(remote) control of: lighting

HVAC (Heating, Ventilation and Air Conditioning)

appliances

and other systems

Home automation system An integrated system (computer-based)

offering home automation functionalities

Integrates electrical devices in a house Through a communication network

Possibly includes devices using different communication protocols

Home Automation System (HAS)

6/11/2012 Dario Bonino - Home and Building Automation Systems 3

Building Automation System (BAS)

Automation Systems

Building Automation System (BAS)

Automation Systems

Building automation

The advanced functionality provided

by the control system of a building

E.g., security & access control, fire

detection & alarms, HVAC, lighting

control, air quality, smoke detection,

intrusion detection, environmental

control, asset location/management

Building Automation System(BAS)

A computerized, intelligent network

of electronic devices designed to

monitor and control the mechanical,

electronic, and lighting systems in a

building

Home vs. Building Automation

Automation Systems

Building

Automation Home

Automation

Home Automation is almost a subset of

Building Automation

Most functionalities are shared

Different protocols and technologies

Main goals

26/03/2010 e-lite - AmI+DOG 7

Interoperability

Intelligence

Man-Environment

interaction

Accessibility

Technology Overview

Automation Systems

RS-485

Automation

Building Automation

Pot-pourri di dispositivi

20/04/2012 e-lite - Smart Buildings 9

Network Technology

Automation Systems

MyOpen

Modbus

Echelon

Powerline

Echelon

Insteon

Wireless

ZigBee

Z-Wave

EnOcean

Main goals

26/03/2010 e-lite - AmI+DOG 11

Interoperability

Intelligence

Man-Environment

interaction

Accessibility

Abstract high-level

(semantic) model

DogOnt

Intelligent Multi-protocol

Gateway

DOG (Domotic OSGi

Gateway)

Modular architecture and

open APIs

Open Source

Application Area

Automation Systems

Automation

Real-time Control

KNX, MyHome, Insteon, Echelon, Modbus, X10, ZigBee, Z-Wave, EnOcean

Lighting

(all Automation)

Metering

RS-485

KNX, Modbus, Echelon, ZigBee, Z-Wave,…

Entertainment

General purpose

Bluetooth

Challenges

6/11/2012 Dario Bonino - Home and Building Automation Systems 13

Integration

Different electrical requirements

Different interaction modalities

Different behaviors (temporal, etc.)

Interoperation

Different protocols

Different interaction modalities

Master/slave

Peer-to-peer

Modeling

Different technologies & assumptions

A single shared, common description

Pot-pourri di applicazioni

Intelligent Domotic Environments

6/11/2012 Dario Bonino - Dog 2.3 15

System Architecture

D D D D D D D D Domotic devices (switches,

buttons, relays, sensors, meters,

Domotic bus (wired, wireless)

GW Bus-to-IP gateway

Bus-to-serial gateway GW

Ethernet, Wi-Fi, USB

Interface

Mobile, Web, Home Display,

Multi Touch, Accessibility,

Natural language, …

analysis

ERP, Web services, Stream

processors, Datawarehouse

Bundles

Device abstraction, Event

abstraction, State abstraction,

Rules engine, …

Interface User

Interface

Appliance

In Ontology We Trust

Devices and networks exposed by means of a formal,

unique, representation

DogOnt (Ontology)

Applications only see DogOnt-based device descriptions

Functionalities

Notifications

Commands

States

State values

Internal representations and drivers must be ontology-

aware, at different degrees

E allora?

D D D D D D D D D D D D

Applicazione Applicazione Applicazione Applicazione

Una lingua franca

Rappresentazione neutrale

Architetture orizzontali e aperte

Funzionalità di base

Driver di interfacciamento ai protocolli

Scambio dati

Elaborazione real time Regole e scenari Servizio

applicativo

Scambio dati

applicativo

Applicazioni consumano

dati e servizi

Scambio dati

applicativo

Applicazioni controllano

DogOnt

26/03/2010 e-lite - AmI+DOG 23

DogOnt [ISWC08]

Ontology-based formal modeling of domotic

environments (OWL)

Technology independent device modeling

Operation semantics for DOG [TCE08]

Interoperation semantics through relations [AMI09]

Automatic generation of inter-operation rules [TCE09]

Semantic Modeling (DogOnt)

26/03/2010 e-lite - AmI+DOG 24

Different Aspects In the same Model

A Lamp is

A thing (BuildingThing)

It is controllable (Controllable)

(On/Off) (OnOffFunctionality)

It can be either Lit or not Lit (OnOffState)

It is located in a Room (isIn Room)

It is connected to a domotic plant (isA

DomoticNetworkComponent)

26/03/2010 e-lite - AmI+DOG 25

Different Aspects in the same Model

26/03/2010 e-lite - AmI+DOG 26

Electric

System

Controllable

Building

Environment

Building Apartment

IsIn / contains

Functionality

Control

Functionality

hasFunctionality

Discrete

hasState

In Practice (Protégé & OWL)

26/03/2010 e-lite - AmI+DOG 27

<owl:Class rdf:about="#SimpleLamp">

<rdfs:comment

rdf:datatype="http://www.w3.org/2001/XMLSchema#string"

>Simple lamp that can be just turn on or turn off</rdfs:comment>

<owl:disjointWith>

<owl:Class rdf:about="#DimmerLamp"/>

</owl:disjointWith>

<rdfs:label

rdf:datatype="http://www.w3.org/2001/XMLSchema#string"

>SimpleLamp</rdfs:label>

<rdfs:subClassOf>

<owl:Restriction>

<owl:someValuesFrom rdf:resource="#QueryFunctionality"/>

<owl:onProperty>

<owl:ObjectProperty rdf:about="#hasFunctionality"/>

</owl:onProperty>

</owl:Restriction>

</rdfs:subClassOf>

<rdfs:subClassOf rdf:resource="#Lamp"/>

</owl:Class>

Sample Room Model in DogOnt

26/03/2010 e-lite - AmI+DOG 28

Sample Room

Switch

OnOffFunctionality

OnNotification

OffCommand

OnOffNotification

Functionality

OffNotification

OnOffState

hasState

OnCommand

hasFunctionality

hasCommand hasCommand

isIn isIn

hasState

hasFunctionality

hasNotification

generatesCmd

DOG - Domotic OSGi Gateway

26/03/2010 e-lite - AmI+DOG 29

http://domoticdog.sourceforge.net/

System Architecture

26/03/2010 e-lite - AmI+DOG 30

Interface

analysis

Bundles

Rules engine, …

The KNX demo box

26/03/2010 e-lite - AmI+DOG 31

The Z-Wave demo box

26/03/2010 e-lite - AmI+DOG 32

Texas Intruments eZ430-Chronos

26/03/2010 e-lite - AmI+DOG 33

DogEye user interface

26/03/2010 e-lite - AmI+DOG 34

DogEye Video

26/03/2010 e-lite - AmI+DOG 35

DogEye with an eye tracker

26/03/2010 e-lite - AmI+DOG 36

Mobile House Control

26/03/2010 e-lite - AmI+DOG 37

System Architecture

26/03/2010 e-lite - AmI+DOG 38

Interface

analysis

Bundles

Rules engine, …

OSGi Alliance

Open Services Gateway Initiative

defines open specifications for

Creating and delivering distributed and

concurrent applications, called bundles

Bundle deployment and installation (core

specification)

Communication among bundles (core specification)

Devices and drivers management (service

compendium specification)

26/03/2010 e-lite - AmI+DOG 39

OSGi definitions

OSGi frameworks

Implementation in JAVA of the OSGi specifications

(equinox, oscar, ..)

Manage the life cycle of the bundles

Resolve the dependences among bundles

Manage Service Registry

Bundles

Jar file

contain java classes and resources

Provide and use services

Services

java objects registered in the framework

Defined by java interfaces

26/03/2010 e-lite - AmI+DOG 40

High-Level Architecture

OSGi - based

2 main layers

3 main bundle groups

Drivers

Add-ons

DogXMLEndPoint DogRESTEndPoint

DogStateMonitor DogScheduler DogExecutor DogDeviceManager DogNotificationManager

DogLogger DogConfigurator

DogDeviceFactory

DogOntLibrary DogDeviceModel

DogSemanticHouseModel

DogSimpleHouseModel

Dog2Library DogJaxBLibrary DogSemanticLibrary MeasureLibrary org.rxtx

Command handling (1)

Application

Command

DogStateMonitor DogScheduler DogExecutor DogDeviceManager DogNotificationManager

DogDeviceFactory

DogOntLibrary DogDeviceModel

DogSimpleHouseModel

Dog2Library DogJaxBLibrary DogSemanticLibrary MeasureLibrary org.rxtx

KNX Modbus Echelon OpenWebNet ZWave

Command handling (2)

Command

Network Message

Notification handling (1)

Application

Notification

DogState

Monitor

Scheduler

Executor

DogDevice

Manager

DogNotification

Manager

DogDeviceFactory

DogOnt

Library

DogDevice

DogSimpleHouseModel

Dog2Library DogJaxBLibrary DogSemantic

Library MeasureLibrary org.rxtx

KNX Modbus Echelon OpenWebNet ZWave

PowerModel

Bundle

PowerBundle

Notification handling

In DogOnt we trust (3)

Building thing

Controllable

Appliances

WhiteGoods

HousePlants

Electrical

System

System Security

UnControllablle

Semantic

Device

Representations

Generation

Configuration Lamp1

Button1

Controls

Consumption

Eletric Power

Consumption

Configuration

Applicatio

DogOnt

Instances

Lamp1Pw = 18W

Device Category (structure)

26/03/2010 e-lite - AmI+DOG 48

public interface Lamp extends Lighting

public static int MATCH_TYPE=100;

public static int MATCH_SUB_TYPE=50;

public static int MATCH_MANUFACTURER=0;

public static String ON_OFF_STATE =

"OnOffState";

public void off();

public void on();

OnOffFunctionality

OnCommand

OnOffState

hasState

OffCommand

hasFunctionality

hasCommand

Lighting

instance

Device Model (Structure)

26/03/2010 e-lite - AmI+DOG 49

public class DogLamp extends DogDevice implements Lamp

public DogLamp(BundleContext context, Properties properties){

super(context);

if(properties==null)

properties=new Properties();

properties.put(Constants.DEVICE_CATEGORY, Lamp.class.getCanonicalName());

this.setDeviceProperties(properties);

this.states.put(Lamp.ON_OFF_STATE,

new DogStatusVariable(

properties.getProperty(DogDeviceCostants.DEVICEURI),

new StatusVariable(Lamp.ON_OFF_STATE, StatusVariable.CM_DER, "-"),

"current state of Lamp",true));

this.registerDevice(Lamp.class.getName());

this.registerStatusVariable();

@Override

public void off(){

((Lamp) this.driver).off();

@Override

public void on(){

((Lamp) this.driver).on();

} Lamp

OnOffFunctionality

OnCommand

OnOffState

hasState

OffCommand hasFunctionality

hasCommand

Lighting

instance

Device Category (Hi-Fi)

26/03/2010 e-lite - AmI+DOG 50

package it.polito.elite.domotics.model.devicecategory; /** * HiFiCategory - automatically generated by DogOnt2Dog */ public interface HiFi extends Entertainment { public static int MATCH_TYPE=100; public static int MATCH_SUB_TYPE=50; public static int MATCH_MANUFACTURER=0; public static String STAND_BY_ON_OFF_STATE = "StandByOnOffState"; public static String SOURCE_SELECTION_STATE = "SourceSelectionState"; public static String PLAY_STATE = "PlayState"; public static String TUNER_STATE = "TunerState"; public static String VOLUME_LEVEL_STATE = "VolumeLevelState"; public void stop(); public void stepUp(); public void rew(); public void set(Object value); public void setVolume(Integer volume); public void next(); … public void goToTrack(Integer trackNumber); }

Automatic Generation

Template based

Common structure of device categories and device models

Fill “placeholders” (possibly empty) with information

extracted from DogOnt

SPARQL

Works on the schema only

26/03/2010 e-lite - AmI+DOG 51

Automatic Generation (recipe)

Device Categories 1. GetAllDevicesWithAncestors

2. Foreach device

1. getDeviceCommands

2. getDeviceStates

3. Write the device category

3. Compile all

4. Jar all with manifest

(bundleize)

5. Use in Dog2.0

Device Models 1. GetAllDevicesWithAncestors

2. Foreach device

1. getDeviceCommands

2. getDeviceStates

3. Write the device model code

3. Compile all

4. Jar all with manifest

(bundleize)

5. Use in Dog2.0

26/03/2010 e-lite - AmI+DOG 52

Queries (1) - Devices

Get all devices with ancestors

SELECT DISTINCT ?x WHERE { ?x

rdfs:subClassOf dogont:Controllable FILTER

(?x!=owl:Nothing)} ORDER BY ?x

Recurse over ?x and call

OntClass deviceClass =

this.modelLoader.getPlainOntModel().getOntCla

ss(deviceURI);

ExtendedIterator iter =

deviceClass.listSuperClasses();

26/03/2010 e-lite - AmI+DOG 53

Queries (2) - Commands Non-Parametric Commands

SELECT DISTINCT ?commandValue WHERE {

dogont:"+deviceClass+" rdfs:subClassOf

[rdf:type owl:Restriction; owl:onProperty

dogont:hasFunctionality; owl:someValuesFrom

?functionality] . ?functionality

rdfs:subClassOf dogont:ControlFunctionality.

?functionality rdfs:subClassOf [rdf:type

owl:Restriction; owl:onProperty

dogont:hasCommand; owl:someValuesFrom

?command] . ?command rdfs:subClassOf

dogont:NonParametricCommand . ?command

rdfs:subClassOf [rdf:type owl:Restriction;

owl:onProperty dogont:realCommandName;

owl:hasValue ?commandValue]} ORDER BY

?commandValue

Parametric Commands

SELECT DISTINCT ?commandValue ?commandParamValue WHERE {

dogont:"+deviceClass+" rdfs:subClassOf [rdf:type

owl:Restriction; owl:onProperty dogont:hasFunctionality;

owl:someValuesFrom ?functionality] . ?functionality

rdfs:subClassOf dogont:ControlFunctionality . ?functionality

rdfs:subClassOf [rdf:type owl:Restriction; owl:onProperty

dogont:hasCommand; owl:someValuesFrom ?command] . ?command

rdfs:subClassOf dogont:ParametricCommand . ?command

rdfs:subClassOf [rdf:type owl:Restriction; owl:onProperty

dogont:realCommandName; owl:hasValue ?commandValue] .

?command rdfs:subClassOf [rdf:type owl:Restriction;

owl:onProperty dogont:commandParamName; owl:hasValue

?commandParamValue]} ORDER BY ?commandValue

26/03/2010 e-lite - AmI+DOG 54

Queries(3) - States

SELECT ?state WHERE{

dogont:"+deviceClass+" rdfs:subClassOf

[rdf:type owl:Restriction;

owl:onProperty dogont:hasState;

owl:someValuesFrom ?state] } ORDER BY

?state

26/03/2010 e-lite - AmI+DOG 55

Accessibility / Disability

26/03/2010 e-lite - AmI+DOG 56

Testing Dog in a Real Home

26/03/2010 e-lite - AmI+DOG 57

Stream processing engine

Stream Processing spChains

Stream

Processing

spChains

Field-level sensors / data sources

Aggregate / Computed

Measures

Pattern Match / Alerts

Data Streams

Data-centric

Application

(e.g., E.R.P.)

Industrial

Stakeholders

(e.g., Management)

Chain Definition

Stream

Processing

Chains

http://elite.polito.it/spchains

Alcuni esempi di spChains

Ogni spChain è definita in XML

Libreria di spBlock parametrici (13++)

Creazione automatica di query sul motore Esper (o altri)

Open Source

Visual rule builder

26/03/2010 e-lite - AmI+DOG 61

Visual Rules Builder: video

26/03/2010 e-lite - AmI+DOG 62

Simulation

26/03/2010 e-lite - AmI+DOG 63

StateCharts

Apache

Commons

SPARQL

Simulation

26/03/2010 e-lite - AmI+DOG 64

SampleLamp1

OnOffFunctionality

OffCommand

OnOffStateLamp1

hasState

OnCommand

hasFunctionality

hasCommand hasCommand

&id;OffState

&id;OnState

Lamp Machine Template

&id;lampOn

&id;lampOff

OffStateValue

Lamp1 OnStateValue

hasStateValue hasStateValue

Lamp isA

Simulation – Bank Door

26/03/2010 e-lite - AmI+DOG 65

Intelligence + Security

26/03/2010 e-lite - AmI+DOG 66

Drools

SPARQL

Rules Generation (DOG1.0)

Query SELECT DISTINCT ?x ?y ?v ?c ?d

?class ?cv WHERE { ?x a

dogont:Controllable . ?y a

dogont:Controllable . ?x

dogont:controlledObject ?y . ?x

dogont:hasFunctionality ?f . ?f

dogont:hasNotification ?n . ?n

dogont:notificationValue ?v . ?n

dogont:generateCommand ?c . ?d

dogont:hasFunctionality ?f2 . ?f2

dogont:hasCommand ?c . ?c

rdf:type ?class . ?class

rdfs:subClassOf

dogont:DiscreteCommand . ?class

rdfs:subClassOf [rdf:type

owl:Restriction; owl:onProperty

dogont:realCommandName;

owl:hasValue ?cv]}

Rule when

dev:Device( id == "http://elite.polito.it/ontologies/simplehome.owl#"?x")

cmd:StateCommand(state =="?v")

status:DeviceStatus( status == cmd, device == dev)

rulesCore:RulesCore()

message: DogMessage(dataType == DogBundle.DataTypeEnum.DEVICE_STATUS, data == status)

Device newDev = new Device("http://elite.polito.it/ontologies/DOGHouseModel.owl#"+?y+");

DeviceStatus newStatus = new DeviceStatus(newDev, new StateCommand("+?cv+”,StatusTypeEnum.DISCRETE));

DogMessage myCmd = new DogMessage(MessageTypeEnum.CMD,new GregorianCalendar(), message.getDataType(), newStatus);

rulesCore.takeActions(myCmd);

26/03/2010 e-lite - AmI+DOG 67

Energy management

26/03/2010 e-lite - AmI+DOG 68

In-Home Display videos

DFP: Direct Feedback of

istantaneous Power GSE: Goal Setting for consumed Energy

26/03/2010 e-lite - AmI+DOG 69

Energy Awareness

26/03/2010 e-lite - AmI+DOG 70

Contacts

26/03/2010 e-lite - AmI+DOG 71

Web site:

http://elite.polito.it

Dog2.0 information, documentation and downloads

http://domoticdog.sourceforge.net

Publications:

http://elite.polito.it/publications-mainmenu-81

Contact e-mail:

fulvio.corno@polito.it

Get involved!

26/03/2010 e-lite - AmI+DOG 72

User interfaces / User experience

Artificial intelligence / Semantic modeling

Wireless [sensor] networks

Ambient Assisted Living applications

Stream data processing

Extension to Buildings and Industrial settings

Energy savings / Energy management

Linked-data information exchange

Device modeling / Environment modeling

Licenza d’uso

26/03/2010 e-lite - AmI+DOG 73

Questa presentazione è rilasciate con la licenza Creative

Commons “Attribuzione-Non commerciale-Condividi allo

stesso modo 2.5 Italia”

Siete liberi di riprodurre e modificare quest’opera, per

scopi non commerciali, e citando la fonte. Eventuali

versioni modificate dovranno essere rilasciate con la

stessa licenza

Testo completo della licenza:

http://creativecommons.org/licenses/by-nc-sa/2.5/it/

Research Activities in Ambient Intelligence

Technology

Ambient Intelligence in Everyday Life

AMBIENT INTELLIGENCE PARADIGM TO CONTROL DEVICES …khaledelleithy.org/Journals/Ambient Intelligence Paradigm to... · AMBIENT INTELLIGENCE PARADIGM TO CONTROL DEVICES THROUGH MOBILE

MAS 961 - Ambient Intelligence - MIT OpenCourseWare · Ambient Intelligence: Vision Ambient Intelligence envisions a world where people are surrounded by intelligent and intuitive

Ambient Intelligence Abdulsalam Alsunaidi. Ambient Intelligence Definition Ambient Intelligence (AmI) is the vision that technology will become invisible,

01QZP - Ambient intelligence - polito.it · AmI Design Process 01QZP - Ambient intelligence Fulvio Corno Politecnico di Torino, 2016/2017

Ambient intelligence präsentation

Ambient Intelligence — Software and Applications

Context Management in Ambient Intelligence using Semantic ... · 24. 06. 2014 LeMAS 2016 Summer School 6 Ambient Intelligence What is Ambient Intelligence (AmI)? In an AmI world,

Ambient Intelligence: An Overview

Carnegie Mellon Ambient Intelligence Labprojects.itsc.uah.edu/datamining/meeting06/docs/Session_4_Cai_revised.pdfCarnegie Mellon Ambient Intelligence Lab Spatiotemporal dynamics of

Definitions of Ambient Intelligence · Definitions of Ambient Intelligence 01QZP Ambient intelligence Fulvio Corno Politecnico di Torino, 2015/2016 ... Adaptive Transparent Ubiquitous

Ambient Intelligence - Parham Beheshti

Middlesex Universityeis.mdx.ac.uk/staffpages/juanaugusto/AITAmI2006.pdf · Artiﬁcial Intelligence Techniques for Ambient Intelligence (AITAmI06) Ambient intelligence [1] (AmI) is

Presentazione csp-ambient intelligence

AMBIENT INTELLIGENCE by Bhagyasri Matta

Ambient intelligence & Ubiquitous Computing

Ao Dai : Agent OrienteD Ambient Intelligence · 2015. 8. 13. · Ao Dai : Agent OrienteD Ambient Intelligence Ambient Intelligence Context-awareness Agents CLAIM Agenti cation Interaction

Ambient 2013: Persuasion Mobility in Ambient Intelligence

Ambient Intelligence: The MyCampus Experience

Ambient Intelligence: Concepts and Applications