D.5.1: Ambient Intelligence in Assistive Technologies...G.A.247447 Collaborative Project of the 7th Framework Programme Work Package 5 AmI and Social Network Services D.5.1: Ambient

G.A.247447 Collaborative Project of the 7th Framework Programme

Work Package 5 AmI and Social Network Services

D.5.1: Ambient Intelligence in Assistive Technologies

(State-of-the-Art)

Fundació Privada Barcelona Digital Centre Tecnològic Version 1.3

16/06/2011

www.BrainAble.org

D5.1_Ambient_Intelligence_in_Assistive_Technologies_v1.3_FINAL (public) page 2 of 66

DDooccuummeenntt IInnffoorrmmaattiioonn Project Number 247447 Acronym BrainAble

Full title Autonomy and social inclusion through mixed reality Brain-Computer Interfaces: connecting the disabled to their physical and social world

Project URL http://www.BrainAble.org

EU Project officer Jan Komarek

Deliverable Number 5.1 Title Ambient Intelligence in Assistive Technologies

Work package Number 5 Title AmI and Social Network Services

Date of delivery Contractual PM04 Actual PM18 (resubmission)

Status Reviewed Final

Nature Prototype Report Dissemination Other

Dissemination Level Public Consortium

Authors (Partner) Fundació Privada Barcelona Digital Centre Tecnològic (BDCT)

Responsible Author Agustin Navarro Email [email protected]

Partner BDCT Phone +34 93 553 45 40

Abstract (for dissemination)

State of the Art about Ambient Intelligence with special emphasis to its application in assisted environments

Keywords AmI, Ambient Assisted Living, Context-awareness, Smart devices, Interoperability

Version Log

Issue Date Version Author Change

31/01/2010 DRAFT - v.0.1 Agustin Navarro First released version for internal reviewers

28/04/2010 Version 1.0 Agustin Navarro Details and feedback from partners

28/04/2010 Version 1.1 Agustin Navarro Formatting, final version released to the P.O.

29/05/2010 Version 1.2 Agustin Navarro Formatting

16/06/2011 Version 1.3 Agustin Navarro Upgraded version released to the PO

The information in this document is provided as is and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. Its owner is

not liable for damages resulting from the use of erroneous or incomplete confidential information.

http://www.brainable.org/�


IInnddeexx

1 INTRODUCTION ......................................................................................................................... 6

2 CONTEXT IN ASSISTIVE TECHNOLOGIES ..................................................................................... 7

3 AMI BASIS AND TECHNOLOGIES ................................................................................................ 7

3.1 SENSORS AND SENSOR NETWORKS .................................................................................................. 8 3.2 PERVASIVE COMPUTING ................................................................................................................ 9 3.3 ARTIFICIAL INTELLIGENCE ............................................................................................................ 10

4 DEVICE NETWORKING AND INTEROPERABILITY ...................................................................... 11

4.1 CONNECTIVITY AND NETWORKING TECHNOLOGIES ............................................................................ 12 4.2 CONTROL AND NETWORKING PROTOCOLS AND STANDARDS ................................................................ 14 4.3 MIDDLEWARE ........................................................................................................................... 19

5 AMI APPLICATIONS ................................................................................................................. 22

5.1 COMMERCIAL PRODUCTS ............................................................................................................ 22 5.2 RELATED APPROACHES AND TECHNIQUES ....................................................................................... 41 5.3 RELATED PROJECTS .................................................................................................................... 44

6 CONSOLIDATING AMI – THE NEXT STEP .................................................................................. 55

7 CONCLUSION ........................................................................................................................... 58

8 REFERENCES ............................................................................................................................ 59

9 LIST OF KEY WORDS/ABBREVIATIONS ..................................................................................... 64


LLiisstt ooff ffiigguurreess FIGURE 1: PERVASIVE COMPUTING ENABLES INTERCONNECTIVITY AND IMPLIES THE AVAILABILITY OF INFORMATION AT

ANY PLACE ANY TIME ......................................................................................................................................................... 10

FIGURE 2: DIAGRAM OF A SIMPLIFIED AMI SYSTEM. THE REASONING COMPONENT PROVIDES THE SYSTEM THE ABILITY TO BE ADAPTABLE AND RESPONSIVE FOR THE BENEFIT OF THE USER. .............................................................................. 11

FIGURE 3: UCH (UNIVERSAL CONTROL HUB) APPLICATION EXAMPLE ................................................................................ 19

FIGURE 4: KADEX SYSTEM DIAGRAM .................................................................................................................................. 24

FIGURE 5: CRESTRON TOUCH PANEL .................................................................................................................................. 24

FIGURE 6: HAI PRODUCT EXAMPLES: OMNITOUCH PANEL (LEFT) AND SCENE SWITCH (RIGHT) ........................................ 25

FIGURE 7: LIFE|WARE SYSTEM (LEFT) AND LIFE|WARE CONTROLLER (RIGHT) .................................................................. 25

FIGURE 8: SUPERNA PRODUCT EXAMPLES: SUPERNA 10” TOUCH PANEL (LEFT), SUPERNA PDA INTERFACE (MIDDLE), AND SUPERNA CONTROLBOX (RIGHT)........................................................................................................................................ 26

FIGURE 9: IMAGES OF THE JURA IMPRESSA F90 + CONNECTIVITY KIT ................................................................................ 27

FIGURE 10: SIEMENS SERVE@HOME EXAMPLES: REMOTE CONTROLLED WASHING MACHINE (LEFT) AND DISHWASHER (RIGHT) 27

FIGURE 11: MIELE MIELE@HOME APPLIANCES ............................................................................................................... 27

FIGURE 12: SYSTEM OVERVIEW OF THE 8-SERIES BY LUTRON ........................................................................................ 28

FIGURE 13: LIGHTING CONTROLLER HELIO LRC 5040/10 ................................................................................................ 28

FIGURE 14: KADEX CANOPY RECEIVER WITHOUT AND WITH A CEILING LIGHT MOUNTED ............................................. 29

FIGURE 15: ADOCO RADIO CONTROLLED POWER SWITCH ............................................................................................. 29

FIGURE 16: ENOCEAN LIGHT ACTUATOR ......................................................................................................................... 29

FIGURE 17: THE BLIND CONTROLLER FROM TELDA ELECTRONICS .................................................................................. 30

FIGURE 18: ABB SHUTTER ACTUATOR, 4 FOLD WITH MANUAL OPERATION ................................................................... 30

FIGURE 19: THE AF24LON DAMPER ACTUATOR FROM BELIMO AUTOMATION LTD. ...................................................... 31

FIGURE 20: SDPASS - IDENTIFIER AND ACCESS CONTROL DEVICE ................................................................................... 31

FIGURE 21: TECOMAT FOXTROT MODULE AND INELS ELEMENTS................................................................................... 32

FIGURE 22: DLINK DCS-5300G ......................................................................................................................................... 32

FIGURE 23: INFRARED READER FOR LOCAL POSITIONING SYSTEMS ............................................................................... 33

FIGURE 24: OPEN SESAME OSHS2 SYSTEM COMPONENTS ............................................................................................. 33

FIGURE 25: TRUTH SENTRY II WINDOW SYSTEM ............................................................................................................. 34

FIGURE 26: ELEVATOR ACCESS CONTROL MODULE ........................................................................................................ 34

FIGURE 27: BLOOD SUGAR METER SMARTLAB®GENIE .................................................................................................... 35

FIGURE 28: BLOOD PRESSURE METER SWEETHEART ...................................................................................................... 35

FIGURE 29: EMERGENCY CALL DEVICE ............................................................................................................................ 35

FIGURE 30: KADEX BRACELET TRANSMITTER .................................................................................................................. 36

FIGURE 31: WINDOWS MEDIA CENTER ........................................................................................................................... 37

FIGURE 32: DREAMBOX DM-8000 ................................................................................................................................... 37

FIGURE 33: THE NOSE MONITOR FROM PURECHOICE .................................................................................................... 38

FIGURE 34: CONTRINEX LLS-4050-000 ............................................................................................................................ 38

FIGURE 35: MICROWAVE-BASED MOTION DETECTOR .................................................................................................... 38

FIGURE 36: RFID READER TSG60 DEISTER ELECTRONICS ................................................................................................. 39

FIGURE 37: FIRST ALERT SA302CN DUAL SYSTEM SMOKE DETECTOR WITH REMOTE TESTING FUNCTION .................... 39

FIGURE 38: ABB WATER DETECTOR ................................................................................................................................. 40

FIGURE 39: ABB BRIGHTNESS SENSOR AND CONTROL DEVICE ....................................................................................... 40

FIGURE 41: THE TIRA 2.1 REMOTE CONTROL RECEIVER/TRANSMITTER FROM HOME ELECTRONICS ............................. 41

FIGURE 40: ABB LIGHT SENSOR ....................................................................................................................................... 41

FIGURE 42: DIAGRAM OF THE AMIGO PROJECT. DYNAMIC INTEGRATION OF HETEROGENEOUS SYSTEMS THROUGH SPECIALIZED MIDDLEWARE. ............................................................................................................................................... 45

FIGURE 43: DIAGRAM OF THE EASYLINE+ PROJECT. ADDITION OF AMBIENT INTELLIGENCE TO THE KITCHEN ENVIRONMENT ENHANCING COMMUNICATION BETWEEN DEVICES. ............................................................................... 45

FIGURE 44: INREDIS ARCHITECTURE: ENHANCING INTERACTION BETWEEN PEOPLE WITH SPECIAL NEEDS AND THEIR ENVIRONMENT. ................................................................................................................................................................. 46


FIGURE 45: DIAGRAM OF THE PROJECT SM4ALL. DESIGN AND DEVELOPMENT OF SMART EMBEDDED DEVICES FOR A PERSON-CENTRIC INTELLIGENT ENVIRONMENT CONTROLLED BY HETEROGENEOUS INTERFACES. .................................. 47

FIGURE 46: I2HOME GENERAL ARCHITECTURE OVERVIEW ............................................................................................. 48

FIGURE 47: VITAL ARCHITECTURE OVERVIEW. ................................................................................................................ 49

FIGURE 48: OVERALL CONCEPT OF BEDMOND PROJECT. ................................................................................................ 50

FIGURE 49: I2LIFE PROJECT GENERAL OVERVIEW. .......................................................................................................... 51

FIGURE 50: MONAMI PROJECT CONCEPT OVERVIEW ..................................................................................................... 52

FIGURE 51: ASK-IT ARCHITECTURE OVERVIEW. ............................................................................................................... 52

FIGURE 52: SOPRANO CONCEPT OVERVIEW. .................................................................................................................. 53

FIGURE 53: UNIVERSAAL VISION AND KEY FUNCTIONALITIES ......................................................................................... 54

FIGURE 54: BRAINABLE FUNCTIONALITIES AND APPLIED TECHNOLOGIES ...................................................................... 57


11 IInnttrroodduuccttiioonn

Ambient Intelligence (AmI) may be considered a recent discipline which is experiencing a dramatic growth and an accelerated development. It is focused on providing an adaptable and context-aware environment by adding intelligence to our surroundings. These intelligent environments represents a futuristic vision where smart, ambient systems react in an attentive, adaptive and proactive way by relating theories and concepts as ubiquitous / pervasive / proactive computing, internet of things and the invisible/disappearing computer [1]. AmI’s main purpose is to aid and facilitate our living conditions improving the way the environment helps us. This is a useful and promising approach, which appropriately applied, is able to revolutionize assistive technologies and significantly improve the lives of people with disabilities. An essential concept in ambient intelligence, as was exposed above, is the context-awareness. It encompasses a notion of context which has been defined differently depending on the application. Referring to the building of smart appliances for intelligent environments, a convenient definition is the knowledge about user’s and device’s states in a determined environment, situation and location [2]. This appears appropriate when having such system features as being sensitive, responsive and adaptive. The proper availability of environment information is therefore one of the central issues in which ambient intelligence relies on. This includes sensor data coming from the surroundings and form biosensors as well, being inputs to the system. Thus, reasoning processes may be carried out in order to supply suitable responses. This ideal availability of the required information everywhere at every time is what ubiquitous or pervasive computing is about. It is also related to the “embedded everywhere” systems and invisible computing, from which behind-the-scene processes are carried out to provide the appropriated response through a natural human-computer interface [3]. The increasing interest in the development of these smart systems has raised a significant investment focused on research. Some of these examples are the I2HOME and SM4ALL, which are European–funded projects aimed at the study and development of innovative interfaces to assist people with special needs through a scalable, user-centric platform composed by interoperable embedded smart devices [4] [5]. This document summarizes these types of examples as well as concepts and applications of these user-centric platforms for ambient intelligence. The assistive nature of this discipline and its inherent potential to improve notably people’s living conditions is our main interest. Our main purpose is to aid disabled and aging people by providing the required tools and support according to their specific needs. This is therefore considered a context-aware assistance through natural interfaces rather than ubiquitous-leisure environments. BrainAble, a European-funded project, serves as a feasible application where this type of advanced assistive technologies will be developed, implemented and validated. This project encompasses context-aware environments and means for self-expression mainly through Brain Computer Interfaces (BCI) within a scalable and interoperable platform of ubiquitous devices. Generic features of interest and differences between other approaches will be further described in this document. The document starts with a preliminary discussion about the notion of context-awareness in assistive technologies and a description of dedicated methods based on sensors, sensor networks, pervasive computing and artificial intelligence, which altogether constitute the technical basis for the perception-action paradigm of ambient intelligence.


22 CCoonntteexxtt iinn aassssiissttiivvee tteecchhnnoollooggiieess

A precise notion of context is essential in an intelligent environment. Its context-awareness feature is the perceptual engine which permits to understand determined actions or situations to produce appropriate responses. One context is identified by one name describing one type of situation under specific features. This understanding of the environment is not explicit and thus not trivial; it depends on its previous knowledge. Experience provides means to classify and highlight specific situations and to relate them with the received stimuli of the sensory system. It is supposed that same situations are promoted by the same stimulus, and that implications resultant from those situations serve as an extension on the recall of experience [6]. Sensing of location, environmental conditions and capturing explicit interactions are the general inputs for context extraction. Nevertheless, it is desirable for a smart environment to interpret the available information by perceptual means similar to those of humans [7]. The assistive property of this discipline is evident. Ambient intelligence applications respond to a series of requirements which could significantly improve aging and disabled people living conditions. Ambient Assisted Living (AAL), a strategic objective of the EU-funded projects, addresses the needs of the ageing population, while lowering social security costs. This example of AmI in assistive technologies rises from the relation with the Universal Access field and is focused on the aid through the inclusion of intelligent products and provision of remote care services. Thus, older people can live in their home environment with an increased autonomy, carrying out daily life activities [3]. In assistive technologies, the notion of context entails a major issue which must be taken into account for the special requirements of ageing and disabled people. Needs, and therefore situations, are defined according to individual requirements rather than generalized for certain tasks. Thus, in the BrainAble project the main objective of ambient intelligence is to assist them in their daily life in order to improve their autonomy and living conditions. This implies that at the design process of an intelligent environment, above the technical features of the system, must be defined fundamental aspects as: how to define a situation, how can that situation be acquired, which situation must be responded and, in that case, how to provide the desired assistance. These aspects define the purpose of the system and conceptualize its context-awareness approach.

33 AAmmII bbaassiiss aanndd tteecchhnnoollooggiieess

The basic principle from which ambient intelligence is based on is the perception of the environment and the action it performs as a consequence. The first step of the AmI algorithm is therefore the perception of the environment and residents through the use of sensors. This information is thus processed using artificial intelligence (AI) techniques, being the reasoning and decision-making step, to finally act upon the environment through controllers and specialized human-computer interfaces (HCI) [8]. This sequence of steps represents the logical perception-action or event-response paradigm handled by an AmI system to fulfil its objectives. However, the core of an AmI system in terms of technical parts can be divided in Sensors and Sensor Networks, Pervasive Computing and Artificial Intelligence as is described in the subsequent subsections.


33..11 SSeennssoorrss aanndd sseennssoorr nneettwwoorrkkss Sensing technologies are essential to perceive physical features of the environment and to extract context. The efficacy of the AmI algorithm to recognize determined contexts depends considerably on the type, distribution pattern and interrelation of sensor data. It is therefore necessary to establish the appropriate strategy to capture the required information in order to identify specific situations. The subsequent step following the context-aware and situations definition is the selection of sensor type, location and distribution. The list below presents some of the most useful situation cues and sensors that permit to obtain a reliable perception of the environment (see [2] and [8] for more detail):

Location, identification and tracking. Initially most of the information required in a smart environment could be reduced to the position of residents and objects. It continues being essential in AmI and includes their identification and tracking. GPS is mainly used in outdoor environments, complemented by GSM cellular networks or RDS (radio data systems); while IR beacons, Ultrasonic Location Systems and RF beacons are useful for indoor environments. Alternatives are wearable sensors for identification and tracking as RFID tags or the I-Button, as well as sound and vision recognition systems through Video Cameras and Microphones. Another alternative is the use of pressure sensors applied, for instance, in beds to know if the user is on it or not.

Motion detection. Simple sensors that provide important information about context are motion detectors as Passive IR, Accelerometers or Gyroscopes. They provide motion, acceleration patterns, angular velocity or inclination information which can be useful in determining efficiently person’s activity.

Video. Visual information captured from Video Cameras, apart from being a source of location, identification and tracking, is useful in situations where complex context and cue recognition as landmarks or gestures are required.

Audio. Microphones and amplifiers are essential tools to capture Sound information. Noise, music and speaking are features that permit identifying determined environments as well as the inherent nature of speaking is a communication mean.

Light. Environment features as light intensity, type of light, density or wavelength

are frequently involved in situation descriptors. It is simple and low cost to get that kind of information though the use of photodiodes, color, IR or UV sensors.

Atmospheric and climate. Temperature, humidity and air pressure can be

considered situation cues that identify atmospheric or climate properties of an environment. Simple temperature sensors are low-cost and are based on thermal resistors. Capacitive and resistive sensors are used for humidity measurement and absolute air pressure sensors are capable of providing altitude or pressure changes.

Proximity and touch. These situation cues are related to the identification of

interaction. It is the physical interaction between people or between people and objects. Capacitive, humidity and force sensitive resistors are commonly used to


capture this information. However, temperature, light sensors or video cameras are used as well.

Gas. The measurement of gas concentration in air and the recognition of a specific

smell through an array of sensors or electronic nose provide specific context information.

Magnetic field. Orientation or direction of movement is an important property that

can be identified by the use of magnetic sensors which act as an electronic compass.

Biosensors. Medical parameters as well as emotional states or even gestures of a person can be obtained from specialized non-invasive biosensors. Pulse or heart rate permits to obtain physiological states (i.e. calm, excited or exhausted). Muscle tension by electrodes provide information about movement or gesture, skin resistance permits to obtain excitement or tension and blood pressure and oxygen saturation present cardiovascular exposure. Another type of biologic measurements can be obtained by the processing of signals obtained from the heart (ECG – electrocardiogram), brain (EEG – electroencephalogram) and muscles (EMG- electromyogram). These are methods of extraction of biological data which offer reliable sources of emotional and medical condition through specialized algorithms.

The presented situation cues and sensors are aimed to provide context information of the environment, thus a similitude to the senses in nature is expected. A centralized or distributed model to manage and analyze this information may be employed, as well as wireless sensors or wireless sensor networks. Nevertheless, an interesting feature to take into account in this context-aware technology, as stated by Schmidt in [6], is the importance of the association of patterns acquired from the sensed information with meaningful concepts. This entails acquiring the real perception of the environment and suggests a direct dependence on memory [9]. Furthermore, other sensor features as portability, power consumption, calibration, setup time, reliability and price and cost, are important design issues that must be considered [2].

33..22 PPeerrvvaassiivvee ccoommppuuttiinngg The concept of pervasive computing could be regarded as the technical representation of Ambient Intelligence itself. This is due to the number of features it covers and the implicit concept it holds, being the availability of information at any place any time. A smart environment would be therefore a scenario where all devices are networked, human-centric, and also communicate and interact with each other [10]. However, it also implies an environment provided with intelligence where special situations are identified and appropriate responses are executed, as discussed above by the introduction of context awareness. This is what highlights Ambient Intelligence and what differentiates pervasive computing as an emerging concept suitable to be applied within. The concept of pervasive computing, also called ubiquitous computing, was formally introduced in 1991 by Mark Weiser in his article “The Computer for the 21st Century” [11]. There he envisaged a technology that does not require active attention and is simple and intuitive. Fundamentally, ubiquitous computing is not a technology itself. It is a concept that implies the combination of new and existing technologies in order to provide individuals


with a series of services through natural interfaces on top of an unobtrusively embedded environment (see Figure 1) [12]. Figure 1: Pervasive computing enables interconnectivity and implies the availability of information at any place any

time

A relevant feature involved in the concept of pervasive computing is the simplicity of use provided by advanced user interfaces. The main objective is to supply the user with a natural interface in which there is no need to learn how to use or configure a determined application [10]. This human-computer interaction (HCI) enables users to have control over technologies and devices in the environment (it might be achieved for example by spoken commands) in such a way they no longer know they were interacting with computers [13]. This invisibility or disappearing feature of the system represents the foundation of this concept. However, it is composed by devices as sensors, processors and actuators that are interconnected and provide the appropriate scenario to deploy this type of applications.

33..33 AArrttiiffiicciiaall iinntteelllliiggeennccee Reasoning represents a fundamental component of AmI, from which the inherent sensing and acting features of the system are based on to provide responsive and adaptive algorithms to the user. These algorithms are essentially focused on the activity prediction and recognition, modelling of user behaviour, decision making and spatial-temporal reasoning (Figure 2) [8]. The recognition of the activity in an environment constitutes one of the basic features of AmI where the identification of users (e.g. residents of a home environment), as well as their actions, permits the system to anticipate their needs and perform an action [14]. This is possible by extracting environment data through sensor


networks, thus actions may be identified by tracking specific patterns [15]. Common methods applied to activity recognition are based on Naive Bayes classifiers, decision trees, Markov models, dynamic Bayes networks and conditional random fields [16].

Figure 2: Diagram of a simplified AmI system. The reasoning component provides the system the ability to be

adaptable and responsive for the benefit of the user.

The ability to model user behaviour constitutes an enhancement of the capabilities of the system. It can provide valuable information such as the identification of anomalies in user behaviour or social interactions through specific data patterns and thus perform fast and appropriate actions. Techniques dedicated to provide this type of service are focused on the ability to refine and learn in order to adapt to changing patterns [8]. Methods as Markov models or fuzzy rules are commonly used to model user behaviour [17]. Decision making and spatial and temporal reasoning techniques are designed as a planning and control system which may also provide a proactive feature and a time reference of meaningful events [18] [19]. Most of the decision making methods applied are based on neural networks, reinforcement learning and fuzzy rules, while spatial and temporal reasoning, besides of the basic reasoning techniques, can alternatively also make use of Allen’s temporal logic [20] [21].

44 DDeevviiccee nneettwwoorrkkiinngg aanndd iinntteerrooppeerraabbiilliittyy

The idea of disposing of the required information at any place and at any time which is inherent to Pervasive Computing, where a network of embedded devices interoperates in an invisible form to perceive situations and react over the environment, could not be possible with the absence of communication technologies. There are different types of communication technologies that may provide the necessary interconnectivity between devices. BrainAble communications are not constrained to a specified type or group of technologies. On the contrary, it is rather open to deal with a flexible and scalable architecture based on the heterogeneity of communications and devices. This section presents an overview of different technologies parting from basic connectivity and networking technologies, control and networking protocols and standards, and finally some examples of middleware for interoperability are shown.


44..11 CCoonnnneeccttiivviittyy aanndd nneettwwoorrkkiinngg tteecchhnnoollooggiieess The basic connectivity and networking technologies correspond to the physical and lower layers of communication in which data can be transferred. A major feature to classify these technologies is the transmission medium, which can be wired or wireless. Nevertheless, independent of this factor, some of the communication technologies briefly described in this subsection includes only lower layers of the full seven-layer OSI network model whereas others tackle all of them. Similarly, some have been designed for networking, while others for peer to peer connections, or for fast or low bit rates transmission. This implies that depending on their features, some technologies are more suitable for specific environments or applications than others, and in some cases, they could be complementary. A brief description of the most important of these technologies is presented below: Wired communications

Ethernet This standard, also known as IEEE 802.3, is intended for data transmission for local area

networks (LAN). This means that all the computers on an Ethernet network are connected to the same transmission line, and communication is carried out using a protocol called CSMA/CD (Carrier Sense Multiple Access with Collision Detection). This is the most widely installed LAN and defines the Physical, Media Access Control (MAC) and Data Link Layer of the OSI networking model. There are different variants of Ethernet technologies distinguished according to the type and diameter of the cable used. The widespread variants are twisted or double-twisted wiring technologies as the 10base-T, which is called the Standard Ethernet with a data rate of 10 Mbps, the 100base-TX, which is the Fast Ethernet at 100 Mbps, and the 1000base-T, which is the Gigabit Ethernet at 1000 Mbps. There is a variant that could supply electricity to devices over the network which is called Power over Ethernet (PoE) [22].

USB

The Universal Serial Bus (USB) is a set of connectivity specifications developed to allow high-speed, easy connection of peripherals to a PC. When plugged in, everything configures automatically. USB is currently the most successful interconnect technology in personal computing. Hi-Speed USB 2.0 provides greater enhancement in performance, up to 40 times faster than USB 1.0, with a design data rate of up to 480 Mbps. One characteristic of USB architecture is that it can supply electricity to devices to which it connects, with a limit of 15 W maximum per device. It was not designed for networking; the maximum length of the cable between two devices is 5 m [23]. The new USB specification, SuperSpeed USB 3.0, which is already in use, allows a data rate of up to 5 Gbps.

FireWire

The FireWire Bus, also called IEEE 1394 or iLink, is a port that exists on some computers that allows you to connect peripherals (particularly digital cameras) at a very high bandwidth. It was designed in order to provide an interconnection system that allows data to circulate at a high speed and in real time. It has the same structure as the USB bus except that it is a cable made up of six wires (2 pairs for the data and the clock and 2 wires for the power supply) that allow it to reach a bandwidth of 3.2 Gbps. The two wires for the clock are the major difference between the USB bus and the IEEE 1394

http://www.intel.com/technology/usb/spec.htm?iid=Tech_usb+spec2�

http://en.kioskea.net/contents/pc/usb.php3�

http://en.kioskea.net/contents/pc/usb.php3�


bus and permit synchronous and asynchronous transfer modes. Furthermore, while USB was designed for low cost and simplicity, FireWire was designed for high performance [24].

Power Line Communications (PLC)

Power Line Communication is any technology that enables data transfer at narrow or broad band speeds through power lines by using advanced modulation technology. It can extend an existing local area network or share an existing Internet connection through electric plugs with the installation of specific units [25]. The principle of PLC consists in superimposing a high frequency signal (1.6 to 30 MHz) at low energy levels over the 50 Hz electrical signal. This second signal is transmitted via the power infrastructure and can be received and decoded remotely. Thus the PLC signal is received by any PLC receiver located on the same electrical network. The widespread solution currently on sale, Homeplug, has a theoretical bandwidth of up to 189 Mbps (HomePlug AV). This communication mean brings mobility, flexibility, ease of installation and complements cable and wireless solutions; however, it is highly dependent on the architecture of the electrical network and might imply interoperability problems and high costs.

Wireless Communications

IrDA The Infrared Data Association (IrDA) is responsible for the specification of the infrared communication standards. In general, IrDA is used to provide wireless connectivity technologies for devices that would normally use cables for connectivity. IrDA is a point-to-point, narrow angle (30° cone), ad-hoc data transmission standard designed to operate over a distance of 0 to 3 meters and at speeds of 9600 bps to 1 Gbps (Giga IR). A full IrDA solution could be accomplished at a very low cost. Nevertheless, it requires line of sight and is useful for determined applications [26].

Bluetooth

Bluetooth is an industrial specification for wireless personal area networks (PANs), also known as IEEE 802.15.1. Bluetooth provides a way to connect and exchange information between devices such as personal digital assistants (PDAs), mobile phones, laptops, PCs, printers, car radios and consoles, digital cameras and video game consoles via a secure, globally unlicensed short-range radio frequency [27]. Bluetooth lets these devices communicate with each other when they are in range. The devices use a radio communications system, so they do not have to be in line of sight of each other, and can even be in other rooms, as long as the received transmission is powerful enough.

o Works in the license-free ISM band at 2.45 GHz

o 79 channels (each 1 MHz wide)

o Speed up to 723.1 kbps (version 1.2), up to 2.1 Mbps (version 2.0) and up to 24 Mbps (3.0 + HS)

o It has been designed for device connectivity, not for networking. Thus, together with limitations of range and cost, this is not considered a dedicated specification for home control.

Wi-Fi


This is a trademark originally licensed by the Wi-Fi Alliance to describe the underlying technology of wireless local area networks (WLAN) based on the IEEE 802.11 specifications (commonly at the 2.4 GHz Band with a range between 30 and 100 m). It was developed to be used for mobile computing devices, such as laptops, in LANs, but is now increasingly used for more services, including Internet and VoIP phone access, gaming, and basic connectivity of consumer electronics such as televisions and DVD players, or digital cameras. Currently, this is the common wireless standard for high data rate (11 - 54 Mbps) applications [28]. Thus, used for internet browsing, PC networking or file transfers. Nevertheless, its power consumption is relatively high compared to other wireless technologies as Bluetooth or Zigbee and even its data rate is far in excess of what a home appliance requires (e.g. a light switch).

LR-WPAN – IEEE 802.15.4

The IEEE 802.15.4 standard defines the characteristics of the physical and MAC layers for Low-Rate Wireless Personal Area Networks (LR-WPAN). It supports three frequency bands: a 2450 MHz band (with 16 channels), a 915 MHz band (with 10 channels) and an 868 MHz band (1 channel) with a data throughput of 250 kbps. The advantages of an LR-WPAN are ease of installation; reliable data transfer, short-range operation, extremely low cost, and a reasonable battery life, while maintaining a simple and flexible protocol stack [29]. Higher layer protocols are left to industry and individual applications, as is the case of Zigbee where network, security and application protocols complement the IEEE 802.15.4.

Mobile Telephony

As an alternative to the Internet for remote communication with home appliances, Mobile Telephony offers a wide range of communication services like voice and data transfer through SMS and other enhanced data transfer protocols like GPRS (21.4 - 171.2 kbps), EDGE (43.2 - 345.6 kbps) and UMTS (0.144 - 2 Mbps) at a wide variety of places on the earth. Furthermore, it includes a security component that is not easy to accomplish using Web Applications [30]. However, due to cost reasons and ease of development, Wi-Fi is currently the simple option to deploy for this type of wireless communications.

44..22 CCoonnttrrooll aanndd nneettwwoorrkkiinngg pprroottooccoollss aanndd ssttaannddaarrddss The information transmitted over the aforementioned communication technologies should not only reach the required device or peer in the network; it also should be understandable by the receptor. Standards are designed to provide this common language, thus a real communication exists. Many different protocols have been designed for a diverse type of applications. Home Automation is also a field where a unified language is required for sensing, controlling and networking of devices. A particularity of this type of application is its openness, which is the publicity of the protocol; its scalability, which refers to the possibility of adding and removing devices without affecting its functionality; heterogeneity, which is the handling of different features as hardware, interfaces and protocols; and topology, which is the way devices are connected to one another [31]. These features are essential when designing ubiquitous systems and therefore, of significant importance within the BrainAble communication architecture. As is presented below, with a brief description of the most relevant protocols deployed in Home Automation, there are differences according to the layers of the networking models involved in each protocol, some have


designed their own full seven-layer model, while others only define the upper layer. Furthermore, there are protocols designed for powerline communications, wired or wireless transmission or all of them. Similarly, there are also protocols designed to be applied by others for tunnelling or device discovery. This general description is presented below:

X10 X10 is one of the oldest international and open industry standards for communication

among devices used for home automation and domotics. It primarily uses power line wiring for signaling and control, where the signals involve brief radio frequency bursts representing digital information. A radio based transport is also defined. X10 was developed in 1975 by Pico Electronics of Glenrothes, Scotland, in order to allow remote control of home devices and appliances. It was the first domotic technology and remains the most widely available. Although a number of higher bandwidth alternatives exist including KNX, INSTEON and LonWorks, X10 remains popular in the home environment with millions of units in use worldwide. A major drawback of X10, besides its slow speed (60 bps), low reliability and the lack of security, is that it uses only one-way communication, i.e. the current status of a device cannot be determined by a controller [32].

HomePlug

This is a new protocol for home automation that uses Power Line Communication (PLC). The encrypted digital communication and bi-directionality are the main features of this protocol. The PLC-BUS has several additional advantages compared to other home automation protocols in particular the reliability and speed of execution of orders executed. In comparative tests it performed 3 times faster compared to the other best known PLC systems, including the X10. In PLC-BUS orders are sent by the electrical network, using signal encryption, to allow the use of up to a maximum of 64,000 addresses in the same facility. The bi-directionality allows getting feedback about the successful execution of the order. The reliability of the commands executed is high, exceeding 99.98%. The encryption of the signal also avoids the passage of signals between buildings and isolates each facility safely. The resistance to noise on the power grid is also higher than the conventional X10. In graphical interfaces control (Touchscreens, TVs, PCs or PDAs or Smartphones) you can get an overview of the state apparatus and the generality of the installation. HomePlug 1.0 can transport data over the powerline at a bit rate of 14 Mbps, while the more advanced HomePlug AV at 200 Mbps, therefore, this is a protocol based for PC networking rather than home control [33].

CHAIN

CHAIN (Ceced Home Appliances Interoperating Network) is a platform, created by the CECED Convergence Working Group, which defines a protocol for connecting different appliances in a single multibrand system. It allows for control and automation of all basic appliance-related services in a home: e.g., remote control of appliance operation, energy or load management, remote diagnostics and automatic maintenance support to appliances, downloading and updating of data, programs and services from the Web. Currently, its command set has been merged to comply with the KNX standard [34].

Konnex (EIB/KNX)

KNX is a standardized network communications protocol for intelligent buildings administered by the Konnex Association (http://www.konnex.org). Currently KNX is the

http://www.konnex.org/�


world’s only open, royalty-free and platform independent standard that applies for all applications in home and building control, ranging from lighting, shutter control to security, heating, ventilation, air conditioning as well as white goods and consumer electronics. KNX is approved as a European (CENELEC EN 50090 and CEN EN 13321) and a worldwide standard (ISO/IEC 14543) and is established upon over 15 years of experience in the market by its predecessors: the European Home Systems Protocol (EHS), the BatiBUS, and the European Installation Bus (EIB). Over 100 member companies worldwide from different branches offer almost 7.000 KNX certified different products on the market. The Konnex Association has partnership agreements with more than 10.000 installer companies in almost 70 countries. The KNX standard defines several physical communication media: o Twisted pair wiring (inherited from the BatiBUS and EIB Instabus standards) o Power line networking (inherited from EIB and EHS - similar to that used by X10) o Radio o Infrared o Ethernet (also known as EIBnet/IP or KNXnet/IP)

Since KNX is designed independently from any particular hardware platform a KNX

Device Network can be controlled by anything from an 8-bit microcontroller to a PC, according to the needs of a particular implementation. In some parts of the world KNX now competes with Clipsal C-Bus (proprietary protocol), known in USA as SquareD [34].

INSTEON

INSTEON is a dual-mesh network that combines wireless radio frequency (RF) with the home's existing electrical wiring, developed by SmartLabs. INSTEON devices are peer-to-peer, and don't require a central controller, all devices can act as a Controller (sending messages), Responder (receiving messages), or Repeater

(relaying messages).

INSTEON is backward compatible to X10, i.e. existing X10 devices will be recognized by INSTEON products. INSTEON products are available from more than 15 manufacturers, including light switches and dimmers, sprinkler controllers, low voltage controllers and sensors and security panel interfaces. For developers, an INSTEON Developer Kit is available [35].

LonWorks

This is a networking platform specifically created to address the unique performance, reliability, installation, and maintenance needs of control applications. The platform is built on a low bandwidth protocol created by Echelon Corporation for networking devices over media such as twisted pair, power lines, fiber optics, and RF [36]. It implements the full seven-layer stack of the OSI model.

It is popular for the automation of various functions within buildings such as lighting

and HVAC. The platform has its origins with chip designs, power line and twisted pair signaling technology, routers, network management software, and other products from Echelon Corporation. In 1999 the communications protocol (then known as LonTalk) was submitted to ANSI and accepted as a standard for control networking (ANSI/EIA709.1-B). Echelon's power line and twisted pair signaling technology was also submitted to ANSI for standardization and accepted. Since then, ANSI/EIA709.1 has been accepted as the basis for IEEE 1473-L (in-train controls), AAR Electro-pneumatic braking systems for freight trains, IFSF (European petrol station control), SEMI


(semiconductor equipment manufacturing), and in 2005 as EN14908 (European building automation standard).

The protocol is also one of data link/physical layers of the BACnet ASHRAE/ANSI

standard for building automation. By 2006 approximately 60 million devices were installed with LonWorks technology. Manufacturers in a variety of industries including building, home, transportation, utility, and industrial automation have adopted the platform as the basis for their product and service offerings. Thus, LonWorks applications are commonly found in industrial applications. However, it is a proprietary and sophisticated standard, needs professional installation and its price is still high to achieve a widespread adoption in the consumer home control market. For product developers, Echelon offers a Node Builder package for 5,995 $, and a Mini PL EVK for 395 $ for powerline applications. Programming of Neuron chips is done in Neuron C.

ZigBee

ZigBee is described in the aforementioned standard IEEE 802.15.4. It works similar to Bluetooth and Wi-Fi, in the 2.4 GHz Band. ZigBee has 16 independent channels. The maximum data transfer rate is 250 Kbit/s; the maximum working distance is 100m. In contrast to Bluetooth and Wi-Fi, ZigBee is much slower in the transfer rate and is not suitable for high bandwidth demands such as video streaming. However, it permits to form mesh networks. Among the public profiles of this standard, there is the Home Automation protocol which specifies common attributes for a large range of devices. ZigBee is expected to be the key of smart home in future [37]. This widely accepted protocol is recommended for wireless sensor networking and Smart Homes. Advantages of ZigBee are:

o High interference resistance

o Low power consumption

o In practice, it works within 100m distance (Bluetooth: 10m, Wi-Fi: 30m)

o Easy handling

o Big number of possible devices in the ZigBee network (216 devices)

Z-Wave

Z-Wave is the interoperable wireless communication standard developed by Danish company Zensys and the Z-Wave Alliance. It is designed for low-power and low-bandwidth appliances, such as home automation and sensor networks. Z-Wave provides radio communication with bandwidth 9,600 bit/s or 40,000 bit/s. Range is approximately 30 meters. Z-Wave Radio uses the 900 MHz frequency bands.

Z-wave uses an intelligent mesh network topology and has no master node. A message

from node A to node C can be successfully delivered even if the two nodes are not within range providing that a third node B can communicate with nodes A and C. If the preferred route is unavailable, the message originator will attempt other routes until a path is found to the "C" node. Therefore a Z-wave network can span much further than the radio range of a single unit. In order for Z-wave units to be able to route unsolicited messages, they cannot be in sleep mode. Therefore, most battery-operated devices will opt not to be repeater units. A Z-wave network can consist of up to 232 units with the option of bridging networks if more units are required. Due to the low bandwidth, Z-


wave is not suitable for streaming audio/video applications but is well suited for sensors and control units which typically only transmit limited information [38].

EnOcean This technology combines Energy Harvesting with Wireless Technology to offer a simple, self-powered wireless communication solution which can be easily installed and also maintenance-free. It is a proprietary environment from the company EnOcean GmbH, a spin-off from Siemens, and is used to transmit very short, low energy radio signals containing data collected by sensors. In general, EnOcean covers the two ISM frequency band frequencies 868.3 MHz and 315.0 MHz for worldwide license-free operation, using ASK (Amplitude shift keying) modulation, ranging up to 300m in the open space or 30m through walls and ceilings inside buildings, sending short wireless telegrams lasting approx 1.2 ms at a data rate of 125 kbps [39].

Jini

Jini is built on Java technology, it is designed to enable users to simply connect any number of digital devices, and to access those valuable services provided by rich, dynamic communities of systems such as personal digital assistants (PDAs), televisions, digital cameras, fax machines, cell phones, even smart card readers. Devices locate the required service by using Jini technology, which employs Java Remote Method Invocation (RMI™), which in turn uses whatever network protocol is supported by the operating system. Jini technology does not require any particular operating system or network transport: A device might use infrared, radio, or be physically plugged in to the local network. Many of the emerging network technology specifications are designed for specific types of networks such as TCP/IP for Ethernet, IrDA for infrared, IEEE 1394 (FireWire), or wireless [40]. Nevertheless, although it has the backing of vendors such as Sony and Philips, Jini remains within the realm of Java purists and programmers prepared to build their own interfaces and hand-code appliance drivers.

CEA-852-A

CEA-852-A specifies a communication method to access and control appliances in the home network over the Internet. Thus, the home network is extended to a client service or device using a zero-configuration secured tunneling mechanism. This is, it might be used as a mechanism to transport different types of protocols through the Internet, as this is the case with LonWorks. The zero-configuration aspect is especially important, since normal users tend to be overwhelmed by the setup of network tunneling tools [41].

UPnP

The Universal Plug and Play (UPnP) Forum, launched by Microsoft, has been around for some years and is constantly growing. The Forum consists of 895 vendors (as of January 2010), including industry leaders in consumer electronics, computing, home automation, home security, appliances, printing, photography, computer networking, and mobile products. The UPnP technology covers two main areas:

o The UPnP device architecture which defines discovery, control and eventing between networked devices, based on HTTP extensions running on TCP/IP and UDP/IP. SOAP is used for remote procedure calls.

o The Device Control Protocols (DCPs) that describe common machine interfaces for devices of multiple categories: AV server and renderer, digital security camera,


HVAC, lighting controls, Internet gateway, Wi-Fi access point, printer (basic and enhanced), remote UI server and client, and scanner.

A list of certified UPnP devices is available from the UPnP Implementers Corporation. The most implemented devices are the Internet gateway and Audio-Video servers and renderers. UPnP compliant products for the digital home are increasingly showing up on the market [42].

44..33 MMiiddddlleewwaarree In distributed systems, the goal of the middleware is to deal with heterogeneity and distribution by (1) providing ways of treating remote resources as if they were local and (2) adding an additional layer of abstraction suitable for a specific type of applications [43]. Currently, as was shown in the previous subsections, the Home Automation and embedded solutions for pervasive applications, dispose of different communication technologies, interfaces and protocols which have experienced different acceptation levels within the market. It is therefore necessary to deal with this heterogeneity through middleware technologies. In BrainAble, an especial focus is on the URC/UCH middleware described below. However, a brief description of different middleware technologies is presented:

URC/UCH A Universal Control Hub serves as “user interface middleware” between home appliances and/or services to be controlled (called “Targets”) and personal devices used as controllers (called “Universal Remote Consoles” or URCs). The rendering of a Target’s user interface on a URC is guided by “Resources” that the Universal Control Hub (UCH) retrieves from Resource Servers local or on the Internet.

Figure 3: UCH (Universal Control Hub) application example

The above figure shows the architecture of the Universal Control Hub. The particular Targets types (right side) and URC types (left side) are examples and are not to be


understood as necessary components of the system. In the centre of the proposed architecture is the Universal Control Hub (UCH) which acts as a gateway between a URC and any network-enabled home appliance or service (Target) that it wants to access and control.

For communication with the Targets, the UCH talks their native network protocols using Target Adapters. For example, it interacts with an UPnP enabled TV or DVR through the UPnP Audio/Video Device Control Protocol; and it interacts with a CHAIN-enabled fridge through the CECED CHAIN standard. These Target network protocols do typically not include any mechanism for specifying a user interface. Thus a manufacturer of a Target device can deploy very simple devices that are not aware of any user interfaces, and solely rely on being controlled through their corresponding standardized network protocols.

On the other end, the URCs do not know anything about the network protocols to be used for controlling the Target devices. To the URCs, the Universal Control Hub acts as a proxy for the target devices. This “proxy” can offer multiple user interfaces served on multiple “user interface channels” for control of the Targets. A URC can pick a channel that it is capable of using. The user interface channels are all based on existing user interface description languages, such as HTML, Flash or Voice XML. This means that the URCs can talk to the UCH in a language which mainstream software components exist for, without the need of special implementations on the URCs. For example, a PDA may have a Flash player installed and use the “Flash Remoting” channel of the UCH. (Macromedia Flash is a widely used framework for rendering dynamic content over the Web.) In this case, the UCH will translate the Target-network specific interaction into appropriate user interface manipulations for the Flash player. In another example, a digital TV used as a URC (controller) could have an HTML browser installed and choose the HTML channel of the UCH to control the fridge (which can only understand commands in the CHAIN protocol). Again, the UCH will act as middleware and provide the necessary translation for the interaction. It is also possible to remotely control devices through audio only, for example over a phone line. A user interface serving this channel may be specified in the W3C’s Voice XML standard language.

The UCH depends on standardized user interface socket descriptions and descriptions of user interface components (called “Resources”) which it retrieves from local or Internet-based “Resource Servers”. By using Resource Servers, user interfaces can be “shipped” and updated even after the Target’s deliverance to the home. Resource Servers are basically public market places for user interfaces where everybody can contribute and which are made available for UCHs in private homes. On Resource Servers, not only the vendor of a Target may register their user interfaces (for different channels), but also vendors of URC devices and 3rd parties specialized in the development of user interfaces for special user groups. For more information see [44].

OSGi

The Open Service Gateway Initiative (OSGi) is a module system and service platform for Java programming language. It defines a process to assure interoperability of applications and services based on its component integration platform. Specifications, reference implementations, test suites and certification are provided by the OSGi Alliance, in order to ensure a valuable cross-industry ecosystem. The lifecycle of the software is also managed in a standardized way.


OSGi offers a standard way to connect devices such as home appliances and security systems to the Internet. With such a standard, home users could, for example, install a security system and be able to change from one monitoring service to another without having to install a new system of wires and devices. It is thus intended to connect new Jini "smart appliances," Bluetooth wireless device groups, as well as TV set-top boxes, cable modems, alarm systems, energy management systems, and other devices to Internet sites that can be used to manage them remotely and interactively. Nevertheless, it is important to notice that there is the limitation of the requirement of the Java Virtual Machine implemented on all devices of the network [45].

openAAL openAAL is a joint open source initiative by FZI Research Center for Information Technologies, Friedrich-Schiller-University of Jena and CAS Software AG. It represents a flexible and powerful middleware for ambient-assisted living (AAL) scenarios and is based on research results of several German and international projects including the SOPRANO2 Integrated Project. The openAAL platform enables easy implementation, configuration and situation-dependent provision of flexible, context-aware and personalized IT services. In openAAL, OSGi serves as a base layer for the three main components: Context Manager, Procedural Manager and Composer. Hence, all interaction between these components and services is managed by OSGi [46].

HYDRA

The HYDRA middleware was developed as an objective of the HYDRA Project, an FP6 IST project. It allows the incorporation of heterogeneous physical devices into developers’ applications, offering easy-to-use web service interfaces. Hydra incorporates means for Device and Service Discovery, Semantic Model Driven Architecture, P2P communication, and Diagnostics. This middleware was developed based on a Service-oriented Architecture (SoA), and includes support for distributed as well as centralized architectures, security and trust, reflective properties and model-driven development of applications. HYDRA middleware provides interoperable access to data, information and knowledge across heterogeneous platforms, including web services, and support true ambient intelligence for ubiquitous networked devices. The middleware is independent of the underlying communication layer [43].

MORE

In the course of the FP6 IST project MORE (Network-centric Middleware for Group communication and Resource Sharing across Heterogeneous Embedded System), the MORE middleware, based on SoA, was developed, deployable on the device level. The objective of the MORE middleware is to overcome heterogeneity and assure interoperability. This middleware has the capability to alleviate of the heterogeneity of devices, not only from hardware point of view, but also from the software point of view. Regarding communication between devices, it supports group policies, and can ensure its security. It also allows resource sharing between devices/services and support service orchestration tailored towards the embedded domain. The middleware provides enabling gateway services that allow incorporating small scale embedded devices into the service domain and therefore increasing range of accessibility [47].

MPOWER

http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci212422,00.html�

http://searchmobilecomputing.techtarget.com/sDefinition/0,,sid40_gci211680,00.html�

http://searchmobilecomputing.techtarget.com/sDefinition/0,,sid40_gci213380,00.html�

http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci212971,00.html�

http://searchsecurity.techtarget.com/sDefinition/0,,sid14_gci211726,00.html�


The M-Power FP6 IST project developed a middleware platform that supports the rapid development and deployment of services for cognitive disabled and elderly. This middleware is intended to be commercially feasible for the general IT industry, not just for SMART HOUSE technology providers that wish to develop distributed integrated applications offering services to cognitive disabled and elderly. The middleware allows the interoperability between systems, bio-sensors and smart house technology. It also copes with various user contexts and usability, as a central issue. Security components where developed, in order to provide storage, communication, access rights and client security. The safety of the information is enabled through components facilitating information distribution and standardized information models [48].

Medusa

Medusa is a middleware that allows users to develop simple applications on their handheld devices, based on the resources discovered in the vicinity. The main goals of the MEDUSA middleware are related to providing end-user support for the creation and customization of applications and for controlling the composition process according to user needs. To achieve these goals, MEDUSA utilizes a composition tool for encoding user intent into applications and a set of control interfaces. Another important issue addressed by MEDUSA is interoperability between heterogeneous devices, networks and platforms. Achieving interoperability is a prerequisite for building an open application composition system, as the services constituting an application need to be able to discover each other, exchange information and indeed interpret this information meaningfully. This is especially important if the environment, in which the application composition system operates, consists of services implemented and deployed by independent providers [49].

55 AAmmII aapppplliiccaattiioonnss

Most of the recent approaches related to Ambient Intelligence are dedicated to support Ambient Assisted Living applications. It is reasonable to focus technologies implied in AmI to provide assistance for the elderly and people with disabilities due to its inherent nature. This is the case of BrainAble, as well as other applications which range from the development of smart devices (e.g. adding communication capabilities to devices in the home environment), to the design of innovative interconnection protocols, middleware and architectures, and techniques focused on the robust extraction of specific context and reasoning methods. BrainAble, particularly, makes use of these existing technologies to provide innovative services to people with functional diversity in order to enhance their physical autonomy and social interaction by using Brain-Computer Interfaces (BCI). In this section, examples of different applications are introduced starting with a summary of commercial devices designed to, among others, measure/control ambient status/conditions with communication capabilities, followed by an introduction of reasoning methods applied through different applications, to finally expose some related projects.

55..11 CCoommmmeerrcciiaall pprroodduuccttss It is not possible to point out what the most relevant protocols/products are, or what are the ones widely used. This is because, since the market has an enormous variety and


possibility of choice, every producer, supplier, retailer, vendor and end-user gives different relevance to each of the product’s characteristics. For example some will choose better quality/price ratio, others will opt for price exclusively, others will simply use the brands represented by their supplier, others will choose the ones with the best design and others can choose based on the protocol or other technical characteristics, providing that it is compatible with already used systems. To summarize, the decision of choosing one product over another is very dependent of the person/entity making the choice. One can assume that big brands will have more impact in the market. In BrainAble, key factors for the product selection, in order to design a feasible system, are related to availability, cost and ease of installation. This section presents some example of products related to the AmI market. For detailed information about the specific product types and system’s architecture implemented in the BrainAble project, please refer to D.5.2. – Technical Specifications for BrainAble AmI Architecture. Home Automation Systems Traditionally, home automation systems require a lot of custom design. Custom installers make a business out of this need, and the results are expensive systems with convenient user interfaces to the wishes of the customer. There is a bunch of custom home control systems, of which Crestron and HAI are introduced as typical examples. The business of custom installations is a constant cash-cow, and so most of these vendors and their installer base do not want an open system for home control. However, the need for making these systems available for the average person is becoming more and more prevalent. BrainAble priorities are therefore focused on commercial devices with standard and open protocols. A general description of different examples of commercial products is presented below:

KADEX Kadex is a system that can be used in new houses, as well as in houses that are being renewed or adapted. This is a proprietary system, but the philosophy regarding this system allows the integration of different devices, through the use of adaptors, just like in an electrical system renewal. This system has alarm and video phone components already available. A central terminal is available, where everything can be controlled using a touchscreen. This system is based on RF, and so no extra wires or cables are necessary. Also, this allows an easy and flexible expansion. Regarding controllers and actuators, this system operates using a proprietary, yet open, protocol. The system can also work using TCP/IP communication, using the RF receiver, and so we can have multi platform access to the system. It is also important to refer that Kadex has specialized solutions for elderly people.


Figure 4: Kadex System Diagram1

CRESTRON

Crestron is a traditional custom installer platform, a proprietary system of controller devices, connectivity devices and controlled devices. A customer who commits to the Crestron system has to buy everything from Crestron (which means it is definitely not an open system). User interfaces are custom-designed and require the work of a custom installer. Crestron provides an integrated development environment for its installers, and teaches them how to use it. Controlled devices include those for lighting automation, climate control, audio distribution, home theater, security and intercoms. A module for controlling digital media on a Windows Media Center PC is available. The system allows for controllers such as small remote controls, hand-held controllers with displays, touch-pads and light switches. More than 100 products are available as touch-panel controllers alone. For bigger systems, a central control box is required between the controllers and the target devices. It communicates to the target devices through one of the following technologies: X-10, infra-red, RS-232, Z-Wave, and SNMP. The Crestron platform does not allow for an open platform of user interfaces for the digital home. Comparable technologies are: AMX, Nevo by Universal Electronics, RTI.

Figure 5: Crestron touch panel2

1 http://www.kadex-domotica.com/ 2 www.crestron.com/features/touchpanels

http://www.kadex-domotica.com/�

http://www.crestron.com/features/touchpanels�


Home Automation, Inc. (HAI)

The system by HAI is similar to CRESTRON, and proprietary, too. Target devices include security, fire alarm, lighting, temperature control, audio-video, pool, and spa. Aside from the OmniTouch controller product line (which are touch-panel displays for indoor and outdoor usage), one can use the telephone (voice), a PDA, a console, or a PC (including a special Media Center interface) to remotely control the target devices. For lighting, scene switches can be used to control multiple lights in concert. An alert pendant is available that can be used by someone who has fallen in the home and cannot stand up any more. Communication options are Cat-5, Z-Wave or the Universal Powerline Bus. HAI provides an IDE for the user interface development which is typically used by custom installers.

Figure 6: HAI product examples: OmniTouch panel (left) and scene switch (right)3

Life|ware

Life|ware is a software-based home automation system that sits on top of the Windows Media Center Edition platform. It allows remote control for lighting, blinds/drapes, climate, security, in addition to the home entertainment system provided by the Media Center Edition itself. So-called "life|scenes" are macros that operate on a set of devices to "set a scene". The controlled devices may come from different manufacturers. Life|ware manifests a trend from custom-made home automation systems to more open technologies and platforms. The Media Center Edition user interface often is used as a new platform for home automation systems. Home appliances and devices (other than those that are part of the MCE, can be connected to a "Life|controller", a Web-service based gateway to other home automation systems. Thus the Media Center platform provides a uniform control platform across vendors and systems. It operates over TCP/IP protocol, using web services to describe the devices’ properties.

Figure 7: Life|ware system (left) and Life|ware controller (right)4

3 www.homeauto.com/products/products_main.asp 4 www.exceptionalinnovation.com

http://www.homeauto.com/products/products_main.asp�

http://www.exceptionalinnovation.com/�


Superna Superna is providing a home automation system based on the UPnP platform. Controller platforms are the PC, laptops and tablet PCs, PDAs and touch panels. The Superna control box comes with connectivity for Z-Wave, Insteon, IR, IP, USB or serial ports. It exposes connected target devices as UPnP devices to the controllers. This is an interesting approach and shows the feasibility of an UPnP-based home control system.

Figure 8: Superna product examples: Superna 10” touch panel (left), Superna PDA interface (middle), and Superna ControlBox (right)5

Siemens – Serve@Home

The serve@Home is a Home Management System from Siemens. With an emphasis on the need for mobility, independence and agility in our modern lives, this home appliance management system can be used to control various systems and appliances (oven, washing machine, air conditioner, dishwasher, etc.) within your home via cellphone or Internet. The system works by networking appliances equipped with a small slot in which a retrofitable System Interface card is inserted. Using a 230 volt power supply and an Ethernet interface, this home management system can communicate with appliances via a wireless network. Using the home automation portal dubbed "cockpit" you can interface with your devices via PC, tablet PC or PDA. All features of your appliances can be changed or checked via this interface, as well as providing normal PC information functions such as email, web, music and television. Customers can choose among products already integrated with this system and can easily add appliances at a later date thanks to the existing infrastructure and standard wireless protocols. This inclusion of new networked appliances carries an additional charge of only 50€, while the system starter solution including infoModule is available from 250€ to 1.000€, depending on the options selected.

White Goods

JURA Elektroapparate AG- Coffee/Tea Machines It is possible to change the machine settings and get them memorized in the machine electronics from a PC. For this connectivity options it is necessary to buy a separate Connectivity Kit, which is connected to the PC through USB. The configurations of the machine are stored in the USB dongle and then you can place the dongle in the machine that will load the existent info. So the machine is not remotely controllable. This machine has a cost of approximately $2,000.

5 www.supernasystems.com

http://www.se-presse.com/index.php?cSID=237aea8d5625317306b222d9dc49ab16&entry=inhalt_pa_detail&menue_offen=292&setlanguage=en&id_detail=19-1037-0506�

http://www.supernasystems.com/�


Figure 9: Images of the Jura IMPRESSA F90 + Connectivity Kit6

Siemens – Serve@Home Appliances

These are the appliances that work with the Siemens Serve@Home system which can remotely control appliances as: cooktops, hobs, ovens, microwave ovens, extraction hoods, freezers, dishwasher, condensed air tumble dryer or washing machines.

Figure 10: Siemens Serve@Home examples: Remote controlled washing machine (left) and dishwasher

(right)

Miele miele@home appliances

This is a set of appliances, mainly kitchen appliances, produced by the german manufacter Miele. Miele@home appliances communicate through the KNX protocol over power line.

Figure 11: Miele miele@home appliances7

6 http://www.jura.com/home_x/products_home_use/f_line/impressa_f90.htm 7 http://www.miele-at-home.de/de/haushalt/produkte/180.htm

http://www.jura.com/home_x/products_home_use/f_line/impressa_f90.htm�

http://www.miele-at-home.de/de/haushalt/produkte/180.htm�


Lighting

Lutron Lutron is all about lighting systems and remote control of them. There are many different models of switches, all for the custom installation home. Communication is based on a proprietary RF protocol.

Figure 12: System overview of the 8-series by Lutron8

The figure shows a schematic illustration of the 8-series architecture. A processor (typically installed in electrical closets and not a visible piece of equipment) has connections to the following components: Remote dimming panels, in-wall master keypads, Sivoia QED-Quite Electric Drive shades and draperies, in-wall dimmers and switches, GRAFIK Eye lighting control, and third-party subsystems & equipment.

Helio one-way controller (with sensor interface)

Helio light controller with single channel for switching and regulating lighting loads up to 5 A, providing a sensor interface for standard IR receivers, pushbutton interface units, movement detectors and light sensors. It implements LonWorks protocol.

Figure 13: Lighting controller Helio LRC 5040/109

8 http://www.lutron.com/homeworks/8series_sys.asp?s=17000&t=17100 9 http://www.echelon.com/productdb/productdetail.asp?prodID=164&manID=122&systemID=14&sysLabel=Actuators

http://www.lutron.com/homeworks/8series_sys.asp?s=17000&t=17100�

http://www.echelon.com/productdb/productdetail.asp?prodID=164&manID=122&systemID=14&sysLabel=Actuators�


Kadex canopy receiver

This receiver can be fitted discretely in the ceiling, directly in the connection point of a ceiling light. Once it receives a command from a transmitter, it switches the state of its output. This device can be operated by various Kadex transmitters and operates using Kadex proprietary protocol, over RF.

Figure 14: Kadex canopy receiver without and with a ceiling light mounted10

Adhoco.S1 radio-controlled power switch (13 A) The adhoco.S1 is a one fold power switch controlled via RF. It can be used to switch any electrical terminal equipment on or off, like lighting. It uses the ZigBee communication protocol. It can also be controlled via a normal switch or push-button. Price: ~90€

Figure 15: Adoco radio controlled power switch11

EnOcean lighting actuator RCM 250 This 1-channel switch actuator is used to activate or deactivate any electrical equipment, such as incandescent lights. It is controlled via RF, using the EnOcean communication protocol.

Figure 16: EnOcean light actuator12

10 http://www.kadex-domotica.com 11 http://www.mkti.eu/shop/micromodule-domotic-zigbee-switch-adhocos1-adhoco-p-3075.html - http://www.adhoco.com/products/actuators/adhoco.s1/ 12 http://www.enocean-alliance.org/en/products/enocean_rcm-250/


http://www.mkti.eu/shop/micromodule-domotic-zigbee-switch-adhocos1-adhoco-p-3075.html�

http://www.adhoco.com/products/actuators/adhoco.s1/�

http://www.enocean-alliance.org/en/products/enocean_rcm-250/�


Shades, Curtains and Blinds

Telda Electronics - Blind Controller The NF850 2ch Curtain Driver controls Blind position and tilt angle through a LonWorks Network.

Figure 17: The blind controller from Telda Electronics13

ABB Shutter Actuator with manual operation – 4 fold

This actuator can control four independent groups for positioning shutters, blinds, awnings and other hangings. The up and down contacts are mechanically interlocked, so that voltage cannot be applied at both contacts at the same time. The communication is performed via KNX protocol. Manual operation is also possible, using the push buttons available in the front of the device. The direction of movement or the current position is displayed via LEDs, next to those buttons. There is, also, an 8 fold version of this actuator.

Figure 18: ABB Shutter Actuator, 4 fold with manual operation14

Heating, Ventilation, Air Conditioning, Air Filter, Air Moisturizer

AF24LON Damper Actuator Actuator for installation in a LonWorks application. It provides the modulation of dampers or valves in HVAC systems.

13http://www.echelon.com/productdb/productdetail.asp?prodID=637&manID=242&systemID=20&sysLabel=Doors+%26+Windows 14 http://www.knx-gebaeudesysteme.de/sto_g/English/_HTML/product_GHQ6310064R0111.htm

http://www.echelon.com/productdb/productdetail.asp?prodID=637&manID=242&systemID=20&sysLabel=Doors+%26+Windows�

http://www.echelon.com/productdb/productdetail.asp?prodID=637&manID=242&systemID=20&sysLabel=Doors+%26+Windows�

http://www.knx-gebaeudesysteme.de/sto_g/English/_HTML/product_GHQ6310064R0111.htm�


Figure 19: The AF24LON damper actuator from Belimo Automation Ltd.15

miele@home based HVAC

See info above about miele@home suite. Security Alarm System

Alarm System "Liana" This is a LonWorks compatible alarm system by FOLDERS Ltd. All units are connected to a single cable.

Door Status Sensors, Door Lock and Gate Opener

SDPASS by CIAC DES SDPASS acts as the ideal interface for LonMark compliant Access Control of one door to be installed as close as possible to it, minimizing cabling. A standard TTL card reader (Clock&Data) can be interfaced to the module, as well as a door state sensor and a request to exit pushbutton. It integrates one relay for electric locks and a second one for user purposes. In addition 2 free digital inputs are available. Physical specifications: Size: 105mm length x 85 mm width x 65 mm depth.

Figure 20: SDPASS - identifier and access control device16

Tecomat Foxtrot

This is a small modular control system from Teco a.s. By the connection of Tecomat Foxtrot to intelligent wiring elements INELS, a distributed management system is created. INELS elements can be divided into switches, buttons and sensors, actors (relays, dimmers etc.). The system can communicate with other systems via OPC open standard.

15 http://www.echelon.com/productdb/productdetail.asp?prodID=1147&manID=345&systemID=1&sysLabel=HVAC 16http://www.echelon.com/productdb/productdetail.asp?prodID=389&manID=172&systemID=24&sysLabel=Integration+Components

http://www.echelon.com/productdb/productdetail.asp?prodID=1147&manID=345&systemID=1&sysLabel=HVAC�

http://www.echelon.com/productdb/productdetail.asp?prodID=389&manID=172&systemID=24&sysLabel=Integration+Components�

http://www.echelon.com/productdb/productdetail.asp?prodID=389&manID=172&systemID=24&sysLabel=Integration+Components�


Figure 21: Tecomat Foxtrot module and INELS elements17

Surveillance Cameras The sector of surveillance cameras has evolved, especially due to an increased demand for surveillance in public areas. This requires a two-way network connection since the surveillance is usually done in a central station where security personal both watches and steers cameras. Available mainstream devices feature video streaming via network and a remote control of camera settings such as image recording, tilting, panning, and zooming. Either these functions are exposed by a proprietary protocol, such as a web-interface, or via an industry standard such as UPnP DSC v1.0 which defines the control of image settings.

DLink DCS-5300G The DLink DCS-5300G is a network-enabled surveillance camera. It supports resolutions up to 640x480 at 10 FPS. The degree of freedom for tilting is 90 deg. and 270 deg. for panning. The device lacks optical zoom, but features a digital zoom up to 4 times. An outdoor housing is available. The camera video is displayed using a proprietary ActiveX plug-in. Camera settings may be modified using the integrated web server or FTP. The device supports both wired and wireless network connections. It integrates into the home network via UPnP. However, it is unclear if the device supports UPnP DSC to control video capturing settings.

Figure 22: DLink DCS-5300G18

AT&T Video Monitoring Service A home monitoring service has been launched by AT&T in the US that includes live video surveillance on a computer or cell phone, as well as lighting controls and

17 http://tecomat.com/ 18 http://www.dlink.com/products/?pid=342

http://tecomat.com/�

http://www.dlink.com/products/?pid=342�


detection sensors for motion, temperature changes and flooding. There is no need to leave a home computer running to operate the system. The system can be programmed to take specific actions or send an alert via e-mail or text message when a sensor detects changes. If there is motion across the camera's field of view or if a sensor detects that a window has been opened, the system can automatically send an alert, turn on a light and start recording video. A Web-based dashboard is used to configure the system, and certain features can be controlled via cell. The remote video monitoring is not full-motion. Viewed on a computer, the picture runs at three to seven frames per second, vs. 24 to 30 in TV and movies. The system won't work if the power fails.

Presence Sensing

Infrared Reader by Visonic Technologies Americas This device is an infrared reader for local positioning systems utilizing a neuron ID and communicating via FTT10A transceiver supported by LNS 3.0. It uses Lonworks protocol.

Figure 23: Infrared reader for local positioning systems19

Mobility

Open Sesame Remote Controlled Door System OSHS2 This automatic door system unlashes, opens, and closes a door with the use of a button or a remote control. This package will release and open a locked (or unlocked) door, pause for entrance or exit, close automatically, and then return the door to its locked position. If you would like to extend the time the door is open, you can do so by pushing the remote control again. The door will stay open as long as you like. To close the door, simply push the remote once again. It leaves the door free-swinging for regular manual use. Price: $1,429.95 at http://www.homecontrols.com/Remote-Controlled-Door-System-OSHS2

Figure 24: Open Sesame OSHS2 System components20

19http://www.echelon.com/productdb/productdetail.asp?prodID=990&manID=320&systemID=22&sysLabel=Hospital+Equipment 20 http://www.opensesamedoor.com/residentialaccess.htm

http://www.homecontrols.com/Remote-Controlled-Door-System-OSHS2�

http://www.homecontrols.com/Remote-Controlled-Door-System-OSHS2�

http://www.echelon.com/productdb/productdetail.asp?prodID=990&manID=320&systemID=22&sysLabel=Hospital+Equipment�


http://www.opensesamedoor.com/residentialaccess.htm�


Truth Sentry II WLS Power Window System

This system is designed for casement and awning windows. A low profile design and quiet running motor are two of the features. No special preparation is required by the window manufacturer, making the Sentry II Power System suitable for on-site installation on new and existing (retrofit) windows. The system is provided with a rain sensor. This system can also be hooked up to a thermostat, an electric smoke detector, or even home control or security systems. The built-in flexibility of the Sentry II Power System allows for a variety of motor control options. Low cost, wireless control (via a RF remote control), multiple control or complete automated control with the Truth's Sentry II Power System can be tailored to meet specific needs.

Figure 25: Truth Sentry II Window system21

APC-300-EAC Access Point Controller for Elevator Access Control

This is an elevator access controller by Circon Systems Corporation. It includes a reader/keypad interface and elevator call button control for access control for one cab and 32 floors. It supports multiple card formats up to 128 bits, stores 10,000 access users, 64 access groups and 64 schedules in power-fail protected database. It includes a 1,500-entry event and alarm log and is focused for commercial buildings.

Figure 26: Elevator access control module22

Medical Devices

smartLAB®genie – Blood Sugar Meter The smartLAB®genie system is an in-vitro diagnostic medical device for self-monitoring blood glucose and can be used at home or by health care professionals. It is manufactured by Heidelberger-Medical-Marketing GmbH (HMM). With an optional available Bluetooth module the smartLAB®genie can be integrated with any

21 http://www.homecontrols.com/Sentry-II-Power-Window-System-TR435100005 22http://www.echelon.com/productdb/productdetail.asp?prodID=1523&manID=188&systemID=27&sysLabel=Monitoring+%26+Security

http://www.homecontrols.com/Sentry-II-Power-Window-System-TR435100005�

http://www.echelon.com/productdb/productdetail.asp?prodID=1523&manID=188&systemID=27&sysLabel=Monitoring+%26+Security�

http://www.echelon.com/productdb/productdetail.asp?prodID=1523&manID=188&systemID=27&sysLabel=Monitoring+%26+Security�


telemedicine system. In addition the measurement data can be transferred to any mobile phone or computer for further usage. The product costs about 40€ plus 15€ for compatible SmartLab Bluetooth receiver.

Figure 27: Blood Sugar Meter smartLAB®genie23

Sweetheart – Blood Pressure Meter

The talking Blood Pressure Meter SweetHeart manufactured by CareTec is specially designed for blind and visually impaired people. The values of each measurement are shown in big letters on the ultra large display, as well as spoken in a clear, natural voice. The SweetHeart device can be connected via an (original manufacturer) data cable to a computer in order to import measurement data to the computer. If the computer does not support such a port an USB adapter cable can be used instead. Price: 349 € + shipping expenses

Figure 28: Blood Pressure Meter SweetHeart24

Emergency Call Devices

Assist Cancel Emergency - Membrane This emergency call device by Merlon Healthcare Communications Pty Ltd is a membrane for Call Points. It is used in hospitals.

Figure 29: Emergency call device25

23 http://www.smartlab.org 24 http://www.marland.info/Blood_Pressure_Meter.708.0.html

http://www.smartlab.org/�

http://www.marland.info/Blood_Pressure_Meter.708.0.html�


Kadex Bracelet transmitter

This device, with the discrete format of a watch bracelet, sends an ON command when both of their buttons are pressed. It is possible to associate any actuator to this ON command, and so activate, for instance, an alarm system. The device uses the Kadex protocol for communication.

Figure 30: Kadex Bracelet transmitter26

Entertainment

Microsoft Windows Vista’s and Seven editions with bundled Windows Media Center Windows Media Center (MCE) based-PCs are very popular all-in-one entertainment devices that you can access music, photos, TV, movies, and the latest in online media all from the comfort of the couch with a remote control. Windows Media Center (MCE) offers built-in support for archiving TV shows directly to video DVDs, faster overall performance, and an improved platform for content owners to deliver new online entertainment services and experiences. MCE supports the following multimedia content (depending on the codecs installed in the machine): Music (Mp3, wma, …); Video (Xvid, Mpeg2, …); Movies; Recorded TV; Images (JPG, PNG, …); Live TV; DVD; Live Radio (FM); Applications MCE uses the following sources to access the multimedia content it supports.

• Music / Video /Images. The files of these content types are maintained in the Windows Media Player library. This library maintains the content organized and has links to the actual real files.

• Live TV / Live Radio o This is achieved with a tuning card. o By default, MCE can only use one tuning source. More tuning card

are supported via o Analog (TV & FM) / ATSC /DVB-T supported by Microsoft. o Some DVB-S supported by manufacturers. (Hauppage & Digital-

everywhere) • MCE applications allow access to other system and to online multimedia. • EPG Guide. This info is retrieved from reliable sources paid by Microsoft.

An important feature is the ability of adding MCE applications. They allow extending MCE functionalities.

o They can be programmed in both MCML + .NET & HTML + JavaScript

25http://www.echelon.com/productdb/productdetail.asp?prodID=1467&manID=351&systemID=22&sysLabel=Hospital+Equipment 26 http://www.kadex-domotica.com





o Examples are included (games & etc) in the installation. o Known as plug-ins; there are many available at the Internet.

Figure 31: Windows Media Center27

DreamMultimedia DreamBox Series

The devices of the DreamBox series feature TV tuners for DVB-T/C/S, a hard disk based DVR via EPG, network connectivity, and an adaptable OSD with support for skinning. The devices are running a Linux operating system which allows the installation of custom plug-ins and background services. Newer models will include more powerful hardware for support of high definition video. DreamMultimedia supports an active community of private developers for the DreamBox. The basis for the software was founded by the TuxBox project (http://wiki.tuxbox.org/Tuxbox). The DreamBox series consists of several models that cost 600 to 1100 EUR. Over the past years, the DreamBox has evolved to a very popular customizable solution for enthusiasts. Main parts of the software are licensed under the GPL and may thus be freely modified.

Figure 32: DreamBox DM-800028

Other sensors

The Nose® Monitor - 5 sensor version The Nose is a small indoor air quality monitoring device that can be mounted on the wall or ceiling of an apartment. The Nose is compact, about the size of a thermostat, and measures the five key indoor air quality (IAQ) constituents - temperature, relative humidity, carbon dioxide, carbon monoxide, and odors and gases (VOCs). The Nose is network-enabled and implements the LonWorks protocol.

27 http://www.microsoft.com/windows/windows-media-center/get-started/default.aspx 28 http://www.dream-multimedia-tv.de/en/dm-8000-hd-pvr

http://wiki.tuxbox.org/Tuxbox�

http://www.microsoft.com/windows/windows-media-center/get-started/default.aspx�

http://www.dream-multimedia-tv.de/en/dm-8000-hd-pvr�


Figure 33: The Nose monitor from PureChoice29

Photoelectric Proximity Switch Photoelectric Proximity Switches are light sensors that signal if anybody or anything is in proximity. The operating distance of these sensors can range from 60 mm to 2 m. The operating distance is virtually independent of target’s color and surface structure.

Figure 34: Contrinex LLS-4050-00030

Motion detection

With motion detection devices we can detect motions in a room. These devices use different technologies, including microwave and IR. Microwave has the advantage that it is independent from the temperature of the moving object. Therefore it can detect movements of bodies which do not have a temperature difference to their environment. Another advantage is that the device can be hidden behind a wall.

Figure 35: Microwave-based motion detector

RFID sensor RFID systems are distinguished based on the following features:

• operating frequency • reading distance

29http://www.echelon.com/productdb/productdetail.asp?prodID=886&manID=305&systemID=21&sysLabel=Fire%2FLife+Safety 30 http://www.contrinex.com/xml/productSheet.aspx?src=ps_LLS-4050-000&langage=English&typeFolder=OPTO

http://www.echelon.com/productdb/productdetail.asp?prodID=886&manID=305&systemID=21&sysLabel=Fire%2FLife+Safety�

http://www.echelon.com/productdb/productdetail.asp?prodID=886&manID=305&systemID=21&sysLabel=Fire%2FLife+Safety�

http://www.contrinex.com/xml/productSheet.aspx?src=ps_LLS-4050-000&langage=English&typeFolder=OPTO�


• penetration characteristics by materials • direction characteristics of the identification range • group reading ability • functionality in metallic surrounding field • designs

Stationary readers are typically deployed at warehouse portals, on conveyor belts or forklift arms, on store shelves. Depending on the application, readers can have different formats and deployed at different locations. An interesting variation is a mat that can be laid underneath a carpet. This mat has RFID tags integrated in a regular pattern. RFID readers can determine their position if they are on the carpet, based on the mat underneath. If a person had an RFID reader integrated into their shoes, it could determine the exact position of the person. Both RFID readers below, from Deister-electronics and from Pepperl & Fuchs, can make high speed detection and have the possibility to mount the transponder directly onto metal surfaces. Both can make group readings. The RIFD Reader TSG60 from Deister-electronic has a reading distance up to 8m, and the RFID Reader from Pepperl & Fuchs (System MV) has a reading distance up to 4m.

Figure 36: RFID Reader TSG60 Deister electronics31

Smoke Detectors

Smoke detectors are devices that detect smoke. Generally they issue a local audible and/or visual alarm from the detector itself. Most smoke detectors work either by optical detection (photoelectric) or by physical process (ionization), while others use both detection methods to increase sensitivity to smoke. Ionization alarms sense fast, flaming fires, while photoelectric models better detect the smoke from slow smolders. For the best protection, experts say you need to have both types of alarms or a model with both technologies.

Figure 37: First Alert SA302CN Dual System Smoke Detector with Remote Testing Function32

G.TEC physiological sensors

G.TEC provides several sensors that allow measuring body functions simultaneously to EEG, ECG, EMG and EOG signals. Analog sensors can be combined with g.Recorder for data acquisition as well as visualization and with the High-Speed Online Processing

31 http://www.deisterrfidportal.de/en/index.htm 32 http://www.jr.com/first-alert/pe/FA_SA302CN/

http://www.deisterrfidportal.de/en/index.htm�

http://www.jr.com/first-alert/pe/FA_SA302CN/�


toolbox for real-time parameter extraction. Some of their products are dedicated to the measurement of the galvanic skin response (g.GSRsensor), piezoelectric respiration sensor (g.RESPsensor), photoelectric pulse sensor (g.PULSEsensor), piezoelectric snoring sensor (g.SNORINGsensor), sensor for the changes in skin temperature (g.TEMPsensor) and oxygen saturation sensor (g.SPO2sensor). For more information see http://www.gtec.at/products/g.Sensors/gsensors.htm.

Pressure sensor

The ForcePLUX sensor is a wearable pressure sensor that can be used, for instance, for postural analysis, strength measurement, and walking patterns evaluation and to determine body weight. The communication mechanism uses a proprietary protocol that communicates to a receiver that communicates then by Bluetooth, which can be received in a PC with a Bluetooth receiver (see http://www.plux.info/force).

ABB Water Detector

This water detector detects water ingress. This can be caused by pipe rupture, ingress of groundwater or sewages, broken home appliances, etc. This sensor is designed to be connected to a security/alarm circuit. Communication to external devices is done using a potential free contact. It uses KNX protocol.

Figure 38: ABB Water detector33

ABB Brightness Sensor

This sensor is used to read the ambient brightness level. Three brightness thresholds can be set. The communication is performed via KNX protocol and it is for outdoor use.

Figure 39: ABB Brightness Sensor and control device34

33 http://www.knx-gebaeudesysteme.de/sto_g/English/_HTML/product_GHQ4030001R0012.htm 34 http://www.knx-gebaeudesysteme.de/sto_g/English/_HTML/product_GHQ6050063R0001.htm

http://www.gtec.at/products/g.Sensors/gsensors.htm�

http://www.plux.info/force�




Figure 40: ABB Light Sensor1

ABB Light Sensor To help control of lightening in rooms with difficult light conditions, one can use this indoor light sensor. When combined with a light controller, it is possible to have constant light control in a room. There can be several light sensors connected to the same light controller. This device uses KNX protocol for communication.

IR Emitters

Tira-2.1: Remote Control Receiver/Transmitter In some situations, it is needed to transmit IR signals to devices. This can be used, for example, when automating a device that was not previously prepared to be integrated in a smart home installation. This way we can have a PC controlling any kind of device that receives IR commands. This IR receiver/transmitter, from Home Electronics, can receive and transmit the IR signals. But it can also record IR signals transmitted to it. This product is listed with a 49 $US price.

Figure 41: The Tira 2.1 Remote Control Receiver/Transmitter from Home Electronics35

55..22 RReellaatteedd aapppprrooaacchheess aanndd tteecchhnniiqquueess This subsection introduces use cases and techniques applied to recognize determined situations for the purpose of providing assistance or any other service. This is the context-awareness feature of Ambient Intelligence. This is supplied by communication technologies, middleware and protocols, and truly enriches the understanding of the environment and therefore, enhances the system’s proactiveness. As mentioned in the previous section, the role of Artificial Intelligence (AI) to achieve a context-aware approach is important. AI techniques are in charge of interpreting multimodal data coming from different sensors. As a result, the system is capable of interpreting the environment’s state; representing the information and knowledge associated with the environment; modelling, simulating and representing entities in the environment; and planning decisions or actions [50]. The use cases presented below describe examples of different applications of AmI. The situations to be recognized thus depend on specific application requirements and can be as simple as the recognition of the light condition of a single room, or as complex as the recognition of human activity and actions. Important issues are therefore the definition of situations (the use of ontologies is a good option), the selection of sensors and appliances which capture the required information to recognize the situation, and the AI techniques and reasoners which interpret this information. Situations of interest in BrainAble are those related to proactive assistance in order to enhance the physical independence and social

35 http://www.home-electro.com/images-ira/tira2.php

http://www.home-electro.com/images-ira/tira2.php�


interaction of people with functional diversities. Functionality and description of BrainAble services are presented in detail in D.2.1 – Requirement Specifications. There, an iterative user-centred design based on the development and validation with real users of up to three prototypes is described. In this sub-section, brief descriptions of some examples use cases are presented below:

Fall detection and anomalous gait recognition

This approach is part of the Confidence project and aims at the development of a ubiquitous care system for monitoring users in order to detect health problems. These problems can be immediate (fall), short-term (limping due to injury or dizziness) or long-term (hemiplegia, Parkinson’s disease, age-related deterioration of movements). Therefore, a fall detection and anomalous gait recognition system was proposed [51]. Users of this system should wear a number of radio tags placed on the body in order to acquire activity information. Thus it is possible to reconstruct the user’s posture and movement to analyze his/her behaviour.

For this study 12 tags were used. They were attached on shoulders, elbows, wrists, hips, knees and ankles, sampled at 10 Hz. The recording of these different body coordinates was performed by the use of a Smart infrared motion capture system. Walking/standing, sitting, lying, the process of sitting down, the process of lying down or falling were the activities that the system was able to classify in order to analyze posture. It was accomplished by training a classifier that could recognize the activity from a one-second interval of the user’s behaviour. The feature vector for machine learning is a concatenation of the features belonging to the ten snapshots of the user’s posture in that interval. From a set of machine learning algorithms used, the Support Vector Machine (SVM) offered the highest accuracy.

Prevention of emergencies, recognizing Activities of Daily Life (ADL)

The aim of this approach is the recognition and generation of preventing solutions of potentially emergency situations. It is part of the Emerge project [52], and is based on the detection of emergency situations at home by embedded ambient and unobtrusive sensors. These sensors are of different types and range from presence detectors, pressure mats or reed contacts, to the monitoring of vital data through an unobtrusive wrist-mounted device, a weight scale, and a blood pressure measurement device. Specific situations to be detected can be distinguished by immediate or short-term events, as sudden falls or cardiac arrest, or by long-term deviations from typical behaviour or critical trends of vital parameters. These long-term trends are useful indicators of changes in the mental or cognitive state, as well as other aged-related chronic diseases.

The reasoning approach deployed by this Ambient Assisted Living (AAL) system is in charge of detecting complex human activities from a continuous sequence of events measured by an array of sensors. Therefore, it is important to detect the specified situations accurately, as reliable and timely as possible. It is important to consider the sensor heterogeneity. Their configuration highly depends on the specific home layout. Furthermore, a typical daily activity is not always represented by a sequence of events that are clearly identified by a temporal distribution. In this approach, an event-driven recognition is applied by dividing complex activities in simpler atomic activities which


are detected by a specialized detection agent that has its own special logical unit. The functions of detection agents are the reception, storage in cache memory and analysis of data, and reporting of determined detected signal or pattern [53].

Monitoring and personalized health care system

This system has been designed to provide users personalized health care services through ambient intelligence. This is part of the VirtualECare project and deploys a multi-agent architecture for monitoring and detecting determined situations in order to report them to interconnected healthcare institutions, leisure centres, patients and their relatives on a common network. The system is therefore, responsible of collecting the information about the environment considered important, the study of the behaviour of the user in terms of the management of the house and inferring of rules for that management.

The information collected may be the temperature, luminosity or humidity of the rooms, the position of the user or other persons inside the house, and the state of the required home appliances. An enhancement of the monitoring capabilities is achieved by adding portable measurement devices worn by the user thus vital data is also collected out of the house. Some reactive and simple decisions can be taken by the system when some predetermined events occur. Moreover, the environmental information can be shared with higher level components of the architecture, so that more complex decisions can be taken as well [54].

Activity recognition in a multi-user scenario

An enhanced recognition of human activities is obtained by analyzing the interaction of the user with the environment and his/her relation with other people within this same environment. Recognizing the activities of multiple users requires a more complex analysis than for one single user. The main challenge is to find a suitable model to capture the interactions between users and perform inference using these observations. In this approach, a temporal probabilistic model was proposed based on Coupled Hidden Markov Models (CHMMs) to model the user interaction and recognize multi-user activities.

Observations of users and their interactions were captured by a specially designed wearable sensor platform. This platform was designed to measure user movements of both hands by using accelerometers, user location, human-object interaction through the sensing of touched RFID identified objects and sound, human-to-human interaction through voice recognition, and environmental information through measurements of temperature, humidity and light. The large amount of information captured by these measurements was converted onto a series of observation vectors which contained all of the data observed in a fixed time interval which was set to one second in their experiments. Different types of sensors require different processing to compute different features [55].

Recognition of food preparation activities

The Slice&Dice system has been developed to detect low-level food preparation through a set of custom made cooking utensils equipped with embedded


accelerometers. This is part of an Ambient Kitchen project, thus this system also makes use of three IP cameras which directly focus on the work surface where the food is prepared. There were 11 basic activities to recognize: chopping, peeling, slicing, dicing, coring, spreading, eating, stirring, scooping, scraping and shaving. Experiments carried out captured data in a set of time intervals ranging from 0.8 seconds to 12.8 seconds. For the data of each interval statistical measures like mean, standard deviation, energy and entropy were calculated. Afterward, the feature vectors computed from the different interval data collections were trained for the Bayesian Network, Decision Tree and Naïve Bayes classifiers [56].

Routing assistance in cognitive impairment situations

The benefits of AmI are also suitable for rehabilitation purposes. In these cases, the main role is to support action by offering creative and effective environmental oriented interventions; adaptable tools according to the characteristics of the user and the provision of feedback from his/her own actions. In this approach, this support was provided to users who have revealed a topographical disorientation disorder. This disturbance impairs the memory of environmental landmarks, in which the physical features are correctly perceived but not remembered. The achieved AmI solution was to provide a sense of location within the environment, both outdoors and at home. Therefore, position and location information was detected by using a wide range of sensors as motion detectors and other ubiquitous computing infrastructure. Furthermore, the system had the capacity of learning everyday behaviour by decoding patterns which recognized signs of distress, disorientation or confusion by monitoring user’s biological signals, voice recognition and behaviour analysis. Thus, as a response, the system could provide help through non invasive sound or visual cues [57].

55..33 RReellaatteedd pprroojjeeccttss AMIGO. (2004–2008) Ambient intelligence for the networked home environment project develops middleware that dynamically integrates heterogeneous systems to achieve interoperability between services and devices. For example, home appliances (heating systems, lighting systems, washing machines, refrigerators), multimedia players and renderers (that communicate by means of UPnP) and personal devices (mobile phones, PDAs) are connected in the home network to work in an interoperable way. This interoperability across different application domains can also be extended across different homes and locations. The AMIGO interoperable middleware architecture is specifically designed to realize an open networked home system that dynamically integrates heterogeneous devices as they join the network The issues related to privacy, rust and security are, however, not addressed by the platform [58]. This project is a reference for interoperability issues and middleware architecture. It is of significant interest the experience acquired with the heterogeneity of devices and their integration. BrainAble tackles the integration layer by using the standard URC/UCH.


Figure 42: Diagram of the AMIGO project. Dynamic integration of heterogeneous systems through specialized middleware.

EasyLine+. (2007–2009) Low cost advanced white goods for a longer independent life of elderly people project brings ambient intelligence to the kitchen environment. It features a central intelligence system that is aware of the status of all white goods in the kitchen without the necessity of replacing existing equipment. Main focus is to provide home appliances the capacity to communicate. Diverse basic sensors are introduced (temperature, fire, smoke, flooding and presence), which together with RFID and EPC (electronic product code) provide useful information as the content of the washer machine or food nearing to expiring date in fridge. The user interface might be through PDAs, smart phones, desktop computers, digital TV or devices in the form of embedded control panels, which supply the interaction of a centralized system coordinated by an e-servant [59]. In this context, EasyLine+ serves as a reference for device and service functionality, as well as a use case for AmI. Though BrainAble’s focus on providing assistance implies special and enhanced services and requirements, EasyLine+ shows examples of Home Automation services and technologies. Figure 43: Diagram of the EasyLine+ project. Addition of ambient intelligence to the kitchen environment enhancing

communication between devices.


INREDIS. (2007–2010) Interfaces for relation between environment and people with disabilities project is focused on the development of basic technologies that allow creating communication and interaction channels between people with special needs and their environment. The project tackles different environments of the daily life of the people, as the domestic and urban environment, work environment, banking and retail, among others. A common interoperability SOA provides the soil for the integration of external and internal services enabling interaction with common use devices using a cellphone. Issues such as accessibility, ubiquitous computing, interoperability, multimodal interaction and security are managed and encompassed in the final solution. The final INREDIS architecture model integrates EDA, OSGI and Multi-Agent Systems approaches [60]. The similarity with BrainAble comes mainly through their assistive nature and device integration technologies of both projects. BrainAble emphasizes on AmI and Context-Awareness in order to provide improved assistive services and BCI-based interfaces. However, the experience acquired in INREDIS related to integration architecture and especial services is significantly useful.

Figure 44: Inredis architecture: enhancing interaction between people with special needs and their environment.

SM4ALL. (2008–2011) Smart hoMes for ALL project aims at studying and developing an innovative middleware platform for inter-working of smart embedded services in immersive and person-centric environments, through the use of composability and semantic techniques, in order to guarantee dynamicity, dependability and scalability, while preserving the privacy and security of the platform and its users. This is applied to the challenging scenario of private/home/building in presence of users with different abilities and needs (e.g., young able bodied, aged and disabled). In particular, in the SM4ALL project P2P, service-orientation and context-awareness are merged in novel ways in order to define


general reference architecture for embedded middleware targeted to immersive scenarios, among which the domotics and home-care have been selected as showcases. The user is able to interact with the house through many interfaces, either centralized (e.g., in a home control station) or distributed and embedded in specific interface devices. In particular, BCIs will be provided for specific categories of users (disabled, etc.), in order to have all really able to exploit the domotic infrastructure [61]. The architecture and intelligence presented in this project, together with the BCI integration, represent an interesting reference. However, BrainAble focus is directed toward a more practical user-centered approach.

Figure 45: Diagram of the project SM4ALL. Design and development of smart embedded devices for a person-centric

intelligent environment controlled by heterogeneous interfaces.

i2home. (2006–2009) Intuitive Interaction for Everyone with Home Appliances based on Industry Standards. The EU-funded project i2home has taken a substantial step towards the long-term vision of an internet with built-in accessible, usable, feasible and affordable user interfaces for everyone. The i2home project’s main objectives have been to implement a standards-based open platform called Universal Control Hub, or UCH for short, and use this platform to implement personal user interfaces for users with special needs so that they can interact with a smart home. This platform only requires that appliances and services are networked, i.e., a computer can communicate with them. In this way, it is possible to integrate not only traditional smart home appliances, such as lamps, heating etc. but also arbitrary software services, such as calendar/reminder systems or video conferencing


systems. Within the project, several personal user interfaces have been evaluated in order to assess their feasibility, usability and accessibility. Many users were actively involved in the project by contributing to requirements on the user interfaces as well as evaluating them. They have been partially-sighted and blind persons, young persons with mild cognitive impairments, elderly persons and, finally, persons with Alzheimer’s disease. The project bases its efforts on industrial standards and mainstream technology: The Universal Remote Console (URC) technology is an international standard (ISO/IEC 24752). The implementation thereof is called the Universal Control Hub (UCH). Activity management, as implemented in i2home, has been standardized as industry standard (ANSI/CEA-2018). The project has also built upon some important mainstream technologies and networked appliances and services, such as Google calendar, Windows Media Center (WMC), and Siemens Serve@home kitchen appliances [62]. This would be an example of URC/UCH application and usage as a platform for device connection and smart home interaction.

Figure 46: i2home general architecture overview

Vital. (2007–2010) Vital assistnce for the elderly. The VITAL project is User centric and for that reason its main objective is to develop a new services platform with the aim to significantly increase the quality of life of the average elderly user. The new platform departs from traditional assistance schemes in the sense that it is not only oriented to satisfying day to day subsistence needs but it considers also other important demands that have deserved little attention up to the moment, such as the need for: information, inter-personal communications, entertainment and self-education.

The VITAL platform is composed of the central services, the user terminals (TV and mobile), the operators' terminals, the software required to run the users' applications and the contents necessary to test and operate such applications. In a typical situation of use, an


elderly person will sit in front of his TV and will make use of his remote controller or speech commands to select the desired service. Then the application software will guide the User through all the steps necessary to complete his demand. Those steps will be translated internally into a number of complex actions that will develop transparently for the Final User until the desired service has been completed. The main aim will be that any person capable of using a standard TV set will be also able to use the VITAL services [63]. In this context, this project presents some interesting applications which serve as reference assistive services for BrainAble.

Figure 47: Vital architecture overview.

Bedmond. (2008–2011) Behaviour pattern based assistant for Early Detection and Management of Neurodegenerative Diseases. BEDMOND is an assistant for the health professional, a daily behaviour information provider to early diagnose mild cognitive impairment (MCI) stages as a first step of neurodegenerative diseases, focused in elderly people while living at home [64]. It consists of:

• A tool for the caregivers at home to be informed about the last activities’ performance;

• An alert tool to inform about home alarm signals and react quickly on high risky situations;

• A main support tool for medics to early diagnose neurodegenerative diseases. This will be done:

• Firstly, by continuous tracking daily activities’ performance of the elder at home, then compared with a set up and recorded behaviour pattern and intelligently ruled with health professionals` criteria to be applied after (diagnosis)


• Secondly, by periodical monitoring of user’s behaviour evolution and trends for the doctor to determine a MCI beginning;

• And when it is determined, a pharmacological treatment can be applied in such an early stage. Then the support tool helps to monitor the delay of the Alzheimer’s disease evolution.

As an assistive approach, this project also presents specialized services which serve as reference for the BrainAble platform.

Figure 48: Overall Concept of BEDMOND project.

i2life. (2008–2010) Real-time Monitoring and Evaluation of Bio-Signals and Remote Assistance. The main objective of the project is to create a solution for Ambient Assisted Living (AAL), which main objectives are:

• Simple and unified interface, capable of controlling and monitoring any appliances and health devices at the patients home

• Permanent, non intrusive, monitoring and control • Follow chronic diseases, special health needs, elderly people, disabled

people • Prevent emergency situations:

• Falls • Cardiac problems • Respiratory problems

• Track patient’s tasks: • Meals eaten • Sleep hours • Medicines taken

• Longer independency for the patients For achieving these objectives a non-intrusive body sensor network (for monitoring heart and respiratory frequency, blood oxygen saturation, body temperature, blood pressure) will


be created and integrated in the URC/UCH platform. It will be developed as an expert system that allows monitoring and analyzing the user behaviour and to detect problems and emergencies. The solution will also contain an alarm management system and will enable the access to the information by health professionals and the users themselves [65]. This exposes the capability of the URC/UCH middleware for the possibility of biosignal monitoring implementation on BrainAble.

Figure 49: I2life project general overview.

MonAmi. (2006–2010) Mainstreaming on ambient intelligence. MonAMI will select bouquets of services to support people at risk of exclusion and loss of autonomy. Step two is then to construct, test and deploy these services and demonstrate that they can be economically mainstreamed into future ambient intelligence technologies. The objective of the MonAMI project is to demonstrate that accessible, useful services for elderly and disabled persons living at home can be delivered in mainstream systems and platforms. This will be done in close cooperation with users and by involving key mainstream actors throughout the whole process. The technology platform to deliver the services will be derived from standard technology. It will integrate elements such as reliable self-organizing networks, wearable devices, and user interaction technology, monitoring capability and service infrastructures that ensure quality of service, reliability and privacy. The platform consists of five main components:

• A set of wireless sensors/actuators • Accessibility: a server of the user interface using an ISO standard: Universal Remote

Control (URC/UCH) • Service: A dynamic service platform based on OSGi • A touch screen as main interface • A Telco Web platform to manage alarms and service updates

The services will be delivered on mainstream devices and services such as digital-TV, third-generation mobile telephones and broadband Internet. To facilitate use and user interaction, MonAMI will develop an innovative interface, involving an embodied conversational agent [66]. In BrainAble, BCI interfaces imply a constraint for mainstream application. MonAMI, however, brings this issue as a reference for the device and service integration.


Figure 50: MonAmi project concept overview

ASK-IT. (2004–2008) Ambient Intelligence System of Agents for Knowledge-based and Integrated Services for Mobility Impaired Users. The ASK-IT project aimed to establish ambient intelligence (Ami) in semantic web enabled services, to support and promote the mobility of people with special needs (MI), enabling the provision of personalized, self-configurable, intuitive and context-related applications and services and facilitating knowledge and content organization and processing. ASK-IT developed an environment that advanced mobile devices as personal guides in leisure, sport, education, work, socialization and tourism and will allow effortless movement of the elderly and disabled people across Europe. Emphasis is on seamless service provision, independent of the media, user location (i.e. indoors, outdoors, in a city, during a trip, etc.), user type and residual abilities [67]. Experience from this project is significant in the fields of AmI and especially Context-Awareness.

Figure 51: ASK-IT architecture overview.


SOPRANO. (2007–2010) Service-oriented Programmable Smart Environments for Older European. SOPRANO aims to design systems which enable significant groups of older people to cope better with everyday life in greater comfort and safety, including supporting the delivery of high quality support care, and to help all Europeans to play a full role in society. Focus is on the use of intelligent ambient systems to support independent living, particularly in the home environment. SOPRANO should act as an informed, friendly agent, taking orders, giving advice or reminders and ready to help, and get help, when needed. Three strands of research and development are to be integrated:

• Stand-alone assistive technology: products designed to compensate for motor, sensory and cognitive difficulties frequently experienced by older adults;

• Smart home technology: networking of ICT in the home environment, with the integration of appliances and devices to provide control of the entire living space;

• Telecare services: applications addressing care-related needs prevalent among older people, with ICT utilized to enable support from professionals and informal carers.

The technical architecture will enable proactive assistance by interpreting information gathered by the system about a user's situation. Responses are to follow agreed rules and seamless access is to be provided to external professionals. Safety and security is strongly enhanced with adherence to stringent reliability standards. Interaction with users by voice and TV helps meet special accessibility and usability needs. Services utilizing SOPRANO capabilities are to be designed for older people, including those with age related cognitive changes. To ensure that services fully meet user needs, developers will work with potential users of SOPRANO systems throughout the project lifecycle, from user requirements analysis, through iterative prototyping, validation of concepts and functionality and usability tests to large-scale field trials involving users in their own homes [68]. OpenAAL, an open source middleware for ambient assisted living resulting from this project is an important reference for the intelligence and context-awareness approach to be implemented on BrainAble.

Figure 52: Soprano concept overview.


universAAL. (2010-2014) Universal open platform and reference specification for Ambient Assisted Living. universAAL aims to reduce barriers to adoption and to promote the development and widespread uptake of innovative AAL solutions. It will benefit end-users (i.e. elderly people and people with disabilities, their carers and family members) by making new solutions affordable, simple to configure, personalise and deploy. It will benefit solution providers by making it easier and cheaper to create innovative new AAL services or adapt existing ones using a compositional approach based on existing components, services and external systems. It should be as simple for users to download and setup AAL services as it is to download and install software applications on a modern operating system. universAAL will establish a store providing plug-and-play AAL applications and services that support multiple execution platforms and can be deployed to various devices and users. Finally the allocation of local human resources is also supported in the store.

universAAL aims to produce an open platform that provides a standardized approach making it technically feasible and economically viable to develop AAL solutions. The platform will be produced by a mixture of new development and consolidation of state-of-the art results from existing initiatives [69]. This ongoing project represents a possible framework for BrainAble to develop and test new services.

Figure 53: universAAL vision and key functionalities


66 CCoonnssoolliiddaattiinngg AAmmII –– TThhee nneexxtt sstteepp Ambient Intelligence is a field that is constantly growing and developing. It has acquired an important role as an aid for assistive technologies and is currently a relevant field of research providing an extensive number of suitable applications. However, a real AmI implementation should deal with several technical obstacles which limit the generalization of its benefits and therefore of its deployment. Some of these limitations come in the form of the lack of flexibility, adaptability and scalability of the systems. Furthermore, as it may be read in previous sections, the heterogeneity of communication technologies and protocols is difficult to integrate and manage. It is useful to dispose of different communication technologies; they fulfil specific requirements according to the application. Nevertheless, their integration through specific bridges or middlewares imposes other constraints that compromise the performance of the system. Section 4, introduced connectivity and interoperability issues, which range from the physical communication media to protocols especially designed for home automation. It was shown that there is no preferred standard. In fact, it is noticeable the fierce competition between different interested companies to promote their own protocol in order to control the home automation market. The spread a protocol is applied depends very frequently on the availability of compliant products on the market, which is proportional to the support given by the owner company or group of companies; and not precisely as a consequence of a better performance. Knowing this, it is already expected that the general interest of these companies is to keep users, and in the case of a standard to keep also developers, to be dependent on their technology. The technical commitments for the consolidation of AmI as a global implementation will therefore come when these limits will be surpassed. AmI is more than the completion of a home automation system which allows users to control home appliances. As described in the first sections, the perception of situations and performance of corresponding actions contribute to the context-aware distinctive feature and should be considered as the core of AmI. That feature is commonly studied and implemented mostly on research projects which are application specific. Thereby, we recommend as a next step for the consolidation of AmI the enhancement of flexibility, adaptability and scalability. It will permit not only the implementation of different types of communications, protocols and middlewares, or the capacity to add or delete devices of the network, but also the compliance of low and high level systems which can be designed by different developers and easily integrated to fulfil the requirements of a given application. This requires the usage of a standard and well-defined description of the capabilities, services and states of any device or subsystem implied, which could be provided by the generic usage of concepts of agents and ontologies. BrainAble – an AmI-based approach The BrainAble project is an ICT-based aid for people with motor disabilities. Its aim is to overcome two of the main shortcomings these people suffer from: exclusion from home and social activities. Hence, the focus of this project is to design and develop a Human-Computer Interface (HCI) composed of Brain/Neuronal Computer Interface (BNCI) sensors combined with affective computing and virtual environments. Ambient Intelligence is essential within BrainAble since people with motor disabilities benefit very much from the inclusion of ubiquitous and context-aware technologies. It is


well known that AmI is, due to its assistive nature, very beneficial for assistive applications. Among the most common types of assistance is the enhancement in the capabilities of controlling some environment features and appliances, as well as the understanding of situations to proactively perform actions such as triggering an alarm in case of emergency. According to the above discussion about the consolidation of AmI, the BrainAble project must contribute on providing means thus a feasible, expandable and adaptable approach is available. Selecting and specifying adequate communication standards and open protocols, as well as the modelling and definition of required contexts should be the first steps. It will bring the flexibility and adaptability in which even disposing of a different group of sensors; the context – ranging from a general to a more detailed - might be detected. This will be made possible due to the level of abstraction of the data extracted from sensors which provides the ambient information available at that moment. The implementation of URC/UCH as middleware should serve as the mean where incoming data from sensors will be translated to a more abstract format defined by the context model. A supplementary, but significantly useful aid may also be provided by enhancing the versatility of the BrainAble user interface presenting menu-options and preferences according to the salience of determined features or situations that occur in the environment. The salience is a concept broadly used in neuroscience and is considered as an attention mechanism from which the organism is able to focus its limited perceptual resources to the relevant sensory information that is available. Therefore, disposing of AmI as a source of saliency of the user’s environment, it is possible to not only manage a reduced amount of information (to be focused only on what is important), but also predict and offer the user with the most probable selectable actions. In BrainAble this vision of AmI might be considered helpful and its role considerable important due to the nature of the user input. It could serve as a complement to the Brain–Computer Interface (BCI) redefining or reinforcing the “hybrid BCI” concept; thus the interaction with the system will become more robust and reliable. BrainAble Functionalities BrainAble aims at designing, implementing and validating a platform to allow disabled people to achieve a higher level of independence by means of BCI, affective computing and virtual reality. An overall diagram of the BrainAble functionalities is depicted in Figure 54. The system enables the user to interact with the Automated Devices present in their home. These devices are to be accessed through an Ambient Intelligence, that is, some decisions are to be made by the user in his own, some other are to be assisted by the system (taking into account the context of the user) and, eventually, some others are to be made by the system according to the user configuration. Users of the BrainAble system are persons with functional diversities from neurological origin who are able to use the BNCI system. Even if many users share the same type of disabilities, group-specific but also person-specific some of the limitations of the users are common, a customisation of the system will be possible like, for example, the functionalities available, alarms for the user or environmental parameters (temperature, humidity, etc.)


Figure 54: BrainAble functionalities and applied technologies

The Central Intelligence implements the context-awareness functionality that recognises the context of the environment. The context is obtained by many different sources such as ambient devices (e.g. light, temperature, humidity) or affective computing which may provide the physiological status of the user. The context will have an effect on the operation of the rest of the functionalities. The Virtual Environment is to provide the system with a virtual environment where the user will be able to interact with. BNCI is envisaged as the tool to control the Virtual Environment which grants the access to all the foreseen users. The user will have many different activities available such as chatting virtually with another BrainAble user, or the so-called self expression tools like composing music. Since some of the BrainAble users are limited in terms of mobility, the Social Network will enable them to meet other people and share experiences. The system will able to connect to the most important social sites like Facebook or Twitter. Although the main objective of the social network is the mitigation of the social isolation, a second objective, not negligible, is also accomplished: a rise in the visibility of the disabled communities. For detailed information about the modular BrainAble architecture and functionalities, please refer to D.5.2. – Technical Specifications for BrainAble AmI Architecture.


77 CCoonncclluussiioonn

Ambient Intelligence is experiencing a rapid growth and is receiving interest not only from the home automation market, but from several different fields which have foreseen its potential. It can be considered a system that perceives specific situations and reacts according to them. This is possible due to the sensing of the environment, ubiquitous communications through pervasive computing and artificial intelligence techniques. A distinctive feature of these types of systems is therefore context-awareness, which is the capacity of recognizing situations; the invisibility of the applied technology and its usage through natural interfaces. It disposes of a large number of communication technologies and protocols, each of them offering advantages for determined applications. Currently, the commercial scope of AmI is tending to satisfy requirements of home automation by providing remote control of appliances and enhancing user’s comfort and entertainment. In most cases, the intelligence of the environment or context aware component is rather simple. On the other hand, research approaches are more focused on the development of this context-aware component including innovative context models, application-specific sensor configurations to recognize situations, AI techniques for learning and reasoning, and the development of suitable standards. It is reasonable to think that the generic usage of these standards is useful to overcome certain limitations towards the consolidation of AmI in our everyday life. Nevertheless, leaving all these technical constraints behind, from the point of view of assistive technologies, any type of AmI application, ranging from a simple switching of a light to the detection of activities of daily life for rehabilitation, could be significantly helpful for a person with disabilities.


88 RReeffeerreenncceess

[1] Streitz, N. and Savidis, A. Mission Statement of the ERCIM Working Group SESAMI (Smart Environments and Systems for Ambient Intelligence), 2006.

http://www.ics.forth.gr/sesami/index.html [2] Schmidt, A. and Van Laerhoven., “How to build smart appliances?” IEEE Personal

Communications, 2001. [3] Stephanidis, C., The Universal Access Handbook, CRC Press Taylor & Francis Group. 2009. [4] Kamel, M.N., Castellot, R., Anastasiou, A., Nugent, C.D., Alexandersson, J., Zimmermann G.,

Cortes, U. and Casas, R., “Connectivity for Healthcare and Well-Being Management: Examples from Six European Projects,” Int. J. Environ. Res. Public Health. 2009.

[5] Catarci, T., Cincotti, F., de Leoni, M., Mecella, M. and Santucci, G., “Smart Homes for All:

Collaborating Services in a for-All Architecture for Domotics,” CollaborateCom, pp. 56-69, 2008. [6] Schmidt, A., Ubiquitous Computing – Computing in Context, PhD Dissertation, 2002. [7] Schmidt, A., Interactive Context-Aware Systems Interacting with Ambient Intelligence, IOS Press,

2006. [8] Cook, D.J., Augustoz, J.C. and Jakkula, V.R., “Ambient Intelligence: Technologies, Applications,

and Opportunities,” Pervasive and Mobile Computing, Vol. 5, No. 4, pp. 277-298, 2009. [9] Goldstein, E.B., “Wahrnehmungspsychologie. Eine Einführung“, Spektrum Akad. Vlg., Hdg., 1997. [10] Stylus Inc. White Paper: Pervasive (Ubiquitous) Computing Today, 2005.

http://www.stylusinc.com/Common/whitepapers/.

[11] Weiser, M., “The Computer for the 21st Century”, Scientific American, vol. 265(3), pp. 94-104, 1991.

[12] Välkkynen, P. Physical Selection in Ubiquitous Computing, VTT Publications, 2007.

[13] Pinkerton, A., PostNote 263, “Pervasive computing”, Parliamentary Office of Science and Technology, 2006

[14] Helal, S., Winkler, B., Lee, C., Kaddourah, Y., Ran, L., Giraldo, C. and Mann, W., “Enabling location-aware pervasive computing applications for the elderly”, Proceedings of the First IEEE Pervasive Computing Conference, 2003.

[15] Maurer, U., Smailagic, A., Siewiorek, D. and Deisher, M., “Activity recognition and monitoring using multiple sensors on different body positions”, Proceedings of the International Workshop on Wearable and Implantable Body Sensor Networks, pages 99–102, 2006.

[16] Doctor, F., Hagras, H. and Callaghan, V.,” A fuzzy embedded agent-based approach for realizing ambient intelligence in intelligent inhabited environments”, IEEE Transactions on Systems, Man, and Cybernetics, Part A, 35(1):55–65, 2005.

http://www.ics.forth.gr/sesami/index.html�

http://www.stylusinc.com/Common/whitepapers/�


[17] Cook, D. and Schmitter-Edgecombe, M., “Activity profiling using pervasive sensing in smart homes”, IEEE Transactions on Information Technology in BioMedicine, 2008.

[18] Simpson, R., Schreckenghost, D., LoPresti, E.F. and Kirsch, N., “Plans and planning in smart homes”, In J. Augusto and C. Nugent, editors, Designing Smart Homes: The role of Artificial Intelligence. Springer Verlag, 2006.

[19] Galton, A., Qualitative Spatial Change. Oxford University Press, 2000.

[20] Hagras, H., Callaghan, V., Colley, M., Clarke, G., Pounds-Cornish, A. and Duman, H., “Creating an ambient-intelligence environment using embedded agents”, IEEE Intelligent Systems, 19(6):12–20, 2004.

[21] Allen, J.F. and Ferguson, F., “Actions and events in interval temporal logic”, Journal of Logic and Computation, 4:531–579, 1994.

[22] Cisco Systems Inc., “Internetworking Technologies Handbook”, Cisco Press, 3th Ed., 2000.

[23] USB Implementers Forum, “USB Specifications”, White Paper at www.usb.org/developers/docs, 2008.

[24] 1394 Trade Association, “1394 Technology”, www.1394ta.org, 2010.

[25] Cacciaguerra, F., “Architecture of powerline communication (PLC) devices”, Kioskea White Paper at www.en.kioskea.net, 2003.

[26] Suvak, D., “IrDA and Bluetooth: a complementary comparison”, Extended Systems Inc. White Paper, 2000.

[27] Bluetooth SIG. “Bluetooth Special Interest Group launches Bluetooth Core Specification Version 2.0 + Enhanced Data Rate”, Press release, 2004.

[28] Wi-Fi Alliance, Wi-Fi Alliance: White Papers, www.wi-fi.org, 2009.

[29] Hydra Project Consortium, “D.2.2. Initial Technology Watch Report - Wireless networks and devices”, 2007.

[30] Shahriyar, R., Hoque, E., Sohan, S.M., Naim, I., Akbar, M. and Khan, M.K., “Remote controlling of home appliances using mobile telephony”, International Journal of Smart Home, Vol. 2, No. 3, 2008.

[31] Aiello, M. and Dustdar, S., “Are our homes ready for services? A domotic infrastructure based on the Web service stack”, Pervasive and Mobile Computing, 2008.

http://www.usb.org/developers/docs�

http://www.1394ta.org/�

http://www.en.kioskea.net/�

http://www.wi-fi.org/�


[32] X10.com, X10 Active Home Controller Pro Resources, www.x10.com

[33] HomePlug Powerline Alliance, “HomePlug AV White paper”, 2005.

[34] Konnex Association, “KNX – World’s first standard for home & building control – EN50090”, Konnex Journal, 2004.

[35] Darbee, P., “INSTEON the details”, Smarthome Technology, pp. 1–64, 2005. http://www.insteon.net/pdf/insteondetails.pdf.

[36] ECHELON Corp. “Building automation technology review”, White paper, 2003.

[37] ZigBee Alliance, “ZigBee Technology: Wireless Control that Simply Works”, Zigbee White Papers, 2003, www.zigbee.org.

[38] Z-Wave Alliance, “Z-Wave - Wireless Home Control Solutions for Your Lighting, Security, Home Entertainment, Energy Management”, Z-Wave White Papers, 2006, www.z-wave.com

[39] Center for Short Range Wireless Technologies, “EnOcean - Dolphin Platform”, White Paper, 2009.

[40] Sun Microsystems Inc. “Jini™ Technology and Emerging Network Technologies”, White Paper, 1999.

[41] CEA-852-A, Tunneling Component Network Protocols over Internet Protocol Channels, 2009, www.ce.org.

[42] UPnP Forum, “UPnP Specifications Named International Standard for Device Interoperability for IP-based Network Devices”, UPnP Forum White Papers, 2009, www.upnp.org.

[43] Hydra Project Consortium, “Networked Embedded System middleware for Heterogeneous physical devices in a distributed architecture – D.2.2. Initial Technology Watch Report”, 2007.

[44] ISO/IEC 24752, Information technology - User interfaces - Universal remote console. International Organization for Standardization (ISO), 2009. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=42309

[45] OSGi Service Platform, Mobile Specification, Release 4, Version 4, OSGi Alliance., 2007.

[46] openAAL, open source semantic middleware web: http://openaal.org

http://www.x10.com/�

http://www.insteon.net/pdf/insteondetails.pdf�

http://www.zigbee.org/imwp/idms/popups/pop_download.asp?contentID=5162�

http://www.zigbee.org/�

http://www.z-wave.com/�

http://www.ce.org/�

http://www.upnp.org/news/documents/UPnPForum_02052009.pdf�

http://www.upnp.org/news/documents/UPnPForum_02052009.pdf�

http://www.upnp.org/�

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=42309�

http://openaal.org/�


[47] MORE Project Consortium, “Network-centric Middleware for group communications and resource sharing across heterogeneous embedded systems - booklet”, 2006.

[48] Mikalsen, M. and Walderhaug, S., “Empowering the Elderly and the Cognitively Disabled”, IEEE

Pervasive Magazine, Vol. 6, No. 1, 2007. [49] Ben Mokhtar, S., Raverdy P.-G., Urbieta, A. & Speicys Cardoso, R., “Interoperable Semantic and

Syntactic Service Matching for Ambient Computing Environments”, Proceedings of the 1st International Workshop on Ad-hoc Ambient Computing, France, 2008.

[50] Ramos, C., Augusto, J.C. and Shapiro, D., “Ambient intelligence – the next step for artificial intelligence”, IEEE Intelligent Systems, Vol. 23, No 2, 2008.

[51] Lustrek, M. and Kaluza, B., “Fall detection and activity recognition with machine learning”, INFORMATICA, 33(2), 2009.

[52] EMERGE: Emergency Monitoring and Prevention, EC Project (STREP), www.emerge-project.eu

[53] Prueckner, S., Madler, C., Beyer, D., Berger, M., Kleinberger, T. and Becker, T., “Emergency monitoring and prevention – EU Project EMERGE”, Proc. Ambient Assisted Living, 2008.

[54] Carneiro, D., Costa, R., Novais, P., Machado, J. and Neves, J., “Simulating and monitoring ambient assisted living”, Proc. ESM, 2008.

[55] Wang, L., Gu, T., Tao, X. and Lu, J., “Sensor-based human activity recognition in a multi-user scenario”, Tscheligi, M. et al. (Eds.): AmI 2009, LNCS 5859, 2009.

[56] Pham, C. and Olivier, P., “Slice&Dice: recognizing food preparation activities using embedded accelerometers”, Tscheligi, M. et al. (Eds.): AmI 2009, LNCS 5859, 2009.

[57] Morganti, F. and Riva. G., “Ambient intelligence for rehabilitation”, In: G. Riva, F. Vatalaro, F. Davide, M. Alcañiz (Eds.) IOS Press, 2005.

[58] AMIGO project website: http://www.hitech-projects.com/euprojects/amigo/amigo.htm

[59] EasyLine+ project website and contents: http://www.easylineplus.com/

[60] INREDIS project website and contents: http://www.inredis.es/

[61] SM4ALL project website and contents: http://www.sm4all-project.eu/

[62] i2home project website and contents: http://www.i2home.org

http://www.emerge-project.eu/�

http://www.hitech-projects.com/euprojects/amigo/amigo.htm�

http://www.easylineplus.com/�

http://www.inredis.es/�

http://www.sm4all-project.eu/�

http://www.i2home.org/�


[63] Vital project website and contents: http://www.ist-vital.org/

[64] Bedmond project DoW and available documentation: http://www.bedmond.eu/

[65] i2life project Description of Work and available documentation.

[66] MonAmi project website and available contents: http://www.monami.info/

[67] ASK-IT project website and available contents: http://www.ask-it.org

[68] SOPRANO project website and available contents: http://www.soprano-ip.org

[69] universAAL project website: http://www.universaal.org

http://www.ist-vital.org/�

http://www.bedmond.eu/�

http://www.monami.info/�

http://www.ask-it.org/�

http://www.soprano-ip.org/�

http://www.universaal.org/�


99 LLiisstt ooff KKeeyy WWoorrddss//AAbbbbrreevviiaattiioonnss AAL Ambient Assisted Living

AAR Association of American Railroads

ADL Activities of Daily Living

AI Artificial Intelligence

AmI Ambient Intelligence

ANSI American National Standards Institute

ASHRAE American Society of Heating, Refrigerating and Air-Conditioning

ATSC Advanced Television Systems Committee

BCI Brain-Computer Interface

BNCI Brain/Neural-Computer Interface

CECED Conseil Europeen de la Construction Electrodomestique

CHMM Coupled Hidden Markov Model

CSMA/CD Carrier Sense Multiple Access / Collision Detection

DCP Device Control Protocols

DVB-S/T Digital Video Broadcasting - Satellite/Terrestrial

DVD Digital Versatile Disc

DVR Digital Video Recorder

ECG Electrocardiogram

EDA Event-Driven Architecture

EDGE Enhanced Data Rates for GSM Evolution

EEG Electroencephalogram

EIB European Installation Bus

EMG Electromyogram

EPC Electronic Product Code

EPG Electronic Program Guide

FP6 Sixth Framework Program of the European Union

FPS Frames per Second

FTP File Transfer Protocol

GPRS General Packet Radio Service

GPS Global Positioning System

GSM Global System for Mobile

HCI Human-Computer Interface

HMM Hidden Markov Model


HTML Hypertext Markup Language

HTTP Hypertext Transfer Protocol

HVAC Heating, Ventilating and Air-Conditioning

IAQ Indoor Air Quality

ICT Information and Communication Technologies

ID Identification

IEEE Institute of Electrical and Electronic Engineers

IFSF International Forecourt Standards Forum

INELS Intelligent Electro-Installation

IP Internet Protocol

IR Infrared

IrDA Infrared Data Association

ISO International Organization for Standardization

IST Information society Technologies

KNX Konnex Standard

LAN Local Area Network

LED Light-Emitting Diode

LNS LonWorks Network Services

LR-WPAN Low-Rate – Personal Area Network

MAC Media Access Control

MCI Mild Cognitive Impairment

MCE Media Centre for Windows

MCML Media Center Markup Language

OPC Object Linking and Embedding for Process Control

OSD On-Screen Display

OSGi Open Services Gateway Initiative

OSI Open System Interconnection

P2P Peer-to-Peer

PAN Personal Area Network

PC Personal Computer

PDA Personal Digital assistant

PLC Power Line Communications

PoE Power over Ethernet

RDS Radio Data System

RF Radio Frequency


RFID Radio Frequency Identification

RMI Java Remote Method Invocation

SEMI Semiconductor Equipment and Materials International

SMS Short Message Services

SNMP Simple Network Management Protocol

SOA Service Oriented Architecture

SOAP Simple Object Access Protocol

TCP Transmission Control Protocol

TTL Transistor-Transistor Logic

UCH Universal Control Hub

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunication System

UPnP Universal Plug and Play

URC Universal Remote Console

USB Universal Serial Bus

UV Ultraviolet

VOC Volatile Organic Compound

W3C World Wide Web Consortium

WLAN Wireless LAN

WMC Windows Media Center

XML Extensible Markup Language

Documents

D.5.1: Ambient Intelligence in Assistive Technologies...G.A.247447 Collaborative Project of the 7th Framework Programme Work Package 5 AmI and Social Network Services D.5.1: Ambient