Middleware for Context-aw are Ubiquitous Computing (In preparation for Distributed System Course)

1/30 Kyunghee University

Middleware for Context-aware Ubiquitous Computing

(In preparation for Distributed System Course)

Hung Q. NgoUbiquitous Computing Group (UCG)

Real-time and Multimedia LABKyung Hee University, Korea

Sept., 2004


Sequence of presentation

• Introduction

• Middleware for Context-Awareness

• CAMUS Architecture

• Feature Extraction Agent

• CAMUS Context Model

• Reasoning Mechanisms

• System Prototyping

• Conclusion and Future Work


Introduction

• Ubiquitous/Pervasive Computing – Calm technology: embedded, invisible, seamlessly,

unobtrusive, intelligent.

Image source:

Friedemann Mattern

(ETH Zürich)


Introduction (cont.)

• Context-awareness– An important aspect of the intelligent pervasive

computing systems – Systems that can anticipate users’ needs and act in

advance by “understanding” their context.



• Context– The specific conditions, external to the application itself,

such as audience, speaker (user), situation (place and its surroundings), time, environmental and network conditions, etc., which determine the application behavior, will be called the ‘context’ of the application.

– “Out of Context: Computer Systems That Adapt to, and Learn from, Context,” H. Lieberman, T. Selker, MIT

– “A Survey of Context-Aware Mobile Computing Research,” by G. Chen, D. Kotz, Dartmouth College

– “Context-Aware Applications Survey,” M. Korkea-aho, Helsinki University of Technology



Context – in our viewpoint:– the situational conditions that are associated with a user

location, surrounding conditions (light, temperature, humidity, noise level, etc), social activities, user intentions, personal information, etc.



• Sensing is a key enabling technology – Heterogeneity– e.g. location-sensing techniques: triangulation, proximity,

scene analysis • (Jeffrey Hightower, “Location Systems for Ubiquitous

Computing”, IEEE Computer August 2001.)

In the environment

Wearable On devices



• Shortcomings of the previous systems

– Context acquisition and use was often tightly integrated into a single application, and could not easily be incorporated into other applications.

– Individual agents are responsible for managing their own context knowledge



• Shortcomings of the previous systems

– Lacking an adequate representation for context modeling and reasoning

• Existing solutions for Context information are – Name-value pairs or entity relation model – Objects to represent context with methods and fields for

retrieval of information– Simple matching mechanisms for context access, and

developers must perform low-level programming for reasoning

– Users often have no control over the information that is acquired by the sensors

• Privacy Concerns


Middleware for Context-Awareness

• Desired Characteristics:

– Support for heterogeneous and distributed sensing agents

• Make it easy to incrementally deploy new sensors and context-aware services in the environment

– Provide different kinds of context classification mechanisms

• Different mechanisms have different power, expressiveness and decidability properties

• Rules written in different types of logic (first order logic, description logic, temporal/spatial logic, fuzzy logic, etc.)

• Machine-learning mechanisms (supervised/unsupervised classifiers)


Middleware for Context-Awareness

• Desired Characteristics (cont.)

– Follow a formal context model using ontology

• To enable syntactic and semantic interoperability, and

knowledge sharing between different domains

– Facilitate for applications to specify different behaviors in different contexts easily, as well as privacy policy and security mechanism

• Graphical development tool to ease developers in writing code.


CAMUS Core Architecture

CAMUS----------------------Context-AwareMiddleware forUbicompSystems


CAMUS Details

• Feature Extraction Agents

– These sensing agents extract the most descriptive features for deducing contexts in upper layers

– Feature - Context Mapping: performs the mapping required to convert a given feature into elementary context based on the meta-information saved in the ontology repository

– Self-configuring components:• Access wrappers• Unification interface for acquiring contexts from sensors

and delivering to consumers.


Sensor signalProcessing

Feature Extraction

FeatureSegmentation

FeatureEncapsulation

ContextRecognition

Table 1. Meta-data for feature encapsulation

Attribute Meaning

Sensor_IDThe unique ID of the sensor board, assigned at development stage & mapped to corresponding location/devices at deployment stage

Type_IDSensor type e.g. audio, temperature etc., for further distinction of context information sources, especially in case multiple sensor types on a single board.

FeatureIDRefers to feature category of each sensor type. It is actually the signature of corresponding feature extraction function in the library of the sensing device.

Feature Value

Symbolic (e.g. light intensity) or absolute numerical value (e.g. temperature absolute value) of the extracted and segmented sensor feature.

Probability The uncertainty or confidence of the context information, could be 0/1 if the feature is segmented using thresholds, or within [0, 1] range if fuzzy sets are applied. This attribute is used to decide which feature values will be sent to backend system (probability > 0) and useful for probability based context recognition mechanisms e.g. Bayesian networks.

Timestamp The absolute time when the feature is extracted. Used to keep consistency of context information, and sometimes useful for temporal reasoning mechanisms.

Feature Extraction Agent (cont.)


Table 2. Examples of features and their semantic meaning. Sensor_IDs {3,7} are mapped to “Bedroom”

Sensor ID

Sensor Type Feature ID

Value

TimeStamp

Numerical Value

Segmented Value (Symbolic, Probability)

31

(Audio)

1(Intensity)

x (dB) 1 (Silent, 0.9)

xxxxxx

2 (Moderate, 0.1)

3 (Loud, 0)

2(ACDCRatio)

y 1 (Low, 0)

2 (Medium, 0.7)

3 (High, 0.3)

7

4(Temperature)

1(AbsValue)

95 (oF)NA

xxxxxx5

(Humidity)1

(Humidity)

z (%) 1 (Dry, 1)

2 (Normal, 0)

3 (Humid, 0)

FT1= {3, 1, 1, 1, 0.9, xxxxx}{Location.Bedroom, Environment.Sound.Intensity = Silent, Probability = 0.9, TimeStamp = xxxxx}

Feature Extraction Agent (cont.)

FT= {Sesnor_ID, Type_ID, Feature_ID, Feature_Value, Probability, Timestamp}


Formal Context Modeling

– In order to have common understanding among HW/SW entities, they must have invariant meanings, i.e. the

context semantics must be formalized

– Advantages• Storing context for long-term• Communicating context universally with other systems• Leads to its testability of being a formalized knowledge

– Result• A growing pool of well-tested context knowledge available

to different context aware systems


Formal Context Modeling (cont.)

• Formal Context Modeling using OWL

– Context entities are concepts in a domain under investigation

– Ontologies are used for above purpose, defined as• Specification of a conceptualization of a domain• Domain contains the vocabularies of concepts and

relationships among them• @syntactic level

– They are XML documents

• @structure level– They consist of a set of triples

• @semantic level– They constitute one or more graphs with partially pre-defined

semantics


CAMUS Context ModelBasic Model

ActivityEvent

LocationContextObject

Event

Place

Activity

+ contel-device.owl

+ contel-time.owl

+ contel-environment.owl+ contel-location.owl+ contel-sweet-space.owl

+ contel-agent.owl+ contel-activity.owl

Agent

Location

Time

Device

Environment


Context Model in detailsAgent ontology

- Agent + Person + Group + Organization + SoftwareAgent

- Profile + PersonProfile + GroupProfile + OrganizationProfile + SoftwareProfile(= SoftwareProfile in Deviceontology)

hasProfile

isProfileOf

- Activity + SingleActivity + GroupActivity + OrganizationActivity + SoftwareActivity(= ApplicationFunction)

+ SeftActivity + ActivityOnObject

+ InstantActivity + IntervalActivity

+ PastActivity + CurrentActivity + IntentionalActivity

+ IndoorActivity + OutdoorActivity

isActorOfhasActor

- ActivityEvent + IndoorActivityEvent + OutdoorActivityEvent

generatesEvent generatedBy

LocationContextObject(from Location ontology)

+ Event * InstantEven * IntervalEvent(from Time ontology)


Context Model in details Device ontology

• Based on FIPA device ontology specification: knowledge reuse

- Device + MemoryDevice + DisplayDevice + AudioDevice + NetworkDevice + PersonalDevice

- DeviceHWProfile + AudioProfile + DisplayScreenProfile + MemoryProfile + NetworkProfile * NetworkParameter * NetworkProtocol

hasHWProfile

- Service

hasService

- SoftwareProfile + ApplicationProfile + OSProfile

hasSWProfile

LocationContextObject(from Location ontology)

- Agent(from Agent ontology)

+ Software * Application * OS

isOwner

hasOwner


Reasoning Mechanisms

• The contextual information provided by the environment leads to only elementary contexts

• Some contexts are useful only when they are combination of some elementary and/or composite contexts, and also need consistency of contextual information


Reasoning Mechanisms (cont.)

• Our framework supports various pluggable reasoning modules and developer of the context Aggregator services can exploit any kind of reasoning mechanism based on application requirements

• These reasoning modules are broadly classified into ontology & context reasoning mechanisms



• Inference services in DL (Description Logic)

Terminological Reasoning– Subsumption check - checks if one concept is a

sub-concept of another– Consistency check - checks for (in)consistency of

concept definitions– Taxonomy construction - explicit concept

hierarchy– Classification - determines the concepts that

immediate subsume or are subsumed by a given concept


• Inference services in DL (Description Logic)

Instance Reasoning– Consistency check - existence of a model of А– Realisation - given a partial description of an

instance, finds the most specific concepts that describe it

– Individual retrieval - finds all instances that are described by a given concept



Examples for Ontology Reasoning


Examples for Ontology Reasoning



*An example of Context

Reasoning:

Using Bayesian Net to deduce User Activity


System Prototyping

• A Smart Home has been set up for system prototyping.

– RFID tags, Door Sensors, Bluetooth and WLAN access points are used for location and identity detection

– Audio/Video based activity recognition


System Prototyping

• A Smart Home has been set up for system prototyping.

– Jini is used as underlay communication and discovery mechanisms

– Jena2 Semantic Web Toolkit is used to implement the Context Repository, Context Reasoner, and Context Query Engine.

– OSGi Gateway for interconnection with other systems


Our Research Contributions

• A Middleware Framework for building Context-Aware applications

– Unified Sensing Framework for Context acquisition from disparate sensors

• Abstractions to relieve the developers of the burden of low level interaction with various hardware devices

– Formal Context Modeling in terms of ontologies using OWL

• To enable syntactic and semantic interoperability, and knowledge sharing between different domains

– Ontology and Context Reasoning Mechanisms• Pluggable Modules


Conclusion and Future work

• We believe that formalizing domains should be seen as emergent phenomenon constructed incrementally, leading to the sharing of contextual information among heterogeneous context-aware systems

• API for application developers to access system’s functionalities while hiding the complexity of underlying context processing

(gathering contexts from sources, managing contexts in knowledge base, handling application queries, and reasoning about contexts based on rules)

• Privacy policy and security mechanism


Selected References• [1] M. Weiser, “The Computer for the 21st Century”, Scientific America (Sept.

1991) 94-104; reprinted in IEEE Pervasive Computing. (Jan.-Mar. 2002) 19-25• [2] M. Satyanarayanan, “Pervasive Computing: Vision and Challenges”, IEEE

Personal Communications (Aug. 2001) 10-17• [3] Dey, A.K., et al., “A Conceptual Framework and a Toolkit for Supporting the

Rapid Prototyping of Context-Aware Applications”, Anchor article of a special issue on Context-Aware Computing, Human-Computer Interaction (HCI) Journal, Vol. 16. (2001)

• [4] S. Jang, W. Woo, “Ubi-UCAM: A Unified Context-Aware Application Model”, Context 2003, Stanford, CA, USA. (Jun. 2003)

• [5] J. Hong, “The Context Fabric”, http://guir.berkeley.edu/projects/confab/• [6] Kumar, M.; Shirazi, B.A.; Das, S.K.; Sung, B.Y.; Levine, D.; Singhal, M,

“PICO: a middleware framework for pervasive computing”, IEEE Pervasive Computing, Vol. 2 Issue 3. (July – Sept, 2003) 72- 79

• [7] Chen Harry, Tim Finin, and Anupam Joshi, “An Intelligent Broker for Context-Aware Systems”, Ubicomp 2003, Seattle, Washington. (Oct. 2003)

• [8] Hung Q. Ngo, Anjum Shehzad, S.Y.Lee, “Developing Context-Aware Ubiquitous Computing Systems with a Unified Middleware Framework”, accepted for publication, The 2004 International Conference on Embedded and Ubiquitous Computing (EUC04), Aizu-Wakamatsu City, Japan, 25-27 August, 2004.

• [9] Anjum Shehzad, Hung Q. Ngo, S.Y. Lee, “Formal Modeling in Context Aware Systems”, KI-Workshop Modeling and Retrieval of Context (MRC2004), German.


• For further discussion:

Hung Q. NgoUbiquitous Computing Group

RT&MM LAB

Kyung Hee Univ. KOREA

Email: [email protected]

www: http://ucg.khu.ac.kr/~nqhung

Documents

Middleware for Context-aw are Ubiquitous Computing (In preparation for Distributed System Course)