Middleware for Wireless Systems - unibo.it · 2012-09-03 · Ubiquitous Computing Middleware 1 Middleware for Wireless Systems Samuele Pasini ([email protected]) Università

Ubiquitous Computing Middleware 1

Middleware for Wireless Systems

Samuele Pasini ([email protected])

Università degli Studi di Bologna Facoltà di Ingegneria

Ubiquitous Computing Middleware:Design and Implementation Principles


Scenario

Many devices and applications

Content and service provisioninganywhereat anytimeon any device


Ubiquitous news: an example

audio

VoIP(phone call)

conversation

text + graphics

Web(HTML pages)

request

response+

text

GSM network(SMS messages)

subscribepublish +

video +audio +

text

DVB-T channel(video carousel)

broadcast

localfilter

+

VoIP(phone call)

conversation

Web(HTML pages)

request

response+

subscribepublish +


Ubiquitous news: an example

text + graphics

text audiovideo +audio +

text

E-mail(PDF file)

GSM network(SMS messages)

DVB-T channel(video carousel)

broadcast

localfilter

+

Web server(Mp3 file)

request

response+

subscribepublish +

request

response+


Challenges

Tailor

services and contents to device characteristics, network conditions and user needs

Permit multiple activities concurrently, monitor and manage them

Minimize technology intrusion into everyday's life.


Goals

Multimodal interfacesinput modalities (mouse clicks, text messages, sensor measurements, dialed tones, voice, …)output modalities (text, graphics, audio, video, …)

Multipattern

interactionsemantics (request/response, publish/subscribe, broadcast,…)implementation (synchronous/asynchronous, connection-less/connection-oriented, message correlation, …)

Multichannel accesscommunication link (IP networks, PSTN networks, DVB-T channels, GPS signals, FM radio, RFID tags, …)exchanged data (HTTP requests, UDP datagrams, video carousel, human voice)


Where to begin

UbiComp

applications are difficult to design and prototypehardware heterogeneitysoftware complexitylack of established techniques

They typically stress many currently debated research fields, variously interconnected and sometimes intermixed.


Content adaptation

Adapt heterogeneous information to heterogeneous devices and users•

type (text, graphics, audio, video, multimedia, …)

•

format (image resolution, audio bitrate, text language, compression codecs, …)

•

customizations (aggregation, enrichment, profiling, …)•

accessibility (colors, contrast ratio, vocal synthesis, …)


Resource composition

Achieve UbiComp

goals by means of structured activities that leverage third- party provided functionalities

•

distributed applications•

software reuse and maintainability

•

business process management


Session management

Let execution span across time and different hardware/software components•

user mobility (across different devices)

•

terminal mobility (across different networks)•

service mobility (across different locations and replicas)


Context-awareness

Acquire and utilize information about the physical and computational environment to provide services that are appropriate to the particular people, place, time, and event•

suitable contents and interfaces

•

value-added services

Nearest station: head southwest

on...


State-of-the-art

Frameworks

and mechanisms

to help maintaining session information (e.g., Web technologies such as Cookie, Servlet container, SEAM, …)

Toolkits

to develop context-aware applications (e.g., Context Toolkit) and models

to define what context is, how to represent it, and how to use it

Guidelines

to develop multimodal interfaces and multichannel access mechanisms (e.g., W3C MultiModal Interaction Framework)

Standards

and tools

to adapt contents (e.g., VoiceXML, MPEG-7) and ad hoc solutions in real applications (e.g., OperaMini)

Languages

and tools

to define business process out of independent software components (BPEL, jBPM, Spring WebFlow) and the semantics of the latter ones (e.g., OWL-S)

Web mashup

platforms

to create value-added services out of existing contents and services available on the Web (e.g., Yahoo Pipes,

iGoogle, Kapow Mashup Server)

…and lots more of vertical and separate approaches, often from a technological-dependent perspective, difficult to exploit together


MiddlewareTo integrate existing

achievements, we need an intermediate software layer

decouple functionalities that have to cooperateabstract heterogeneityovercome distributionintegrate third-party resourcesdefine business processesmonitor activities and react to eventsprovide common facilities


Middleware examplesDomain-specific middleware services: focus on vertical aspects by

providing implementations of domain-specific functionalities (e.g., IBM WebSphere

Voice Application Access)

Common middleware services: manage general aspects of distributed applications –

such as transactions, security, resource pooling, caching

or scheduling –

by means of reusable software components (e.g., J2EE JPA, JPA JMS, CORBAServices, .NET Services)

Distribution middleware: defines high-level programming model and abstract differences of heterogeneous software components (e.g.,

Java

RMI, CORBA, DCOM, SOAP)

Host infrastructure middleware: executes on top of existing operating systems and hardware resources to access underlying functionalities by means of a uniform API (e.g., JVM)


Ubiquitous Computing Middleware

More-specific middleware services usually build on top of more-general middleware layers

e.g., IBM..Voice

Access → J2EE → JavaRMI

→ JVM

All middleware layers are needed to support Ubiquitous Computing applications

Anyway, this leads to a plethora of domain-specific middleware platforms: general purpose solutions tend to become

too complex to adopt in the real world


Towards a comprehensive solution

Modularity. Suitable separation of responsibilities

Encapsulation. No assumptions about the implementation of components, which must be treated as black boxes

Recursiveness. Allow for nesting, so that a complex functionality – provided by leveraging several components within the Ubiquitous

Computing infrastructure –

can appear as a single and autonomous facility to other software components

Extensibility. Facilitate the integration of components that support new interaction modalities, without modifying the existing deployments


“MAGIC” middleware disappears

Furthermore, middleware intervention should be transparent

users dislike tedious configuration procedures

programmers and software architects wish to avoid complex integration issues, in addition to their own business logic

it must be possible to also integrate legacy “stuff”, unaware of both middleware existence and the overall Ubiquitous Computing scenario


A multimodal middleware architecture

W3C Multimodal Interaction Framework

Input information can be collected at

client/network/server side to demand processing

Data undergo analysis and evaluation by means of

dedicated interaction managers

Manager deals with enacting

the required business

logic

Session and

context information are relevant

to adapt contents / services

Results are delivered to their final destination


Interaction managers

Receive data fromautonomous applications running on client devicesremote services and server applicationssensorsany kind of “user interface”

Deal withidentification / authenticationdata normalizationenacting application functionssupporting the desired pattern of interaction

?


Application Functions, Session and Environment

Altogether, realize

the actual

Ubiquitous

Computing logic

process incoming information

keep track of ongoing activities

exploit context information to perform operations

produce results

Permit

to

leverage/integrate also

non-middleware

resources

and functionalities

{user position}

{update user status}

{nearby POI}

{send SMS}

+ + +


Content operations – Don’ts

As long as a middleware

provides too much specific “facilities”

to operate on contents, it also becomes

“less easy” to learn

“more difficult” to adopt in general purpose scenarios

As well, non-middleware components that demand middleware operations to perform, become

harder to develop (API learning curve)

harder to test (mock-up of middleware features)

harder to reuse (middleware dependencies)


Content operations – Do’s

To disappear in the background, middleware must define the overall integration framework characteristics, but…

push content-related logic outside its core layer

only keep management and coordination responsibilities

It is thus a best practice to delegate content-related logic to third-party provided functionalities, by leveraging middleware to compose them together and orchestrate their execution


Service-Oriented Architecture

“Services” can realize all kind of content-related functionalities, whereas middleware plays a role of coordinator and provides limited integration features to only overcome resource distribution and heterogeneity

Already a robust trend in the Enterprise Application Integration

(EAI) and Business-toBusinss

(B2B) world, also gaining momentum to provide

value-added applications to final users:

services are “self describing, open components that support rapid, low-cost composition of distributed applications”, providing “a distributed computing infrastructure for both intra- and cross-enterprise integration and collaboration”

Service-Oriented Architecture (SOA) provides means to model complex application scenarios


SOA – Technologies

SOA

is not a single technology or system, but a methodology to design a group of systems. Most often, Web-Services related technologies are employed, since they provide the most flexibility

Services are exposed via Web-based technologiesWeb Services Interoperability (WS-I) Organization defines standards that ensure interoperability regardless of underlying operating systems and programming language (e.g., WS-Security and WS-Transaction specification)WSDL documents define service characteristics (name, arguments, binding, …)UDDI registries store information about service to enable discovery (if needed)Protocols such as SOAP or XML-RPC define exactly how to pass messages (typically over HTTP) and encode/decode data

Other standards exist, that are not part of the WS family (e.g.,

Jini/Apache River, WCF, EJB, CORBA, XML-RPC w/o Web Services, …)

typically more efficient vendor-specific features and implementations (lock-in)


Business Process Management

We can organize services into “workflows” that model the business processes of Ubiquitous Computing scenarios

Services should not directly interact with each other; rather, their operations should be organized into manageable business processes..

…where activities are structured in form of workflows, made up of both control and business logic…where services provide the actual implementation of business logic

Synthe-sizer

XMLparser

Podcastcreator

Webserver

SMSgateway

VoIPserverplain text phone call

resource publishing

SMS sending

abort


Business Process Management

We can organize services into “workflows” that model the business processes of Ubiquitous Computing scenarios

Middleware supports business process execution by invoking services, according to workflow definitions

reliable data transportation, monitoring, QoS, exception handling, failover mechanisms, …“passivation” and resumption of long lasting processes

…and specifically to UbiComp scenarios…

execution upon synchronous and asynchronous eventsavailability of session and context information, managed by the middleware, throughout all business processesbuild-time (and possibly run-time) modifications of processes, according to changed conditions of users and services


Experience: project SWIMM

From the Italian: “Servizi

Web Interattivi

e Multimodali per la Mobilità”

Design and prototype development of a software platformto provide customized and differentiated services,

on the basis of current user preferences, communication channel, and exploited device, as well as session and context information

Some informationProject was funded by the Emilia Romagna ICT plan (PRRIITT)Project is part of the Emilia Romagna technological network (ASTER)Project involved collaborations with Public Administrations and several enterprises


Actual use cases

SWIMM prototype has been engaged inaggregating and dynamically adapting contents of RSS feeds and HTML pages, from several sources of information (such as on-line newspapers, personal blogs, legacy Web sites), for use on heterogeneous device types (e.g., handheld PCs, notebooks, smartphones, tradizional cell phones, mp3 players, digital-TV set-top-boxes) in different formats (e.g., e-mail messages, SMS text messages, podcast channels, MhP applications, VoIP phone calls)

determining user geographical position and leveraging it to dynamically provide location-driven services and contents (e.g.,news about train delays and traffic jams) by means of SMS text messages and customized Web pages

aggregating customized, adapted and location-aware weather forecasts, “provided” by legacy information sources and in heterogeneous formats, by means of podcast channels and customized Web pages

selecting and adapting tourist guide information on the basis of current user coordinates, by means of podcast channels and customized Web pages

delivering multimedia contents via Bluetooth, on the basis of user proximity to different areas of a museum (and similar use cases)


Prototype architectureLegacy resources are allowed via proxies

homogeneous lifecycle management

read/write access to managed session and context information

Scenario requirements drive composition of resources into workflows

static resource metadatadynamic context conditions

Each Interaction Module models a specific pattern of interaction

input normalizationworkflow(s) execution


Implementation details

Based on the J2EE frameworkdistributed components (EJB3)messaging facilities (JMS) naming service (JNDI)persistence layer (JPA)transactional environment (JTA)

Running on JBoss Application Serverextensible architecture (JMX-based)

Leveraging additional JBoss

facilitiesjBPM business process definition enginejPDL process definition language

Leveraging additional SpringFramework featuresInversion of Control (IoC) techniques (dependency injection)Aspect-Oriented Programming (AOP) (cross-cutting concerns in workflows)


Achievements and innovationsAutomatic resource composition

template-based approachmathematical constraints on resource metadata and context conditionsautomatic and proactive calculation, as soon as changes happen in context and requirements

Per-process configuration proxies permit different configurations of a single resource in different workflowsdynamic reconfiguration of resources (when recalculating compositions that have become invalid)support for replication (equivalent proxies) and fault-tolerance (re-enact proxy configurations of failed nodes)

Context information distributionproducers and consumers of context information that belong to the same process access the same copya single resource can output copies of the same context information in every process for which its proxy provides a configuration

Session information scopingsession information is handled on a per-resource basissupport for execution in different processes via incremental visibility levels(resource lifetime, proxy instance, proxy configuration)


Multimodal interaction supportSeparation between

supporting different request formatssupporting different patterns of interaction

↓Request Normalization Algorithms (RNA)

translate raw request data to middleware commands (who, what, when)each proxy of a request-generating resource must register the RNA that corresponds to its request “Syntax”

Interaction Modules (IM) evaluate request content through the corresponding RNAenact the corresponding middleware commandsprovide support for specific patterns of interaction (and corresponding feasible transportation mechanisms)

es: conversational → correlate messages and maintain statuses: accept data via SOAP messages, Java RMI, JMS queues, …

hot-pluggable like all other middleware parts (including Orchestration Engine and Workflow definitions)


Multimodal interaction support

…resources that exploit the same pattern of interaction can leverage the same IM, though they convey data in different formats…

Request-only Interaction Module

(1) notification

RSS feed

ResourceWeb site

RSS document

Proxy:RSS poller

Request-response Interaction Module

(1) request

URL

Resourceclient-sideapplication

Web browser

Proxy (2) response

HTML

raw data source URLRSS feed

business process ID

syntax RSS normalization algorithm

raw data HTTP request

syntax HTTP normalization algorithm


Request/response interaction on traditional web browsers

WebSiteProxy

(es: HTTP filter)

PULL IM

(request/responsepattern)

request = http://site.com/mynews.jsp?matter=politics&JSESSIONID=3224234

syntax = HTTPNEWS User XYZ wants to run WF #3123123 with args: [politics]

WorkflowRSS

readerRSS

reader...

RSS aggr

RSS toHTMLContainer level

(RMI, caching, clustering) Intercommunication level

TransactionsNaming Security Persistency Messaging

Registry level MappingDomainRegistry

Template Registry

ExpressionDomainRegistry

Resource Registry

SyntaxRegistry

Engine level

OrchestrationEngine

Composition Engine

ReificationEngine

Normalization Engine


PUB/SUBIM

Request-only interaction via SMS

G-SMSProxy

(es: ad hoc gateway

application)

request = from +39328977… NEWS 18:00 POLITICS MAIL ON

syntax = SMSNEWS User XYZ wants to run WF #3123124 with args: [on,politics,18:00,mail]

Workflow...

timer setup

18:00,on

Label 7d9a8s7

politics,

mail, XYZ

subscmngr

Container level




Template Registry


Resource Registry

SyntaxRegistry

Engine level

OrchestrationEngine

Composition Engine

ReificationEngine


1-WAY IM

(request-onlypattern)


Publish/subscribe interaction: SMS

SystemClockProxy

1-WAY IM


request = label 7d9a8s7

syntax = TIMEREVENTUser XYZ wants to run WF #3123125 with args: [politics]

WorkflowRSS

readerRSS

reader...

RSS aggr

RSS totext

SMSgw

Container level




Template Registry


Resource Registry

SyntaxRegistry

Engine level

OrchestrationEngine

Composition Engine

ReificationEngine



Publish/subscribe interaction: e-mail

SystemClockProxy

1-WAY IM


User XYZ wants to run WF #3123125 with args: [politics]

Workflow


syntax = TIMEREVENTUser XWY wants to run WF #3123126 with args: [politics]

RSS reader

RSS reader...

RSS aggr

RSS totext

SMSgw

RSS reader

RSS reader...

RSS aggr

RSS totext

SMTPsrv @

Container level




Template Registry


Resource Registry

SyntaxRegistry

Engine level

OrchestrationEngine

Composition Engine

ReificationEngine



Publish/subscribe: podcast

1-WAY IM


User XYZ wants to run WF #3123125 with args: [politics]

Workflow


syntax = TIMEREVENTUser XWY wants to run WF #3123126 with args: [politics]

RSS reader

RSS reader...

RSS aggr

RSS totext

SMSgw

RSS reader

RSS reader...

RSS aggr

RSS totext

SMTPsrv

...

User SYS wants to run WF #3123126 with args: [politics]

RSS reader

RSS reader

RSS totext

txt

2 speech

podcast

XML

RSS aggr

Container level




Template Registry


Resource Registry

SyntaxRegistry

Engine level

OrchestrationEngine

Composition Engine

ReificationEngine


SystemClockProxy


ConclusionsUbiquitous Computing scenarios are inherently challenging…

several aspects to take into considerationintegration of brand new and legacy assets is required

Middleware adoption can help dominating complexitydecoupling, separation of responsibilitiessupport and integration facilities

Current standards and solutions…tend to have vertical approaches and address specific problems (only useful in limited application domains)otherwise, are either too complex to learn or too general to exploit effectively

A “magic” middleware, instead, should “disappear” in the background while providing support

define framework characteristics and the overall integration mechanismsmanage and orchestrate potentially unaware computational resources (including those that support specific UbiComp issues)enforce crosscutting facilities (e.g., session management, context distribution, service parameter resolution, …) that are common to most UbiCompscenarios, …possibly without requiring additional efforts to developers and architects


ReferencesW. M. P. Van Der

Aalst, A. H. M. Ter

Hofstede, B. Kiepuszewski, A. P. Barros , “Workflow Patterns”, Distributed and Parallel Databases, Volume 14 Issue 1, Pages 5-51, Springer Netherlands, July 2003.

IONA Technologies, “Using Artix

and Service-Oriented Architecture for Multi-Channel Access”, http://www.iona.com/devcenter/artix/articles/0304soa.pdf, February 2008.

Girma

Berhe, Lionel Brunie, Jean-Marc Pierson , “Modeling service-based multimedia content adaptation in pervasive computing ”, Proceedings of the 1st conference on Computing frontiers , CF '04. ACM, April 2004.

Bernstein, P. A. 1996. Middleware: a model for distributed system services. Commun. ACM 39, 2 (Feb. 1996), 86-98.

M. Boari et al., “Middleware for automatic dynamic reconfiguration of context-driven services”, Microprocessors and Microsystems, No. 32, Pages 145-160, September 2007.

M. Comerio, F. De Paoli, S. Grega, C. Batini, C. Di Francesco, A. Di Pasquale, “A service re-

design methodology for multi-channel adaptation”, Proceedings of the 2nd International Conference on Service Oriented Computing, ICSOC 2004, November 2004.

Cheyer, A. and Martin, D. 2001. The Open Agent Architecture. Autonomous Agents and Multi-

Agent Systems 4, 1-2 (Mar. 2001), 143-148.

R. Gorrieri, C Guidi, R. Lucchi, "Reasoning About Interaction Patterns in Choreography", from "Formal Techniques for Computer Systems and Business Processes",

Pages 333-348. Springer Berlin / Heidelberg, 2005.


ReferencesO. Haase

, “Session maintenance ”

from “The Handbook of Mobile Middleware ”, P. Bellavista, A. Corradi, CRC Press, 2007 .

S. Kumar, P. R. Cohen, H. J. Levesque, "The Adaptive Agent Architecture: Achieving Fault-

Tolerance Using Persistent Broker Teams," Multi-Agent Systems, International Conference on, pp. 0159, Fourth International Conference on Multi-Agent Systems (ICMAS'00), 2000.

D. Martin et al., “OWL-S: Semantic Markup for Web Services”, W3C Member Submission, 22 November 2004. Available from: http://www.w3.org/Submission/OWL-S/.

J. M. Martínez, "MPEG-7 Overview (version 10)", Palma de Mallorca, October 2004.Milanovic, N.; Malek, M., “Current solutions for Web service composition ”, Internet

Computing, IEEE, Volume 8, Issue 6, Nov.-Dec. 2004. Page(s): 51 –

59.S. Monti, S. Pasini, A. Corradi, E. Lodolo, M. Boari, “An eXtensible

middleware for Multichannel, Multimodal, and Multipattern

services (X3M)”, 5th International Workshop on Next Generation Networking Middleware (NGNM'08), Samos Island, Greece, September 2008.

“Message Exchange Pattern”, http://www.serviceoriented.org/message_exchange_pattern.html, ©

2006 Momentum s.r.l.Z. Obrenovic, D. Starcevic, “Modeling Multimodal Human-Computer Interaction”, IEEE

Computer, Vol. 37 , No. 9, September 2004.


ReferencesS. L. Oviatt, “Ten myths of Multimodal Interaction”, Communications of the ACM, pp. 74-81,

November 1999.M. P. Papazoglou

and D. Georgakopoulos, “Service Oriented Computing”, Comm. ACM, vol. 46, no. 10, 2003, pp. 25–28.

D. Salber, A.K. Dey

and G.D. Abowd, “The Context Toolkit: Aiding the development of context-enabled applications”, in Proceedings of the SIGCHI conference on Human factors in computing systems, Pittsburgh, Pennsylvania, United States. Pages: 434 -

441, 1999. ISBN:0-201-48559-1.

R. E. Schantz, D. C. Schmidt, "Middleware for Distributed Systems -

Evolving the Common Structure for Network-centric Applications", 2001.

C. Schroth, O. Christ, “Brave New Web: Emerging Design Principles and Technologies as Enablers of a Global SOA”, in Proceedings of the 2007 IEEE International Conference on Services Computing (SCC 2007), Salt Lake City, USA, 2007.

Schroth, C.; Janner, T., “Web 2.0 and SOA: Converging Concepts Enabling the Internet of Services”

, IT Professional, Volume 9, Issue 3, May-June 2007, Page(s):36 –

41. W3 Consortium, “W3C Multimodal Interaction Framework”, W3C Note,

http://www.w3.org/TR/mmi-framework/, May 2003.M. Weiser, “The Computer for the 21st Century”, Scientific American, Sept., 1991, pp. 94-104;

reprinted in IEEE Pervasive.

Documents

Middleware for Wireless Systems - unibo.it · 2012-09-03 · Ubiquitous Computing Middleware 1 Middleware for Wireless Systems Samuele Pasini ([email protected]) Università