Integration of Hand-Integration of Hand-Held Devices into Held Devices into

Collaboration Collaboration EnvironmentsEnvironments

IC’02 Las Vegas, NVIC’02 Las Vegas, NV June 27 2002June 27 2002

Geoffrey Fox, Sung-Hoon Ko, Kangseok Geoffrey Fox, Sung-Hoon Ko, Kangseok Kim, Sangmi Lee, Sangyoon OhKim, Sangmi Lee, Sangyoon Oh

Community Grids Laboratory, Indiana UniversityCommunity Grids Laboratory, Indiana UniversityComputer Science, Indiana UniversityComputer Science, Indiana University

Computer Science, Florida State UniversityComputer Science, Florida State University

ContentsContents

Garnet Collaborative systemGarnet Collaborative system Universal Accessibility in GarnetUniversal Accessibility in Garnet Integration Handheld devices into Integration Handheld devices into

collaboration environmentcollaboration environment Shared Applications in Garnet for Shared Applications in Garnet for

Universal AccessibilityUniversal Accessibility Experimental ResultExperimental Result Conclusions and Future WorksConclusions and Future Works

CoCollaborative Systemsllaborative Systems Collaborative SystemsCollaborative Systems supports supports

coordination and cooperation of two or coordination and cooperation of two or more people who attempt to perform a more people who attempt to perform a task or solve a problem together.task or solve a problem together.

CoCollaboratingllaborating directly, shar directly, sharinging resources.resources.

The collaboration featuresThe collaboration features:: Basic interactive tools - Text Chat, Instant Basic interactive tools - Text Chat, Instant

Messenger, White Board Messenger, White Board Shared resources – Shared Display, Resource Shared resources – Shared Display, Resource

Exporting, Sharing I/O devicesExporting, Sharing I/O devices Real-time Multimedia tools- Audio-video Real-time Multimedia tools- Audio-video

ConferencingConferencing

GarnetGarnet Collaborative SystemCollaborative System Purpose : Support distance Purpose : Support distance Education, TrainingEducation, Training and if and if

possible Computing as Grid(Web) Servicespossible Computing as Grid(Web) Services Integrate Integrate SynchronousSynchronous and and AsynchronousAsynchronous

collaborationcollaboration Support Support universal accessuniversal access including PDA’s including PDA’s

collaboration with desktopscollaboration with desktops Uniform Uniform XML eventXML event (message) based architecture (message) based architecture

All data structuresAll data structures defined in defined in XML Schema GXOSXML Schema GXOS XML for all metadata (Users, documents, computers) and XML for all metadata (Users, documents, computers) and

object changes -- from text chats to display changes etc.object changes -- from text chats to display changes etc. MyXoSMyXoS manipulate GXOS objects manipulate GXOS objects

We build on We build on GMS/JMSGMS/JMS (Java Message Service)(Java Message Service) as as industry standard to implement industry standard to implement publish/subscribepublish/subscribe modelmodel

Support collaborative features : basic interactive Support collaborative features : basic interactive features (textchat, whiteboard, etc.), shared features (textchat, whiteboard, etc.), shared resources (shared display, shared export), AV resources (shared display, shared export), AV conferences.conferences.

GMS (GMS (Garnet Message SGarnet Message Service) Systemervice) System All message publication labels and subscription All message publication labels and subscription

profiles are defined in XMLprofiles are defined in XML

Publishers Subscribers

Message QueueLabeled by (XML)

Topic Object

SubscriberProfile Objects

Specify Query to Event Label

DatabaseSubscribes to all events to getpersistence

Integration Handheld devices Integration Handheld devices into collaboration into collaboration

environmentenvironment Garnet providesGarnet provides universal accessuniversal access including including

PDA’s collaborating with desktopsPDA’s collaborating with desktops GMS is not able to support light-weight clients.GMS is not able to support light-weight clients.

Limited computing power – CPU, memory size, Limited computing power – CPU, memory size, display display

Network Environment – unreliable, unstable Network Environment – unreliable, unstable wireless communication, various network protocolswireless communication, various network protocols

Developing Environment – various operating Developing Environment – various operating systemssystems

New universal collaboration and access New universal collaboration and access architecture is required: GMSME (Garnet architecture is required: GMSME (Garnet Message Service Micro Edition)Message Service Micro Edition)

GMSMEGMSME

Supports heterogeneous clients by Supports heterogeneous clients by customizing message depending on the customizing message depending on the client specifications.client specifications.

User profile and mobile device profile User profile and mobile device profile (screen size, device type) is defined with (screen size, device type) is defined with XML object metadata specification XML object metadata specification (GXOS)(GXOS)

Consists of PDA Adaptor, Hand Held Consists of PDA Adaptor, Hand Held Message Service (HHMS) protocol, and Message Service (HHMS) protocol, and API for application processors.API for application processors.

GMSME – PDA AdaptorGMSME – PDA Adaptor PC PDA Adaptor PC PDA Adaptor

PDA Adaptor looks like a conventional client PDA Adaptor looks like a conventional client to GMS and adapts data to specified to GMS and adapts data to specified client/user specifications client/user specifications

““Listens” to all the messages and events on Listens” to all the messages and events on the Garnet collaboration systems by the Garnet collaboration systems by subscribing to GMS.subscribing to GMS.

PDA Adaptor PDA ClientPDA Adaptor PDA Client Parses the GMS XML message body and Parses the GMS XML message body and

delivers it to the mobile clients over HHMS.delivers it to the mobile clients over HHMS. HHMS is a specially designed optimized HHMS is a specially designed optimized

protocol for light-weight devices.protocol for light-weight devices.

PDA Adaptor (Personal PDA Adaptor (Personal Server)Server)

GMSME – Application GMSME – Application TransportsTransports

Instant Messenger Instant Messenger supports to access from PDA Jabber clients to supports to access from PDA Jabber clients to

Jabber server.Jabber server. Parsing XML messages between PDAs and Parsing XML messages between PDAs and

Jabber Server.Jabber Server. Managing Presence information based on Managing Presence information based on

XML presence protocol (XXMP).XML presence protocol (XXMP). Text ChatText Chat

extends Text Chat feature built in Garnet to extends Text Chat feature built in Garnet to Mobile clients.Mobile clients.

Translates GMS messages between PDAs and Translates GMS messages between PDAs and Garnet collaborative systems.Garnet collaborative systems.

IM and Text Chat IM and Text Chat TransportsTransports

GMSME – Application GMSME – Application Processors Processors Shared Display Shared Display

Sharing frame-buffer corresponding to either a Sharing frame-buffer corresponding to either a windows or entire desktop replicated among the windows or entire desktop replicated among the clients.clients.

customizable display resolution ( smaller resolution, customizable display resolution ( smaller resolution, different display types etc.)different display types etc.)

Data compression, incremental update.Data compression, incremental update. Shared ExportShared Export

Convert (rendering of) object to some standard form Convert (rendering of) object to some standard form that is more flexible than bitmap of Shared Display.that is more flexible than bitmap of Shared Display.

Replicated appropriate viewer for each client (PDF Replicated appropriate viewer for each client (PDF HTML SVG Browsers)HTML SVG Browsers)

Adapts W3C’s SVG (Scalable Vector Graphics) for Adapts W3C’s SVG (Scalable Vector Graphics) for exporting 2D graphics. - Inherits the benefits of Vector exporting 2D graphics. - Inherits the benefits of Vector graphics, XML document based model, SVG universal graphics, XML document based model, SVG universal accessibility features (stylesheets, alternatives).accessibility features (stylesheets, alternatives).

The Architecture of Shared The Architecture of Shared Export Export

The workflow of shared SVG Export in Garnet System

Well Formed SVG Document

A

Well Formed SVG Document

AAAConvert to SVG

Convert to SVG

Power Point Slides

Power Point Slides

HHMP

HHMP

HHMP

HHMP

HHMP

JMS/GMS Server

GMS

JMS/GMS Server

GMS

Grid Shared SVG Browser

Grid Shared SVG Browser

Grid Shared SVG Browser

GM

S

Grid Shared SVG Browser

Grid Shared SVG Browser

Grid Shared SVG Browser

Grid Shared SVG Browser

Grid Shared SVG Browser

GM

SG

MS

GMSGMS

GMSGMS

Personal Server

Parser

Rendering Engine

Stylesheets

Personal Server

Parser

Rendering Engine

Stylesheets

My_Professor My_Professor Collaborative User Interface providing Collaborative User Interface providing

integrated GUI to Garnet users accessing from integrated GUI to Garnet users accessing from mobile devices.mobile devices.

Implements HHMS client for wireless devices.Implements HHMS client for wireless devices. Provides user information about device, user, Provides user information about device, user,

application to PDA adaptor.application to PDA adaptor. Designed specially for various devices : network Designed specially for various devices : network

protocols, display types, etc.protocols, display types, etc. Delivers messages from PDA Adapter to each Delivers messages from PDA Adapter to each

application modules;application modules; Display IM message converted to HHMS by PDA Display IM message converted to HHMS by PDA

Adaptor.Adaptor. Draw ready-to–use image for Shared Display and Draw ready-to–use image for Shared Display and

Export. Export.

Garnet : Shared DisplayGarnet : Shared DisplayPCs

Garnet : SVGGarnet : SVGPCs

Experimental ResultsExperimental Results GMSME : iPaq H3650, WinCE 3.0, Personal-Java1.1GMSME : iPaq H3650, WinCE 3.0, Personal-Java1.1

Wireless 11 Mbit/s IEEE 802.11bWireless 11 Mbit/s IEEE 802.11b

0.1

1

10

100

1000

10000

Message Size (bytes)

mill

i-sec

onds

GMSMEGMS

Total Latency

Conclusions and Future Conclusions and Future WorksWorks

A collaborative environment Garnet designed a A collaborative environment Garnet designed a universal event model using the pub-sub universal event model using the pub-sub paradigm.paradigm.

A integration of PDAs into synchronous A integration of PDAs into synchronous collaboration.collaboration.

PDA Adaptor maps system events into a form PDA Adaptor maps system events into a form suitable for the PDAs and uses an optimized suitable for the PDAs and uses an optimized protocol. protocol.

Future WorksFuture Works supporting various Handheld device including PalmOS supporting various Handheld device including PalmOS

and Cellular Phone.and Cellular Phone. Web Service for Universal Access CollaborationWeb Service for Universal Access Collaboration