Upload
jo-vermeulen
View
3.452
Download
1
Embed Size (px)
DESCRIPTION
We present a rendering engine for displaying graphical user interfaces on multiple devices. The renderer interprets a standardized XML-based user interface description language: the User Interface Markup Language (UIML). A generic architecture for the renderer is defined so that deployment of the engine on different devices implies only little effort. We show that our rendering engine can be used on iDTV set-top boxes, mobile phones, PDAs and desktop PCs, and smoothly integrates with both local and remote application logic. As a test bed for the UIML specification we also explore support for extensions to UIML that enable the user interface designer to maximize accessibility and target multiple devices and different types of users at once. Best talk award at CADUI 2006, the 6th International Conference on Computer-Aided Design of User Interfaces.
Citation preview
A Generic Approach for Multi-device User Interface
Rendering with UIML
Kris Luyten, Kristof Thys, Jo Vermeulen and Karin Coninx
{kris.luyten, kristof.thys, jo.vermeulen, karin.coninx}@uhasselt.be
Expertise Centre for Digital MediaAffiliated with the Institute for BroadBand Technology
Hasselt University - transnationale Universiteit LimburgBelgium
A Generic Approach for Multi-device User Interface
Rendering with UIML
A Generic Approach for Multi-device User Interface
Rendering with UIML
What is UIML?
description of auser interface
• high level
• generic
• no specific tags (e.g. <button>)
• allows to build a custom user interface language
What does UIML look like?
Structure
Style
Content
Behavior
Logic
Presentation
InterfacePeers
Vocabulary UIML document
<uiml> <interface> <structure> <part>...</part> </structure> <style> <property>...</property> </style> <behavior> <rule>...</rule> </behavior> </interface> <peers>...</peers></uiml>
A Generic Approach for Multi-device User Interface
Rendering with UIML
Renderer vs Compiler
Compiler
Executable codeUIML
Renderer
• runtime environment
• allows for rapid prototyping
• dynamic changes at runtime
A Generic Approach for Multi-device User Interface
Rendering with UIML
Multiple renderers
• Mono/MS .NET/MS Compact .NET (Uiml.net)
• Java MHP
• Java MIDP
Multiple devices
• Desktop PC
• PDA
• iDTV
• Smartphone
• ...
A Generic Approach for Multi-device User Interface
Rendering with UIML
Common renderer architecture
• requires a VM
• one rendering core
• multiple rendering back-ends
Rendering core
• process UIML
• build internal representation
• mappings
• dynamically at runtime; or
• ad-hoc (when no reflection is available)
Rendering back-ends
• limited responsibility
• process parts that require widget-set specific knowledge
The rendering process
• three stages
• pre-processing
• main processing
• post-processing
Pre-processing
• transform one UIML document into another
• examples
• adaptation by user profile
• applying layout constraints
Main processing
• interpret the UIML file
• build a concrete instantiation using the available widget set
Main processing details
1. look up the required back-end
2. build an internal representation
3. for every part, attach its style and behavior
4. render the generated tree to a concrete instantiation
<structure> <part id=”all” class=”Container”> <part id=”rubens” class=”Button”/> </part></structure>
<style> <property part-name=”rubens” name=”label”> Rubens Barichello </property></style>
rubens
all
label
Post-processing
• manipulate the concrete instantiation
• examples
• bind the UI with the application logic
• adaptable navigation
The demo
Overview
• industry case
• participate in TV show by using different devices
• functional interactive quiz
• with scorekeeping and an evaluation
• adapted to the user’s profile
• on different devices
UIML allows us to reuse the quiz logic
Shared logic
PdaQuiz.Browser
mcdp.CanvasClient
Xlet.UimlXlet
<logic> <d-component id=”Quiz“ ...> <d-method id=”Answered“ ...> <d-param ... /> </d-method> <d-method id=”GotoPage“ ...> <d-param ... /> </d-method> <d-method id=”Exit“ ... /> <d-method id=”GetPoints“ ... /> <d-method id=”Question1Correct“ ... /> <d-method id=”Question1Incorrect“ ... /> <d-method id=”Question2Correct“ ... /> <d-method id=”Question2Incorrect“ ... /> <d-method id=”Question3Correct“ ... /> <d-method id=”Question3Incorrect“ ... /> </d-component></logic>
Smartphone
iDTV
PDA
DesktopDesktopQuiz.Browser
Completely independent of the user interface
GTK# desktop demo
SWF desktop demo
Can we reuse the entire interface
description?
Almost
• interactors
• interaction
• application logic
• content
• layout
Layout constraints
• widget set independent layout specification
• plastic
• extension to UIML
• handled in the pre-processing step
(0,0) (50,0)
“rubens” above “giancarlo”
“rubens” left-aligned-with “giancarlo”
(0,0)
(0,10)
<style> ... <property part-name=”rubens” name=”position”>0,0</property> <property part-name=”giancarlo” name=”position”>0,10</property> ...</style>
Add position properties
MHP iDTV demo
PDA demo
Smartphone demo
User adaptation
• Intelligent User Interface Agent
• knows about user’s preferences
• transforms the interface accordingly
• currently MPEG-21 based
• handled in the pre-processing step
Use case
• visually impaired users
• user agent adapts the UIML description
• larger, bolder fonts
Before
After
Conclusions
• functional client-side UIML renderer on different devices
• .NET version is freely available
• http://research.edm.uhasselt.be/uiml.net
Questions?