Embed Size (px)
DESCRIPTION
In ubiquitous computing, interactive applications are shipped with different variations of its user interface depending on the constraints imposed by the context in which they are running, such as the user, the computing platform and environment. A multi-presentation user interface is composed of a series of in-terconnected user interfaces for the same task to be carried out in different contexts of use. When access to software applica-tions must be guaranteed in more than one context of use, it is necessary to automatically adapt the interface in order to pre-serve their usability when context switching occurs, for instance, a switch from a desktop to a pocket computer. To achieve this goal, this paper proposes a model and a visualization technique to express and manipulate the plasticity domains of a multi-presentation user interface. The plasticity domain denotes the set of contexts of use it is able to cover while preserving its usabil-ity. This paper focuses primarily on one aspect of the context of use: the computing platform and its screen size: when the di-mensions of a graphical user interface change, the multi-presentation interface automatically switches to the presentation which is the most adapted to this screen. The model supports the definition of this plasticity domain in terms of window size and location. The visualization technique helps in both making ob-servable the set of presentations that fit the available space, and perceiving which operations could help in switching from one presentation to another one. The model has been integrated into a user interface description language and is supported by an in-telligent editor, because it infers from plasticity domains all the constraints and conditions required for context switching.
Citation preview
1 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
An Intelligent Editor forMulti-Presentation User Interfaces
Benoît Collignon1, Jean Vanderdonckt1, Gaëlle Calvary2
1Université catholique de Louvain (UCL)Louvain School of Management (LSM)
Belgian Laboratory of Computer-Human Interaction (BCHI)http://www.isys.ucl.ac.be/bchi
2Université Joseph Fourier – Grenoble I, Laboratoire LIG385, rue de la Bibliothèque BP 53 - F-38041 Grenoble Cedex 9 (France)
2 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Introduction
• The problem of designing multi-target user interfaces– Context of use = {user, computing platform, physical environment}– Multiple contexts of use => multiple variations of users, platforms, and
environments• Independently• Simultaneously
• Notion of plasticity– Plasticity = user interface ability to change according to a change of the
context of use while preserving predefined usability properties• Context-aware adaptation = mere adaptation depending on context
changes• Plasticity = more than context-aware adaptation since usability should
be guaranteed at a minimum level of satisfaction
• To address the problem, we combine– Model-driven engineering of user interfaces– Plasticity as a mean to address context-aware adaptation
3 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Multiplicity of contexts of use
Location Role Device Experience
Sporting Multimedia Travel programme
Working Travel booking sitePowerful interface for complex operations
Travelling Booking notificationEverywhere connectivity for simple data exchange
Family TV is multi-media family device #1
4 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Example of a Plastic User interface
• Property of plasticity = adaptation to the context of use while preserving predefined usability properties of interest
[Grolaux et al.,2002]
5 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Example of a multi-target UI
• Traditional approach: Atomica
HTML/Java UI code
Target 1:Web terminal
Target 2:Personal computer
Visual C++ UI code
Target 3:Pocket PC
Mobile Visual Basic UI
6 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Related work
• Some solutions to address this problem:– The mediator solution
(model-based approach)
[Eisenstein et al.,2001]
7 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Related work
• Some solutions to address this problem:– Multiple variations based on a model (here, a platform model)
[Märtin et al.,1990]
8 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Related work
• Some solutions to address this problem:– Adaptive UI layout: widgets are (un)displayed and laid out
according to resolution
[Plomp & Keranen,2002]
9 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Related work
• Some solutions to address this problem:– Artistic resizing : widgets have different variations according to
resolution
[Dragicevic et al.,2005]
w
h
wL
hL
yL
xLr
hB
wB xL = (w-wL) / 2 yL = (h-hL) / 2 wL = 20 hL = 10
wB = 5 hB = 5
r = 20
10 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Our solution
• Combine model-driven engineering and plasticity domains
11 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Case study
• Case study: multi-target plastic UI for date & time (FlexClock)
[Grolaux et al.,2002]
12 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Case study
• Dialogue modeled through a Moore machine– 16 alternative variations– 6 resizing operations
• Observations– Visualization is hard to achieve
13 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
14 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
15 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
16 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
17 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
18 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
19 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W1HH:MM
W2HH:MM:SS
W3HH:MMJJ/MM
W4HH:MM:SSJJ/MM/AAAA
W5HH:MM:SS
DAY JJ/MM/AAAA
W6HH:MM:SS
DAY JJ MONTH AAAA
W7
W8
HH:MM
JJ/MM
W9 HH:MM:SS
JJ/MM/AAAA
W10 HH:MM:SS
DAY JJ/MM/AAAA
W11 HH:MM:SS
DAY JJ MONTH AAAA
W13
HH:MM:SS
DAY JJ/MM/AAAA
W12
HH:MM:SSJJ/MM/AAAA
W14
HH:MM:SS
DAY JJ MONTH AAAA
W15
HH:MM:SS
DAY JJ MONTH AAAACALENDAR
W16HH:MM:SS
DAY JJ MONTH AAAACALENDAR
20 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Case study
• Dialogue modeled through a Mealy machine
W2 W1
W2 W2
W2 W2
W2 W1
2 / 1 1
11 / 12
17 / 133
30/15 4
19 / 95
42 / 16 6
98 / 167
102 / 16
8
21 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Case study
• Main idea: design one UI variation for one context at a time, connect related variations afterwards
Horizontal screen resolution
Vertical screen resolution
Plasticity domain
22 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Case study
• Main idea: design one UI variation for one context at a time, connect related variations afterwards
23 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Tool support: PlastiXML
• Graphical editor for plasticity domains– Direct manipulation of plasticity domains (here, rectangles)– Assignment of a UI variation to each plasticity domain– Automated generation of UsiXML for
• Layout• Navigation between variations
<contextModel id="test-contextModel_14" name="test-contextModel"> <contextRessource contextId="cont1"> <Environment id="envir1" name="envir1" … /> <userStereotype id="ustr1" stereotypeName="ustr1" … /> <Platform id="plat1" name="plat1"> <HardwarePlatform …screenSize="640x480"/> <SoftwarePlatform … /> <NetworkCharacteristics … /> <WapCharacteristics … /> <BrowserUA … /> </Platform> </contextRessource> <PlasticityDomainSet id="plasts1"> <PlasticityDomain id="plast1" contextId="cont1"> <PlatformPlasticityDomain id="platp1" PlatformId="plat1"> <ScreenSizeAspect id="scrd1" shape="rectangle" corners="{(80,10);(80,30);(ScreenSizeXLimit,10); (ScreenSizeXLimit,30)}" allowedOperations="{vertical shrinkage}"/> </PlatformPlasticityDomain> </PlasticityDomain> <PlasticityDomain id="plast2" contextId="cont1"> <PlatformPlasticityDomain id="platp2" PlatformId="plat1"> <ScreenSizeAspect id="scrd2" shape="polygon" corners="{(80,30);(ScreenSizeXLimit,30); (ScreenSizeXLimit,60);(100,60); (100,ScreenSizeYLimit); (80,ScreenSizeYLimit)}" allowedOperations="{vertical shrinkage}"/> </PlatformPlasticityDomain> </PlasticityDomain> </PlasticityDomainSet> </contextModel><mappingModel id="map1" name="map1" …> <isShapedFor id="adp1" name="adp1"> <source sourceId="W2"/> <target targetId="plast1"/> </isShapedFor> <isShapedFor id="adp2" name="adp2"> <source sourceId="W4"/> <target targetId="plast2"/> </isShapedFor> <source sourceId="W12"/> <target targetId="plast4"/> </isShapedFor></mappingModel>
24 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Scenario
• First plasticity domain
25 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Scenario
• Second plasticity domain
26 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Scenario
• Third plasticity domain
27 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Running applications
• Example #1: country selector
28 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Running applications
• Example #2: multi-presentationmedical teaching application
29 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Running applications
• Example #3: multi-presentationsimple information systems
30 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Conclusion (1/2)
• Advantages– One UI variation is assigned to one context (here, a resolution) at a time– Connect related variations afterwards– Visual design of connected variations insted of separate (unrelated) design– Automated generation of UsiXML specifications
• Presentation• Navigation
– Mealy machine– Moore machine
• Limitations– Apart from copy/paste, no direct reuse of components from one UI
variation to another– No factoring out of common components
• Possible extension– Generate a default UI variation to feed the various resolutions to be
supported– Edit afterwards
• Generalization needed
31 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Conclusion (2/2)
• Feedback from designers– Like the visual, colourful design aspects– Dislike the modelling aspects (fortunately generated by software)– Limited to GUIs for the moment
• Feedback from users– Adaptation always induce some user disruption– No explanation of the adaptation
• But would it be better is explanation is provided
32 ACM SAC’2008 – Fortaleza, Ceará, Brazil, March 16-20, 2008
Thank you very much for your attention
For more information and downloading,http://www.isys.ucl.ac.be/bchi
http://www.usixml.orgUser Interface eXtensible Markup Language
http://www.similar.ccEuropean network on Multimodal UIs
Special thanks to all members of the team!
