Intelligent User Interfaces Intelligent User Interfaces Research GroupResearch Group

Directed by: Frank ShipmanDirected by: Frank Shipman

Visual Knowledge Builder (VKB): Visual Knowledge Builder (VKB): Supporting Personal CollectionsSupporting Personal Collections

Problem: information tasks require a combination Problem: information tasks require a combination of location, comprehension, and modificationof location, comprehension, and modification• Current systems have emphasized the location of Current systems have emphasized the location of

information (e.g. search engines, portals)information (e.g. search engines, portals)• There are few tools supporting the activity of There are few tools supporting the activity of

comprehending and modifying the found contentcomprehending and modifying the found content

Approach: Spatial hypertextApproach: Spatial hypertext• Spatial hypertext expresses inter-document Spatial hypertext expresses inter-document

relationships via visual and spatial cuesrelationships via visual and spatial cues• Users develop personal visual languages during the Users develop personal visual languages during the

course of their activitiescourse of their activities• The system can recognize parts of the visual language The system can recognize parts of the visual language

and interpretation to support the user’s task. and interpretation to support the user’s task.

Frank ShipmanFrank Shipman

VKB

Expressing Relations VisuallyExpressing Relations Visually

VKB provides: A hierarchy of two-dimensional workspaces called

collections for placing information Easy manipulation of

visual properties of information

Information objects pointing to external content

Attribute/value pairs for attaching metadata

VKB

Experience with VKB UseExperience with VKB Use

VKB

Project Management

ConferencePlanning

Writing Papers & Creating Presentations

VITE : Manipulating Structured VITE : Manipulating Structured Information in a Visual WorkspaceInformation in a Visual Workspace

Problem: computers generate/require lots of Problem: computers generate/require lots of structured information but people prefer natural structured information but people prefer natural representationsrepresentations• Incomplete representation –Incomplete representation –

formal representations abstract real contentformal representations abstract real content• Insufficient intermediate representation –Insufficient intermediate representation –

formal representations miss transient states during formal representations miss transient states during problem solvingproblem solving

Approach: 2-way mappings in a visual workspaceApproach: 2-way mappings in a visual workspace• Two-ways: (1) visualization and (2) visual parsingTwo-ways: (1) visualization and (2) visual parsing• Editable visualization and mappingEditable visualization and mapping• Direct manipulation of content in the workspaceDirect manipulation of content in the workspace

Haowei HsiehHaowei Hsieh

VITE

Problem Solving in a Visual WorkspaceProblem Solving in a Visual Workspace

Visual mapping is created in the Mapping Designer. Structured information is rendered as information objects in the workspace.

Users accomplish tasks by adjusting mapping designs and then manipulating information objects in the workspace.

VITE

Generating Mappings with EaseGenerating Mappings with Ease

The Mapping Assistant generates mappings based on:• a brief description of the

user’s task,• a statistical analysis of the

data set, and • a mapping design

knowledge-base combining results from the VITE evaluation with general graphic design principles.

The Mapping Assistant helps users generate initial mappings quickly so users can start working on the task right away.

VITE

GRC: Graphical Requirements GRC: Graphical Requirements CollectorCollector

Problem: Software requirements elicitation• Questionnaires enable feedback from a large

group but do not elicit rich design information.• Interviews and observations generate rich

content but are expensive in time and money.

Approach: Collect design information from probable end-users through:• End-user graphical user interface mock-ups• Textual descriptions and rationales for the

widgets and windows that they create

J. Michael MooreJ. Michael Moore

GRC

Examples of End-user Mock-upsExamples of End-user Mock-ups

Above: Other users relied more on Above: Other users relied more on textual description. The text dialog textual description. The text dialog

explains the role of the “Design” button.explains the role of the “Design” button.

Below: Some users depend Below: Some users depend heavily on graphical organization.heavily on graphical organization.

GRC

Generating Design Information from Mock-upsGenerating Design Information from Mock-ups

Term-frequency view of mock-up designs

Main analysis interface provides access Main analysis interface provides access through:through:• Filtering mock-up componentsFiltering mock-up components• Grouping components based on visual Grouping components based on visual

and textual similarityand textual similarity

Algorithms analyze spatial, textual, Algorithms analyze spatial, textual, and temporal information to and temporal information to generate pattern-based views of generate pattern-based views of design data to help construct design data to help construct domain modelsdomain models

GRC

MASH / WARPMASH / WARPLuis Francisco-RevillaLuis Francisco-Revilla

MASH

Problem: information needs to be adapted based Problem: information needs to be adapted based on the use contexton the use context• Information systems can employ multiple models in Information systems can employ multiple models in

order to adapt content and presentation.order to adapt content and presentation.• Conflict may occur as different models propose Conflict may occur as different models propose

contradicting suggestions.contradicting suggestions.

Approach: identifying mechanisms for dealing Approach: identifying mechanisms for dealing with conflictswith conflicts• Deliver a flexible context-sensitive solution to this issue

within the field of adaptive spatial hypermedia• MASH (Multi-model Adaptive Spatial Hypermedia) is a

framework for dynamic and adaptive behaviors.

Architectural Framework

Spatial Hypermedia

Platform

SpatialParser

Spatial Analyzer

Spatial Transformer

Composites

Metrics

Transformations

Behaviors

M1

M2

Mn

Spatial Hypermedia Generator

Contents

Dynamic

Adaptive

Generative

Interactive

Models

The MASH framework consists of three parts: a high-level abstraction of objects and relationships, a generic architectural framework, and a theoretical ontology of spatial adaptations.

Ontology of Adaptations

Object Abstraction

MASH

Space Dynamic/Static

Dimensions

Homogeneous/Heterogeneous

Object Composite

Atomic

Document

Relationship Association

Quality

Scope

Explicit

Implicit

Absolute

Relative

Extra-space

Intra-space

Adaptive mechanisms can

hide or show different objects

within the presentation in

accordance to the user’s preferences.

Conflict between multiple models can be resolved to fit relevant aspects of the use context, such as activity and situation.

WARP is a first implementation of a MASH-based system.

WARP can discover the implicit structure of a

document as defined by the spatial relationships

between objects.

The spatial structure provides a useful

definition of context that facilitates the adaptation

of the document.MASH