Automated Generation of Visual Discourse Michelle X. Zhou Department of Computer Science Columbia University New York, NY 10027 Supported in part by DARPA

Automated Generation of Automated Generation of Visual DiscourseVisual Discourse

Michelle X. ZhouMichelle X. ZhouDepartment of Computer ScienceDepartment of Computer Science

Columbia UniversityColumbia UniversityNew York, NY 10027New York, NY 10027

Supported in part by DARPA Contract DAAL01-94-K-0119, New Supported in part by DARPA Contract DAAL01-94-K-0119, New York State Science and Technology Foundation, NSF Grant ECD-York State Science and Technology Foundation, NSF Grant ECD-

88-11111, and ONR Contract N00014-97-1-083888-11111, and ONR Contract N00014-97-1-0838

2

ProblemProblem Designing effective visual presentations is Designing effective visual presentations is

difficult and costlydifficult and costly Designing customized visual presentations in a Designing customized visual presentations in a

timely manner is more difficulttimely manner is more difficult

ApproachApproachDevelop computer techniques to automate Develop computer techniques to automate visual design process visual design process

Automated Visual DesignAutomated Visual Design

3

Automated Visual DesignAutomated Visual Design

??????InputInput

4

Single displaysSingle displays

Previous WorkPrevious Work

APT Mackinlay 86APT Mackinlay 86SAGE Roth & Mattis 91SAGE Roth & Mattis 91ANDD Marks 91ANDD Marks 91

APEX Feiner 85APEX Feiner 85IBIS Seligmann 91IBIS Seligmann 91WIP Andre et al. 93WIP Andre et al. 93

A series of displaysA series of displays

. . .. . . . . .. . .

discretediscrete

5

Our Goal: Visual Discourse Our Goal: Visual Discourse DesignDesignA series of connected displaysA series of connected displays

RotateRotate Scale+Move+...Scale+Move+... OpenOpen

OpenOpenRotateRotateScaleScaleMoveMove

. . .. . .

6

ThesisThesis. . .. . .. . .. . .

Thesis WorkThesis Work

Visual DiscourseVisual DiscourseModelingModeling

• CoherenceCoherence• VersatilityVersatility• InteractivityInteractivity

System DesignSystem Design

• Knowledge baseKnowledge base• Inference engineInference engine• Visual realizerVisual realizer• Interaction handlerInteraction handler

IMPROVISEIMPROVISE

• Computer network Computer network managementmanagement

• Patient medical Patient medical record summaryrecord summary

• Data characterizationData characterization• Visual task hierarchyVisual task hierarchy• Presentation design Presentation design languagelanguage• Inference paradigmInference paradigm

DesignDesignFoundationFoundation

SystemSystemDevelopmentDevelopment

GeneralGeneralApproachApproach

7

ThesisThesis. . .. . .. . .. . .






IMPROVISEIMPROVISE







8

Visual Discourse ModelingVisual Discourse Modeling

CoherenceCoherence

VersatilityVersatility

InteractivityInteractivity

ContinuityContinuityConsistencyConsistencyUnityUnity

Wide range of informationWide range of informationWide variety of visual media/techniquesWide variety of visual media/techniques

InterruptibleInterruptibleResponsiveResponsive

9


ThesisThesis. . .. . .. . .. . .





IMPROVISEIMPROVISE







10

Design Foundation: Design Foundation: Design ProcessDesign Process OutputOutput

++++

PresentationPresentationDataData

PresentationPresentationIntentsIntents

PresentationPresentationContextContext Design EngineDesign Engine

Design KnowledgeDesign Knowledge

InputInput

11

Design Foundation Design Foundation OutputOutput

++++





InputInputDataDataCharacterizationCharacterization

12

Presentation-relatedPresentation-relateddata propertiesdata properties

Data CharacterizationData Characterization


……..……....

……..……..

GoalGoal

Characterizing Quantitative dataCharacterizing Quantitative dataMackinlay 86Mackinlay 86Roth & Mattis 90Roth & Mattis 90

Characterizing Qualitative dataCharacterizing Qualitative dataArens et al. 93Arens et al. 93

Data Data Visual ElementsVisual Elements

13

PresentationPresentation-related-relateddata propertiesdata properties

Our Approach: Characterizing Our Approach: Characterizing Heterogeneous Data Heterogeneous Data

Data DimensionsData Dimensions

……..……....

……..……..

DomainDomainTypeTypeAttributeAttributeRelationRelation

RoleRoleSenseSense

(Jones (is-a PATIENT)(Jones (is-a PATIENT) (type ATOMIC)(type ATOMIC) (property (Form …)...)(property (Form …)...) (relation (rel …)...)(relation (rel …)...)

(role LOCATE)(role LOCATE) (sense SYMBOL))(sense SYMBOL))

14

Data Data CharacterizationCharacterization

TaxonomyTaxonomy

AtomicAtomicCompositeComposite

EntityEntityConceptConceptMeasurementMeasurementEventEvent

FormFormMaterialMaterialLocationLocationTransienceTransienceImportanceImportanceComposite AttributesComposite Attributes

FDFDConstituencyConstituencyAttributeAttributeEnumerationEnumeration

LabelLabelListListPlotPlotSymbolSymbolPortraitPortrait

OrderingOrderingScalabilityScalabilityContinuityContinuity

AssociateAssociateBackgroundBackgroundCategorizeCategorizeClusterClusterCorrelateCorrelate. . .. . .

TypeType

DomainDomain

AttributesAttributes

RelationRelation

RoleRole

SenseSense

15

Design FoundationDesign Foundation

++++






ContextContextModelingModeling

OutputOutput

16

Context ModelingContext Modeling

Our ApproachOur Approach AudienceAudience OccasionOccasion EnvironmentEnvironment

Previous WorkPrevious Work Situation space Friedell 83Situation space Friedell 83 Display categories Mackinlay 86Display categories Mackinlay 86

GoalGoal

ContextContextInformationInformation

VisualVisualTechniquesTechniques

17


++++






IntentIntentModelingModeling


OutputOutput

18

Intent ModelingIntent Modeling

Presentation IntentsPresentation Intents Wehrend & Lewis 90Wehrend & Lewis 90 Casner 91Casner 91 Marks 91Marks 91 Roth & Mattis 91Roth & Mattis 91 Ignatius & Senay 94Ignatius & Senay 94

Visual TechniquesVisual Techniques Hunter 87Hunter 87 Levin 87Levin 87

Maybury 93Maybury 93 Seligmann 93Seligmann 93 Sutcliffe et al. 94Sutcliffe et al. 94


VisualVisualTechniquesTechniques

GoalGoal


19

Visual TasksVisual TasksVisualVisual

TechniquesTechniquesPresentationPresentation

IntentsIntents

Our Approach: Visual Task Our Approach: Visual Task CharacterizationCharacterization

FocusFocusEnlargeEnlarge

HighlightHighlightZoomZoom

SearchSearchElaborateElaborate

ImplyImplyAchieveAchieve

Abstract Visual TechniqueAbstract Visual Technique

AchieveAchieve ImplyImply

(Visual Effects)(Visual Effects)

20

BackgroundBackgroundAssociateAssociate ColocateColocate ConnectConnect UniteUnite AttachAttachCategorizeCategorize MarkDistributeMarkDistributeClusterCluster OutlineOutline MarkDistributeMarkDistributeCorrelateCorrelate PlotPlot MarkComposeMarkComposeCompareCompare DifferentiateDifferentiate IntersectIntersectDistinguishDistinguish MarkDistributeMarkDistribute IsolateIsolateEmphasizeEmphasize FocusFocus IsolateIsolate ReinforceReinforce

GeneralizeGeneralize MergeMergeIdentifyIdentify NameName PortrayPortray IndividualizeIndividualize ProfileProfileLocateLocate PositionPosition SituateSituate PinpointPinpoint OutlineOutlineRankRank TimeTimeRevealReveal ExposeExpose ItemizeItemize SpecifySpecify SeparateSeparateSwitchSwitch

EncodeEncode LabelLabel SymbolizeSymbolize QuantifyQuantify IconifyIconify PortrayPortray TabulateTabulate PlotPlot TraceTrace StructureStructure MapMap

Visual Task TaxonomyVisual Task Taxonomy

21


++++







Design KnowledgeDesign KnowledgeModelingModeling

OutputOutput


22

Design Knowledge ModelingDesign Knowledge ModelingGoalGoal Computational representation of designComputational representation of design knowledgeknowledge

Previous WorkPrevious Work Visual formalismsVisual formalisms

Marks 91; Lohse et al. 94Marks 91; Lohse et al. 94

Visual techniquesVisual techniquesFriedell 84; Seligmann 93; Keller&Keller 94Friedell 84; Seligmann 93; Keller&Keller 94

Visual design principlesVisual design principlesWinn&Holliday 82; Bertin 83; Mullet&Sano 95Winn&Holliday 82; Bertin 83; Mullet&Sano 95Mackinlay 86; Ignatius&Senay 94; Tufte83, 90, 97Mackinlay 86; Ignatius&Senay 94; Tufte83, 90, 97

23

Our Approach: Presentation Our Approach: Presentation Design LanguageDesign Language

Visual objects Visual objects Represent various visual formalismsRepresent various visual formalisms

Visual techniquesVisual techniquesCompose/manipulate visual objectsCompose/manipulate visual objects

Visual design principlesVisual design principles Guide the visual object composition and Guide the visual object composition and

manipulation manipulation

24

Visual Object RepresentationVisual Object Representation

SyntaxSyntax

SemanticsSemantics

PragmaticsPragmatics

Patterns/compositionsPatterns/compositions

Meanings Meanings

Specific meaningsSpecific meanings

++redred

human hearthuman heart

lovelove

I New YorkI New York

25Visual PrimitivesVisual Primitives

Shape/Color/Size/Orientation/...Shape/Color/Size/Orientation/...

Visual Object HierarchyVisual Object Hierarchy

Visual UnitiesVisual Unities

BostonBoston

Visual LexiconVisual Lexicon(atomic objects only)(atomic objects only)

Visual StructuresVisual Structures

TablesTablesChartsCharts

DiagramsDiagramsVisual FramesVisual Frames

Static/DynamicStatic/DynamicDiscourseDiscourse

26

ExamplesExamples

Categorized by functionCategorized by functionFormationFormation

TransformationTransformation

CameraCamera

Visual TechniquesVisual Techniques (DesignTableChart(DesignTableChart (is-a FormationTech)(is-a FormationTech) (operands (operands ?data-obj?data-obj ?table)?table) . . .). . .)

(Move (Move (is-a TransformationTech)(is-a TransformationTech) (operands ?obj)(operands ?obj) (source)(source) (destination)(destination) (startTime)(startTime) (endTime))(endTime))

(SetCamera(SetCamera (is-a CameraTech)(is-a CameraTech) (operand ?camera)(operand ?camera) (position)(position) (orientation)(orientation) . . .). . .)

Categorized by usageCategorized by usagePrimitivePrimitive

CompositeComposite

27

Visual Design PrinciplesVisual Design Principles

Expressiveness rulesExpressiveness rules

Effectiveness rulesEffectiveness rules

Comprehensiveness & distinctivenessComprehensiveness & distinctiveness Generality & discreteness Generality & discreteness IntegrityIntegrity x

28

Visual Design PrinciplesVisual Design Principles

Expressiveness rulesExpressiveness rules

Effectiveness rulesEffectiveness rules

Comprehensiveness & distinctivenessComprehensiveness & distinctiveness Generality & discreteness Generality & discreteness IntegrityIntegrity

Accuracy & clarityAccuracy & clarity AppropriatenessAppropriateness ImmediacyImmediacy ContinuityContinuity ConsistencyConsistency UnityUnity

30


++++







Design KnowledgeDesign KnowledgeModelingModeling

InferenceInferenceModelingModeling

OutputOutput


31

Inference ModelingInference Modeling

GoalGoal Flexible and efficient design methodFlexible and efficient design method

Previous WorkPrevious Work Constructive designConstructive design

Mackinlay 86; Roth & Mattis 90; Marks 91Mackinlay 86; Roth & Mattis 90; Marks 91

Parametric designParametric design Zdybel et al. 81; Robertson 91; Ignatius&Senay 94Zdybel et al. 81; Robertson 91; Ignatius&Senay 94

32

Our Approach: Hybrid Our Approach: Hybrid Inference ParadigmInference Paradigm

Constructive synthesisConstructive synthesis

Parametric synthesisParametric synthesis

Planning presentations from scratchPlanning presentations from scratch

Efficiently create visual models for Efficiently create visual models for atomic data objectsatomic data objects

A least-commitment constraint-based approachA least-commitment constraint-based approach

33

Planning Elements & FeaturesPlanning Elements & FeaturesVisual DiscourseVisual Discourse

Collection of framesCollection of framesPlanPlan

Collection of actionsCollection of actions

Visual TechniquesVisual TechniquesVisual/Domain ObjectsVisual/Domain Objects

Design PrinciplesDesign Principles

OperatorsOperatorsObjectsObjects

ConstraintsConstraints

DPOCLDPOCL(Young et al. 95)(Young et al. 95)

SIPESIPE(Wilkins 88)(Wilkins 88)++ PREVISEPREVISE

(Zhou 97)(Zhou 97)

PracticalPracticalhierarchical-decomposition partial-order planninghierarchical-decomposition partial-order planning

34

Extended FeaturesExtended Features

DPOCLDPOCL(Young et al. 95)(Young et al. 95)

SIPESIPE(Wilkins88)(Wilkins88)++ PREVISEPREVISE

(Zhou97)(Zhou97) Top-down strategyTop-down strategy Action decompositionAction decomposition +Object decomposition+Object decomposition

+Dynamic numerical +Dynamic numerical constraintsconstraints

+Domain heuristics+Domain heuristics

VariablesVariables +Enriched Variables+Enriched Variables

+Spatial reasoning+Spatial reasoningTemporal reasoningTemporal reasoning

36

ThesisThesis. . .. . .. . .. . .






IMPROVISEIMPROVISE







37

System FrameworkSystem Framework

Knowledge BaseKnowledge Base

Domain dataDomain data Situation dataSituation data Visual dataVisual data Meta dataMeta data

Inference EngineInference Engine

PREVISEPREVISE Visual lexical chooserVisual lexical chooser

Visual RealizerVisual Realizer

Interface language designInterface language design

Interaction HandlerInteraction Handler

Present interactivityPresent interactivity Control interactivityControl interactivity

38

ThesisThesis. . .. . .. . .. . .






IMPROVISEIMPROVISE







39

Stand-alone graphics systemStand-alone graphics systemComputer network managementComputer network management

Graphics generator in a multimedia Graphics generator in a multimedia presentation systempresentation systemHealthcareHealthcare

IMPROVISE IMPROVISE (Illustrative Metaphor (Illustrative Metaphor PROduction in VISual Environments)PROduction in VISual Environments)

40

ArchitectureArchitecture

Task Task AnalyzerAnalyzer

Task Task AnalyzerAnalyzer

Presentation PlannerPresentation Planner

InteractionInteractionHandlerHandler

InteractionInteractionHandlerHandler

ConverterConverter

RendererRenderer

Knowledge BaseKnowledge Base

Content PlannerContent Planner

ChooserChooserOrganizerOrganizer

DesignerDesigner

StylistStylistCoordinatorCoordinator

Mes

sen

ger

Mes

sen

ger

41

Example: Present Patient Example: Present Patient Information to NurseInformation to Nurse

42

Sample Visual Discourse Sample Visual Discourse RepresentationRepresentation

(Discourse1 (is-a VISUAL-DISCOURSE)(Discourse1 (is-a VISUAL-DISCOURSE) (frames [Frame1] [Highlight2] . . .))(frames [Frame1] [Highlight2] . . .))

(Frame1 (is-a VISUAL-FRAME)(Frame1 (is-a VISUAL-FRAME) (frameElements [StructDiag1])(frameElements [StructDiag1]) (startTime 0.0)(startTime 0.0) (endTime -1.0))(endTime -1.0))

(Highlight2 (is-a Highlight)(Highlight2 (is-a Highlight) (operands [Table1-demo])(operands [Table1-demo]) (style MARKER)(style MARKER) (color . . .)(color . . .) (startTime 5.0)(startTime 5.0) (endTime 8.0)(endTime 8.0) . . .). . .)

(StructDiag1 (is-a STRUCTURE-DIAGRAM)(StructDiag1 (is-a STRUCTURE-DIAGRAM) (heading [Table1-Demo)(heading [Table1-Demo) (core [Unity1-body])(core [Unity1-body]) (elements [Unity2-device1] . . .) (elements [Unity2-device1] . . .) . . .) . . .)

43

Input: Presentation DataInput: Presentation Data

((Jones-demoJones-demo (is-a CONCEPT)(is-a CONCEPT) (type COMPOSITE)(type COMPOSITE) (convey [Jones-name](convey [Jones-name] . . .). . .) (role Identify)(role Identify) (sense LIST) . . .)(sense LIST) . . .)

((JonesJones (is-a PATIENT) (is-a PATIENT) (type ATOMIC)(type ATOMIC) (role LOCATE)(role LOCATE) (sense SYMBOL) . . .)(sense SYMBOL) . . .)

((Swan-GanzSwan-Ganz (is-a DEVICE) (is-a DEVICE) (type ATOMIC)(type ATOMIC) (convey [sw-name] (convey [sw-name] [sw-loc][sw-loc] [sw-content])[sw-content]) (sense SYMBOL). . .)(sense SYMBOL). . .)

((Jones-infoJones-info (is-a CONCEPT) (is-a CONCEPT) (type COMPOSITE)(type COMPOSITE) (convey [Jones] [Jones-demo] [Swan-Ganz] . . .) . . .)(convey [Jones] [Jones-demo] [Swan-Ganz] . . .) . . .)

(Jones-nameJones-name (is-a NAME-ATTR)(is-a NAME-ATTR) (value “S. Jones”)(value “S. Jones”) (role IDENTIFY)(role IDENTIFY) (sense LABEL). . .)(sense LABEL). . .)

44

Input: Presentation ContextInput: Presentation Context

(Context-info (Context-info ((audienceaudience

(identity NURSE) . . .)(identity NURSE) . . .) ((occasionoccasion

(presentation ON-LINE)(presentation ON-LINE)(medium VISUAL-SPEECH) . . .)(medium VISUAL-SPEECH) . . .)

((environmentenvironment(display (color COLOR) (size . . .) . . .)(display (color COLOR) (size . . .) . . .)(platform (cpu SGI R4400) . . .)))(platform (cpu SGI R4400) . . .)))

45

Intent:Intent:Summarize<Jones-info>Summarize<Jones-info>

Identify<Jones-info, Jones-demo>Identify<Jones-info, Jones-demo>Locate<Jones-line, Jones>Locate<Jones-line, Jones>Symbolize<Jones-line>Symbolize<Jones-line>

Name<Jones-demo, Jones-name>Name<Jones-demo, Jones-name>Itemize<Jones-demo>Itemize<Jones-demo>

Name<Jones-line, line-name>Name<Jones-line, line-name>Associate<Jones-line, line-content>Associate<Jones-line, line-content>

Structure<Jones-info>Structure<Jones-info>

Input: Presentation Intent & Input: Presentation Intent & Visual Task FormationVisual Task Formation

46



Itemize<Jones-demo>Itemize<Jones-demo>Name<Jones-demo, Jones-name>Name<Jones-demo, Jones-name>



Fulfill Visual TasksFulfill Visual Tasks

DesignStructureDiagramDesignStructureDiagram<Jones-info, ?diag><Jones-info, ?diag>

Patient InfoPatient Info

StructureDiagram

48








DesignVisRep<Jones-demo, ?h>DesignVisRep<Jones-demo, ?h>

DesignVisRep<Jones, ?core>DesignVisRep<Jones, ?core>

DesignVisRep<Johns-line, ?el>DesignVisRep<Johns-line, ?el>

HeadingIdentify

DemographicsDemographics

PatientPatient

CoreLocate

DeviceDevice

DeviceDevice

DeviceDevice

DeviceDevice

DeviceDevice

ElementSymbolize

ElementSymbolize

ElementSymbolize

ElementSymbolize

ElementSymbolize

StructureDiagram

50

DesignVisRep<Jones-demo, ?h>DesignVisRep<Jones-demo, ?h>

DesignVisRep<Jones, ?core>DesignVisRep<Jones, ?core>

DesignVisRep<Johns-line, ?el>DesignVisRep<Johns-line, ?el>








DesignTable<Jones-demo, ?t>DesignTable<Jones-demo, ?t>

DesignVisRep<Jones-name, ?th>DesignVisRep<Jones-name, ?th>

Heading(Table)

PatientPatient

CoreLocate

StructureDiagram

DeviceDevice

DeviceDevice

DeviceDevice

DeviceDevice

DeviceDevice

ElementSymbolize

ElementSymbolize

ElementSymbolize

ElementSymbolize

ElementSymbolize

DesignTable<jones-demo, ?table>DesignTable<jones-demo, ?table>

DesignTableDesignTable<jones-identification, ?table11><jones-identification, ?table11> DesignTableDesignTable

<jones-medHistory, ?table12><jones-medHistory, ?table12>

DesignTableDesignTable<jones-operation, ?table13><jones-operation, ?table13>

DesignVisRep<jones-name, ?table-heading>DesignVisRep<jones-name, ?table-heading>DesignVisRep<jones-identification, ?table-cell1>DesignVisRep<jones-identification, ?table-cell1>DesignVisRep<jones-medHistory, ?table-cell2>DesignVisRep<jones-medHistory, ?table-cell2>DesignVisRep<jones-operation, ?table-cell3>DesignVisRep<jones-operation, ?table-cell3>

DesignVisRep<jones-age,?table11-cell1>DesignVisRep<jones-age,?table11-cell1>DesignVisRep<jones-gender,?table11-cell2>DesignVisRep<jones-gender,?table11-cell2>DesignVisRep<jones-mrn, ?table11-cell3>DesignVisRep<jones-mrn, ?table11-cell3>

DesignVisRep<op-name,?table13-cell1>DesignVisRep<op-name,?table13-cell1>DesignVisRep<op-surgeon, ?table13-cell2>DesignVisRep<op-surgeon, ?table13-cell2>

DesignWord<jones-age, ?age-rep>DesignWord<jones-age, ?age-rep>DesignWord<jones-gender, ?gender-rep>DesignWord<jones-gender, ?gender-rep>

DesignWord<jones-mrn, ?mrn-rep>DesignWord<jones-mrn, ?mrn-rep>

DesignWord<op-name, ?op-name-rep>DesignWord<op-name, ?op-name-rep>DesignWord<op-surgeon, ?surgeon-rep>DesignWord<op-surgeon, ?surgeon-rep>

. . .. . .

Heading(Table)

StructureDiagram

Core(Unity)

DeviceDevice

DeviceDevice

DeviceDevice

DeviceDevice

DeviceDevice

ElementSymbolize

ElementSymbolize

ElementSymbolize

ElementSymbolize

ElementSymbolize

Core(Unity)

Heading(Table)

StructureDiagram

NameAssociate

Element(Unity)

Element(Unity)

Element(Unity)

Element(Unity)

Element(Unity)

leftalignment

spacing2spacing1

spacing

min. line length

AppropriatenessAppropriatenessInteractivityInteractivity

ClarityClarityConsistencyConsistency

STM (Gleicher 95)STM (Gleicher 95)

56

ContributionsContributions

Definition and modeling of visual Definition and modeling of visual discoursediscourse

Methodologies for automated visual Methodologies for automated visual discourse designdiscourse design

Approaches to practical system design Approaches to practical system design Design and implementation of Design and implementation of

IMPROVISEIMPROVISE

57

Future WorkFuture Work

Conversational capabilityConversational capability

Explanation capabilityExplanation capability

Adaptive capabilityAdaptive capability

58

Thank You

Thesis CommitteeMAGIC Group

Project Students & Graphics GroupMike Gleicher

60

Exploring Network EntityExploring Network Entity

(Node1 (is-a NETWORK-NODE)(Node1 (is-a NETWORK-NODE) (type COMPOSITE)(type COMPOSITE) (components [node-ports])(components [node-ports]) (convey [node1-name] [node1-loc])(convey [node1-name] [node1-loc]) (sense SYMBOL) . . .)(sense SYMBOL) . . .)

(Link1 (is-a PHY-CONNECTION)(Link1 (is-a PHY-CONNECTION) (type COMPOSITE)(type COMPOSITE) (components [vpath-segments])(components [vpath-segments]) (sense SYMBOL)(sense SYMBOL) (source [node1])(source [node1]) (destination [node2]) . . .)(destination [node2]) . . .)

61

(Task (Elaborate (link23)))(Task (Elaborate (link23)))

Exploring Network LinkExploring Network Link

62

(Task (Elaborate (link23)))(Task (Elaborate (link23)))


(Elaborate (link23))(Elaborate (link23))

(Expose (link23))(Expose (link23))(Focus (link23))(Focus (link23))

63


Composite Tech: Focus (?objX)Composite Tech: Focus (?objX)

Decomposition2:Decomposition2:Highlight (?objX)Highlight (?objX)

Decomposition1:Decomposition1:Enlarge(?objX)Enlarge(?objX)

Preference:Preference: low-intersection(?objX)low-intersection(?objX)

Decomposition3:Decomposition3:Separate(?objX, ?rest)Separate(?objX, ?rest)

Preference:Preference: reduce-intersection(?objX)reduce-intersection(?objX)

64


Composite Tech: Expose (?objX)Composite Tech: Expose (?objX)

Decomposition2:Decomposition2:CutAway (?objX)CutAway (?objX)

Decomposition1:Decomposition1:Open (?objX)Open (?objX)

Preference:Preference: Openable (?objX)Openable (?objX)

Decomposition3:Decomposition3:SetTransparency (?objX)SetTransparency (?objX)

Preference:Preference: ok-show-all (?objX)ok-show-all (?objX)

65

Focus(link23) Expose (link23)Focus(link23) Expose (link23)

Display (link23-vpathSegs)Display (link23-vpathSegs) Display Decomposition:Display Decomposition:Display (link23-Seg1)Display (link23-Seg1)Display (relation (link23-Seg1, Capacity))Display (relation (link23-Seg1, Capacity)). . .. . .

Causal LinkCausal Link

Separate (link23, network1)Separate (link23, network1)Open(link23)Open(link23)

Scale (link23, node1, node5)Scale (link23, node1, node5)


Separate Decomposition:Separate Decomposition:Align (network1)Align (network1)

Move (link23, node1, node5)Move (link23, node1, node5)

Scale (link23, node1, node5)Scale (link23, node1, node5)

Display Decomposition:Display Decomposition:Display (link23-Seg1)Display (link23-Seg1)Display (relation (link23-Seg1, Capacity))Display (relation (link23-Seg1, Capacity)). . .. . .

Open(link23)Open(link23)

Focus Decomposition3:Focus Decomposition3:

Separate (link23, network1)Separate (link23, network1) Open(link23)Open(link23)

Expose Decomposition1:Expose Decomposition1:

66

Align (network1)Align (network1)

Move (link23, node1, node5)Move (link23, node1, node5)Scale (link23, node1, node5)Scale (link23, node1, node5)Display (link23-Seg1)Display (link23-Seg1). . .. . .Display (relation (link23-Seg1, Capacity))Display (relation (link23-Seg1, Capacity)). . .. . .

Open(link23)Open(link23)

Complete Plan:Complete Plan:


TimeTime

67

Align

Open

Move + Scale + . . .


Documents

Automated Generation of Visual Discourse Michelle X. Zhou Department of Computer Science Columbia University New York, NY 10027 Supported in part by DARPA