Upload
xiaoyu-wang
View
534
Download
0
Tags:
Embed Size (px)
DESCRIPTION
This is my work on using visual analytic
Citation preview
Designing Visual Analytics Systemsfor Organizational Environments
Xiaoyu WangResearch Associate at UNC Charlotte
Visiting Research Scientist at PARC
A Framework and Its Guidelines
Motivation
Introduction Background Framework Case Study Evaluation Contribution Future Work
GTDVis
OpsVis
Taste
IRSV
Visual Analytics Systems
Evaluations and
Statistical Analysis
Familiarity with
Domain Users
Computation and
Automation
Knowledge Management
and Organizationa
l learning
What’s a systematic approach to design a user-centered visual analytics system in
organizational environment?
Two-stage Visual Analytics Design Framework
Introduction Background Framework Case Study Evaluation Contribution Future Work
Domain General Analysis Process
Individual Analysis Process
Observation and Design
Observation and Analysis
Design Artifacts Specification
User-centric RefinementSystem Deployment and User
Training
Usage Collection and Customization
VisualAnalyticsSystem
Goals: Generalize domain analytical workflows to present high-level problem-solving direction
Construct a design framework to incorporate both general domain analytical process and individual analysis approaches
Bridge the gap between high-level design concepts and fine-grain implementation of such concepts
Augment organizational information analyses through modeling domain users’ reasoning approaches
Two-step Research Progression
• First step:• Summarize design knowledge
learnt from all my previous research activities
• Identify similarities and unique of each analytical domains and system design correspondingly
• Understand the analytical workflows
• Resulted in guidelines this paper
Introduction Background Framework Case Study Evaluation Contribution Future Work
• Second step:• Top-down approach to create
design framework that encapsulate the knowledge gained
• Utilize existing systems for external evidents to verify and validate the framework
• Apply the framework to further design and research practices
• Resulted in A Two-stage Framework for Designing Visual Analytics System in Organizational Environment (to appear in IEEE VAST 2011 )
Collaborators and Settings
Introduction Background Framework Case Study Evaluation Contribution Future Work
• Bridge Management Project• Team: The US Department of Transportation & Civil Engineering
Department• Scope: Research on techniques for innovative bridge maintenance
planning process
• Document Management Project• Team: Palo Alto Research Center & Xerox Corporation• Scope: Research on efficient visual abstraction for recalling and
managing personal document activities
• Network Operation Management Project• Team: Microsoft Research & Microsoft Cloud Service Team• Scope: Research on effective methods for monitoring and responding
to cloud service
Observation and Design stageObjectives: Characterize general domain analytical processes
Identify design artifacts for visual analytics implementation
Introduction Background Guidelines Evaluation Contribution Future Work
Visual Analytics SystemSummative Evaluation
Ob
serv
atio
n a
nd
Des
ign
Sta
ge
Analytics Requirements
User Analysis Task Analysis Context Analysis
Data Requirements
Data Analysis Process Analysis
Domain Characterization and Analysis Generalization
Domain Observation and
Analysis Formative Evaluation
Evaluation Metrics(key specifications for assessing the system)
Analysis Encapsulation and Visual Encoding
Formative Evaluation
Domain Analysis Dissemination
Interaction Specification
Visualization Specification
Alternative Visualization/Interaction Combinations
Design Artifacts Specification
Fa
il
Observation and Analysis
Objective: Domain Characterization and Analysis Generalization
Introduction Background Framework Case Study Evaluation Contribution Future Work
Analytics Requirements
User Analysis Task Analysis Context Analysis
Data Requirements
Data Analysis Process Analysis
Formative EvaluationEvaluation Metrics
• Analyze organizations for their technical, analytical, and collaborative context where the visual analytics system will be applied to
Context Analysis
• Specify the tasks and analytical workflow in an organizational environment.
• Verify the design specifications and reduce design costs
Task Analysis
• Domain users’ information (e.g. Demographics, Personal Traits)• Distinguish users broadly by expertise, task experiences, usage
constrains
User Analysis
• Specify the expected analysis goals from the domain users for the designed visual analytics systems
• List key specifications for assessing the system (details in the paper)
Evaluation Metrics
Example---U.S. DOT: Domain Characterization
Introduction Background Framework Case Study Evaluation Contribution Future Work
Analytical Workflow and its
Actionable Knowledge
Methodology: Survey: Nationwide state DOTs
Observations: North Carolina and Charlotte DOT
Gather inspection information
Select bridges within jurisdiction
Analyze collected data
Compare it with prior inspection cycles
Consulting with structural specialist for maintenance necessities
Collaborate with colleagues to balance budgets Prioritize maintenance plan
Prepare maintenance proposal
Submit for final approval
Follow-up work on the execution of the maintenance
Update existing database
Domain Analysis GeneralizationBridge the gap between high-level design concepts and fine-grain implementation of such concepts
Introduction Background Framework Case Study Evaluation Contribution Future Work
Design Artifacts and SpecificationObjectives: Analysis Encapsulation and Visual Encoding
Disseminate high-level task activities into actionable knowledge
Transform actionable knowledge into visual encoding
Introduction Background Framework Case Study Evaluation Contribution Future Work
Formative Evaluation
Domain Analysis Dissemination and Transformation
Interaction SpecificationVisualization Specification
Alternative Visualization/Interaction Combinations
Visual Analytics System
• Specifies the pragmatic view of knowledge utilization and application towards specific analytical ends
• Details the relations between domain analytical tasks and the related knowledge actions• Presents the analytics activities from domain users’ perspectives• Is widely accepted in organizational learning and practices• It IS the design artifacts that represents the fine-grain domain analysis processes
Actionable Knowledge Personalized content and information Easy ‘slice and dice’ information and direct content exploration Examine and depict information from multiple aspects Make sense of significant data patterns and trends
Create hypothesis based on analysis Identify evidence that supports both thesis and antithesis Depict information from multiple aspects Annotate evidence with clear statements Group evidence with reasoning logic
Content Filtering and Customization
Evidence Collection and Hypothesis
Generation
Common Task Activities Key Actionable Knowledge Personalized content and information Easy ‘slice and dice’ information and direct content
exploration Examine and depict information from multiple aspects Make sense of significant data patterns and trends
Create hypothesis based on analysis Identify evidence that supports both thesis and
antithesis Depict information from multiple aspects Annotate evidence with clear statements Group evidence with reasoning logic
Key Actionable Knowledge
Deliver contents in straightforward representation Enable facet filtering for information personalization Interactive content exploration and filtering (Optional) Employ sophisticated data structures
Allow evidence collection and annotation Support storytelling and enable interactive grouping of the
evidence with users’ reasoning logic (Optional) Trace interactions and system usage for future
automation
Visualization and Interaction Specifications
Summary of Designing VA for General Analysis
Introduction Background Framework Case Study Evaluation Contribution Future Work
Domain General Analysis Process
Observation and Design
Observation and Analysis
Design Artifacts Specification
VisualAnalyticsSystem
User-centric RefinementObjectives: Provide a “feedback” loop to incorporate individual analysis routines
and to customize (personalize) visual analytics system
Deploy visual analytics systems and provide trainings to domain users
Enable organizational communication and collaboration
Introduction Background Framework Case Study Evaluation Contribution Future Work
Summative Evaluation
Visual Analytics System
Domain Analysis Workflow Data Infrastructure Visualization
CombinationInteraction
Combination
Usage Collection (both individual level and organizational level)
Interaction Logging Annotation Tracking
Refine Analysis Focuses
Update Data Model
Customize Visualization Combination
Analysis Evaluation and Knowledge Validation
Usage Pattern Analysis and Customization
User-centric Refinement stage II
Pa
ss
DocumentationSupport
Installation
System Deployment and User Training
Training
General Visualization Concepts Analysis Scenarios
Usage Pattern and Customization StepObjectives: Support individual tasks routines and analysis preference
Enable individual’s to collect analytical findings and their analysis provenance
Establish organizational collaborations and facilitate collective decision-making
Introduction Background Framework Case Study Evaluation Contribution Future Work
Visual Analytics System
Domain Analysis Workflow Data Infrastructure Visualization Combination Interaction Combination
Usage Collection (both individual level and organizational level)
Interaction Logging
Refine Analysis Focuses
Update Data ModelCustomize Visualization
Combination
Analysis Evaluation and Knowledge ValidationSystem
Deployment and User Training
Interaction Logging and Capturing User’s Analysis Provenance
Objective: Reveal the relationship between problem solving and interactions
Represent the analysis trail for domain users
Indicate domain users’ analytics preferences
Introduction Background Framework Case Study Evaluation Contribution Future Work
Empirical Proof*: Suggest the clear connection between interactions and the type of strategies users tend to develop
* Empirical study can be found in Dou et al. (2010) : “Comparing different levels of interaction constraints for deriving visual problem isomorphs”
How-to: Log Focus Log Elements
Tracing details of analysis sessions
Low level event (e.g. MouseClick, Key Stroke)
Replay key analysis frames Visual States (e.g. visualization parameters)
Reconstructing user's analysis process
Low-level events and Contextual information and etc.
Interaction Logging ExampleIntroduction Background Framework Case Study Evaluation Contribution Future Work
Log Element:
Visualization Parameters
Data Parameters
Frequencies of views used
Dwell time on each visualization
Analytical Sessions
Utilization of Interaction Log
Example
Visualization Combinations
System places more frequent used visualization combination for individual user based on analyzing the logged frequency and dwell time
Visual Mappings Update
System records the most frequently used visual encoding and its data operators. System makes suggestions if the pattern is repeated
Data and Statistics Model Update
System rearranges the priority of the data based on its usage frequency. System also connects the data with its related statistics for future analysis suggestions
Usage Pattern and Customization StepObjectives: Support individual tasks routines and analysis preference
Enable individuals to collect analytical findings and reveal their analysis provenance
Establish organizational collaborations and facilitate collective decision-making
Introduction Background Framework Case Study Evaluation Contribution Future Work
Visual Analytics System
Domain Analysis Workflow Data Infrastructure Visualization Combination Interaction Combination
Usage Collection (both individual level and organizational level)
Interaction Logging
Refine Analysis Focuses
Update Data ModelCustomize Visualization
Combination
Analysis Evaluation and Knowledge ValidationSystem
Deployment and User Training
Annotation Tracking and Content Sharing
Objective: Enable users to attach semantic meaning to analysis findings
Share analysis findings and results in an organization
Create environment to encourage collective decision-making
Introduction Background Framework Case Study Evaluation Contribution Future Work
How-to:Sharing Mechanism
Content Shared Efficiency Effectiveness Information Sharing Flow
Sharing static annotations
Fixed Image;Textual Information;Drawing;
Easy to construct Can be add-on to existing visual analytics system
More effective in a small-to-medium-sized group
Typically one-way. Information comes from original analyst and shared with other colleagues
Exchanging dynamic annotation
Parameters that can be applied to in another instance of the visual analytics system
Need to be considered at the initial system design. Could be difficult to add onto a existing system
Support larger collaboration groups or departments
Bi-direction, both original analysts and peers can collectively modify and extend the analysis results
Annotation Example: DOT Web
Introduction Background Framework Case Study Evaluation Contribution Future Work
Multiple Evidence
Collections
Freeform Selection and
Graph Connection
Detailed Annotation
Instant Sharing with Colleagues
Summary of Designing VA for Individual Analysis processes
Introduction Background Framework Case Study Evaluation Contribution Future Work
Individual Analysis Process
User-centric Refinement
System Deployment and User Training
Usage Collection and Customization
VisualAnalyticsSystem
Contributions• Constructed a two-stage visual analytics design
framework to incorporate both general domain analytical process and individual analysis approaches
• Generalize domain analytical workflows to present high-level problem-solving direction
• Bridge the gap between high-level design concepts and fine-grain implementation of such concepts
• Augment organizational information analyses through modeling domain users’ reasoning approaches
Introduction Background Framework Case Study Evaluation Contribution Future Work
Impacts
Academia Presented a more theoretical approach to design visual analytics system
Encourage academic research for the foundation of visual analytics design
Educational purpose, a syllabus for graduate school teaching material
Industry Concrete and practical guidelines and considerations for designing visual analytics system
Present general ground to bridge research and industry on design and development
Introduction Background Framework Case Study Evaluation Contribution Future Work
Future Work• Continue working interactive learning from domain users’
interaction logs• Machine learning• Reactive (emotion) visualization
• Contribute to the evaluation foundation of visual analytics• Create standard evaluation metrics• Identify key measures for assessing knowledge-gain through using
visual analytics
Introduction Background Framework Case Study Evaluation Contribution Future Work
Questions
Charlotte Visualization Center
http://webpages.uncc.edu/~xwang25
Xiaoyu Wang
Probably on Skype Now..
Case: Design Artifacts and Specification
Introduction Background Framework Case Study Evaluation Contribution Future Work
Summary of Observation and Design stage• Domain observation and Analysis
• Generalization of Domain Analysis Processes• Elements needs to be considered during observation and domain
characterization• Evaluation Metrics that are useful throughout the design as an
assessment to the function
• Design artifacts• Actionable knowledge is a fine-grain items to analytically examine
the domain’s general analytical workflow• Disseminate general task activities into design artifacts through
actionable knowledge• Design considerations that are generated based on design
artifacts.• Visual analytics design needs to follow these artifacts
Introduction Background Framework Case Study Evaluation Contribution Future Work
Example---Xerox: Domain Characterization
Introduction Background Framework Case Study Evaluation Contribution Future Work
Analytical Workflow
Methodology: Semi-structured interview: 30 Knowledge workers
Observations and Interactions: Groups of Xerox employees
Challenges: The date and time when events occurred
(e.g., when documents were received, read, created, or modified);
Content keywords that users associate with activities
(e.g. the title of a document or the name of a person or company);
Document types or applications are used to perform a particular type of activity
(e.g., Microsoft Excel or Apple Mail)