Piccolo.NET: A Scalable Structured Graphics Toolkit

Piccolo.NET: A Scalable Structured Graphics Toolkit

Ben BedersonUniversity of Maryland

Human-Computer Interaction LabComputer Science Department

University of Maryland

Human-Computer Interaction LabPiccolo.NET

Piccolo.NET: What is it?

Toolkit that supports: structured canvas of graphical objects hierarchical scenegraph model creation of “controls” for use within Windows Forms

Supports 2D object-oriented graphics hierarchies (transformation, transparency, etc.) animation, event handling cameras, layers, views efficiency mechanisms

Missing structure that relies on underlying renderer GDI+, Direct3D, OpenGL, Java2D, etc.

=> Open, Extensible and Efficient



What Is It Good For?

=> Structured graphics Custom interaction Zoomable User Interfaces

Gentleware.com - Poseidon

PaperLens – Microsoft Research

Demo




Structured graphics => Custom interaction Zoomable User Interfaces

Range Slider

International Children’s Digital Librarywww.icdlbooks.org

This Presentation Tool

Demo




Structured graphics Custom interaction => Zoomable User Interfaces

PhotoMesawww.photomesa.com

U. Victoria – Ontology Visualization

Demo



Where Does It Run?

Almost everywhere Piccolo.NET

(C#) .NET for Windows PocketPiccolo.NET

(C#) Compact Framework for Pocket PC Piccolo.Java

(Java) for everywhere else



How Does It Work?

“Monolithic” design Simple class hierarchy Simple runtime structure Utilities for easy startup

PNode

PRoot PLayer PCamera

PText

PPath

PImage

piccolo piccolo.nodes

1.n

Class Hierarchy

PRoot

PLayer PCamera

PNode PImage

PText PPath

Typical run-time structure

using UMD.HCIL.Piccolo;using UMD.HCIL.Piccolo.Nodes;using UMD.HCIL.PiccoloX;

public class PHelloWorld : PForm { public override void Initialize() { PText text = new PText("Hello World!"); Canvas.Layer.AddChild(text); } static void Main() { new PHelloWorld(); }}

Code



What Does It Do?

Let’s look at the “Feature” demos:

Demo



Bigger Example: GraphEditor

Nodes and Edges Highlightable Draggable

Approach: Structure of objects Relate nodes & edges Event handlers

PRoot

PCameraPLayer

PNode PNode

PPathPPath

PPathPPath

PPath

PPathPPath

PPathPPath

PPath

Edges Nodes

Code



This Presentation Tool

Similar structure to GraphEditor: Scenegraph Event handlers

But this time: Hierarchical transforms Animation

PRoot

PCameraPLayer

PNode

Slides

Slide Bar

PMultiSizeImagePMultiSizeImage

PMultiSizeImagePMultiSizeImage

PMultiSizeImage

PImage

Current Slide Indicator



PocketPiccolo.NET

Same great taste – less filling! Essentially identical except for:

Lower quality rendering No graphical paths or transparency No rotation Poorer performance

Still all the cool stuff: Hierarchical, transformable objects Animation Event handling Efficiency mechanisms

Demo



History Lesson – ZUI ToolkitsSpatial Data Management System (SDMS): Donelson, MIT

1978 – SDMS Tour de force

Ubiquitous computing ZUIs Novel input devices

Totally specialized system



History Lesson – ZUI ToolkitsPad: Perlin, NYU

1993 – Pad Ran on Sun 2 Written in C X Graphics

Jump-zoom, B&W Impressively fast Moderately extensible



History Lesson – ZUI ToolkitsPad++: Bederson, UNM

1994 – Pad++ Designed for prototyping Used C++ and Tcl/Tk and

X or OpenGL graphics

Smooth zooming Full color Bitmaps and vector objects

Didn’t scale up well The API was defined in Tcl and the C++ code was

efficient, but messy…[Bederson & Meyer - SPE 1998]



History Lesson – ZUI ToolkitsJazz: Bederson, UMD

1998 – Jazz Wanted ability to build “serious” applications Moved to Java and object-oriented design Inspired by 3D graphics:

We built “polylithic” scenegraph Different than “monolithic” GUI toolkits

Node

Rectangle

Node

FadeRectangle

FadeRectangle

MonolithicPolylithic

Root

FadeRectangle

Monolithic

FadeRectangle

FadeRectangle ...

Root

Polylithic

Fade FadeFade ...

Rectangle Rectangle Rectangle...



Polylithic Potential

Simpler objects, easier to maintain Decoupled objects, easier to extend More run-time control

Could better support design environments

But … More objects to manage Introduced “editor” to manage object chains Still not good enough for app programmers



History Lesson – ZUI ToolkitsPiccolo: Bederson, UMD

2003 – Piccolo Decided app builder more important than toolkit builder Added support for C# and Pocket PC Went back to “monolithic”, but OO design

Finally happy This is our “last” toolkit Now have commercial apps being built using Piccolo



How Do Toolkits Compare?

Tough comparison – many design differences aside from polylithic / monolithic architectures

Did case studies and a performance analysis

Custom Piccolo Jazz Scene render time 1.5 msec 2.1 msec 2.2 msec Lines of Code 272 lines 171 lines 219 lines Class file size 10.4 kbytes 10.4 kbytes 13.2 kbytes Memory usage our code (full application)

7.5 k (484k) 8.0 k (516k) 10.2 k (535k)

DateLens Mockup

Piccolo.dll [120 kb] PiccoloX.dll [68 kb]

DateLenswww.datelens.com



Task Custom Piccolo Scene graph Overhead

Jazz Scene graph Overhead

10,000 rectangles 265.0 msec 270.3 msec 2 % 282.8 msec 7 % 1,000 groups of 10 rectangles

273.4 msec 3 % 281.2 msec 6 %

100 groups of 10 groups of 10 rectangles

267.2 msec 1 % 281.3 msec 6 %

10 groups of 10 groups of 10 groups of 10 rectangles

270.4 msec 2 %

278.1 msec 5 %

Performance Analysis

Task Piccolo Jazz 10,000 rectangles Build 10,000 nodes 16.0 msec 219.0 msec Translate 10,000 nodes 0.4 msec 23.5 msec Remove 10,000 nodes 5.3 msec 5.3 msec 1,000 x 10 rects Build 10,000 nodes 16.0 msec 218.0 msec Translate 10,000 nodes 0.4 msec 50.8 msec Remove 10,000 nodes 5.3 msec 5.3 msec 100 x 10 x 10 rects Build 10,000 nodes 15.0 msec 226.5 msec Translate 10,000 nodes 0.4 msec 62.5 msec Remove 10,000 nodes 5.0 msec 10.6 msec 10 x 10 x 10 x 10 rects Build 10,000 nodes 16.0 msec 226.5 msec Translate 10,000 nodes 0.4 msec 82.3 msec Remove 10,000 nodes 5.0 msec 10.3 msec

Rendering Speed

Scenegraph manipulation speed

[Bederson, Grosjean, Meyer - TSE 2004]



Architectural Reflections

Toolkits are usually worth their weight Polylithic and Monolithic each have merits

Similar performance and code sizes Base arch. on expected life cycle

More programmer oriented => monolithic More design program oriented => polylithic



Language Reflections: Java vs. C#

Languages C# “usability” advances important for us:

properties, events, indexers, foreach, using, attributes Libraries

Similar, but multiplatform has significant cost Platform-specific feature use common Java tortured its APIs with backwards compatibility

IDEs Eclipse: Free, refactoring support, pluggable, rapid changes VS.NET: Expensive, but fast, form designer, integrated

device emulator, and cheap for academics Platforms

Multiplatform required for UMD’s required CS courses But C# is used in some senior/grad courses

Demo



Conclusions / Directions

Are toolkits beneficial? => Clearly yes (sometimes)

Does zooming work? Is animation helpful?

Good small representations needed Animation helps maintain object constancy Understand domain and users

Piccolo’s Future: Finish website, tutorials, etc. Hook up to other renderers (Direct3D, OpenGL) Build more apps!

www.cs.umd.edu/hcil/piccolo

Funded by: Microsoft

Initially by: DARPA NSF

Documents

Piccolo.NET: A Scalable Structured Graphics Toolkit