High Resolution Displays at PPPL

High Resolution Displays at PPPL

Mike [email protected]

[email protected]

Topics

• Single Machine Applications– PowerPoint Presentations

– PSpice / ExpressPCB• Stepper Motor Control

Circuit

– AVS Express

• Limitations

• Parallel Visualization– OpenGL concepts

– Systems currently in use:• (Custom solutions)

• WireGL

• Chromium

– How to execute

– Performance

Single Machine Applications

• Hardware/Software Required– Pentium 4 1.5GHz

– 1 GB Ram

– 40 GB Hard Disk

– 3 * Matrox G200MMS Quad• $699 each

– WinNT Required• Matrox Virtual Desktop Hack

• Limitations– PCI Bus

• 33 MHz * 32bit = 132 MB/sec• Total for all devices

– Image Size• 4096 x 2304 = 9 megapixels• 18 MB/image @ 16bit

– 65,536 colors– 7.33fps max

• 27 MB/image @ 24bit– 16,777,216 colors (standard)– 4.89fps max

Implementing Parallel Visualization

• Hardware– Dual P4 / 1.5GHz

– Myrinet Networking• 1280 megabits / second

• 13 s latency– mostly consistent

• Uses GM, not TCP/IP

– 100 Mb switch• 100 megabits/second

• 30 s latency– varies by size

– NVIDIA Quadro 2 Pro• 1024 x 768 @ 60Hz

– Total Resolution: 4096x2304

• Software– Debian 2.2r4 ‘testing’

– GM 1.5-pre2b

– vncwall 1.0

– WireGL 1.2.1

– Chromium (nightly CVS)

OpenGL Overview

• High level graphics API• Not focused on low level

(pixel operations), but it’s possible

• The user defines geometric primitives

• All primitives have properties associated with them

• Examples:– Primatives

• Pixels

• Vertices– Create triangles

– Properties• Material

• Lighting

http://www.opengl.org/

Example: Atlantis

• DefinitionglBegin(GL_POLYGON);

glVertex3fv(P001);

glVertex3fv(P002);

glVertex3fv(P003);

glNormal3fv(N01);

glEnd(GL_POLYGON);

…

• RenderingglPushMatrix(MASK) /

glPopMatrix(MASK)glRotatef(…)glLightfv(…)glMaterialfv(…)glEnable(MASK)etc…

How do we Distribute the Display?

• Customized Solution– Used by main campus– Custom Graphics Apps

• Each application is rewritten to facilitate selective rendering

• Time Consuming• Impossible for closed source

software

– Virtual Display Driver• Buggy, hard to implement in

Windows

• Intercept Solutions– vncwall

• Virtual Network Computing

– WireGL• Intercepts OpenGL calls

– Chromium• WireGL code base, many

additions

VNC over a Display Wall

• Standard VNC– Virtual Network Computing

– Developed by AT&T

– Allows remote execution & display of applications

– Similar to:ssh <remote>, export DISPLAY=<local>

• Distributed VNC– Developed by NCSA

– Allows the viewer to run on multiple machines / only display certain tiles

– Relatively efficient

– Does not implement GLX

http://www.ncsa.uiuc.edu/TechFocus/Deployment/DBox/

WireGL

• Replaces OpenGL libraries– opengl32.dll

(Windows)– libGL.so (Linux / IRIX)

• wgl helper application– Copies files to

/tmp/wglXXXXX– Executes code in above dir.

• pipeserver– Runs on the “servers”

• rey=client, paredXX=servers

• How it works– libwiregl_client.so intercepts

an OpenGL call

– Encodes using a custom packet

– Sends it to the appropriate machine

– Pipeserver converts that WireGL code back to an OpenGL call

Normal OpenGL vs WireGL

Program

libGL.so

OpenGL

Video Card

GLX/DRI

Monitor

Analog / Digital Video Out

Program

libwiregl_client.so

OpenGL

Normal Program WireGLed Program

Myrinet (Using GM)

Machine 1 Machine 2

Machine 3 Machine 4

Machine 5 Machine 6

pipeserver

libGL.so

Video Card

Projector

OpenGL

GLX/DRI

Analog Out

GLX vs. WireGL

• GLX– Designed to interface

OpenGL into the X protocol

– Minimum state tracking– Not designed for high

speed operation– Often replaced by DRI for

desktop applications

• WireGL– Designed for efficient state

tracking– Especially good with

Vertex Arrays• Allows OpenGL to retrieve

information directly from the application’s memory

• Thus, the state tracker can fit it in here.

Chromium

• What’s an SPU?• Common SPUs

– Pack SPU• Send commands directly

– Tilesort SPU• Distributes “Tiles,” (in our case,

12)

– Render SPU• Direct Output

– Print SPU • Logging

Program

libcrutil_client.so

OpenGL crserver

libGL.so

Video Card

Projector

OpenGL

GLX/DRI

Analog Out

SPU #1

SPU #2

SPU #n

http://sourceforge.net/projects/chromium/

Distributed Visualization

• General– Many graphics commands

must be run in a set order

– glFinish() might be too much, since we don’t need actual execution

– Use separate contexts to avoid state conflicts

• WireGL– Supported, but not documented

• Chromium– Creates multiple threads, each

aware of the other– Use “Barriers” to maintain

order– Barriers only affect graphics

rendering, not program execution

Access Grids: Extending the Computational Grid

Components of an AG NodeComponents of an AG Node

Digital Video

Digital Video

Digital Audio

NETWORK

MixerControl

Computer

NTSC Video

RGB Video

Analog Audio

Video Capture

Computer

DisplayComputer

AudioCapture

Computer

EchoCanceller

•Group-to-group interactions are different from and more complex than individual-to-individual interactions.

•Large-scale scientific and technical collaborations often involve multiple teams working together.

• The Access Grid concept complements and extends the concept of the Computational Grid.

• The Access Grid project aims at exploring and supporting this more complex set of requirements and functions.

•An Access Grid node involves 3-20 people per site.

•Access Grid nodes are “designed spaces” that support the high-end audio/video technology needed to provide a compelling and productive user experience.

•The Access Grid consists of large-format multimedia display, presentation, and interaction software environments; interfaces to grid middleware; and interfaces to remote visualization environments.

• With these resources, the Access Grid supports large-scale distributed meetings, collaborative teamwork sessions, seminars, lectures, tutorials, and training.

Over 70 AG sites (PPPL Over 70 AG sites (PPPL will be next!)will be next!)

AG: Providing New Capabilities

• Capabilities will include– high-quality multichannel digital video and audio,

– prototypic large-format display

– integrated presentation technologies (PowerPoint slides, mpeg movies, shared OpenGL windows),

– prototypic recording capabilities

– integration with Globus for basic services (directories, security, network resource management),

– macroscreen management

– integration of local desktops into the Grid

– multiple session capability

How to execute: VNC

• Rey (server)

– Start a vncserver• Similar to an X server with no

display• vncserver –geometry 4096x2304

– Run vncwall (Java)• Automatically launches

tileviewer via vncd

• Pared Cluster (clients)– Run vncd

– Let vncwall (on server) automatically launch tileviewer

How to execute: WireGL

• Rey (client)

– wgl <commandname>

• Pared Cluster (servers)– Start pipeserver

• Use a repeating script

while (1)

pipeserver -f

end

– Wait for WireGL packets

How to execute: Chromium

• Rey (client)– python2 <conffile> <args>

– crappfaker

• Pared Cluster (servers)

– crserver

Implementation Issues

• Projector Alignment– 72 degrees of freedom

– Manual Alignment

– Automated Alignment• Software based

– Image warping

• Hardware based– Stepper Motor Project

• Projector Discrepencies– Brightness / Contrast

• Lamp Age

– Color balance

• Trapezoidal Projections– Screen curvature

Performance

Source: http://graphics.stanford.edu/papers/clust_render/clust_render.pdf

4003 dataset = approximately 1.5 million triangles

102 patches @ 1800 triangles each = 183,600 triangles

<5000 triangles– large polygons, large textures

References

• OpenGL– http://www.opengl.org/

• WireGL– http://graphics.stanford.edu/software/wiregl/

• Chromium– http://sourceforge.net/projects/chromium/

• DisplayWall-in-a-Box– http://www.ncsa.uiuc.edu/TechFocus/Deployment/DBox/

• Princeton Main Campus Wall– http://www.cs.princeton.edu/omnimedia/

Documents

High Resolution Displays at PPPL