Gled a ROOT based framework for distributed computing and dynamic visualization http :// / Matevž Tadel, CERN

Gled

a ROOT based framework for distributed computing and

dynamic visualization

http://www.gled.org/

Matevž Tadel, CERN





CHEP-2006 M. Tadel: GLED - Generic Lightweight Environment for Distributed computing 2

Overview

1. Introduction

2. Example: Visualization of ALICE Distributed Analysis movie

3. Basics of Gled classes & objects

data-model, method execution, example class

4. Distributed-computing model of Gled

5. Conclusion


Gled mission statement

Gled is a framework for management of object collections in a distributed,

multi-user environment.It allows fast development of:1. class libraries for scientific computation &

visualization2. distributed applications for collaborative

object-space access & management3. protocols for intra/inter cluster communication

and data transport4. advanced visualization systems for single and

multi-user applications


What is Gled?

1. C++ development framework based on ROOT uses: serialization, CINT, networking Gled classes are instrumented with:

bindings for creation & execution of RPC requests reference counting & optional reverse-references auto-generated object GUI; rendering via OpenGL

allows easy wrapping of existing code

2. Distributed run-time environment object-collection management

in a hierarchic server-client structure of nodes management of RPC requests

routing/delivery, execution-steering, broadcasting cluster introspection & management


Movie introduction

Visualization of ALICE Distributed Analysisrevival of a demo we have been showing on

SuperComputing-2005 in Seattle

AliEn used as front-end for job control Jobs report monitoring info to MonALISA Gled connects to MonALISA

obtains list of jobs, job details playback of a finished job on-line monitoring of a running job


Comments on movie

Visualization elements: one connection one running process ball exchange state update bars per-site & overall progress [events done/all events]

Gled related: note GL user-interface 3 running threads

1. MonALISA communication & processing2. animation steering (creation of objects, movement)

a) auto-update mode (change driven redraw)b) key-frame capture (time quantized; disk/net output limited)

3. GL rendering (render time 1/400 s)


Comments on Gled workspace

Objects can be shown in: full-view – showing all registered class members for an object object-browser – showing selected members (note layout spec.) rendered with GL – render classes written manually

Notes:1. “MonaLisa Client” object wrapping access to MonALISA

(note full-view: a user can edit/submit queries directly!)

2. “Animator” interface to user-thread that does steering of animation (note thread controls and time variables in full-view)

3. “Main Scene” contains visualization objects4. “Overlay” contains GL interface (can be edited on run-time)

5. “Pupil” GL window representation (note that camera LookAt and UpReference are pointers to Gled objects)

6. context-menu in GL window (note Export to CINT option)


Gled data-model

Base-class of Gled ZGlass : public TObjectidentification in a cluster context, reference counting, RPC bindings

1. data-members: can be basic types, ROOT classes, ...rootcint used all features of ROOT serialization available

2. object aggregation: addressing other Gled objects

relevant for ref-counting, serialization, GUI links smart pointers AList virtual base for container classes

(only requires forward iterator)

3. data manipulation & method execution cluster context must be forwarded for replicated objects multi-threaded execution locking system provided

(GUI and GL run in a separate thread anyway!)


Gled method execution

MIR Method Invocation RequestSaturn Gled node or a processQueen mother and ripper of lenses does MIR sanity-check, authorization, locking

Lens a Gled objectEye a viewer; GUI or other thread notification (Ray) delivered via TCP socket

MIR is just a buffer ZMIR : public TMessage1. Routing header: caller, recipient, request for exit code/result

2. Execution instructions: 1. ID of execution lens & other lens-type arguments

2. specification of library/class/method ID

3. serialized arguments

3. Arbitrary datagram can be used for data transport!

Execution modes: ASync & Sync, detached (in a dedicated thread)

class Cylinder : public ZNode // Example of a Gled class{ void _init();

public: enum Orientation_e { O_X, O_Y, O_Z };

protected: Orientation_e mOrientation; // Xport{GST} 7 PhonyEnum(-join=>1) Float_t mPhiOffset; // Xport{GST} 7 Value(-range=>[-0.5,0.5, 1,1000])

Float_t mHeight; // Xport{GST} 7 Value(-range=>[0,1000,1,1000], -join=>1) Bool_t bRnrDisks; // Xport{gST} 7 Bool() Float_t mRInBase; // Xport{GST} 7 Value(-range=>[0,1000,1,1000], -join=>1) Float_t mRInTop; // Xport{GST} 7 Value(-range=>[0,1000,1,1000]) Float_t mROutBase; // Xport{GST} 7 Value(-range=>[0,1000,1,1000], -join=>1) Float_t mROutTop; // Xport{GST} 7 Value(-range=>[0,1000,1,1000]) ZColor mColor; // Xport{PGST} 7 ColorButt() Int_t mLodH; // Xport{GST} 7 Value(-range=>[1,100,1,1], -join=>1) Int_t mLodPhi; // Xport{GST} 7 Value(-range=>[1,100,1,1])

public: Cylinder(const Text_t* n="Cylinder", const Text_t* t=0) : ZNode(n,t) { _init(); }

#include "Cylinder.h7“ // Include auto-generated member function declarations. ClassDef(Cylinder, 1); // Simple cylinder, possibly hollow.}; // endclass Cylinder

Ctors for PERL objectsthat generate widget code


Example: processing of the class

The header file parsed by project7 (written in perl) creates .h7 and .c7 files (included back into .h, .cxx) creates code for class-GUI (class CylinderView)

GUI elements emit MIRs, which are properly routed in cluster context

GL renderer written manually (class Cylinder_GL_Rnr) render-classes access glass-data and call OpenGL directly

During build three libraries are produced: Core library code + ROOT/Gled glue GUI library auto-generated + custom elements Render library all GL specific code

Loaded separately at run-time (servers, pure computing nodes)

// Auto-generated code, snip from Cylinder.h7:Orientation_e GetOrientation () const { return mOrientation; }

// Auto-generated code, snip from Cylinder.c7:void Cylinder::SetOrientation (Orientation_e orientation){

mOrientation = orientation; mStampReqTring = Stamp(FID());}ZMIR* Cylinder::S_SetOrientation (Orientation_e orientation){

ZMIR* _mir = new ZMIR(mSaturnID);_mir->SetLCM_Ids(1, 167, 1);*_mir << (Int_t)orientation;return _mir;

}void Cylinder::ExecuteMir (ZMIR& mir){

static const Exc_t _eh("Cylinder::ExecuteMir ");static const Exc_t _bad_ctx("ctx argument of wrong type");switch(mir.Mid) {case 1: { Orientation_e orientation; { Int_t _e; mir >> _e; orientation = (Orientation_e)_e; } Cylinder::SetOrientation(orientation); break;}// More code follows ...

}

Gled uses FLTK GUI toolkit. FLTK lives in harmony with ROOT GUI (runs in a separate thread on a different display)

Gled widgets can be resized at run-time (font-size, spacing) Browser can swallow all class-view elements, e.g. ZNode(Pos[16])

: Cylinder(Height[5],RnrDisks[3], RInBase[5],RInTop[5],ROutBase[5],ROutTop[5],Color[4])


Hierarchic server-client model & distributed object-spaces

1. server-space (sun): master copy of objects ZKing object rules over the object space; administrative role ZQueen objects manage sub-spaces and interact with a) each queen can be mirrored independently b) a queen can depend on other queens

2. proxy/client space (moon): mirrored objects3. local/private space (fire): local data, GUI, visualization

Kings & Queens are Gled objects:a) can be introspected with standard Gled GUIb) all object-space management implemented via MIR exchange

S0

S1

S2


Conclusion

Gled – an acronym:Generic Lightweight Environment for

Distributed computing[but truly, Gled is a mountain temple from an ancient legend]

History – in development since 1998 3 people, about 5 man-years

J.J. Javoršek: meta-development, web, documentation

A. Mrak-Tadel: ALICE visualization, animation first public release June 2003 about ½ year release cycle (we’re late right now)

Many thanks to the ROOT team andALICE computing group!


ENDsome additional info follows


Status / Usage

Stable if used correctly thread safety of CINT/ROOT (will be fixed this year!)

Real-life usage – mostly graphics: visualization of ALICE distributed analysis prototype of ALICE Event Visualization Environment

Usage for distributed applications: limited to simple applications and stress-tests

cluster monitoring (n-tupling load-avgs, mem/cpu usage) execution of local commands wrapped in Gled threads numerical demos (ODE integration, stochastic minimization) data-transfer benchmarking

must find a customer to prove it worthy!


GL rendering

Render driver: aware of object aggregation (links can be used

as render state modificators) Gled-object graph represents the

hierarchical scene model scene build and updates are automatic

Capture of images and making movies: Rendering into off-screen buffers Support for key-frame animation

Free-time use: (collaborative)editing of calligraphic symbols

hand drawn

Gled

POV-Ray

Documents

Gled a ROOT based framework for distributed computing and dynamic visualization http :// / Matevž Tadel, CERN