Metisse is not a 3D desktop!

Olivier Chapuis, Nicolas RousselProjet In Situ (CNRS, Universite Paris-Sud & INRIA Futurs)

LRI, Batiment 490, Universite Paris-Sud91405 Orsay Cedex, France

chapuis| roussel @lri.fr

ABSTRACTTwenty years after the general adoption of overlapping win-dows and the desktop metaphor, modern window systemsdiffer mainly in minor details such as window decorations ormouse and keyboard bindings. While a number of innovativewindow management techniques have been proposed, few ofthem have been evaluated and fewer have made their wayinto real systems. We believe that one reason for this is thatmost of the proposed techniques have been designed usinga low fidelity approach and were never made properly avail-able. In this paper, we present Metisse, a fully functionalwindow system specifically created to facilitate the design,the implementation and the evaluation of innovative windowmanagement techniques. We describe the architecture of thesystem, some of its implementation details and present sev-eral examples that illustrate its potential.

Categories and Subject Descriptors:D.4.9 [Systems Pro-grams and Utilities]: Window managers. H.5.2 [User Inter-faces]: Graphical user interfaces, Screen design, Windowingsystems.

Additional Keywords and Phrases:application redirection,OpenGL, X Window system, VNC.

INTRODUCTIONOverlapping windows that users can freely move and resizehave been described more than thirty years ago and havebeen available to the general public for more than twentyyears [14]. Over the time, various interaction techniqueshave been proposed to control the placement, size and ap-pearance of application windows. Yet, from a user perspec-tive, the most popular window systems differ mainly in minordetails such as window decorations or mouse and keyboardbindings, and not in their fundamental operation principles.As B. Myers already put it in 1988, “there is not a great dealof difference among different window managers” [13].

The growing range of activities supported by interactive com-puter applications makes it more and more difficult to re-member these activities and to organize them. At the sametime, recent advances in computer graphics and display tech-

nologies combined with decreasing costs are changing thenature of the problem. High performance graphics cards,high definition displays, big screens and multiple monitorsystems are becoming common place. From the original timewhen window systems were too demanding and had to becarefully tuned for performance, we have now moved to asituation where a lot of software and hardware resources areavailable. The question is: how should these resources beused?

Little research has been performed on understanding peo-ple’s space management practices [9]. While a number ofinnovative window management techniques have been pro-posed by HCI researchers over the last few years [23], veryfew of these techniques have been formally evaluated andeven fewer, if any, have made their way into current windowsystems. We believe that these two points are strongly relatedto the fact that most of the techniques proposed by the HCIcommunity were designed using a low fidelity approach andwere never made properly available in a real window system.

Building a whole new window system is a hard task, one thatfew HCI researchers are willing to do. At the same time, ex-isting systems are either closed boxes, inaccessible to devel-opers, too limited for the envisioned interaction techniquesor too complex to program. How would you implement azoomable window manager? One that would strengthen thepaper and desktop metaphor? One that can be used on atabletop display? One that would support bi-manual inter-action?

In this paper, we present Metisse, a fully functional windowsystem specifically created to facilitate the design, the imple-mentation and the evaluation of innovative window manage-ment techniques. The paper is organized as follows. Afterintroducing some related work, we describe Metisse by pro-viding an overview of its design and architecture as well assome implementation details. We then present several ex-amples that illustrate its potential for exploring new windowmanagement techniques. Finally, we conclude with a discus-sion and some directions for future research.

RELATED WORK

In this section, we briefly describe the current state of thethree most popular window systems as well as several re-search projects related to the exploration of new windowmanagement techniques.

Apple Mac OS X, Microsoft Windows and X WindowThe Apple Mac OS X graphics system is based on three dif-ferent libraries : Quartz for 2D graphics (a rendering enginebased on the PDF drawing model), OpenGL for 3D graphicsand QuickTime for animated graphics and video. Window-ing services are available through a software calledQuartzCompositor[1]. This software handles the compositing of allvisible content on the user’s desktop: Quartz, OpenGL andQuickTime graphics are rendered into off-screen buffers thatthe compositor uses as textures to create the actual on-screendisplay.

Among other features, Quartz Compositor supports windowtransparency, drop shadows and animated window transfor-mations, which are used to create various effects such as thescaleandgenieeffects used for window (de)iconification, thefast user switchinganimation and the threeExpose modes.From a developer perspective, however, Quartz Composi-tor is a closed box. Most of its functionalities are availablethrough a private, undocumented and probably unstable APIthat only a few highly-motivated developers are willing touse1. Gadget applications using this private API are inter-esting because they show that the compositor is much morepowerful than it seems and that services such asExpose arein fact the result of a careful selection of its features. At thesame time, this is very frustrating since this compositing pol-icy and the associated design space remain out of reach forthe HCI researcher.

The window system of Microsoft Windows is tightly coupledwith the operating system, which makes it difficult to accessand modify. Several applications such as SphereXP2 allow toreplace the traditional desktop by a 3D space in which arbi-trary objects can be painted with 2D images from applicationwindows. However, the implementation details of these sys-tems are not available. The next version of Microsoft Win-dows will most probably include a composite desktop basedon DirectX 9 [3]. Details of what will be available to usersand developers remain uncertain. However, one can reason-ably imagine that the compositing policy will probably beout of reach for the average developer and HCI researchers.

A key feature of the X Window System [19] (or X) is thatany application can act as a window manager. As a con-sequence, a large number of window managers have beendeveloped for this system, providing a range of appearancesand behaviors. Recent X extensions make it now possible forthese window managers to use a compositing approach [8]:Composite, that allows windows to be rendered off-screenand accessed as images;Damage, that allows an applicationto be notified when window regions are updated; andEventInterception, that allows keyboard and mouse events to bepre-processed before being sent to their usual targets. An-other extension,Xfixes, provides the data types and functionsrequired by these extensions.

Experimental X window managers are slowly taking advan-tage of these extensions to provide eye candy effects similarto the ones proposed by Mac OS X. However, the numerous

1http://cocoadev.com/index.pl?WildWindows2http://www.hamar.sk/sphere/

extensions required make it hard for developers unfamiliarwith the X architecture to implement their own compositingwindow manager. Moreover, implementing a fully functionaland standard-compliant X window manager requires muchmore than simple window image compositing and event pre-processing.

Window management researchMany of the window management solutions proposed by theHCI research community have been designed using a low-fidelity approach and have never been implemented as partof a real window system.Elastic windows[12], for example,were only implemented within custom applications.Peeledback windowshave been demonstrated within specific Tcl/Tkand Java prototypes [2, 7]. Window shrinking operationsand dynamic space management techniques have also beendemonstrated within specific Java prototypes [10, 4].

A notable exception to the low-fidelity approach is Microsoft’sTask Gallery [17], a system that uses input and output redi-rection mechanisms for hosting existing Windows applica-tions in a 3D workspace. The redirection mechanisms re-quire several modifications of the standard window man-ager of Windows 2000. They provide off-screen renderingand event pre-processing facilities similar to those becomingavailable in X [24]. However, as the Windows 2000 modi-fications have never been publicly released, researchers out-side Microsoft have not been able to experiment with thishigh-fidelity approach.

Several recent projects have tried to move from low-fidelityprototypes to real functional systems. Scalable Fabric [16],mudibo [11] and WinCuts [22], for example, are imple-mented as “real” applications supplementing the legacy win-dow manager of Windows XP. However the fact that thesesystems are developed outside the window system makesthem unnecessary complex, potentially inefficient and harderto combine with other window or task management tech-niques. As an example, since they can’t be notified of win-dow content updates, the three mentioned systems resort toperiodically calling a slowPrintWindowfunction to get win-dow images, which is both inefficient and unsuitable for in-teractive manipulation of the window content.

Experimental desktop environmentsAmetista [18] is a mini-toolkit designed to facilitate the ex-ploration of new window management techniques. It sup-ports the creation of new OpenGL-based desktop environ-ments using both a low-fidelity approach, using placehold-ers, as well as a high-fidelity approach based on X appli-cation redirection through a custom implementation of theVNC [15] protocol. Several 3D environments such as Sun’sLooking Glass3 and Croquet [20] are also using the X exten-sions we already mentioned to host existing applications.

The main problem of these new environments is that al-though they provide the fundamental mechanisms for im-plementing compositing window managers, they implementonly parts of the standard X protocols related to windowmanagement (e.g. ICCCM, EWMH) that define the interac-

3http://wwws.sun.com/software/looking_glass/

tions between window managers, applications, and the vari-ous utilities that constitute traditional desktop environmentssuch as GNOME or KDE. As a consequence, these envi-ronments are hardly usable on a daily basis, since they donot support common applications such as mail readers, Webbrowsers, media players or productivity tools.

METISSEMetisse is an X-based window system designed with twogoals in mind. First, it should make it easy for HCI re-searchers to design and implement innovative window man-agement techniques. Second, it should conform to existingstandards and be robust and efficient enough to be used on adaily basis, making it a suitable platform for the evaluationof the proposed techniques. Metisse is not focused on a par-ticular kind of interaction (e.g. 3D) and should not be seenas a new desktop proposal. It is rather a tool for creating newdesktops.

The design of Metisse follows the compositing approach andmakes a clear distinction between the rendering and the inter-active compositing process. TheMetisse serveris a modifiedX server that can render application windows off-screen. Thedefault compositor is a combination of a slightly modifiedversion of a standard X window manager,FVWM, with aninteractive viewer calledFvwmCompositor. As we will see,the use of FVWM provides a lot more flexibility and reliabil-ity than custom-made window managers such as those usedby Ametista or Looking Glass. In the next section, we willalso show that other compositors can be used in conjunctionwith the Metisse server.

Metisse is implemented in C and C++ and runs on the Linuxand Mac OS X platforms. Figure 1 shows the communica-tion links between the various software components. TheMetisse server sends window-related information, includingwindow images, to FvwmCompositor. FvwmCompositordisplays these images with OpenGL, using arbitrarily trans-formed textured polygons, and forwards input device eventsto the server. FVWM can solely handle basic window opera-tions such as move, resize or iconify, issuing the appropriatecommands to the X server. It can also delegate these opera-tions to FvwmCompositor. New window operations can beimplemented either as FVWM functions, using a scriptinglanguage, or in FvwmCompositor.

The following subsections will provide more implementationdetails about the Metisse server, our modified FVWM andFvwmCompositor.

Metisse serverThe Metisse server is a fully functional X Window server de-rived from Xserver4, a software used by the X community forexploratory developments. It uses Xserver’s rootless exten-sion to provide off-screen rendering of application windows:each top-level window is rendered in a separate pixmap – animage stored in a single contiguous memory buffer – that isdynamically allocated when the window is created and re-allocated when it is resized. The server stores along witheach window image the coordinates of its upper-left corner

4http://www.freedesktop.org/

Figure 1: General overview of Metisse.

in the compositor’s display. These coordinates are the onesreported to applications that issue geometry requests.

Each time an application updates a window content, theserver sends an update notification to the compositor. Thecorresponding region of the pixmap can be transmitted to thecompositor using a custom protocol similar to VNC. It canalso be copied in a shared memory space if the two processesare running on the same machine. These off-screen render-ing and update notification mechanisms are quite similar totheCompositeandDamageX extensions. In fact, the mainreason why we didn’t use these extensions is that they werestill in early development stages and heavily discussed whenwe started implementing Metisse.

The server sends various other notifications to the compositorto indicate the creation, destruction, mapping or unmappingof a window as well as geometry modifications, changes inthe stacking order and cursor changes. It provides the com-positor with the actual bounds of shaped (i.e. non rectan-gular) windows. It also indicates a possibly related windowfor all transient and override redirect windows (e.g. pop-upmenus). All these notifications make it easy for the compos-itor to maintain a list of the windows to be displayed and toapply to pop-up menus the transformation used for the corre-sponding application window.

Window visibility is usually an important concern for Xserver implementations, since a window can be partially oc-cluded by another one, or be partially off-screen. Traditionalservers generateExposeevents to notify applications of vis-ibility changes and clip drawing commands to the visibleregions. Since the Metisse server renders windows in sep-arate pixmaps, partial occlusion never happens. Moreover,since the actual layout of windows is defined by the com-positor, the notion of being partially off-screen doesn’t makeany sense in the server. As a consequence, the Metisse servernever generatesExposeevents.

Traditional X servers receiving a mouse event use the pointerlocation and their knowledge of the screen layout to decidewhich window should receive the event. Again, in the caseof Metisse, since the actual layout is defined by the compos-itor, the server cannot perform this computation. As a con-

sequence, the mouse events transmitted by the compositormust explicitly specify the target window. When the screenlayout changes, X servers usually look for the window un-der the pointer and, if it has changed, sendLeaveandEnterevents to the appropriate windows. In the case of Metisse,this process is left to the compositor.

Metisse compositor: FVWM and FvwmCompositorFVWM5 is an X window manager created in 1993 and stillactively developed. Originally designed to minimize mem-ory consumption, it provides a number of interesting featuressuch as GNOME and KDE compatibility, customizable win-dow decorations, virtual desktops, keyboard accelerators, dy-namic menus, mouse gesture recognition, as well as variousfocus policies. All these features can be dynamically config-ured at run-time using various scripting languages.

Scripted functions are a powerful and simple way of extend-ing the window manager. As an example, one can easilydefine a new iconification function that raises the window,takes a screenshot of it with an external program, definesthis image as the window icon and then calls the standardiconification command. Commands can be executed condi-tionally, depending on the nature and state of a window, andcan be applied to a specific set of windows. Commands andscripted functions can be easily bound to a particular mouseor keyboard event on the desktop, a window or a decorationelement. They can also be bound to higher-level events suchas window creation or focus changes.

FVWM can also be extended by implementingmodules, ex-ternal applications spawned by the window manager witha two-way communication link. FvwmCompositor is anFVWM module implemented with the Nucleo6 toolkit, whichprovides a simple OpenGL scenegraph and a basic asyn-chronous scheduler for multiplexing event sources. It usesthe window images as textures that can be mapped on arbi-trary polygons, these polygons being themselves arbitrarilytransformed (Figure 2).

Implementing the Metisse compositor as an extension ofFVWM has several advantages over developing one fromscratch. First, almost nothing needs to be done to replicatethe standard window operations of existing window systems:FvwmCompositor simply needs to display window images atthe positions given by the Metisse server, and to forward in-put device events to it. Second, since FVWM reparents appli-cation windows in new ones containing the decorations, thesedecorations are automatically made available in the compos-itor through the server. This has proved to be much moreconvenient than writing OpenGL code to display and inter-act with title bars and borders, buttons and pull-down menus.

FvwmCompositor displays its composition in an OpenGLwindow of the native window system (a GLX window onLinux and a Carbon/AGL window on Mac OS X). Althoughnot mandatory, this window is usually set to be full-screen,so that FvwmCompositor visually replaces the native win-dow system. The current implementation uses a perspectiveprojection. The third dimension (i.e. Z axis) of OpenGL is

5http://www.fvwm.org/6http://insitu.lri.fr/˜roussel/projects/nucleo/

Figure 2: Basic composition showing a rotated Mozillawindow with a pop-up menu, a downscaled xemacswindow and a translucent GIMP window. Note that thesame transformation is used for the pop-up menu andthe Mozilla window.

used to enforce the stacking order of the windows definedin the server by FVWM. In order to avoid intersections be-tween windows that would not be on parallel planes, large Zdistances are used between windows, especially for the bot-tom and top ones. Consequently, all windows are rescaledto keep their original size despite their distance to the viewerand the perspective projection.

Keyboard events are simply forwarded to the Metisse server,the keyboard focus policy being handled by FVWM. Whenreceiving a mouse event, FvwmCompositor uses OpenGL’sselection mode and picking to find the window under thepointer. It then uses the transformation matrix associated tothat window and its position on the server’s virtual screen totransform the mouse coordinates into the server’s coordinatesystem. The event is then forwarded to the server with theseadjusted coordinates and additional information specifyingthe target window.

In some situations, FvwmCompositor needs to use the trans-formation matrix of a particular window even if the mousepointer is not over it. This happens, for example, if the useris interactively resizing the window in a movement so fastthat the pointer leaves the window. To avoid this particularsituation, FvwmCompositor uses “infinite” polygons whendrawing windows in selection mode.

EXAMPLESIn the previous section, we have described the architectureof Metisse and explained how this design provides a windowsystem that is both fully functional and highly tailorable. Inthis section, we present several examples that illustrate howMetisse facilitates the implementation of innovative windowmanagement techniques.

Basic operationsThe following code sample shows how FVWM can be con-figured to scale windows by clicking on one of the buttons oftheir title bar or pressing some keys:

Mouse 3 4 A SendToModule FvwmCompositor Scale 0.7

Key minus W C SendToModule FvwmCompositor Scale 0.9Key plus W C SendToModule FvwmCompositor Scale 1.11

The first line requests FVWM to send the string “Scale 0.7”to FvwmCompositor when the user does a right mouse click(third button) on the minimize icon of the title bar (fourthicon), no matter the active keyboard modifiers (A is for “anymodifier”). In addition to the specified string, FVWM willsend the X id of the window that received the click. A sim-ple parser implemented in FvwmCompositor will decode themessage and perform a 30% reduction of the representationof the specified window. Similarly, the two other lines re-quest FVWM to send a scale command to FvwmCompositorwhen the user presses Ctrl+ or Ctrl- while the mouse pointeris on the window.

Other commands implemented in FvwmCompositor allowto rotate a window around different axes, to scale it non-uniformly and to set, lower and raise its opacity and bright-ness levels. Metisse provides a default configuration file forFVWM with menus and various bindings (mouse, keyboardor high-level events) for all these operations. These bindingsmake it possible, for example, to lower the brightness of allwindows except those of the application having the keyboardfocus. One can also specify that all windows of a certain typeshould be semi-transparent (e.g. those of an instant messag-ing application) and become fully opaque when the mousepointer comes over them.

FvwmCompositor commands can also be combined with tra-ditional window operations in interesting ways. As a firstexample, one can easily replace the usual maximizing oper-ation by a function that zooms in the window so that it takesthe whole screen space instead of resizing it. Another inter-esting combination is the ZoomOutAndMaximizeY functionillustrated by Figure 3. This function zooms out a windowuniformly and then resizes its height so that it takes the wholescreen. It is available in Metisse as a toggle switch (i.e. call-ing the function a second time returns the window back to itsprevious state).

The ZoomOutAndMaximizeY function is particularly inter-esting when working on a large text document. When acti-vated, it allows to see a larger part of the document whichin turn makes it easier to navigate. Calling the function asecond time restores the original scale and size of the win-dow, providing a more detailed view of the selected part ofthe document. A notable fact about this function is that ithas been designed and implemented in a few minutes by theauthors on a laptop during a subway trip between Paris andOrsay. The final implementation is only a few lines long tobe added to the configuration file of Metisse.

Interactive window manipulationInteractive window manipulations such as the peel-back op-eration described in [2] (Figure 4) cannot be implementedwith FVWM scripts. This kind of complex operations needto be handled directly by FvwmCompositor. In order to fa-cilitate this, we have created a new FVWM command calledMetisseInteractiveManip.

Figure 3: ZoomOutAndMaximizeY funtion applied ona Web browser showing a large document. The upperimage shows the browser before calling the function,the lower one shows the resulting transformation.

Figure 4: Xemacs window being peeled back.

MetisseInteractiveManip abstracts the concept of interactivemanipulation from the FVWM point of view. It takes a win-dow operation name and a cursor name as arguments. Whenexecuted, it grabs the mouse and the keyboard (i.e. forbidsother applications to use them), changes the cursor for theone specified, and checks that the specified window is stillvalid. FVWM then sends to FvwmCompositor the operationname along with the cursor position and the target window,and enters a simple event loop, waiting for the operation tocomplete. Upon completion, FVWM releases the mouse and

keyboard and reenters its main loop.

Interactive move, rotation and scaling are implemented inthe same way, as FvwmCompositor operations called in re-sponse to an FVWM message sent by MetisseInteractiveMa-nip. Here’s how the folding operation might be configured inFVWM:

# Immediately (I) raise the window and start the# fold operation (Fold) if the mouse is dragged (M)# or hold (H)AddToFunc FoldWindow+ I Raise+ M MetisseInteractiveManip Fold FOLD_CURSOR+ H MetisseInteractiveManip Fold FOLD_CURSOR

# Bind the folding function to a right mouse button# click (3) on the window border (F A)Mouse 3 F A FoldWindow

The interactive scale operation is in a certain way similarto the usual resize operation and can be bound to the ma-nipulation of the borders of the windows with a given key-board modifier. Rotation around the Y axis can be bound toa mouse drag on the left or right border of the window with akeyboard modifier. The top and bottom borders can be usedfor rotations around the X axis. The corners of the windowmight be used for rotations around the Z axis.

We believe that window scaling might offer some interestingnew ways of managing overlapping windows. As opposedto the traditional resize operation, scaling a window reducesoverlapping while preserving the layout of window contents.This has proved to be useful, for example, for checking thelayout of a Web page in one or more browsers while editingit in a text editor. Temporarily scaling down two applica-tions also allows to quickly perform a series of interactionsbetween them, such as drag-and-drop or copy/paste opera-tions. Note that when using a perspective projection, rota-tions around the X and Y axis produce a non-uniform scalingeffect that can also be used to reduce overlapping.

Unlike low-fidelity environments usually used to implementinnovative window management techniques, Metisse allowsall these techniques to be used for real on a daily basis.This can help adjusting the details of a particular technique.The peel-back operation, for example, became much moreinteresting after we decided to make the back side of thepeeled-back window translucent (Figure 4). Daily use alsohelped realize that the ability to put back windows into a“normal” state after some transformation was very important.As a consequence, the default Metisse configuration allowsto cancel the transformations applied to a window by right-clicking on its title bar, a simple animation being used toease the transition. We are also adding a history mechanismwith an undo/redo mechanism that should make it easier tounderstand manipulation errors and to capture interaction se-quences to create new commands.

Animations and temporary transformationsAnimations have long been used in window managers to pro-vide feedback of ongoing operations. Specifying animationsin Metisse is quite simple. It doesn’t require much program-ming skills and could probably be done by experienced users.

The following code sample shows how to create an animatediconification function. It uses two lines of Perl to send mul-tiple Scalecommands to FvwmCompositor to produce theanimation effect. Note that this animation is implemented asa script that can be parsed at run-time by FVWM. No modi-fication of FvwmCompositor is necessary:

AddToFunc myIconify+ I PipeRead ’for ((i=0; $i<20; i++)) ; do \

echo "SendToModule FvwmCompositor Scale 0.9"; \done’

+ I State 1 True

AddToFunc myDeIconify+ I PipeRead ’for ((i=0; $i<20; i++)) ; do \

echo "SendToModule FvwmCompositor Scale 1.11"; \done’

+ I State 1 False

AddToFunc myToggleIconify# deiconify if iconified+ I ThisWindow (State 1) myDeIconify# iconify if not iconified+ I TestRc (NoMatch) myIconify

The functions for moving, rotating and scaling windows havebeen implemented in two forms. The first one correspondsto the usual operation mode: the user presses a mouse but-ton and moves the mouse to define the transformation. Whenthe button is released, the window stays where it is, usingthe last transformation. The second form is a temporary one:when the mouse button is released, the window returns to itsoriginal position with an animation. This second form pro-vides interesting alternatives to the folding operation. Whileexperimenting with translucency effects, we also found outthat temporary modifications of the opacity level also offersnew interesting possibilities. As an example, when movinga window, making that window or the other ones translucenthelps finding the right place to put it.

Position-dependent window transformationsUntil now, we have presented techniques that put the userin total control of every detail of the transformations appliedon windows. However, combining two or more elementarytransformations, such as a move and a rotation, can be quitetedious. A simple and powerful way of solving this problemis to define the transformation to be applied on a window asdependent of its position. This way, the user will indirectlyapply these transformations by simply moving the window.A position-dependent transformation can be described as afunction

f : S4 −→ {Finite sequence of OpenGL transformations}

whereS is the screen (or a set of screens with its layout fora multi-monitor setting). Given a window we apply to it asequence of OpenGL transformationsf(a, b, c, d) wherea =(a1, a2), . . . d = (d1, d2) are the coordinates of the cornersof the window.

As a first example, we use this approach to scale down win-dows as they approach the left or right borders of the screenso that they remain fully visible instead of becoming partly

off-screen (Figure 5). A minimum window size is imposedso that at some point, moving the window further towards theborder of the screen has no effect. A special FvwmCompos-itor command allows to restore the original position and sizeof a window before it was moved and scaled. This new opera-tion provides a simple and continuous way, as opposed to theusual iconification operation, of moving a window from theforeground to the background. Although different in spirit,it is in some ways similar to the window manipulation tech-niques provided by Scalable Fabric [16].

Figure 5: The top left window has been pushed againstthe left border of the screen and scaled down. Theright window has the same size but is not scaled. Thesmall windows on the right have been either draggedto the black region by hand or sent there by clicking ona button in their title bar.

Daily use of this move-and-scale operation also gave us theidea of implementing a second version of it, a temporary onefollowing the approach described in the previous subsection.This version allows users to grab a window with the mouse,move it to the side of the screen (and thus reduce its size) andthen simply release the mouse button to restore the window’soriginal size and position. This new operation proved to bequite useful to see what’s behind a window while keeping itscontent visible.

Our second example of position-dependent transformationhas been designed for tabletop interactive displays [6]. Inthis example, we split the screen into two equal partsA (thebottom part) andB (the top part). When a window is totallycontained inA no transformation is applied to it. When it istotally contained in partB, it is rotated around the Z axis by180 degrees. When a window is between theA andB parts arotation between0 and180 degree is applied to it dependingon the distance to the splitting line. The rotation is appliedclockwise if the center of the window is on the left part ofthe screen and counterclockwise if on the right. This way, ifa window is moved fromA to B, it is progressively rotatedupside down (Figure 6).

One nice feature of FvwmCompositor is that it can dupli-cate a window. Users can interact with a duplicated windowexactly as if it were the original one (see [21] for more de-tails). This feature can be combined with the tabletop inter-

Figure 6: Tabletop interface featuring automatic win-dow orientation and on-demand window duplication.

actions we just described: the top-left window of Figure 6 isa zoomed duplicate of the one in the middle of the lowerpart. Window duplication is also interesting in multiple-monitors configurations. Although the current implementa-tion of Metisse does not support multiple simultaneous input,this example has already proved to be useful in situationswhere one user needs to show something to other people.

Interactive desktop manipulation

Global operations that transform all the windows can alsobe implemented in Metisse. As an example, we have im-plemented a zoomable desktop that allows to navigate in avirtual space nine times bigger than the physical one (ninevirtual screens arranged in a 3x3 matrix). This desktop sup-ports standard panning techniques that allow to move fromone virtual screen to adjacent ones by simply moving themouse towards the corresponding edge. It also supports con-tinuous zooming with the mouse wheel, which provides anoverview of several adjacent screens at the same time up to acomplete bird’s eye view of the virtual desktop (Figure 7).

Figure 7: Bird’s eye view of a virtual desktop made ofnine virtual screens.

In the current implementation of this zoomable desktop,clicking on a particular virtual screen from an overview ini-tiates an animated transition that zooms into it. Note thatall applications remain accessible while in overview mode:they can be moved between virtual screens, resized or closedby the user who can also interact with them as usual. Wehave also implemented a simple version of Apple’sExpose,which scales down the windows on the screen and tiles themto make them all visible. Like our zoomable desktop and asopposed toExpose, this version lets the user interact with ap-plications as usual. The zoomable desktop and ourExpose-like could probably be combined, the latter allowing users toaccess windows otherwise unreachable.

DISCUSSION AND DIRECTIONS FOR FUTURE WORKWe believe that a well chosen subset of the techniques de-scribed above could significantly improve window manage-ment tasks. The main problem we have when we demon-strate a Metisse-based desktop to some people is that theytend to assume that Metisse is what they see (e.g. a 3Ddesktop). Metisse is not a 3D desktop! It is a highly tai-lorable, graphics-rich, out-of-the-box replacement for exist-ing X desktops. Which makes it a perfect tool for rapid high-fidelity prototyping of window management techniques. Aswe already stated above, we believe this is an important pointand a great improvement over other experimental desktopsbecause it should make it possible to conduct longitudinalstudies of new window management techniques.

Metisse raises some interesting questions. As an example,when the cursor comes over a transformed window, shouldthe same transformation be applied to the cursor itself? Con-versely, when a transformation is applied on a window andthe cursor is on that window, should the cursor be automat-ically moved to stay at the same place? Should a pop-upmenu or a transient window follow the transformation of itsparent window? In some cases, this seems desirable but insome others not... Our experience indicates that a distinctionbetween windows in the user’s focus and peripheral windowsmight help finding answers to these questions.

Metisse allows to easily add rendering artifacts to windows,like shadows. Amon other things, we plan to investigate thedesign of new artifacts that would help users to understandthe relations between the various windows. As an exam-ple, one could show the relation between a dialog box andits parent windows by drawing translucent lines between thecorners of the two windows. This, like the other windowmanagement techniques described in this paper, should beevaluated as part of a future longitudinal study.

Performance and preliminary evaluationOne of the authors uses Metisse on a regular basis. In par-ticular, a large part of FvwmCompositor has been devel-oped inside FvwmCompositor itself (modify the code, com-pile and restart FvwmCompositor!). Metisse works perfectlywell for day to day activities such as e-mail and instant mes-saging, digital pictures and web browsing, text and imageediting, as well as small-sized videos. Rotations and scal-ing are often used to reduce overlapping. The overviewmode of the zoomable virtual desktop is used for rearrang-ing windows. One limitation of the current implementation

is that OpenGL applications cannot be hardware-acceleratedin Metisse, which makes them slower than usual. The cur-rent way of dealing with this is to switch from Metisse to thenative desktop for OpenGL applications (and high-resolutionvideos).

The computer used by the author is a two years old Laptoprunning Linux, with a 2 GHz Pentium IV, 768 MB of mem-ory and a Radeon Mobility M6 graphics card with 32 MBof memory. On that machine, applications making an ex-tensive use of the X drawing API run at up to fifty framesper second while high-resolution videos, as we explained,can’t be played at their nominal frame rate. As an ex-ample, a 720x576 Divx video is displayed at only twelveframes per second. As illustrated by Figure 7, many applica-tions can run together without problem. The limited amountof video memory sometimes causes temporary performanceproblems. However, iconification of a few windows is usu-ally enough to free some memory and return to the standardperformance level. Several tests with a more recent graphicscard, a Nvidia GeForce with 128 MB of memory, doubledthe frame rate of drawing-based applications and allowed toview the Divx video at nominal frame rate.

A preliminary version of the Metisse source code has beenpublicly released in June 2004. A few maintenance releaseshave followed. The last release has been downloaded morethan 4500 times and the Metisse web site serves around 800pages by day. E-mail messages from about eighty peoplewere sent to the authors asking for support, giving some feed-back and reporting a few bugs. Most people who contactedus were very positive and a few of them actually use Metisseas their default desktop. We are currently preparing an eval-uation survey that will be distributed with the next release.We are also working on an extension of Metisse that wouldallow the recording of all window operations for later replayand analysis.

Towards a variety of compositorsThe Metisse server is currently being used by a group of peo-ple from Mekensleep7, a company developing an OpenGL-based on-line poker game. Their original motivation was tobe able to integrate an external chat application in the game.Once they had implemented a basic Metisse compositor intheir application, they realized that it could also be used tobring 2D interfaces built with traditional GUI toolkits suchas GTK+ into their OpenGL scene (Figure 8).

This idea of using Metisse to integrate 2D interfaces in 3Denvironments seems very interesting to us. We hope thatMetisse will be used by other researchers in similar ways.As a consequence, we are currently developing a library tofacilitate the use of the Metisse server and the implementa-tion of new compositors. FvwmCompositor has a now rela-tively long history. Some code clean-up will also be made,which will probably facilitate the implementation of newexperimental window management techniques by other re-searchers.

As we said in the previous section, FvwmCompositor can

7http://www.mekensleep.com/

Figure 8: Poker3D as a Metisse compositor: the bluewindows contain GTK+ interface elements and are ren-dered by the Metisse server.

display multiple instances of a window. But it can do more:it can duplicate a window region (as described in [22]), cre-ate holes in a window (as suggested in [9]), create a newwindow from pieces of others, and embed part of a windowinto another one. In fact, FvwmCompositor should be seenas awindow regioncompositor (details are available in [21]).The next natural step is to move to awidget compositor. Weare currently investigating the use of accessibility APIs toget the widget tree associated to a particular window anduse it to support new interaction techniques. As an exam-ple, since FvwmCompositor already handles all user input,this should make it possible to use semantic pointing [5] forwindow management.

CONCLUSIONIn this paper, we have described Metisse, a window systemcreated to facilitate the design, the implementation and theevaluation of innovative window management techniques.We have presented the general architecture of the system anddescribed some of its implementation details. We have pre-sented several examples of uses that illustrate its potentialand several directions for future research.

Metisse is available from http://www.lri.fr/ chapuis/metisse/

REFERENCES1. Mac OS X v10.2 Technologies: Quartz Extreme and

Quartz 2D. Apple Technology brief, October 2002.

2. M. Beaudouin-Lafon. Novel interaction techniques foroverlapping windows. InProceedings of UIST 2001,pages 153–154. ACM Press, November 2001.

3. J. Beda. Avalon Graphics: 2-D, 3-D, Imaging AndComposition. Presentation at the Windows HardwareEngineering Conference, May 2004.

4. B. Bell and S. Feiner. Dynamic space management foruser interfaces. InProceedings of UIST 2000, pages239–248. ACM Press, 2000.

5. R. Blanch, Y. Guiard, and M. Beaudouin-Lafon. Se-mantic pointing: Improving target acquisition with

control-display ratio adaptation. InProceedings of CHI2004, pages 519–526. ACM Press, April 2004.

6. P. Dietz and D. Leigh. DiamondTouch: a multi-usertouch technology. InProceedings of UIST 2001, pages219–226, New York, NY, USA, 2001. ACM Press.

7. P. Dragicevic. Combining crossing-based and paper-based interaction paradigms for dragging and droppingbetween overlapping windows. InProceedings of UIST2004, pages 193–196. ACM Press, 2004.

8. J. Gettys and K. Packard. The (Re)Architecture of theX Window System. InProceedings of the Linux Sym-posium, Volume 1, pages 227–237, July 2004.

9. D. Hutchings and J. Stasko. Revisiting Display SpaceManagement: Understanding Current Practice to In-form Next-generation Design. InProceedings ofGI 2004, pages 127–134. Canadian Human-ComputerCommunications Society, June 2004.

10. D. Hutchings and J. Stasko. Shrinking window oper-ations for expanding display space. InAVI ’04: Pro-ceedings of the working conference on Advanced visualinterfaces, pages 350–353. ACM Press, 2004.

11. D. Hutchings and J. Stasko. mudibo: Multiple dialogboxes for multiple monitors. InExtended abstracts ofCHI 2005. ACM Press, April 2005.

12. E. Kandogan and B. Shneiderman. Elastic Windows:evaluation of multi-window operations. InProceedingsof CHI 1997, pages 250–257. ACM Press, March 1997.

13. B. Myers. A taxonomy of window manager user in-terfaces. IEEE Computer Graphics and Applications,8(5):65–84, sept/oct 1988.

14. B. Myers. A brief history of human-computer in-teraction technology.ACM interactions, 5(2):44–54,march/april 1998.

15. T. Richardson, Q. Stafford-Fraser, K.R. Wood, andA. Hopper. Virtual Network Computing.IEEE InternetComputing, 2(1):33–38, Jan-Feb 1998.

16. G. Robertson, E. Horvitz, M. Czerwinski, P. Baudisch,D. Hutchings, B. Myers, D. Robbins, and G. Smith.Scalable Fabric: Flexible Task Management. InProc.of AVI 2004, pages 85–89, May 2004.

17. G. Robertson, M. van Dantzich, D. Robbins, M. Cz-erwinski, K. Hinckley, K. Risden, D. Thiel, andV. Gorokhovsky. The Task Gallery: a 3D window man-ager. InProceedings of CHI 2000, pages 494–501.ACM Press, April 2000.

18. N. Roussel. Ametista: a mini-toolkit for exploringnew window management techniques. InProceedingsof CLIHC 2003, pages 117–124. ACM Press, August2003.

19. R.W. Scheifler and J. Gettys. The X Window System.ACM Transactions on Graphics, 5(2):79–109, 1986.

20. D. Smith, A. Raab, D. Reed, and A. Kay. Croquet: Amenagerie of new user interfaces. InProceedings ofC5 2004, pages 4–11. IEEE Computer Society, January2004.

21. W. Stuerzlinger, O. Chapuis, and N. Roussel. User in-terface facades: towards fully adaptable user interfaces.Submitted for publication.

22. D. Tan, B. Meyers, and M. Czerwinski. Wincuts: ma-nipulating arbitrary window regions for more effectiveuse of screen space. InExtended abstracts of CHI 2004,pages 1525–1528. ACM Press, 2004.

23. M. Tomitsch. Trends and Evolution of Window Inter-faces. Diploma thesis, University of Technology, Vi-enna, December 2003. 132 pages.

24. M. van Dantzich, G. Robertson, and V. Ghorokhovsky.Application Redirection: Hosting Windows Applica-tions in 3D. InProceedings of NPIV99, pages 87–91.ACM Press, 1999.