Unit Testing for Graphical User Interfaces

Unit Testing for Graphical User Interfaces

A Paper Submitted in Partial Fulfillment

of the Requirements for CSCI E-275

Michael Jastram

http://jastram.de

[email protected]

Harvard University, May 19, 2003

Table of Contents1 Introduction.........................................................................................................................3

1.1 Conventions...................................................................................................................31.2 History...........................................................................................................................3

2 GUI Testing.........................................................................................................................42.1 Rendering technologies..................................................................................................4

2.1.1 HTML....................................................................................................................42.1.2 Postscript...............................................................................................................42.1.3 Swing.....................................................................................................................42.1.4 X11........................................................................................................................42.1.5 Microsoft Windows................................................................................................52.1.6 Console applications...............................................................................................5

2.2 What to test...................................................................................................................52.2.1 Look and Feel.........................................................................................................52.2.2 Functionality...........................................................................................................72.2.3 Compliance.............................................................................................................7

2.3 Technologies and Frameworks.......................................................................................82.3.1 JUnit.......................................................................................................................82.3.2 Interacting with Swing Components.......................................................................82.3.3 java.awt.Robot.......................................................................................................92.3.4 Abbot.....................................................................................................................92.3.5 Non-Swing Technologies......................................................................................10

2.4 Architectures................................................................................................................102.4.1 MVC....................................................................................................................102.4.2 Document-View...................................................................................................112.4.3 PAC.....................................................................................................................11

3 Case Study: Workflow Designer GUI Testing.................................................................123.1 Architecture.................................................................................................................12

3.1.1 DesignerContext...................................................................................................123.1.2 ActivityTable........................................................................................................123.1.3 WorkflowRelevantDataTable................................................................................133.1.4 ActivityDetailView...............................................................................................133.1.5 RuleEditor............................................................................................................133.1.6 TextEditor............................................................................................................13

3.2 Methodology................................................................................................................133.3 Testing.........................................................................................................................13

3.3.1 Model testing........................................................................................................133.3.2 View and Controller testing..................................................................................143.3.3 DesignerContext and Client testing.......................................................................153.3.4 TextEditor testing.................................................................................................17

3.4 Conclusion...................................................................................................................20

4 References..........................................................................................................................21

© 2003 Michael Jastram (http://jastram.de) May 14, 2003 - 2/21

Introduction Introduction

1 Introduction

Software is created for humans. At some point, we will have to interact with it. Software hasUser Interfaces for this purpose, and more and more software these days has Graphical UserInterfaces (GUIs).In this CSCI E-275 Term Paper, I discuss GUI unit testing, with the main focus on Java /Swing. I was a member of the Designer Group, and one of our deliverables was the WorkflowDefinition Editor (also called Workflow Designer, or just Client, Figure 3), a graphical tool thatallowed users to create and edit Workflow Definitions. I ended up to be the person in chargeof the Client GUI, and I learned a number of insightful lessons about testing in the process.This paper consists of two parts. The first part takes a conceptual look at GUI unit testing andinvestigates approaches for various architectures (MVC, PAC, etc.), technologies (Swing, web-based, console-based, etc.), and frameworks (JUnit, Abbot, j ava. awt . Robot , etc.). Whilevarious GUI frameworks will be investigated, the focus will be on the ones that interoperatewith Java / Swing.The second part consists of a case study on how this knowledge has been put to use for theBPM-275 Workflow Designer Client. The workflow designer has been written in Swing, andthe unit tests explore different testing techniques.

1.1 Conventions

Code, code fragments, and class names are printed in Tel et ype.

1.2 History

User interfaces for software evolvedfrom plugs, switches and light bulbsto punch cards and line printers, todumb terminals, interactive characterbased applications to the graphicalapplications we are used to today.Where do we draw the line of what aGraphical User Interface is? It couldbe argued that even a console basedapplication can have a GUI, if itdoesn't operate in command linemode. In fact, the IBM created anASCII extension consisting of blockand line symbols for exactly thatpurpose on the original PC [IBM].

Most people agree that the GUI was “born” in Xerox PARC in 1975, although the firstexperiments with GUIs have already been done in the sixties (at MIT, amongst other places).See [Gla00] for a complete time line.


Figure 1 Example of a Console "GUI"

GUI Testing GUI Testing

2 GUI Testing

In this chapter, I will explore the various Graphical User Interfaces we may have to deal withtoday. First, we have to find out what a GUI is. According to Meriam-Webster, a GUI is “acomputer program designed to allow a computer user to interact easily with the computertypically by using a mouse to make choices from menus or groups of icons” [MW03].

2.1 Rendering technologies

Ultimately, there will be pixel on the screen, and there are cases where in fact individual pixelsneed to be inspected. However, in many cases it is feasible to probe the data before it isconverted into pixel. What is the “pre-stage” of pixel? Below is an (incomplete) list ofcommon technologies, and their “pre-pixel” representation.

Technologies that have been developed for a networked environment (HTML, X11) usuallymake it easier to access the pre-pixel representation than non-networked environments (e.g.Microsoft Windows).

2.1.1 HTML

HTML is a good starting point, as even non-programmers these days have some idea of whereit fits into the picture. HTML is a mark-up language that is interpreted by a rendering engine(built into the web browser) to produce the GUI. Also, the HTML is very accessible (allbrowsers offer a “View Source” functionality).

In practice, the HTML is not only accessed directly for testing, but for other purposes as well.HTML “Screen Scrapers” extract useful information by looking at the HTML, not the renderedscreen, for example. Still, screen scraping is awkward, and breaks easily (if the original page'slayout changes, for example). More and more sites are offering an XML data feed, if theyapprove of their content is being reused..

2.1.2 Postscript

Using Postscript on the OS level to render a screen has been done successfully with Mac OS X,and has been tried, more or less successfully, before. While it is not necessarily easy to accessthe screen Postcript, it is another example where a high-level language is used to render a GUI.

2.1.3 Swing

Swing components are accessible, but only from inside the JVM. But if the code (e.g. the testcode) runs within the same JVM, it is easy to get an alternate representation of the GUI. Forexample, we can always enumerate all Components in a Frame, inspect their properties, changetheir values, etc. Some of my test cases use the Abbot framework that has been created forexactly that purpose (see section 2.3.4).

2.1.4 X11

X11 has been designed from the start in a client-server fashion, which makes it more complexand slower than other display technologies, but also makes it much more flexible and versatile.

The implications on testing are twofold. For one, the X11 communication can be used directlyfor interfacing with the test script. This makes the system robust against system differences


GUI Testing Rendering technologies

(e.g. different window decorations), as the X11 protocol allows the identification of individualwidgets.

The second advantage of testing on an X11 system is, that test scripts that need to “pop up”frames can do that on any X11 display, not only the screen of the machine the code is runningon. This is useful if the test system is used as a desktop, where people are doing work (itwould be irritating, to say the least, if the moment an automated test script starts up, the screenwould be flooded with windows). If we would have been interested in it, we could have set upElmo (the BPM-275 development machine) that way.

2.1.5 Microsoft Windows

Windows has been developed from the very beginning as a single user system, so it is muchmore integrated than X11, which gives it much better performance due to the tighterintegration, but less accessibility for testing. Nevertheless, it's possible to gain access toindividual screen widgets with the right Software. Of course, this should be a last resortsolution. As with Swing, Microsoft provides a toolkit for developing applications for Windows(Microsoft Foundation Classes, MFC). If the test cases are written in the same environment, itsimplifies testing significantly, as individual widgets can be accessed directly from the test code.

2.1.6 Console applications

It's debatable whether console applications even should be mentioned. Command lineapplications should certainly be excluded from this category. However, there are variousapplications around that create “pseudo-graphical” user interfaces, some even support themouse.

The main difference is that these applications are never concerned with individual pixels, butwith individual characters. For testing, we can go directly to the screen buffer and probeindividual characters.

It should be mentioned that there are screen scrapers for console based applications as well,and in fact these tools play an important role these days, as they are sometimes used tointerface legacy applications (that expect to talk to a dumb terminal) with newer applications.

2.2 What to test

There are at least two broad categories of what we can develop if we talk about GUIdevelopment: We can create components (widgets) that are intended for reuse, or we candevelop GUI applications, which usually means to reuse and to customize components that arepart of an existing framework. In the following, we will keep these two very differentdevelopment efforts in the back of our heads. The BPM-275 effort fell into the secondcategory.

2.2.1 Look and Feel

What do we mean by testing the look and feel? There are various things to test:

� Layout of components on the screen (and the presence of components)� Appearance of individual components (especially custom components)


GUI Testing What to test

� Behavior (on select, on focus, processing generic key strokes, etc.)� Accessibility

The Look and feel should definitely be tested; however, it should be decided on a case by casebasis whether it needs to be unit tested. Instead, many of these tests could be performed duringacceptance testing or usability testing.

A related question is whether these tests should be automated. Instead, at least some of thesetest can be tested with test checklists that are being executed by a human.

Having said that, there are certainly ways to automate the testing of Look and Feel:

There are various aspects of the layout that can be tested. Wake provides a hands-on exampleon how the relative location of components to each other can be tested, using JUnit [Wa00] –the data required for this can be extracted directly from the j ava. awt . Component .

asser t ( " l abel l ef t - of quer y" , sp. sear chLabel . get Locat i onOnScr een( ) . x < sp. quer yFi el d. get Locat i onOnScr een( ) . x) ;

Inspecting Component further unveils more useful methods (e.g. we can extract theComponent 's size, which is useful if the size is set by a Layout Manager ). See section2.3.2 for more testing techniques.

Testing the appearance of a Component is a custom job, and often it may be more effective tofocus on testing the underlying structures (e.g. the MVC components), rather than the renderedComponent itself. Still, short of testing individual pixels, at least the standard behavior of thecomponent can be tested. For example, how does the Component respond to resizerequests? Will color changes be propagated correctly? Etc.

Testing the behavior of the system in the context of Look and Feel applies to “standardbehavior” , vs. problem specific behavior (which is called functionality). For example, Dialogboxes are acknowledged with the Ret ur n key, and dismissed with the Escape key. Thus,this kind of test is only necessary to test building blocks, as predefined components alreadybehave in a system-compliant manner (e.g. j avax. swi ng. JBut t on).

Accessibility testing is an extension of behavior testing, except that it is too often neglected.Fortunately this is changing, and most GUI frameworks have dedicated hooks for accessibility.

2.2.2 Functionality

The Functionality of the system can be tested relatively easily, but we should remember thatthis is often beyond unit testing, as an action often affects multiple components in the GUI. InSwing it's fairly easy to inject events (keystrokes, mouse clicks, etc.) into the system. There arevarious testing tools out there for almost all technologies that perform this kind of test – forHTML, Windows applications, etc. In fact, the same technologies are being used from screen-scraping as well.

Wake shows how simple Swing applications can be tested using the JUnit framework [Wa00].Here is an example from that paper:


GUI Testing What to test

publ i c voi d t est 0( ) { Sear chPanel sp = new Sear chPanel ( ) ; sp. set Sear cher ( new Test Sear cher ( ) ) ; sp. quer yFi el d. set Text ( " 0" ) ; sp. f i ndBut t on. doCl i ck( ) ; asser t ( " Empt y r esul t " , sp. r esul t Tabl e. get RowCount ( ) == 0) ; }In this test, a new Sear chPanel is instantiated, andpopulated with a Test Sear cher (a mockimplementation of a Sear cher that returns well-defined results). The test then performs the same stepsa user would perform: Populate a textbox, click on thebutton, and inspect the result.

While I find this test very elegant (I am using similartests for the WorkflowDesigner), I think that Wakecheated a little: He could only write such an eleganttest, because the Component s he needs to accesswere package private (sp. quer yFi el d,sp. f i ndBut t on, and sp. r esul t Tabl e). If itwasn't for the tests, these fields could be private. As wewill see later in section 2.3.4, there are test frameworksthat can work around this issue.

There is a good rule of thumb to determine whether you are still doing unit testing, or whetheryou started integration testing: If managing state becomes an issue, it's not unit testing anymore. The above code is very instructive: It doesn't depend on state, because aTest Sear cher has been created and used. Without the Test Sear cher , statemanagement would have been an issue (the state of the mock database), and we would haveleft the realm of unit testing.

2.2.3 Compliance

If we develop components rather than a custom application, we don't know how thecomponents will be put to use, and compliance with the framework contract are crucial andneeds to be tested. Even if we don't develop custom components, we still want to becompliant, to be nice to the engineers coming after us.

The functional contract (expected behavior) of new Swing Components is defined in theJavaDoc of the superclasses we use for our new Component (e.g. Component ,JComponent , JPane, etc.). The documentation is excellent, but unfortunately, compliancecannot be enforced (as these classes are designed for subclassing, it would actually be great tohave abstract test cases, using the abstract test pattern, for such classes [Geo02]).

2.3 Technologies and Frameworks

This section will discuss specific technologies and frameworks for accessing components,which is necessarily in order to build unit tests that directly interact with the application'scomponents.


Figure 2 Example GUI used by Wake

GUI Testing Technologies and Frameworks

2.3.1 JUnit

JUnit is a generic framework for unit tests that is completely indifferent to which tests are beingexecuted through it. The frameworks described in the following all work seamlessly withJUnit, as long as they are Java frameworks.

2.3.2 Interacting with Swing Components

The most “generic” way of testing a GUI application is by interacting directly with itscomponents. Generic, because usually no access to source code is required. Instead, we needa framework to find components, and to read out their values, essentially a “screen injector”and a “screen scraper” . However, this is a last resort for GUI unit testing, as the wholeapplication is rarely a unit.

For Java and Swing (and AWT), we have much more direct access to the components, as longas our test code runs in the same JVM. This has been demonstrated in section 2.2.2. Buttonsare clicked, test fields read, by directly accessing the Component s . It is worth to take sometime and browse throught the JavaDoc for both, Component and Cont ai ner [Sun03], aswell as the specific classes your are testing (e.g. JLabel , JText Fi el d). Here are thehighlights from Component :

Component Method Use for Testingget Backgr ound( )get For egr ound( )

Useful if the component's color has dependency (e.g. if it istaken from its Cont ai ner ).

get Locat i onOnScr een( ) Extremely useful for checking the location of Component srelative to each other.

get Si ze( ) Useful for checking on minumum or maximum size, or fortesting the impact of the Layout Manager .

hasFocus( ) Can be used if an action influences the focus (e.g. “tabbing”through Component s in the right order).

r equest Focus( )t r ansf er Focus( )

Useful for test setup.

Specific subclasses of Component have additional useful methods. JBut t on, for instance, hasa method to click on the button; JText Fi el d has methods to read and set the content of thefield. And so on.

Cont ai ner is important, as it allows us to find Component s , even if we don't have areference to them. A Cont ai ner knows about all its Component s . Here is a list ofrelevant methods that can be useful for testing:

Container Method Use for Testingf i ndComponent At ( )get Component ( )get Component At ( )get Component s( )

Various methods of finding Component s , either by position,by a numeric id, or all in the form of an array.

get Component Count ( ) Useful if the count may change (e.g. if new Component canbe added a run time, a list of text boxes, for example).



Note that Cont ai ner and Component are part of the AWT framework. There is also theSwing object JComponent .

Now that we know how to communicate with Component s , we can put these methods towork, in JUnit or otherwise. We just have to somehow get hold of the Component s we needfor our tests. This can be done by keeping references in package private instance variables.But there are frameworks available that don't even require that, as discussed in the followingsections.

2.3.3 j ava. awt . Robot

The creators of Java already thought about the problem of testing1, and provided a tool for“test automation, self-running demos, and other applications where control of the mouse andkeyboard is needed” [Sun03]. This tool is j ava. awt . Robot . A Robot can move and usethe mouse, press keys, wait, and inspect the screen (snapshot or individual pixels). This is notmuch, but may be enough for small applications.

Note that a Robot doesn't know anything about Components! It operates exclusively on apixel basis. This restricts its use even further: Many applications use dynamic layouts (e.g.Bor der Layout ), where the size and location of components may differ on differentplatform, or even with different “Themes” (and let's not even start talking about colors).

2.3.4 Abbot

While I don't feel qualified to judge j ava. awt . Robot (after all, I never used it), others did.Abbot stands for “A better Bot” , and the group states as its objective “Improving upon the veryrudimentary functions provided by the j ava. awt . Robot class” [Abb03]. From theJavaDoc: “Abbot builds a level of abstraction on top of java.awt.Robot, mirroring the waySwing Component s are a level of abstraction above basic windows.” This abstraction layerallows to dynamically look up Component s , thereby making it unnecessary to either nowtheir location, or having a reference to them. In the simplest case, the tag can be a Stringassociated with a Component (e.g. the text on a JBut t on), but the lookup can becomemore and more sophisticated (which is necessary, for example, if the UI contains more than oneJBut t on with the same text).

This framework can be put to use in unit tests. But in addition to this, the Abbot frameworkprovides a scripting engine that allows the creation of test scripts that don't require compilation,which can be really useful for functional testing and integration testing. In a biggerorganization, it will make it easier to decouple QA's dependency on the developers.

I played around a lot with the JUnit extensions, and abbot . Component Fi nder inparticular. This interface provides powerful methods for finding Component s . One of themore interesting methods is f i ndComponent ( ) , which takes a Component Ref er enceas an argument. And a Component Ref er ence takes up to eight parameters to uniquelyidentify a Component – see the Abbot API documentation for details.

1 Actually, they thought about this problem fairly late and after the fact: java.awt.Robot got introduced onlywith JDK 1.3.



2.3.5 Non-Swing Technologies

As mentioned earlier this paper focuses on Swing. But I briefly want to mention some GUItesting frameworks that are not Swing specific.

There is no shortage of framework for Web applications (HttpUnit [HU03], HtmlUnit[HmU03], jWebUnit [jWU03]). While these are useful tools, it should be remembered thatthey are more appropriate for functional/acceptance testing, rather than unit testing, as the“units” are not easily accessible with these tools.

Even Windows has a Macro Recorder build in, but it's not at all adequate for testing. But thereare plenty of commercial testing tools available.

2.4 Architectures

All the tests frameworks described so far had one thing in common: They interacted directlywith the Component s of the application. But I would argue that this is not always the mostimportant unit test, or the most efficient way to perform GUI unit testing.

If we want to perform unit testing of the GUI before the GUI is rendered, the approach totesting depends on the architectural pattern used.

2.4.1 MVC

Swing is based on the Model-View-Controller (MVC) architectural pattern [Bu96]. Each ofthe three pieces has its own set of responsibilities, and thus each piece can be testedindividually. Furthermore, For off-the-mill Components, we may not even change all three.For example, it is common for a JTabl e to simply provide a domain-specific Tabl eModel ,while using the unmodified Swing JTabl e.

How do we test the model? We usually have two representations of the same data: Theapplication-specific data structure, and the representation of that data from the model to theview. Let's look at a table again. The application-specific data structure could just be a twodimensional array. The Tabl eModel , however, has a well-defined API to make this dataavailable to the view (through the methods get Col umnCount ( ) , get Val ueAt ( ) , etc.).If we make the assumption that the Swing Table MVC framework is bug-free, and if we furtherassume that we only replace the model, than testing the model should be sufficient to unit testour table. And the aspect to test is the data integrity between the data that underlies the model,and the data that the model provides to the view.

If a custom view is employed, things get trickier; again, with Swing, we may be lucky enoughnot having to write the complete view from scratch (with JTabl es , for example, we may justhave to provide a new Tabl eCel l Render er . This interface has only one method, whichmakes it fairly easy to unit test.).

If the view is completely custom, things get even trickier to test, as the controller and view arerelatively tightly coupled (and in this case, there will be implications for the model as well):“The MVC triad components are tightly coupled (in the static sense), but are loosely coupled interms of the runtime notification mechanism” [Rob03]. While the dependencies between thethree components can be tested, the “outcome” (i.e. what's rendered on the screen) is moredifficult to test.

It should be noted that Struts is using MVC as well, and that the ideas in this section


GUI Testing Architectures

conceptually apply to Struts as well. However, the view in Struts is more limited than inSwing, as Struts can merely produce HTML, while Swing can actually control every singlepixel on the screen. This limitation on the view is a good thing if it comes to testing. Ofcourse, that's exactly the reason why HTML is too limited for some applications (e.g. no drag-and-drop support, no drawing canvas, etc.). And of course, HTML in itself is architecturallyneutral (especially small sites use the JSP/ASP/PHP/Perl “Spaghetti” architecture).

2.4.2 Document-View

The Document-View architectural pattern is similar to MVC, where “the document componentcorresponds to the model in MVC, and also implements a change-propagation mechanism”[Bu96]. For testing, it is nice to still have the graphical aspect to be isolated, which makes the“rest” (the Document) more testable. If we would use an existing Document-View framework,it is likely that just the Document would require customization (as we mainly discussed thecustomization of the Model in the previous section).

2.4.3 PAC

Unfortunately, I never had the pleasure to work with PAC(Presentation-Abstraction-Control)in practice, nor did my literature research return much information about PAC unit testing.Having said that, here are my personal conclusions:

As with MVC, The piece in PAC that is most likely to get customized (and thus requirestesting) is the abstraction (corresponding to the Model). This assumes that a PAC frameworkexists that is being used for implementing a GUI. In order to take advantage of the features ofPAC, the control needs to be adapted (and thus tested) as well – to take advantage of cachingin the abstraction, for instance). If we develop a PAC framework, this obviously doesn't holdtrue. I couldn't find any Java PAC frameworks (only some very modest referenceimplementations [Bu00]).


Case Study: Workflow Designer Case Study: Workflow Designer

3 Case Study: Workflow Designer

As we split up the work in the Designer team, I ended up doing a lot of the GUI work. TheGUI consisted of a Swing application that allowed users to create and edit BPM-275 WorkflowDefinitions (Figure 3).

3.1 Architecture

The GUI was build in Java, using Swing. Swing provides an MVC based componentframework. The application Frame contains a menu bar, a status bar, and three other elementscontained in JSpl i t Panes . All three elements were custom components:Act i v i t yTabl e (top left) , Wor kf l owRel evant Dat aTabl e (right), andAct i v i t yDet ai l Vi ew (bottom left). In addition, we created two modal custom dialogs:Rul eEdi t or and Text Edi t or . Last, we created the interface Desi gner Cont ext thatprovided access to “system-wide” data and services. All these components will be brieflydiscussed in the following section, to provide a foundation of the discussion of the GUI tests.

3.1.1 Desi gner Cont ext

During the devleopment of the client we realized that there were some things several objectsneeded access to. Refactoring the code at that point resulted in the Desi gner Cont extinterface. This interface was eventually implemented into Wor kf l owDesi gner , the JFr amethat would eventually hold the entire GUI, and that would also contain the mai n( ) method(the entry point into the Client). Desi nger Cont ext has three methods: one to access thecurrent Workflow Definition, one for retrieving the system Logger, and one for accessing thestatus line.

3.1.2 Act i v i t yTabl e

The Act i v i t yTabl e is a subclass of JTabl e, and we use a corresponding


Figure 3 Workflow Designer GUI

Case Study: Workflow Designer Architecture

Act i v i t yTabl eModel , which is a subclass of Abst r act Tabl eModel . The JTabl ehad to be subclassed to implement fairly standard configuration stuff: set up the columnsproperly (e.g. plugging in a custom cell editor), etc. It also contained code to manage the detailview of the selected activity. Last, it implemented the interface RowAct i onLi st ener ,which allowed us to access all tables in the system consistently through the menu bar (to whichtable the menu actions would be directed, depended on which GUI element had the focus).

The table model had to be subclassed as well, to map between the internal data structure of aWorkflow Definition (accessed through the Desi gner Cont ext ), and the row-column-representation, as defined by the table model.

3.1.3 Wor kf l owRel evant Dat aTabl e

The Wor kf l owRel evant Dat aTabl e and the corresponding model were builtcorresponding to the Act i v i t yTabl e architecture. As already mentioned in the previoussection, the similarities between the two tables got exploited for reuse. For example, bothtables use a subclass the abstract class NameCel l Edi t or , which allows to edit a St r i ng,while providing instant validation feedback on the status line.

3.1.4 Act i v i t yDet ai l Vi ew

The Act i v i t yDet ai l Vi ew was tricky, as our hope was to just provide another view of theactivities. That wasn't really possible, however, as the Act i v i t yTabl e's view determinedwhich activity details we had to show, and not the model. Thus, we had to hand control to thethe Act i v i t yTabl e, which was the object tracking the currently selected activity.

3.1.5 Rul eEdi t or

The Rul eEdi t or is a modal dialog that allows to edit, add and delete rules associated withan activity. The fact that the dialog was modal made it straightforward: While the rules werebeing edited, everything else was on hold. The code calling the Rul eEdi t or had to updatethe corresponding models, of course, to reflect the changes the user may have done.

3.1.6 Text Edi t or

In various situations, a simple text editor was required: For editing the repository properties,and for editing the workflow and activity descriptions. Putting the modal editor together wasstraight forward, and because it was so simple, it was a good opportunity to experiment withthe testing frameworks.

3.2 Methodology

I had not too much trouble applying the write-the-test-first XP approach for non-GUI classes(e.g. the repository code); but with the GUI code, I had a hard time, and often didn't knowwhere to begin. It got easier over time, with practice.

Wake [Wa00] created an excellent tutorial that shows how to apply XP methodologies toSwing GUI development. I used this approach where I could (see below). As I mentionedearlier in section 2.2.2, Wake assumes that the Component s required for testing are at leastpackage private, if not more accessible. I sometimes found this hard to achieve.

There were some tutorials provided by the Abbot group [Abb03], but they were only useful to


Case Study: Workflow Designer Methodology

a degree. This is partly because Abbot has a strong focus on functional testing by creating testscripts, using their script editor. As my focus was on unit testing, I ignored the scriptingcapabilities of this tool. And last, their approach to testing was ignoring the design aspect,which is central to XP. They were more focusing on the needs of QA groups.

3.3 Testing

In the following, I will show example code and discuss the testing approach.

3.3.1 Model testing

Testing the Model in the MVC paradigm felt easy compared to some other tests, as the testingobjective was clear: Make sure that changes in the underlying data structures get propagatedproperly to the model data accessors, and the other way around. This test applied toAct i v i t yTabl eModel Test and Wor kf l owRel evant Dat aTabl eModel Test .

In order to make the tests truly unit tests, they had to be decoupled from the rest of the system.Specifically, The models required a Desi gner Cont ext to access the underlying data (theWor kf l owDef i ni t i on object). As Desi gner Cont ext is an interface, we provided atest implementation. Considering that in the production code the Desi gner Cont ext isimplemented by Wor kf l owDesi gner , a class that had a visual representation, using ourown Desi gner Cont ext implementation had the additional advantage of removing Swingelements from the test case2. The implementation Test Desi gner Cont ext is extremelysimple (50 lines, including comments).

The structure of the test cases is very straightforward: First, we create a well-defined model,associated with a well defined Wor kf l owDef i ni t i on. This is done by set Up( ) :

/ * * * Cr eat es a c l ean model . * /pr ot ect ed voi d set Up( ) { cont ext = new Test Desi gner Cont ext ( ) ; model = new Act i v i t yTabl eModel ( cont ext ) ;}

Now every test checks that changes from the WorkflowDefinition get propagated properly tothe model, or the other way around. The following test, for example, manipulates the model,and checks the effect on the WorkflowDefinition:

/ * * * Makes sur e t hat i nser t i ng one Row updat es t he * wor kf l owDef i ni t i on. * /publ i c voi d t est Def i ni t i onAf t er I nser t OneRow( ) { model . i nser t Row( ) ; asser t Tr ue( cont ext . get Wor kf l owDef i ni t i on( ) . get Act i v i t i es( ) . hasNext ( ) ) ;

2 In fact, the model test cases can run without requiring a Graphics context, in contrast to some other unittests in this package.


Case Study: Workflow Designer Testing

}

It turns out that model testing is easy, systematic, and can be done without requiring a Graphicscontext.

3.3.2 View and Controller testing

It is not possible to test the view without a Graphics context (at least I didn't figure out how).As the control is usually instantiated together with the view, they can be tested as one unit (andin the BPM-275 code, we never touched the default control code in the first place).

There are two fundamental ways for testing the view: We can make sure that changes in theview get propagated properly to the model, by checking the data stored in the model. Or wecan simply check the view, to see whether it returns the expected results when probed.

What do I mean by that? Imagine we have the view of a table (a subclass of JTabl e), andwant to make sure that we can change the value of a cell. Let's assume the table has exactlyone row and one column, so the cell at (0, 0) exists. We can manipulate the value of that cellas follows:

t abl e. set Val ueAt ( “ New Val ue” , 0, 0) ;

How can we test whether the model now holds the proper value? We can check the modelitself:

asser t Equal s( “ New Val ue” , t abl e. get Model ( ) . get Val ueAt ( 0, 0) ) ;

However, as the table visualizes the value of the underlying model, we could also perform thefollowing test:

asser t Equal s( “ New Val ue” , t abl e. get Val ueAt ( 0, 0) ) ;

The second option gives more the feeling of a unit test, as the model doesn't seem to beinvolved (which is not true, but the interaction is not visible, as we only access JTabl emethods). If we had a dummy Tabl eModel implementation with a set Val ueAt ( )method that does nothing, both assert statements would fail.

Which of the two methods to use ultimately depends on the details of the implementation, forinstance on where validation takes place. Validation can be provided as the user types. We usethis technique for the activity name by providing a custom Cel l Edi t or (calledNameCel l Edi t or ) that updates the status line as the user types. This editor itself preventsdata from being propagated to the model in the first place, if it is not valid. However,validation can also take place in the model. Either way, the tests would be set up differently.

3.3.3 Desi gner Cont ext and Client testing

As the Desi gner Cont ext is an interface, an abstract test case has been provided toimplement the abstract test pattern. As quite a bit of the functionality cannot be tested againstthe interface alone (e.g. we can't check whether the status bar is set properly), it feels a bit likeoverkill to apply the pattern.

The actual implementation of Desi gner Cont ext is the application itself. This creates two



problems. First, we would like to test much more than just Desi gner Cont ext specificfunctionality. This is not really a problem in this case, but imagine we would implementmultiple interfaces? Then we would also have multiple abstract test cases. As a class can onlysubclass one class, we would have to implement multiple test classes for the same class.Possible, but potentially confusing (and it may duplicate code, specifically in the set Up( ) andt ear Down( ) methods).

The second problem is to instantiate the Wor kf l owDesi gner . The class has been designedto run as a standalone application, containing a mai n( ) method. In addition, the application isdesigned to use configuration information read out of a file stored in the user's home directory.This level of dependency is not acceptable for unit testing.

Eventually, this dilemma has been resolved by refactoring the constructor from a privateconstructor with no arguments to a package private constructor that takes a Pr oper t i esobject as an argument. Whether the properties have been provided by the test case, or beenread from a configuration file is now irrelevant.

Even though we now have a way to instantiate the workflow designer, we have to dispose itagain. This is not as easy as it sounds, as Swing applications are threaded. The call to theconstructor returns immediately, yet the frame containing the application stays on the screen,running on its own thread. This is where the Abbot framework provides useful services. Inorder to find Component s , we need to get a Fi nder instance:

Component Fi nder f i nder = Def aul t Component Fi nder . get Fi nder ( ) ;

With the finder we can get references to visible Component s , as shown later. For now, wesimply want to close all visible Windows, which can be done with one simple call:

f i nder . di spl oseWi ndows( ) ;

This call goes conveniently into the t ear Down( ) method. Now we can write our test casesagainst the Wor kf l owDesi gner we instantiate in the setUp() method, e.g. the following:

/ * * * On a St at us Li ne r eset , er r or f l ag shoul d be i gnor ed. * /publ i c voi d t est Set St at usLi neToNul l Er r or ( ) { desi gner . set St at us( nul l , f al se) ; asser t Equal s( " Ready. " , get St at usLi ne( ) . get Text ( ) ) ; asser t Equal s( Col or . BLACK, get St at usLi ne( ) . get For egr ound( ) ) ;}

So far, so good3. But wait – how do we get hold of the status line? As there is no need in theapplication to read out the value of the status line, there is no “get” method for the status, andthe JLabel holding it is private. Instead, the test class provides the methodget St at usLi ne( ) :

/ * *

3 Well, Pat would object to this test code, as it contains two assert statements. I left the test code intact, as thisis how it appears in the repository. However, we could easily break it into two test cases.



* Thi s hel per f i nds t he St at us l i ne. One assumpt i on i s t hat * t her e' s onl y one JLabel ar ound i n t he f i r st pl ace. * @r et ur n t he St at us Li ne. * /pr i vat e JLabel get St at usLi ne( ) { Component Ref er ence r ef = new Component Ref er ence( " " , JLabel . c l ass) ; r et ur n ( JLabel ) f i nder . f i ndComponent ( r ef ) ;}

This method demonstrates how the Abbot framework can be put to work. We collect aminimum set of properties of the Component we are looking for, and instantiate aComponent Ref er ence with this description. The finder then returns the Component thatmatches the description. Note that the finder would throw an exception, if multipleComponent s would match the description. In this particular case, we know that in the wholeUI there is only one single JLabel . Providing the class is enough to find it.

3.3.4 Text Edi t or testing

The Text Edi t or is distinct fromother pieces of the client code. Whilemost classes are specialized classes,this is a general-purpose class thatcould easily be reused elsewhere (andin fact, it is used in three places in theWorkflow Designer: to edit workflowdefinition descriptions, activitydescriptions, and repositoryproperties). The interface to thesystem is fairly simple: the class hasonly one public method – everythingelse, including constructor, is private:

/ * * * Shows a modal di al og t hat al l ow t he user t o edi t t he gi ven * t ext . Ret ur ns t he modi f i ed t ext , or nul l i f t he user * cancel ed. * /publ i c st at i c St r i ng showText Di al og( Fr ame f r ame, St r i ng t i t l e, St r i ng t ext )

The JavaDoc shown above doesn't provide a complete description of the functionality: TheFr ame is required, as the class subclasses JDi al og, which uses the Fr ame to center theJDi al og properly. In addition, the text editor features an OK and a Cancel button. If thetext has been changed, and the cancel button has been clicked, a confirmation dialog pops up(see Figure 4).

How do we unit test such a class? The first question is, whether the class is well-designed, and


Figure 4 TextEditor


whether there would have been a better design if we had written the unit tests first. This classhas been written in the spirit of JOpt i onPane that also offers a number of static show*methods (showConf i r mDi al og( ) , showI nput Di al og( ) , etc.). It's hard to tell, asJavaDoc doesn't show package private methods and members; however, as JOptionPane hasbeen designed to be extended, it offers quite a number of protected members and methods thatcould be used at least to a degree for testing. Still, if the show* methods are being used, thereis still no way to get hold of the instance that's shown to the user.

There is another dilemma: as the TextEditor dialog is a modal dialog, the call blocks until thedialog is dismissed. In order to test such a dialog, the test code must run on its own thread.

So in order to test, I used the Abbot framework again, and this time more extensively. As I justmentioned, the dialog must run on its own thread, so that the test code can manipulate thedialog. For this purpose, I used the following private class:

/ * * * Thi s hel per spi ns of a Di al og on i t s t hr ead. Once t he * di al og i s c l osed, t he r esul t i s avai l abl e t hr ough t he member * r et ur nVal ue. * /pr i vat e c l ass Di al ogThr ead ext ends Thr ead{ / / Hol ds t he r et ur n val ue af t er t he Thr ead t er mi nat es. St r i ng r et ur nVal ue;

/ / The i ni t i al val ues St r i ng t i t l e; St r i ng t ext ;

publ i c Di al ogThr ead( St r i ng t i t l e, St r i ng t ext ) { t hi s. t i t l e = t i t l e; t hi s. t ext = t ext ; }

/ * * * @see j ava. l ang. Runnabl e#r un( ) * / publ i c voi d r un( ) { r et ur nVal ue = Text Edi t or . showText Di al og( nul l , t i t l e, t ext ) ; }}

This class was supplemented with a helper method that was actually spinning off aDialogThread:

/ * * * Thi s f act or y met hod shoul d be used t o cr eat e a new Thr ead.



* I t aut omat i cal l y st ar t s t he t hr ead, and wai t s f or hal f a * second t o make sur e t hat t he di al og i s bui l t pr oper l y. * /publ i c Di al ogThr ead makeDi al og( St r i ng t i t l e, St r i ng t ext ) t hr ows I nt er r upt edExcept i on{ Di al ogThr ead t hr ead = new Di al ogThr ead( t i t l e, t ext ) ; t hr ead. st ar t ( ) ; Thr ead. sl eep( del ay) ; r et ur n t hr ead;}

The static variable delay has been determined experimentally to find the balance betweenwaiting long enough for the UI to build, and short enough for human impatience. Arbitraryconstants like that are always unsatisfying, as this could result in the unit test being “broken” ona slower system. Still, once this framework was in place, the tests were straight forward towrite, and readable:

/ * * * Open wi t h NULL val ue * /publ i c voi d t est Wi t hNul l Text ( ) t hr ows I nt er r upt edExcept i on { Di al ogThr ead t hr ead = makeDi al og( " t i t l e" , nul l ) ; get OkBut t on( ) . doCl i ck( ) ; t hr ead. j oi n( ) ; asser t Equal s( " " , t hr ead. r et ur nVal ue) ;}

Things became really complicated, once the confirm dialog popped up. To elaborate: assumethe following code is executed:

Di al ogThr ead t hr ead = makeDi al og( " t i t l e" , nul l ) ;get Text Ar ea( ) . set Text ( " Repl ace t he t ext " ) ;get Cancel But t on( ) . doCl i ck( ) ;

Now, the doCl i ck( ) blocks until the confirm dialog is dismissed! And to make things evenmore complicated, finding the right button also got more difficult, as we now have two visibleOK buttons and two visible Cancel buttons. Abbot's ComponentFinder needs more informationto identify the proper Component. Here is the code that eventually does all this:

/ * * * Edi t t he t ext , c l i ck Cancel , c l i ck OK t o conf i r m. * /publ i c voi d t est Edi t Cl i ckCancel Ok( ) t hr ows I nt er r upt edExcept i on{ Di al ogThr ead t hr ead = makeDi al og( " t i t l e" , " t ext " ) ; get Text Ar ea( ) . set Text ( " Repl ace t he t ext " ) ;

/ / As c l i ck i ng pops up anot her di al og, we have t o t hr ead



/ / i t . . . Thr ead t hr ead2 = new Thr ead( ) { publ i c voi d r un( ) { get Cancel But t on( ) . doCl i ck( ) ; } } ; t hr ead2. st ar t ( ) ; Thr ead. sl eep( del ay) ;

/ / Fi nd t he Conf i r m Di al og OK Di al og Component Ref er ence r ef = new Component Ref er ence( " OK" , JBut t on. cl ass, " OK" , " OK" , " Cont ent got modi f i ed. Di scar d cont ent ?" ) ; ( ( JBut t on) f i nder . f i ndComponent ( r ef ) ) . doCl i ck( ) ; t hr ead2. j oi n( ) ; t hr ead. j oi n( ) ; asser t Nul l ( t hr ead. r et ur nVal ue) ;}

In order to find the proper button, we use the title of the dialog containing the button.Unfortunately, this makes the code brittle against title changes. Overall, I am not too happywith this code, even though it does the job.

3.4 Conclusion

Testing within the MVC paradigm felt easy and straight forward, and testing custom Swingapplications fortunately falls into this category. It is also consistent with the idea of unittesting, where just one Component is tested, isolated and taken out of its context.

This ease disappeared with the Text Edi t or . I blame mostly the fact that this class had asimple Java interface (one method), but a complex human interface. There was no easy way toget to the human interface. Wake [Wa00] avoided this dilemma by using package privateComponent s . Doing so would saved me only half the headache, as there still would havebeen the problem of threads, and the second dialog popping up. I can't think of a way to takecare of the second dialog, except through Abbot.


References References

4 References

[Abb03] Abbot Framwork, as of 2003, http://abbot.sourceforge.net/

[Bu96] Buschmann, Meunier, Rohnert, Sommerlad, Stal, “Pattern-Oriented SoftwareArchitecture” , 1996

[Bu00] Buschmann, Meunier, Rohnert, Sommerlad, Stal, “Pattern:Presentation-Abstraction-Control” , as of 2000,http://www.vico.org/pages/PatronsDisseny/Pattern%20Presentation%20Abstra/

[Geo02] Eric George, “Testing Interface Compliance with Abstract Test” , as of 2002,http://www.placebosoft.com/abstract-test.html

[Gla00] Jonathan Gladden, “Xerox PARC and the GUI” , Research Paper, 2000,http://www.accad.ohio-state.edu/~jgladden/GradCourses/ComputerGraphicsHistory/ResearchPaper/parcgui01.html

[HmU03] HtmlUnit, as of 2003, http://htmlunit.sourceforge.net/

[HpU03] HttpUnit, as of 2003, http://www.httpunit.org/

[IBM] The ASCII extensions by IBM for the PC:http://www.mkssoftware.com/docs/man5/ascii.5.asp

[jWU03] jWebUnit, as of 2003, http://jwebunit.sourceforge.net/

[MW03] Abridged Meriam-Webster's Dictionary, Online version, as of 2003, http://www.m-w.com/

[Rob03] Robinson, “Architecture Principles and Patterns” , as of 2003,http://www.people.fas.harvard.edu/~robinson/cscie275/slides/m02-principlesandpatterns.pdf

[Sun03] Sun Microsystems , “Java 2 Standard Edition, version 1.4” , 2003http://java.sun.com/j2se/1.4.1/docs/api/index.html

[Wa00] William C. Wake, “The Test/Code Cycle in XP: Part 2, GUI” , as of 2000,http://www.xp123.com/xplor/xp0001/


Documents

Unit Testing for Graphical User Interfaces