THINKING ABOUT VISUAL PROGRAMS

T R G Green and A F Blackwell

Background: The Dream

Programming is hard, fidgety, and intricate. The older languages (e.g. early Fortran) were full of idiosyncrasies. To unde r sad what a program meant, you had to understand how it worked. The corre- spondence between the problem world and the program world was very poor. The problem world might include simultaneous equations and matrix algebra; the program world dealt with loops, tempo- rary variables, overflow conditions, indexes and registen. Programmers had to leam to translate between them.

The dream of computer scientists has always been to improve the match between these two worlds - if possible, to make the program world identical to the problem world. Each new form of programming has been hailed as a step forward. Algebraic programming, logic programming, functional and object- oriented programming have each been seen as ways to bring the two worlds closer. Visual programming has been hailed in much the same terms.

The theme of this paper is that simple comparisons betwleen diagrams and text are inappropriate. at least in the programming area. Simple text-graphic comparisons are certainly very popular among computer scientists; we surveyed the literature (approximately 140 papers) and found that the vast majority made the simple assertion that visual programming would be better. Various reasons were cited; here are some of them.

Straighvonvard Advantages: Visual programming is claimed by various authors to be more user friendly, helpful, satisfying, intuitive, simple, readable, familiar, appealing, reliable, accessible, straightforward, pleasant, memorable, alluring, immediate and obvious.

Absrracrion: “People find it easier to deal with the concrete than the abstract, and that solutions are therefore easier to perceive if abstract information is converted to a concrete (i.e. visual) form.”

Expression of Structure: “Relationships between entities are more explicitly represented and easily recognised in pictures than in text.”

Expressiveness oj Pictures: “Pictures contain more infoimation than text does,” or “a picture is worth 1000 words”.

Mental Imagery: “Visual programming languages will support mental imagery better.”

Cognirive Faculries: ‘The human mind is optimised for vision, making shapes easier to process than

Avoidance of synrar: “A visual programming language makes semantic relationships explicit in

words and making 2D representations better than linear ones.”

pictorial form. This makes it possible for the programmer to perceive a solution directly, without syntax. ”

MRC Applied Psychology Unit, 15 Chaucer Rd, Cambridge CB2 2EF [ thomas.green, alan.blackwell)@ mrc-apu.cam.ac.uk http://www.mrc-apu.cam.ac.uWpersonal/( th0mas.g” alan.blackwell) /

5/ 1

Could The Dream be true?

The idea that visual programming might always be better than textual programming - in all conditions, for all tasks - turns out to be empirically wrong. Green, Petre, and Bellamy (1991) showed that answering conditionals was harder in a visual language than in a textual language. In a second study, Green and Petre (1992) showed that the advantage of text was not just because it was more familiar.

What these two experiments also showed was the importance of structure. To get best results, the structure of the notation and the type of question had to match. If a language was expressed in a sequential manner then it was good for answering ‘forwards’ questions: if A. B and C are true, what happens? But it was bad for answering backwards questions: X happened, what must have been true?

Exactly the same was true of both the visual and the textual languages. Forwards questions were better than backwards for one kind of structure, but it was the other way round for a different kind of structure. Evidently, what matters is not anything as crude as ‘visual or textual’, but instead, the infor- mation structure.

Structural Analysis of Notations

It seems that the dominant feature that determines whether it is easy to reason about a program may be the structure of the information, rather than whether it is presented visually or textually. Visual languages encourage certain types of structuring, so they are good for certain purposes. So, rather than pose such a brutally simple question as ‘Is visual programming better than textual?’, we shall set out a competing agenda of the pros and cons of visual programming as a tool to support reasoning. This agenda is part of a framework of Cognitive Dimensions of Notations (Green and Petre, 1995).

Spatial relationships can support useful mental representations

Under favourable conditions, representations allow their components to be laid out in ways that capture aspects of the problem domain. Green and Petre call this secondary notation. The spreadsheet is an excellent example, in which real life users take advantage of the layout possibilities.

Ironically enough, visual programming languages are sometimes very poor at supporting secondary notation. In box-and-wire languages, the spatial layout is dominated by trying to avoid a rat’s nest of crossing lines. The so-called two-dimensional freedom simply does not exist in any useful degree. It is easier to use paragraphing and indentation as a form of secondary notation in Basic than it is to use layout in a box-and-wire language.

Accessibility of related information

Related pieces of information need to be juxtaposable, so they can be viewed side by side for compari- son and contrast. Not all visual languages make this easy. In Prograph, for example, every conditional spawns a new window containing a subroutine (‘method’). These windows quickly accumulate, so that searching through the structure to find out what the program really does in some particular situation can become a serious problem; managing the display becomes the centre of cognitive activity, rather than understanding the program. There is no available empirical data on this problem but experienced professional Prograph users (and supporters) have criticised this aspect.

Visual programs can hold ‘think traps’

Certain diagrams are extremely difficult to interpret, because they force their readers to perform particularly hard mental operations. Figure 1 illustrates one of these‘think traps’. Empirical data from the experiments mentioned above showed that such questions were answered slowly and erratically, with much use of their fingers to keep track of what they were doing.

I Arauel ............ t "" +$..--.-m."." ......... ..... iel

..e not-gate

Figure I : Hard mental operations in LobvIEw. Problem: if Tired' and Thirsty' are both false, which of the three outputs receives a true signal? (Only one of them ever receives a true, for any input data.)

W h y is this structure so difficult? The reason is that the diagram is a visual maze, and the reader has to keep track of which paths have been followed as the data signals are propagated from left to right. Answering the problem given above requires a process like this:

Take first truth-value. Tired=False. Scan to find Tired in program. Propagate False along the wire. Place finger at choice point and consider each direction in turn.

- Propagate horizontally first. On reaching next component, Sleep, abandon this branch, since we haven't sent a True to an ouqm yet.

- Now propagate downwards. On reaching the invemr, start to propagate True. On reaching junction, place finger at choice point and consider each direction in turn.

- Propagate horizontally first. On reaching and-gate, place finger on it and stan to search backwards. Search reaches input Thirsty. Look up truth-value. which is 12alse. Return to most recent finger, abandon this branch since the conjunction is false.

- Return to previous finger. Start to propagate downwards. On reaching and-gate place finger on it and start to search backwards. On reaching invertor, remember to invert next signal found. Search reaches Thirsty which is set to False so invertor is set to True. Return to most recent finger and continue propagating True. Search reaches Argue. Stop with output 'Argue'.

mj (Tlrcdj

....... ........................

........... I".. ...................

10

Figure 2: the choice points rhat must be traversed to answer the problem.

513

Premature commitment in VPLs

Textual languages frequently encouragesor even compel programmers to attempt to make decisions about the future content of their program before they are ready to do so. Thus, the programmer must either guess, or build the program as a two-stage process - which militates against speedy construction and bends the focus of cognitive activity away from the problem solution towards the program structure.

VPLs sometimes reduce premature commitment because they allow the process of program construc- tion to go on in any order. There is no encouragement to work on one particular point; a multitude of paths can be left open and any one of them can be picked up, as desired.

Yet even in VPLs premature commitment can occur. Modugno et al. (1994) showed that in the first design of a visual programming-bydemonstration system it was necessary for the user to look ahead and plan the demonstration; if the required data cases had not been selected at the start of the demon- stration, the effort was wasted and had to be repeated. In a redesigned version the look-ahead was eliminated.

Conclusions

To speak of ‘thinking visually’ or ‘thinking with diagrams’ is not enough. It is tempting indeed to assume that diagrams are different from text, and we have listed quotations where exactly that type of thinking occurs; but diagrams, like text, contain information structures, and these structures control the accessibility and utility of information in inescapable ways.

The problems we have raised may seem self-evident with hindsight, but they are certainly not SO. The overwhelming belief amongst the computing fratemity is that what matters about visual programming is its visualness. rather than its structure.

The Cognitive Dimensions of Notations framework is an attempt to analyse structural aspects. More importantly, it is an attempt to give engineers and computer scientists a set of terms that they can use to compare the pros and cons of designs. These terms form discussion tools for thinking about diagram- matic (and other) notations.

References

Green, T. R. G. and Petre, M. (1992) When visual programs are harder to read than textual programs. In G. C. van der Veer, M. J. Tauber, S . Bagnarola and M. Antavolits (Eds.) Human-Computer Interaction: Tasks and Organisation, Proceedings of ECCE-6 (6th European Conference on Cognitive Ergonomics). CUD: Rome.

Green, T. R. G. and Petre, M. (1995) Usability analysis of visual programming environments. Submitted to IEEE Computer.

Green, T. R. G., Petre, M. and Bellamy, R. K. E. (1991) Comprehensibility of visual and textual programs: a test of ‘Superlativism’ against the ‘match-mismatch’ conjecture, Belliveau, T. Moher, and S. Robertson (Eds.), Empirical Studies of Programmers: Fourth Workshop. Norwood, NJ: Ablex. Pp. 121-146.

Modugno. F. M., Green, T. R. G. and Myers, B. (1994) Visual programming in a visual domain: a case study of cognitive dimensions. In G. Cockton, S . W. Draper and G. R. S . Weir (Eds.) People a d Computers IX. Cambridge University Press.

In J. Koenemann-

0 1996 The Institution of Electrical Engineers. Printed and published by the IEE, Savoy Place, London WC2R OBL, UK.