Learning a first computer language: strategies for making sense

Int. J. Man-Machine Studies (1~"32) 16, 449-486

Learning a first computer language: strategies for making sense

M. J. COOMBS, R. GmSON AND J. L. AL'rY

Computer Laboratory, University of Liverpool, Liverpool L69 3BX, U.K.

It is a common observation that people difIer greatly in their ability to make use of computers. In controlled experiments on the writing and debugging of programs, for example, large discrepancies in performance have been found even at the professional level, and in universities it is often noted that some individuals make more effective use of facilities than others who have undergone the same training and whose needs are just as great. This paper reports a study in which individual differences found in the learning of FORTRAN as a first computer language by a university population are used as a source of information on the nature of computing skills.

The study employed two classes of task: a "target" task consisting of tests of programming skill and an "indicator" task being a measure of learning style devised by Pask. Novice programmers completed these tasks following a standard introductory FORTRAN course. Comparison of performance by each subject on the two tasks was then used to draw inferences on the nature of successful strategies for learning a first programming language. Successful learners worked from "inside" the language, paying close attention to the procedural representation of logical relations between individual language structures. Less successful learners sought to determine important structural detail with reference to factors external to the program language itself, e.g. features of the local machine, and to represent this knowledge in descriptive rather than procedural terms.

1. The instruction and guidance of university computer users

The ability to make effective use of a compu te r is fast becoming a pre-requis i te for research in most academic disciplines. However , in c o m m o n with o ther " b a c k g r o u n d " research skills such as statistical or numerical analysis, comput ing is rarely taught in depth outside its specialist depar tment . Pos tgradua te s tudents and researchers are expected to develop compe tence in the use of a comput ing facility th rough trial and error following a short in t roductory course. For scientists and engineers this is usually on a high-level language such as F O R T R A N , while social scientists and arts users usually learn to use a re levant applications package. Such courses usually avoid any detailed discussion of the fundamenta l principles of comput ing, limiting themselves to syntactic details and simple job submission procedures . Af te r an in t roductory course, a user 's main official sources of suppor t are the informat ion and guidance services provided by their university compute r centre (Aity & Coombs , 1981). Users thus work in an env i ronment where their main learning takes place while work ing on their applicat ion problem, new informat ion being sought as the direct consequence of a p rog ram failure or a processing requi rement .

M a n y compute r users find this learning env i ronmen t entirely congenial . Th rough purposeful interaction with the machine , documen ta t ion and p r o g r a m m i n g advisory

449

0020-7373/82/040449 + 38503.00/0 �9 1982 Academic Press Inc. (London) Limited

450 M. I. C O O M B S , R. G I B S O N A N D J. L. A L T Y

service users often become computer experts in their own right. Indeed, it is common for a number of users to acquire sufficient expertise over a sufficiently wide range of facilities to be used by their local computer centre as an informal "debugging service" for new items of software. However , such success is balanced by a significant number of people who fail to acquire more than rudimentary computing skills, and often after considerable effort. Moreover , personal accounts of the early computing history of the less successful learners usually reveal no obvious differences in the instructional and guidance facilities they have employed or in their need to use the machine. People from both groups are found to have at tended the same introductory courses, approached learning with the same diligence, had application requirements of similar complexity and consulted the same documentat ion and computer professionals.

The popular explanation for these differences is that the users who have difficulty acquiring computing skills come from disciplines new to computing such as social science and medicine, the implication being that their problems are due to an inability to express clearly their application requirements (which often involves a numeric computat ion) and an unfamiliarity with general machine processes. However , although research carried out at Birmingham and Liverpool Universities has shown this to be partially true (Lang, Alty & Lang, 1979), a considerable number of unsuccessful learners also come from scientific and engineering backgrounds. Moreover, such people often shown considerable mathematical and technical skill within their own academic area.

In addition to the considerable informal evidence on differences in the ability of users to acquire computing skills, the existence of related difference in p rogrammer performance has also been reported in the literature. For example, Gould (1975), in a study of debugging, noted that there were significant individual difference in both the number and type of bugs located by experienced programmers , although he hesitated giving an account of these differences in terms of detailed variation in processing strategy. A second relevant study was conducted by Bell (1976) into the value of a number of standard tests of cognitive performance for predicting error propagat ion and debugging skill. This research produced the interesting finding of a high negative correlation between debugging ability and a test of critical thinking in naturalistic situations widely used for p rogrammer selection. This would suggest that in order to perform effectively, programmers need to be able to work without reference to common-sense inference patterns.

The traditional solution to the existence of differences in aptitude for computing skills is to a t tempt to minimize their effect by providing application packages which address the user in terms of his discipline and which shield him from the computer operat ing system. Although the need for software which is sympathet ic to the user has been recognized for some time, its production begs many questions which do not have obvious answers. What constitutes a "sympathet ic" user interface? Should the software completely hide the workings of the system? If it does, will it be possible for the user to diagnose and remedy program failures? In addition, there is a strong possibility that even if these questions can be successfully answered (see du Boulay & O'Shea, 1981), the same users will continue to have difficulty due to some fundamental incompatabili ty between their style of mental processing and the structure of computing information.

LEARNING A FIRST COMPUTER L A N G U A G E 451

The existence of marked individual differences in the learning of computing skills is of considerable interest for both theoretical and practical reasons. In the first instance an analysis of the contrasting learning strategies used by successful and unsuccessful learners should provide data on the nature of computing information itself and on the cognitive skills required for its acquisition. Such knowledge would aid in the design of all systems for educating and guiding users, including documentation, machine-based "help" systems and training courses. Secondly, their study should enable individuals needing special attention to be identified in advance so that they can either be given an individual training program or be instructed at an early stage in the relevant basic learning skills. It was therefore decided to undertake a study of individual differences in the acquisition of computing skills with the objective of gaining insight into ways of effectively supporting university computer users.

2. A methodology for the study of computing skills

There are various well-established methods which may be employed for identifying the strategies people employ to acquire computing information and assessing those which are most successful. It would be possible, for example, for subjects to be given some computing tasks and to record the actions they take or the thoughts they have while completing it. Such so-called "protocol" analysis has been undertaken successfully in classic studies both by Bruner, Goodnow & Austin (1956) in a study of concept formation and by Newell & Simon (1972) in a study of problem-solving. However, we considered that this method would be unsatisfactory for our purposes. Any realistic computing task would require a very diverse range of activities, hence the protocols would be expected to be very complex and thus difficult, if not impossible, to analyse without some well-founded theory of programming to help identify significant details. However, no suitable theory exists. We have therefore adopted an alternative approach.

Subjects were presented with two tasks--a " target" task and an "indicator" task. The objective of the method was the characterization of subject performance on the target task about which little was known, and this was achieved by careful choice of an indicator task selected to have some predefined relationship with the target task. Information from the indicator task was then used to generate hypotheses about performance on the target task which were to be tested in a conventional manner. This paper confines itself to the first stage of this approach, namely the generation of hypotheses on the acquisition of computing skills. The second stage, the testing of such hypotheses, is currently in progress at Liverpool and will be reported elsewhere.

2.1. THE P R O G R A M M I N G TESTS (TARGET TASK)

The computing tasks selected for the present research were concerned with programming itself. This was not only for the obvious reason that programming is central to all computing activity but also because a user with significant programming skill should find debugging simpler (Youngs, 1974) and should find it easier to search for relevant information in documentation. Unlike other researchers (e.g. Mayer, 1976; Sime, Green & Guest, 1977), it was decided not to devise a special learning task but to study the learning of a wide lyused language (FORTRAN) by a normal intake of

452 M. J. C O O M B S , R . G I B S O N A N D J. L. A L T Y

novice university users. All courses used in the research were fairly typical of the F O R T R A N courses taught in most computer centres, emphasis being placed on the description of language structures rather than on the principles behind the design of the language or on its method of implementation. Students were encouraged to practise the use of these structures on standard problems at the end of each lecture.

The devising of a method for assessing learning raised a number of problems. The measurement of programming skill offers the investigator a heterogeneous set of variables ranging from individual language structures to the logic of the final program. Thus it might be concluded that an adequate test of learning would need to be very complex. However, it is often found in learning experiments that the ability of the subjects to handle complex material is determined by a small number of different underlying skills. For example, in previous work by Coombs (1977) into the learning of physical manipulative procedures, it was found that the ability of a subject to recall the units of a procedure was independent of ability to recall the units in the correct order. These two capabilities proved to be the product of two very different coding strategies.

The program reads a time in the format of a 24 hour

clock, for example: 1436

and outputs the result in the format of a 12 hour

clock, for example: 2 36 p.m.

SUBCALL TIME (TVALU ,HRS ,MINS ,PERIOD)

* INPUT DATA IS TURLU THE TIME AS 24 HR. cLOCK.

C OUTPUT DATA IS THE HRS AND MINS OF 12 HR. CLOCK,

C WIITH INDICATION

C PERIOD = i FOR A.M.

C (I~s = 2 FOR P.M.

INTEGER TVALU, HRS, MINS, -PERIOD

MINS=TVALU- (TVALU/100) * i00

HRS=TVALU-MINS

C DETERMINE IF A.M. OR P.M.

IF (HRS.G.E.13) GO TO WELL

5PERIOD= 1

RETURN TO CALLER

i 0 HOURS=HRS - 12

PERIOD=2

RETURN

END

FIG. 1 A sample question from the Statement Tes t .

L E A R N I N G A FIRST C O M P U T E R L A N G U A G E 453

Following a consideration of the standard programming tasks students might be expected to be able to under take at the end of a F O R T R A N course, it was observed that there were strong parallels with the manipulative procedures studied by Coombs (1977). Most importantly they both involved:

(1) learning about a large number of individual operat ions and (2) the assembling of these operat ions in a particular order so as to achieve some

given processing objective.

It was therefore decided to test separately for the above two activities in our assessment of programming skills. This decision was supported by programming professionals at Liverpool who maintained that many students had difficulty in assembling language structures into a working program even though they were able to grasp the workings of individual structures. Two target tests were accordingly devised to assess pe r fo rm- ance on the sequential and the non-sequential aspects of programming. These were known as the "Logic Tes t" and the "S ta tement Test" . The same basic test formats were maintained for all individual p rogramming studies, although changes in detail were made as necessary.

The Statement Test consisted of three short F O R T R A N programs, together with a specification of the functions of each (Fig. 1). The instructions were in the correct order to run the program, but contained a number of errors (21 in 56 lines of program) randomly distributed throughout the complete test, but no more than one per line. The errors were all contained within a single s ta tement and were of the type that were likely to be caused by mispunching or minunderstandings concerning the syntax of F O R T R A N . The programs were presented on standard computer cards, the statements being printed out at the top of each card.

The Logic Test (Fig. 2) consisted of three short (19 line) F O R T R A N programs which were also punched and printed on cards. Each deck comprised a complete set of correctly-written instructions but were supplied in random order. Working f rom a specification of the operat ion of the program (including the function of the sub- routines), the subject had to reconstruct the original. Any permissible deviation from the "s tandard" order was allowed, although the programs were written so as to minimize the number of variations.

2.2. T H E S E A R C H FOR AN I N D I C A T O R TASK

It is reasonable to assume that individual differences in the learning of computing skills are related to the strategies adopted by the learner in handling computing information. A pattern of such strategies will make up an identifiable cognitive style.

The literature on cognitive style is extensive, although most writers make their definitions in terms of polar dispositions which seem to be variations on a theme. Examples of these are:

convergent thinking vertical thinking analytic verbal sequential field independence

- divergent thinking - lateral thinking - gestalt - spatial - simultaneous - field dependence

(Hudson, 1966) (de Bono, 1967) (Levy-Agrcsti & Sperry, 1968) (Paivo, 1971) (Luria, 1966) (Witkin, Dyk, Faterson,

Goodenough & Karp, 1962)

454 M. J. C O O M B S , R. G I B S O N A N D J, L. A L T Y

The problem is to produce a program which reads the three lengths of the

sides of a triangle, and decides if it is a right-angled triangle.

The input data comprises: the three lengths of the sides and an acceptable

tolerance (or error) such that

2 2 2 (A -B -C ) ~< Error

for a right-angle triangle, where A is the length of the largest side and

B, C are the lengths of the other sides.

10

100

200

300

MASTER MAIN

READS (5, I00) A,B,C,ERR

BIG=A

PUT THE LARGEST VALUE (SIDE) IN BIG

IF (B.GT.BIG) BIG=B

IF (C.GT.BIG) BIG=C

EVALUATE THE TEST CONDITION FOR RT. ANGLE TRIANGLE

TEST=2.0*BIG*BIG- (A*A+B*B+C*C)

IF (ABS (TEST) .LE.ERR) GOTO 10

WRITE (6,200) A,B,C

STOP

WRITE (6,300) A,B,C

STOP

FORMAT (4FI0. I )

FORMAT ( IX, I IHNO TRIANGLE, 3FI0.1 )

FORMAT (IX, 1 IHTRIANGLE OK,3FI0. i )

END

FINISH

FIG. 2. A sample program from the Logic Test .

It may be tempting to add, from within computer science, the distinction between "bottom-up" and "top-down" approaches to programming (e.g. Dahl, Dijkstra & Hoare, 1972; Hoc, 1977).

The above dichotomies appear to have many features in common but they are by no means identical. Each can be described in terms of two contrasting modes of cognitive functioning: (a) a mode that is active, analytical, articulated, specific and critical; (b) a mode that is passive, global, vague, diffuse and uncritical. However, as is pointed out by Wallach (1962), mental operations given an identical interpretation

L E A R N I N G A FIRST COMPUTER I . A N G U A G E 455

on this scheme do not always correlate and similar tests yield evidence for a given theoretical placing on one occasion but not on another. It was therefore considered important that any attempt to assess the correlation of computer users' cognitive style with computing performance should use a test of style which had several significant features in common with the computer learning task.

One writer who has devised a classification of individual differences in cognitive style using tasks which are in some measure similar to our programming tasks is Pask (1976). Pask makes his description of style within two basic dimemsions of human information processing: (a) a dimension concerned with the management of data selected from the world (attention); (b) a dimension concerned with the mental representation of that data (mental model building). The attention dimension draws the binary distinction between the local features of a subject material and the global features. The representation dimension draws the distinction between the representation of new information in terms of a description or in terms of a procedure to be used for its generation from previously acquired information. By relating these two aspects of information processing we get four cognitive modes (Fig. 3) and two general classes of cognitive style. Learners working primarily within the two procedure- building modes are termed operation learners and learners working with the description building modes are termed comprehension learners.

Local features

Operation Learning

Procedure budding

Operation Learning

Comprehension Learning

Descriplion building

Comprehension

Learning

Global features

FIG. 3. Schematic representation of information processing dimensions.

Pask (1976, pp. 84-85) gives the following description of operation and comprehension learning styles, which are seen as "clearcut but not dichotomous".

Comprehension learners pick up an overall picture of the subject matter, for example, in a taxonomy the number of classes, the type and number of items in a class, redundancies in the taxonomic scheme, relations between the distinguished classes, a clear picture of where information about items can be discovered. These learners may or (significantly) may not be able to perform the operations required

456 M. J. C O O M B S . R. G I B S O N A N D J. I_. A L T Y

to use the subject matter information . . . . Often enough, comprehension of a layout or framework exists in the absence of rules or operational meaning and perhaps in ignorance of details that have to be filled in if the taxonomy (or whatever) is to be used in practice.

Conversely, operation learners pick up rules, methods and details but are often unaware of how they fit together, still less of why they do fit together. Typically, operation learners have at most a sparse mental picture of the material. Their recall of the way they originally learned (insofar as they learned at all) is guided by arbitrary numbering schemes or accidental features of the tutorial information frames.

Pask argues that procedure building is usually most effectively undertaken using low-level, local information while description building is best undertaken with attention to global features. Operation learners therefore tend to polarize towards the procedure building/local features quadrant and comprehension learners polarize towards the description building/global features quadrant. In addition to operation and comprehension learning, Pask identifies a third class of learning activity. This is seen as independent of style and is thus described as "incidental" in the sense that it does not result from a systematic method of working through the learning material. The outcome of such learning will be memory for surface, and to some extent isolated, items of information.

The procedural nature of computing provides a conceptual link between a student's approach to learning and the two programming tests described above. As formerly stated, F O R T R A N is usually taught by first introducing students to individual language structures (DO loops, conditionals, etc.) and then requiring them, without significant assistance, to assemble the structures into a logical order to solve a problem. This latter activity is not formally taught. It was our contention that students who were primarily operation learners would have a dual advantage on the Logic Test but have no advantage on the Statement Test.

With reference to the Logic Test, an operation learner would be expected to pay close attention during the learning of a language structure to its internal logic. Information on such internal logic may come from testing the operation of the structure under varying conditions. The comprehension learner, on the other hand, would only attempt to remember the global features of the structure as given by the lecturer, making no close distinction between their essential features and those arising from their illustrative context. For example, an operation learner given illustrations of the use of the DO loop in which the counter variable was always incremented by one would be likely to infer that this may not necessarily be the case from considering the possible uses of iteration. However, the comprehension learner would be expected to accept the example as given and always work within that framework until he was presented with a different example. The operation learner would thus have an advantage when faced with the Logic Test, because he would have a clear idea of the essential features of the structure independent of context. Structures could therefore be readily assembled, without interference, to complete the problem program. The operation learner would also be expected to gain a second advantage by applying an operation strategy to the problem itself which would help him to isolate its essential structure.

L E A R N I N G A FIRST C O M P U T E R I . A N G U A G I " 457

The Statement Test would not be expected to be so sensitive to learning style. In this test all errors were contained within individual statements and were not related to the logic of the programs. It should therefore be possible to identify them from incidental learning of the surface features of the language. They would not " look right" to anyone who had been exposed to F O R T R A N , irrespective of his understanding of the workings of the language or of its global organization.

2.3. THE SPY-RING HISTORY T E S T - - A C A N D I D A T E FOR T H E I N D I C A T O R TASK

Pask provides an instrument for assessing the mix of comprehension/operat ion learning a student uses to complete a complex learning task. It is called the Spy-Ring History Test. The test is completed in one session of some length (221-h), the time being divided between a learning phase and a test phase. The learning material concerns changes in a communication network between five spies in three countries over three years. In the learning phase some background information relating to the character of each of the spies, and the polit ical/economic situation in each of the countries, is given to the subjects to read. This is followed by representations of the communication network between the spies. The information is presented one year at a time in both the form of a sample list of eight successive transactions and a directional graph showing the channels open between the five spies. The lists are of the form shown in Fig. 4.

1986

R = Ruritania T = Transylvania

O - Olympia

From To Dryden (T) Euclid (O) Euclid (O) Caesar (T) Byron (R) -- Ajax (R) Dryden (T) Euclid (O) Euclid (O) -- Caesar (T) Byron (R) . . . . Ajax (R) Caesar (T) -, Dryden (T) Ajax (R) * Byron (R)

FIG. 4. Representation as a transaction list.

The first three transactions would be interpreted as "The spy Dryden, in Transyl- vania, sent a message to Euclid, in Olympia; then Euclid, in Olympia, sent a message to Caesar, in Transylvania; then Byron, in Ruritania, sent a message to Ajax, also in Ruritania". Information concerning who can send to whom (as deduced from the occurrences of actual transmissions in the lists) can also be represented as a graph (see Fig. 5).

In this representation the order information is lost. Lists and graphs are presented simultaneously on an overhead projector.

Each subject is asked to reproduce both the list and the graph correctly as soon as it is removed from view. It is unlikely that subjects can do this after one exposure as they are deliberately overloaded with information, with the intention that they should

458 M. J'. COOMBS, R. GIBSON AND J. L. ALTY

AJOX

Dryden 0 ...~___....,..~0 Caesar

FIG. 5. Representation as directed graph.

explore alternative representations and adopt one convenient to themselves. Subjects are thus given as many exposures as they wish with the instruction to learn each year's list and graph to a criterion of one perfect recall. With group administration this poses something of a problem if, as is likely, the subjects learn at different rates.

Following the learning phase, subjects are asked to complete a test booklet. The questions test for rote learning of background information, reproduction of lists and graphs, and ability to make deductions from the lists and graphs (which may well have been forgotten).

3. Programming study I

3.1. METHOD

Subjects were all volunteers from a standard 2-week Introductory F O R T R A N Course run by the Computer Laboratory, University of Liverpool, for staff and postgraduates. Each Course session lasted for 3 h and included a 1�89 h lecture followed by a 1�89 h practical period. The 28 people attending the Course were asked on their first day to volunteer to take part in the study, which could "help the Laboratory design better programming courses". Sixteen people agreed to participate, although only 11 people completed both the Spy-Ring History Test and the computing tasks. All 16 had no previous experience of computing and were all scientists or engineers.

The Spy-Ring History Test was administered to small groups of 3-5 people during the first 4 days of the Course and the programming tests were administered on the final day. Half the people attending on the final day completed the Statement Test first, while the other half completed the Logic Test. Subjects were given a maximum of 30 min to complete each test. Three subjects who were identified as being strongly biased towards operation learning and three subjects who were biased towards comprehension learning (two of them strongly biased) were observed closely during the practical sessions on days 5-7 and were interviewed in some depth at the end of the Course. The same six subjects were also interviewed on the methods they employed to complete Spy-Ring.

3.2. RESULTS

The Statement Test was scored using the P(,~.) measure employed in signal-detection theory (McNicol, 1972, pp. 31-40). This measure took a line of text as a unit, a line

L E A R N I N G A F I R S T C O M P U T E R L A N G U A G E 459

containing an error being regarded as a "signal" trial and one which did not as a "noise" trial. Any line marked as containing an error was taken as a "yes" response, the absence of such a declaration was taken as a "no". A mark on a line containing an error is thus a "hit" and a mark on a line containing no error is a "false alarm". These variables were used to compute a non-parametric measure of sensitivity P(,~). Being a measure of probability the range of possible scores was from 0 to 1.

The Logic Test was scored using a measure based upon transitional error (Coombs, 1977). The cards returned by each subject were numbered according to their position in the original program. The error score was then calculated by counting the number of incorrect transitions in the program, e.g. the following permutation of the first five integers contains three incorrect transitions--1 2 4_ 3_ 5. Total transitional errors for all three test programs were then expressed as a proportion of possible transitions using the following formula

(max. T E - observed TE)/max. TE,

where TE is the transitional error score; the max. TE for a list of l eng thn = n - 1. This method of scoring only takes into account the first-order sequential dependen-

cies. Furthermore, it does not take account of possible reasons for misplaced cards, nor does it rate the "grossness" of any individual error. However, what it lacks in sensitivity it makes up for in objectivity and ease of scoring. This method of scoring has also been found to be most adequate in work on serial order recall in the learning of manipulative procedures.

The scoring of the Spy-Ring History Test is complex. The scoring scheme provided with the test provides for summarization at two levels. Raw scores to sets of individual questions are combined to give seven sub-scores, and these are in turn combined to give four main scales: neutral score (incidental learning), operation learning, comprehension learning and versatility (the ability to employ either strategy as appropriate). All scores are expressed as a proportion of the maximum possible score.

Means and standard deviations for the two F O R T R A N tests and the four Spy-Ring scores are given in Table 1. Comparisons were made between the four Spy-Ring scores and the two F O R T R A N tests using Pearson r Correlation Coefficients (Table 2). There proved to be no significant correlations between the Statement Test and any of the Spy-Ring measures. However, scores on the Logic Test correlated with operation learning and versatility at better than the 0.01 level. It was also noted that

TABLE 1 Summary of test scores for Programming Study I

Scores i s N = 11

Statement Test 0.92 0.050 Logic Test 0.58 0.180 Neutral score 0.70 0.1~3 Operation learning 0.57 0.132 Comprehension learning 0.63 0.138 Versatility 0.16 0.160

All scores range from 0 to 1

460 M. J. COOMBS, R. GIBSON AND J. L. ALTY

TABLE 2 Summary of comparisons between the programming tests and the spy-ring

scores

Comparisons r Sig. (N = 11) (d.f. = 9)

Statement Test with Neutral score 0-40 ns Operation learning 0.44 ns Comprehension learning 0.11 ns Versatility 0.14 ns

Logic Test with Neutral score 0.11 ns Operation learning 0.83 p < 0'01 Comprehension learning 0.13 ns Versatility 0.74 p < 0"01

there was no significant correlation between the two F O R T R A N tests (Pearson r = 0-30 ns).

3.3. DISCUSSION

It can be seen from "Fable 2 that our hypotheses concerning the Logic Test were supported, the operation learning score of the Spy-Ring test being correlated with it sufficiently at above the 0-01 level. On the other hand, there was no significant correlation between the Logic Test and either comprehension learning or the neutral score. It therefore appeared reasonable to seek guidance on the psychological requirements for the sequential aspect of programming from the theoretical basis of the operat ion/comprehension distinction. It should also be noted that there was a significant correlation at the 0.01 level between versatility and the Logic Test. This does not contravene any particular expectation. Although versatility was intended to represent an ability to use either of the learning strategies as appropriate, the Spy-Ring data indicated that the operation learning component predominates.

The results concerning the Statement Test were less clear. While there was no significant correlation between the test and either of the two styles of learning, there was also no significant correlation with the neutral score (incidental learning). However, it was noted that operation learning and the neutral score were both correlated with the Statement Test at a higher level than comprehension learning, although the two correlations were well below the 0.05 level of significance. It could thus be concluded that the operation and neutral scores tapped a greater proportion of the learning attributes that contributed to success on the Statement Test than comprehension learning but the variance accounted for was small. There could be two reasons for these correlations and it is not possible to choose between them on present evidence. The tirst concerns the operation of a ceiling effect on the Statement Test-- i t may be noted from Table 1 that the mean success was 92%--whi le the second concerns the presence of a fourth learning factor not measured by Spy-Ring. Given this problem it was decided to delay further interpretation of the results--including analysis of the observational and interview data--unti l we had tested their reliability by a replication on a second group of F O R T R A N learners.

LEARNING A FIRST COMPUTER LANGUAGE 461

4. Programming Study II

4.1. METHOD

The study was repeated using a second standard Compute r Labora to ry F O R T R A N Course, although on this occasion no formal a t tempt was made to observe a sample of subjects during practicals or to interview subjects on their learning activities at the end of the Course. Twenty-one users at tended the Course, all of them being completely new to programming. The backgrounds of the students were similar to those in the first study, being mainly physical scientists. All tests were administered by a different investigator to the initial study.

The only modification made to the test materials was the presentat ion of the Statement Test on a sheet of computer printout rather than on cards. The three programs were listed in order down the page, subjects being asked to underline errors on the sheet.

The drop-out rate unfortunately proved to be greater than in the initial study. Only 12 people stayed to complete the course and of those only eight had taken all our tests. We therefore had just sufficient numbers to serve as a replication but with much smaller numbers than intended.

4,2. RESULTS

The scoring of all tests was identical to that used in the initial study. A summary of results is given in Table 3.

TABLE 3 Summary of test scores for Programming Study H

Scores $ s N = 8

Statement Test 0.69 0.210 Logic Test 0-43 0-240 Neutral score 0.49 0.295 Operation learning 0.60 0.166 Comprehension learning 0.64 0.226 Versatility 0.16 0.148


Comparisons between the two programming tests and the Spy-Ring scores were again made using Pearson correlations. All comparisons are summarized in Table 4.

The replication again found that there was significant correlation between operat ion learning and the Logic Test but not between the comprehension learning and the Logic Test. However , it also produced a significant correlation between the two programming tests ( r = 0 . 8 3 , p < 0 . 0 1 ) , and between operat ion learning and the Statement Test.

4.3. DISCUSSION

The above replication may be judged to be partially successful. The study supported the earlier finding of a significant positive correiation between the operat ion learning scale of the Spy-Ring History Test and the test requiring the ordering of s tatements

462 M. J. COOMBS, R. GIBSON AND J. L. AI.TY

TABLE 4 Summary of comparisons between the programming tests and spy-ring

scores

Comparison r Sig. (N = 8) (d.f. = 6)

Statement Test with Neutral score -0-17 ns Operation learning 0.74 p < 0.05 Comprehension learning 0.22 ns Versatility 0.58 ns

Logic Test with Neutral score 0.03 ns Operation learning 0.74 p < 0.05 Comprehension learning 0.47 ns Versatility 0.49 ns

to form a working program (Logic Test). However, on this occasion the correlation between the Logic Test and versatility failed to reach significance at the 0.05 level. It should also be noted that there were other complications. Firstly the replication again did not support the hypothesis of a significant correlation between the Statement Test and the Spy-Ring neutral score. Secondly, on this occasion the Statement Test was found to correlate significantly with operation learning and with the Logic Test.

Taken alone the replication may be considered to provide less-than-convincing support for the hypotheses proposed at the end of section 2.2. Nevertheless, it does not negate the findings of the initial study; it suggests the usual picture of complications and qualifications absent from the first set of data. Overall, it does seem that the Spy-Ring History Test does isolate a component of cognitive performance which is relevant to the learning of computing skills. That it is able to do this with such a heterogeneous sample population is quite encouraging. Given these problems in interpretation, it was necessary to continue our analysis further. However, because of the complexities encountered in interpreting performance on the Statement Test, it was decided to reserve consideration of this task for a further study. We therefore concentrated on the Logic Test.

The two studies potentially generated rather more data than is reflected in the gross scores in the form of the individual items of the Spy-Ring History Test. The author of Spy-Ring provides for a set of rather tentatively labelled sub-scales but the nature of the test is such that the relationship of the individual items to the theoretical distinctions they purport to assess is often less than obvious. A further analysis of the Spy-Ring History Test was therefore undertaken to clarify such relationships.

5. Further analysis

5.1. A FACTOR ANALYSIS OF THE SPY-RING HISTORY TEST+

Advantage was taken of the availability of raw scores on individual question items of the Spy-Ring Test to investigate its internal structure, The sub-scales in the official

- All analyses in this section were performed using the Statistical Package for the Social Sciences (SPSS), Version 5, implemented on the 1906S at the University of Liverpool Computer I.aboratory.


scoring scheme are to some extent hypothetical, so it was of considerable interest to see if they were statistically valid. A total sample of 19 subjects was obviously inadequate for such an investigation. Although these were the only subjects for which computing skills scores were available, many more Spy-Ring Tests had been administered at Liverpool to computer users. A number of these were collected together to make a total sample of 34. This gave a 34 • 47 matrix which is, of course, not analysable by factor analysis. However , the 47 raw scores were easily reduced to 23 by elimination of some items (where questions gave scores which were perfectly correlated) and others could be summed (where grouping seemed both acceptable on a priori grounds and was supported by correlational data).

The 34 • 23 matrix was analysed using a principal components analysis followed by a varimax rotation using SPSS. Unity was used for the est imated communal i ty for each variable. This produced nine factors (criterion: A1.0); a scree test did not strongly indicate that the number of factors should be limited further, so this full set was retained.

Consideration of the factor loadings gave rise to the following interpretations.

Factor 1. Recall of both lists and graphs for the first two years of the spy-ring

There were wide variations in patterns of recall per formance over the three years. This factor may be interpreted as discriminating between individuals who failed to find a suitable representat ion for the spy-ring information, and thus suffered from increasing interference as they a t tempted to r emember information concerning each new year, and those who found a representat ion and improved their recall with each new year (see Factors 7 and 8 ) - - 1 9 . 5 % of variance.

Factor 2. Declared use of uninterpreted patterns and symmetries

This factor loads mainly on the questions asking the subject to declare whether he looks for patterns and symmetries in the spy-rings and uses these to work out the type of patterns which would occur later in the history of the spy- r ing - -11 .3% of variance.

Factor 3. Declared use of message lists rather than graphical information

This factor again loads on the subject 's declaration that he t reated the test as "a memory task relying upon messages lists and their o rde r ing" - -10 -0% of variance.

Factor 4. Declared use of the rules governing the transmission of messages

The main loading is on the declaration that the subject used "the rules for transmitting messages between spies in a given y e a r " - - 8 . 7 % of variance.

Factor 5. The rote learning of background information making up the neutral score

7.7% of variance.

Factor 6. Interpreted systematic changes over time

This factor mainly loads on a set of questions concerned with changes in the spy-ring over t i m e - - 6 . 3 % of variance.

464 ~I. J. COOMBS, R. GIBSON A N D J. L. ALTY

Factor 7. Use of systematic representation based on lists

This factor loads on recall of the list for the tinal year and the generation of a list for the fourth year of the spy- r ing- -5 .7% of variance.

Factor 8. Systematic representation based on graphs

This factor loads on recall of the final g r a p h - - 5 . 3 % of variance.

Factor 9. Declared insight into the rules governing temporal changes in the spy-ring

4.6% of variance. The groupings of Spy-Ring History Test items within the nine factors is in no way

similar to the groupings that make up the sub-scales in the official scoring scheme. It was therefore necessary to reassess the relationship between performance on Spy-Ring and the computing skills tests, particularly the Logic Test. To do this the factor score coefficients were used to compute factor scores for each of the 19 students in the two studies and these were correlated with the Logic Test scores. A summary of results is given in Table 5.

TABLE 5 Correlations between factor scores and the Logic

Test

Logic Test

Factor Sig. (N = 19) (d.f. = 17)

1 0.21 ns 2 0.10 ns 3 0.04 ns 4 0.43 ns 5 -0.24 ns 6 0.19 ns 7 0.55 p<0 .05 8 -0.04 ns 9 -0.12 ns

It can be clearly seen f rom Table 5 that factor 7 correlates with the Logic Test at above the 0.05 level and factor 4 correlates at just below the 0.05 level. It may be recalled that factor 7 is labelled as the "use of systematic representat ion based on lists", and factor 4 is labelled as the declared "use of rules governing the transmission of messages". We thus find additional support for our former interpretation of the significant correlation between operat ion learning and the Logic Test.

5.2. SOME P R E L I M I N A R Y CONCLUSIONS

Sufficient information has been collected from our three programming studies and the factor-analysis of the Spy-Ring History Test to a t tempt to characterize the strategies used by successful and unsuccessful learners of F O R T R A N . To do this we also have available data f rom the observation of six subjects with distinct biases towards operat ion or comprehension learning on their use of practical sessions, and


interviews with the same six subjects on their learning techniques and methods for completing the Spy Ring History Test. This data was collected during the first programming study. In order to focus our analysis, it was decided to consider first performance on the Spy-Ring Test.

Both operation and comprehension learners relied principally upon rote learning for the first year graphs and lists; they looked for simple patterns in the materials and constructed simple mnemonics to aid recall. However, subjects with a strong operation bias soon abandoned this method and sought to reduce the complexity of material, while subjects with a strong comprehension bias occupied themselves with the construction of increasingly elaborate mnemonics. These mnemonics were often constructed with reference to tire graphical representation, although this was by no means always the case.

The extreme operation learners appeared to pursue the reduction of complexity as a conscious goal rather than as a result of some other goal such as mnemonic building. They were also fairly uniform in their methods of reduction, which usually fell into two stages. Firstly, they abandoned any at tempt to remember the real-world scenario, including the names of spies, early in the test and concentrated upon learning an abstract version of the lists. Secondly, they sought to reduce the abstract lists to a small number of types of transactions between spies and a set of transformation rules which would generate the complete lists from the list of transaction types.

Contact with students during the F O R T R A N courses again indicated that those with a strong operation bias differed in definable ways from those with a strong comprehension bias:

operation learners gave priority during lectures to "getting the facts down on paper", while comprehension learners often took time out to work at understanding the information; operation learners completed more of the practical exercises than comprehension learners; operation learners accepted the exercises as given and solvable while comprehension learners often questioned their validity; operation learners often extended the problems once they had solved them as presented, while comprehension learners rarely experimented with the language outside the set exercises; operational learners rarely asked general or conceptual questions about the computer, while these were common from comprehension learners.

It is possible to explain all of the above differences in terms of the different mental coding strategies adopted by operation and comprehension learners for storing Spy- Ring material. Tire application of an operation learning strategy to F O R T R A N information gathered during lectures would produce a representation of language structures which was more abstract, and so less bound to the context of specific examples, than a comprehension strategy. It would therefore be expected that the operation learner would be more flexible in his problem-solving. He would thus be more likely to be able to solve a problem in the form given; the abstract nature of his representation of a language structure would also make him more likely to "play" with different applications of the structure. The comprehension learner, on the other hand, would be expected to represent the language structures in relatively concrete

466 M. J. C O O M B S , R. G I B S O N A N D J. L. A L T Y

terms, often with some detail f rom the examples used during instruction. Such representations would be difficult to apply to new problems. This may account for the questioning by comprehension learners of the validity of the problems.

From the results reported above it is possible for us to make four assertions concerning the learning of F O R T R A N as a first computer language.

(1) It is possible to define at least two different learning styles in a population of novice computer users.

(2) Students exercising one of the s ty les- -opera t ion learn ing- -are more successful at assembling language structures into an effective algorithm.

(3) The successful learning style is defined by close attention to detail and a preference for procedural representation.

(4) Success in the correct identification of individual language structures is independent of learning style.

However , these results beg a number of important questions which need to be resolved before they can be applied with confidence to helping computer users.

The initial problem relates to the statistical validity of the programming studies themselves. Both studies were conducted with very small samples of users (a total of 19) who were to some degree self-selected; we were only able to include those who possessed sufficient stamina to attend all lectures and complete all tests. The tests themselves, and in particular the Spy-Ring History Test, required a considerable investment in t ime and might only have been tolerated by able or highly motivated students. A second problem relates to the unsatisfactory nature of the Statement Test, in particular the presence of ceiling effects due to the insertion of rather obvious errors. A test of the full set of hypotheses concerning program construction and debugging would therefore require the preparat ion of an improved Statement Test containing errors which are more realistic and span a wider range of difficulty. A third problem is that each of the studies was conducted by a single investigator, thus there is the possibility of investigator bias in the collection of the data from practicals concerning the extreme operat ion and comprehension learners. In order to address all three of the weakness of the previous studies, we considered it necessary to under take a further replication using a revised Statement Test and more rigorous control of data collection.

6. Programming Study I I I

6.1. METHOD

A third study was planned using postgraduates and staff attending a similar Introduc- tory F O R T R A N Course to that employed for the previous two studies. Fourteen students volunteered out of a course membersh ip of 15, although only 10 students completed all tests. These included four postgraduate students from engineering, five physical science postgraduates and a postgraduate student from business studies. The Course was taught by a different lecturer to the two earlier courses but used similar instructional material. Students attended for nine afternoon sessions over 2 weeks.

In almost all respects the study employed a similar procedure to that of the previous work. Early in the course small groups of 2-3 subjects were administered the Spy-Ring History Test, and on the final day all subjects were given the Logic Test and a


re-designed Statement Test. The new Statement Test used three programs taken from an earlier unpublished study of F O R T R A N problem-solving and the errors seeded into the programs were selected from a list of real, single-line bugs found in these programs. The errors were selected to include bugs with various degrees of persistence. Ten bugs were seeded into each program (of approximately 35 lines). A sample question is given in Fig. 6.

MASTER MAIN C PROGRAM CALCULATES COMPOUND INTEREST OVER ONE YEAR, WITH THE INTEREST

C BEING COMPOUNDED AT FIVE DIFFERENT FREQUENCIES THROUGH THE YEAR.

DIMENSION FREQ 5 C SPECIFY DIFFERENT PERIODS IN THE yEAR

DATA FREQ(-I) /I.0/ ,FREQ(2) /2.0/,FREQ(3) /4.0/ ,FREQ(4) /12.0/ , [FREQ (5) / 3 6 5 . 0 / READ{S,100) PRIN,RATE

C LOOP THROUGH THE DIFFERENT PERIODS

DO 200 I=1,5

~ATE/100,0=R

X=PRIN

R=I.0+(R/FREQ(I))

JTOT:FLOAT (FREQ (I)) C WORK OUT SUM AT END OF YEAR AND WRITE OUT THE TOTAL RETURN

FOR 50 J=I,JTOT

50 X=X*R 200 WRITE(6,300) PRIN,RATE, JTOT,X

STOP

i00 FORMAT 2FI0.2) 300 FORMAT(XI,16HFOR A PRINCIPAL=,FI0.2~I211 AT A RATE=,FI0.2,

12H COMPOUNDED ,13,16H TIMES IN I YEAR,IIH RETUP2q IS,F$0.2)

END

FINNISH

1000.00 12.50

FIG. 6. A sample question from the new Slatement Test used in Programming Study III.

The Logic Test was administered on computer cards and the Statement Test was administered as a listing. No time-limit was imposed on the tests, although subjects were strongly urged to finish after spending half an hour on each. Half the subjects were administered the Logic Test first, while the other half were administered the Statement Test.

Subjects were also given two questionnaires not used in the two previous studies these being designed to collect the same data as was previously taken by observation and unstructured interviews. On the first day they were given a "Post-Registration Information Sheet" which was used to assess their prior knowledge of computing and ability at activities commonly held to influence the learning of computing. After the first 6 days of the Course, subjects were verbally administered a "Learning Activities Questionnaire". This was designed to assess the extent to which they engaged in learning activities which had in the previous studies been identified as being related to operation and comprehension learning strategies. Subjects were also verbally questioned on the techniques they had used for completing the Spy-Ring History Test. So far as possible, each data collection activity was supervized by a different person. Moreover, in order to prevent subjects from being influenced by experimenters' prior knowledge of their placing on the operation and comprehension learning scales, the Spy-Ring Tests were not scored until the Course had ended.

468 M. J. COOMBS, R. GIBSON AND J. L. ALTY

6.2. RESULTS

The Logic and S t a t e m e n t Tests were scored in an ident ica l m a n n e r to tha t used in the p rev ious two s tudies . A s u m m a r y of resul ts is given in T a b l e 6.

TABLt:. 6

Scoring of test scores for Programming Study III

Scores 2? s N = 10

Statement Test 0.81 0.073 I.ogic Test 0.52 0.164 Neutral score 0.63 0.294 Operation learning 0.67 0.177 Comprehension learning 0.65 0.183 Versatility 0.20 0.113


C o m p a r i s o n s b e t w e e n the two p r o g r a m m i n g tests and the S p y - R i n g scores were again m a d e using Pea r son r C o r r e l a t i o n Coefficients . These are givcn in Tab le 7.

TABLE 7 Summary of comparisons between the programming tests and the spy-ring

scores

Comparisons r Sig. (N = 10) (d.f. = 8)

Statement Test with Neutral score 0'25 ns Operation learning 0"49 ns Comprehension learning -0 .01 ns Versatility 0.06 ns

Logic Test with Neutral score 0.56 p < 0'05 Operation learning 0.63 p < 0-05 Comprehension learning - 0 . 0 6 ns Versatility 0 '40 ns

It can be seen f rom Tab le 7 that the re was again a signif icant co r re la t ion be tw e e n o p e r a t i o n l ea rn ing and the Logic Test . It was also no ted that on this occas ion the Logic Tes t co r r e l a t ed signif icant ly (p < 0 - 0 5 ) with the neut ra l score . T h e r e were no signif icant co r re l a t ions b e t w e e n the new S t a t e m e n t Tes t and e i the r l ea rn ing style, so r ep l i ca t ing the resul ts of the first p r o g r a m m i n g s tudy.

6.3. DISCUSSION

T h e a b o v e results conf i rm our h y p o t h e s i z e d r e l a t ionsh ip b e t w e e n o p e r a t i o n learn ing and the act ivi ty of p r o g r a m cons t ruc t ion , which a p p e a r s as the one s tab le re la t ionsh ip in all t h r ee p r o g r a m m i n g s tudies .

[ . E A R N I N G A FIRST C O M P U T E R L A N G U A G E 469

No significant correlation was found between the Statement Test and any of the Spy-Ring scales, including the neutral score. This replicates the findings of the initial study and, given the statistical acceptability of the new Statement Test, may be taken to indicate that the skills required for within-statement debugging are outside those evaluated by the Spy-Ring History Test.

Two notable deviations were observed between the three programming studies which need to be mentioned at this point. The most marked deviation between the earlier work and the present study is the reduction of the comfortable, although not necessarily significant, correlation between the Logic Test and the versatility score. However, the importance of this should not be over-estimated given the origin of the versatility measure. Sixty-seven percent of the score is obtained from deriving Pask's predictions for a fourth year of the spy-ring. There are, however, several possible predictions, all of which are consistent with both the rules for passing messages and the structure of the ring for previous years. With the present study three of the high-scoring operation learners made correct, although not allowable, predictions and so gained very low versatility scores. The second notable deviation was the finding of a significant correlation (p < 0.05) in the present study between the neutral score and the Logic Test. We are unable to explain this result, particularly given the absence of any indication in subjects' accounts of their method of completing the Spy-Ring History Test of any at tempt to use or remember the realistic details concerning the spies and their countries.

7. The acquisition of programming skills

7.1. A R E - A N A L Y S I S OF T H E SPY-RING H ISTO RY TEST

Before attempting a detailed account of the actual learning strategies employed by operation and comprehension learners during the F O R T R A N course, it was decided to check the reliability of the Spy-Ring History Test components isolated in section 5.1 and the reliability of the correlations between such components and the programming tests. A replication of the earlier correlations would add considerably to our confidence both in interpreting the Learning Activities Questionnaire and in defining strategies.

A factor analysis was conducted in an identical manner to that reported in section 5.1 but using an increased sample of 47 Spy-Ring Tests. The 4 7 x 2 3 matrix was analysed using a principal components analysis with a varimax rotation. This again gave nine factors (criterion: A 1.0), a scree test not indicating any further reduction in the number of factors. A consideration of the factors indicated that they were in almost all reports identical to the previous set but were differently ordered . t

Factor 1. Recall of both lists and graphs for the first two years of the spy-ring

19.2% of variance (old factor 1).

Factor 2. Use of rule based, systematic representation of lists

12.2% of variance (old factor 7 plus old factor 4).

+ See section 5.1 for an interpretat ion of factor names.

4 7 0 M . J . COOMBS, R. GIBSON AND J. L. ALTY

Factor 3. Interpreted systematic changes over time


Factor 4. Declared use of uninterpreted patterns and symmetries


Factor 5. Correct placement of spies in the appropriate countries

Without the use of rules (for transmitting messages) to reconstruct the spy-r ing--6 .9% of variance (new factor).

Factor 6. Declared use of message lists rather than graphical information


Factor Z Systematic representations based on graphs

5.3% of variance (this is similar to old factor 8 but is rather more uninterpretable).

Factor 8. The rote learning of background information making up the neutral score


Factor 9. Declared insight into the rules governing temporal changes in the spy-ring


Given the similarity between the above list of factors and the original set, it might be expected that a similar relationship would be found between the factor scores and the Logic Test over all 29 subjects who had completed the test. The relevant correlations are given in Table 8.

T A B L E 8

Correlations between factor scores and the Logic Test

Logic Test

Factor r Sig. (N = 29) (d.f. = 27)

1 0'23 ns 2 0'57 p<0.01 3 0-14 ns 4 -0"02 ns 5 0.10 ns 6 -0"03 ns 7 0'05 ns 8 0"02 ns 9 -0 '03 ns

It can be clearly seen that the only significant relationship is with factor 2, which includes both of the earlier factors which were found to correlate highly with the Logic Test (old factor 7 "Use of systematic representation based on lists" and old factor 4 "Declared use of the rules governing the transmission of messages"). It may

LEARNING A FIRST COMPUTER L A N G U A G E 471

thus be concluded, following the definition of operat ion learning proposed by Pask (1976), that the successful completion of the Logic Test is in some measure related to:

the systematic abstraction of critical features of programming structures after close attention to detail; the representat ion of structural relations in rule form.

It may be recalled that because of doubts concerning the validity of the Statement Test used in programming Studies I and II, no correlations were computed with the Spy-Ring factors. However , such a comparison was now possible using the more carefully constructed test used in Study III. Results are given in Table 9 for the small sample of 10 subjects who completed the study.

T A B L E 9

Correlations between factor scores and the State- ment Test

Statement Test

Factor r Sig. (N = 10) (d.f. = 8)

1 0.59 ns 2 0.35 ns 3 -0.72 p<0-05 4 -0.02 ns 5 0.52 ns 6 -0.07 ns 7 0.36 ns 8 0.13 ns 9 0.16 ns

The only significant correlation is between the Statement Test and factor 3. This factor accounts for systematic changes in the graphical representat ion of the spy-ring over time, the emphasis being upon the perceptual comparison between two or more graphs. This suggests that there ought to be a significant correlation between the Statement Test and comprehension learning, which we failed to find.

A second puzzle was set by a validatory replication computed at the same time as the above between the factor scores for the 10 subjects of Programming Study III and the Logic Test. Here, it was discovered that in addition to the significant correlation with the Logic Test and factor 2, there was also a significant correlation at over the 0.05 level with factor 1 "Recall of both lists and graphs for the first two years of the spy-ring" (Pearson r = 0.68).

7.2. AN ANALYSIS OF Q U E S T I O N N A I R E DATA

In section 5.2 evidence was presented of differences in the use of lecture and practical sessions by operat ion and comprehension learners attending a conventional F O R - T R A N programming course. This evidence was gathered from close observation of and discussion with three subjects with a strong operat ion bias and three subjects

472 M . J . C O O M B S , R . G I B S O N A N D J. L. A L T Y

with a strong comprehension bias. It appeared that the operat ion learners used the lectures to gather "un in te rpre ted" facts about F O R T R A N structures. These were recorded as presented, with little a t tempt made to understand the structures outside the context of examples used by the lecturer. However , operat ion learners made extensive use of the practical sessions to help abstract the essential features of individual structures and to elaborate the possible functional relationships between them, i.e. to exper iment with possible combinations of structures to achieve some given computa- tion. The comprehension learners, on the other hand, tended to concentrate most of their learning activity on the lectures. Here they would frequently take " t ime-ou t" from attending to the lectures in order to decide on the essential features of a structure or to a t tempt to understand the use of a structure. Little use would be made of the practicals to test the validity of conclusions reached during the lectures.

The evidence for the above assertions was collected by observat ion and by interview, the exper imenter concentrating upon six subjects who were known to have a strong bias towards operat ion or comprehension learning. Such methods are inherently unreliable, there being a strong possibility of exper imenter bias. It was therefore decided to a t tempt to collect data on learning techniques in a more objective manner. As repor ted in section 6.1, questionnaires were employed to this end. These included:

a "Post-Registrat ion Information Sheet" (PRIS) designed to assess subjects ' prior knowledge of computing and their ability at activities commonly held to influence the learning of computing (see Appendix A) and a "Learning Activities Quest ionnai re" (LAQ), which was designed to assess the extent to which subjects differed in their use of practicals and lecture t ime (see Appendix B).

The PRIS indicated that none of the students on the F O R T R A N Course had any significant prior experience of computer operat ions or programming, with the excep- tion of one person who had occasionally run pre-prepared programs (this person failed to complete all tests and so was dropped from the study). This finding was confirmed by the question asking students to assess their knowledge of computing terms. Only three students out of the original 14 volunteers (21%) claimed to have a " G o o d Unders tanding" of more than three of 20 common computing terms. The student with the greatest knowledge was also the person who had run pre-prepared programs and who was dropped from the study. Although a large proport ion of the students (43%) claimed "Some Unders tanding" of many of the terms (50%), this was usually found to amount to little more than recognizing them. Finally, the PRIS was used to assess students ' knowledge and experience of activities commonly held to be related to computing. There was found to be no significant correlation (Pearson r = 0.24, ns) of such knowledge with experience and knowledge of computing terms.

From the above, it appears that we can discount prior contact with computers and computing f rom influencing subjects ' learning of F O R T R A N . With this assumption we continued with an analysis of the "Learning Activities Quest ionnaire" . Following the earlier study of learning activities, it was decided to at first restrict the analysis to subjects with a strong bias (20% or above) towards operat ion or comprehension learning and to consider the full sample only if a substantial number of differences were found with subjects showing marked stylistic bias. The analysis was thus initially conducted on a small sample of three operat ion learners and three comprehension


learners. One additional learner with a strong operation bias was discounted because he scored at a very low level on both scales and therefore could not be taken as exemplifying either style. While the LAQ offered some support for the differences in strategy discussed in section 5.2, the analysis was on the whole disappointing and so was largely restricted to the sub-sample of six. Statistical comparison was therefore not possible. Results should thus be taken as a basis for speculating upon differences in learning strategy rather than as sound indications of the nature of such differences.

The first interesting finding was that all three operation learners declared that they expected the practicals to provide their main source of F O R T R A N learning, in contrast to the three comprehension learners who favoured either lectures alone or a balanced combination of lectures and practicals. This finding held when extended to the full sample of 10 subjects. However, in contrast to the earlier study, comprehension learners proved in reality to be as frequent in their at tendance at practical sessions as the operation learners. The most likely explanation for this was the high quality of conceptual support given by an experienced demonstrator at these sessions. The effect of this support on attendance was particularly noticeable after the third day of the Course, comprehension learning often dropping out of the practicals at this point in the earlier studies.

The LAQ offered some evidence for the contention that operation and comprehension learners make different use of lecture and practical sessions. All three operation learners declared that they regarded lectures as primarily an opportunity to record facts about F O R T R A N , while one comprehension learner declared that he aimed at obtaining a general overview of the language and only one concentrated on F O R T R A N facts (Q6). This difference in emphasis is also reflected in the action taken by a learner when he thought of a query or comment during a lecture. It appeared that while comprehension learners would often take t ime-out during the lecture to work at the problem or write it down for later consideration, operation learners were prepared to forget the point. For them the task of recording facts had priority (Q7 and Q10).

All people on the F O R T R A N Course appeared to seek considerable support during the practicals from demonstrators. However, the LAQ indicated that operat ion and comprehension learners may well have sought different types of information. The three operation learners declared that they usually asked for factual information about F O R T R A N , while the comprehension learners asked for facts, conceptual information and details about the Liverpool computer (Q16). Moreover, the LA Q also indicated that while the operation learners tended to begin the programming exercise by mapping out and coding the critical procedures of the program, the comprehension learners tended to start from I / O procedures----often including the F O R M A T s ta tements- - and worked through the program in a sequential manner (Q18).

8. Strategies for making sense

The present research was motivated by two objectives:

(i) to provide data on the nature of computing information and on the cognitive skills required for its acquisition and

(ii) to enable the prior identification of users needing special attention so that they could either be given an individual training program or be instructed in the appropriate study skills.

474 M . J . C O O M B S , R . G I B S O N A N D J. L. A L T Y

At the outset we considered the first of these to be our main objective and developed our research methodology accordingly. However, in the event we were more successful at achieving the second objective. The research has thus provided a method, albeit a rather inelegant one, of identifying users who have difficulty learning F O R T R A N from conventional courses and has indicated possible means of improving their learning. Moreover, the fundamental nature of these difficulties makes it highly likely that the results will generalize to the acquisition of other computing skills and languages.

The initial hypotheses were framed in terms of relationships between learning style, as defined by Pask (1976), and two classes of learning activity which we considered to be necessary for programming competence:

the learning of individual language structures; learning to assemble these structures into a particular order to achieve a given processing objective.

The original study was thus couched in terms of individual predispositions towards learning specific classes of information, these being determined by learning style. The study was thus expected to yield correlations between pre-defined information processing characteristics of the learner and performance at given programming skills, including the debugging of individual statements and the assembly of programs. The stylistic distinction used (operation learning and comprehension learning) was defined in terms of the level of detail to which the student habitually attends during instruction and the form of representation he adopts for processing information (rules or descriptions).

At the stylistic level the research produced one stable relationship which ran through all three programming studies. This was between operation learning and a test of program construction (Logic Test). Interpreting this result using Pask's (1976) theory and our own factor analysis of the test of style (Spy-Ring History Test), we concluded that students who best acquired the skill of assembling statements into programs were those who:

paid close attention to detail; systematically abstracted the critical features of programming structures; represented structural relations in rule form.

Using correlations between cognitive style and programming performance, it was thus possible to identify some of the elements of a successful learning strategy. However, using this evidence alone, it was not possible to identify the dynamic aspects of a successful strategy. Such a dynamic account is necessary if guidance is to be provided for unsuccessful students.

There is an alternative approach to building a dynamic model of F O R T R A N learning. Our studies have provided a description of two different classes of strategic element: those concerned with mental processing (arising largely from the correlational comparisons between learning style and program performance) and those concerned with learning activities (arising from observation, interview and questionnaire). From these two classes of element it is possible to assemble model strategies for operation and comprehension learners. Such model strategies are in many respects similar to the competence models of linguistics in that they offer idealised but self-consistent and phychologically plausible accounts of successful and unsuccessful processing. In


the presence instance our model strategies also have sufficient empirical foundation to provide a basis both for the planning of further research and for guiding instruction.

Learning to use any symbol system, including a programming language, requires the student both to identify acceptable structures within the system progressively and to apply these structures to some processing objective. Moreover , there is also evidence that the competent use of a symbol system implies the ability to perceive structures at many different levels (Van Dijk, 1977). A symbol system may, therefore, be said to have both microstructure (e.g. an assignment statement in F O R T R A N ) and a macrostructure (e.g. the set of statements which, taken together, will conduct a bubble-sort). Green, Sime & Fitter (1981) emphasize the importance of macrostructure for the writing and debugging of programs, arguing that a novice programmer needs to learn not only the basic structures of a language but also useful global arrangements of code. We would further propose that the strategies employed by operation and comprehension learners will result in differences in the level of structure learned and hence to differences in programming skill.

How may the strategies employed by operation and comprehension learners during an introductory F O R T R A N course be characterized? Firstly, operation learners appear to make a clear distinction between the learning activities to be used in lecture and practical sessions. The lecture sessions are largely used for recording facts, great care being taken to ensure that the statements and examples of the lecturer are recorded accurately and in full. Examples are also recorded with great care, the operation learner's priorities being (a) to get the example copied, (b) to follow the lecturer's analysis of the logical relations between statements and (c) to consider alternative methods of achieving the same processing objective. In (b) and (c) the operation learner works from "inside" the code, paying close attention to the logical relations between statements. By following these activities, the student leaves the lecture with an accurate and relatively uninterpreted record of the structures discussed and the examples given to illustrate their use. Moreover, the internal logic of most of the examples will be understood in detail.

The major learning activity of operation learners appears to take place during practicals. Here they continue to build their knowledge of the functional relations between low-level language structures so leading to a knowledge of useful macrostructures. This again appears to be achieved from the "inside" by, for example, seeking various different combinations of statements which will achieve the same processing objective. Operation learners thus build up considerable knowledge of the computa- tional possibilities of combinations of statements and rapidly develop a rich sense of macrostructure. During this work the computer is itself used as the instructor, reference only being made to external information such as the principles behind the local implementation of F O R T R A N or the design of the local operating system when prompted by a program failure which cannot otherwise be corrected. Through use of the practicals, the operation learner develops an understanding of the logic of program design (including knowledge of various levels of program structure) and skills related to debugging and the avoidance of errors.

Comprehension learners, on the other hand, appear to adopt a very different distribution of activity. During lectures the comprehension learner attempts to achieve what the operation learner reserves for practical sessions, i.e. an understanding of the critical features of F O R T R A N structures and their relations. The implication of this


is that the comprehension learner engages in a significant amount of "on line" processing and therefore is rarely fully engaged in simply reporting the facts and examples given by the lecturer. He rarely takes verbat im notes, rather recording the abstracted or generalized results of his processing of the lecture material. Such processing is, of course, carried out without the aid of the computer . Significant structure is therefore determined by reference to external sources of information from other related domains such as F O R T R A N design principles or details concerning the local machine. Where relevant information is not available, the comprehension learner tends to create determining principles for himself rather than store the low-level information in unintegrated form. The comprehension learner thus leaves the lecture with representations of language structures which have been determined to some extent by factors external to F O R T R A N and which have never been validated by an actual computer run. Moreover , his knowledge of structure does not develop beyond this point because he also fails to take advantage of practicals to conduct such validation. The comprehension learner thus leaves a course with a descriptive knowledge of individual, low-level structures but no relaible knowledge of macrostructure. Macrostructure will be represented as a generalized outline of how a processing objective may be reached, with supporting external information, but no knowledge of the operat ional details.

These descriptions of strategy, one successful and one unsuccessful in the context of the type of introductory course considered in this paper, pull together our various findings within the f ramework of operation and comprehension learning style. The accounts are relatively specific and can readily be t ransformed into hypothesis for further research. Such research is at present being undertaken at Liverpool, using different computing tasks and classes of user.

The above accounts of strategy lead to some specific recommendat ions concerning the design of courses. It may be argued that all students should be lead towards using an operational strategy by:

pre-instruction in the strategy as a technique for learning; closely supervising all students during practicals, ensuring that they engage in the range of activities which are naturally undertaken by operat ion learners; pegging the lectures to a hand-out giving the details of all language facts and structures to be used in practicals.

However , it has been found by Pask (1976) that it is very difficult for students to change their learning style and when they do they are significantly less effective than those for whom it is "natural" . In this case it would perhaps be more satisfactory to leave comprehension learners with their natural style but to:

provide them with accurate conceptual and supporting information, and encourage them to complete the problem solving exercises in a structured environ- ment with significant guidance from tutors.

The choice between these alternatives must remain a matter for empirical research. Finally, it may be objected that it is p remature to make general proposals for the

design of courses on the basis of the study because of the unique characteristics of F O R T R A N as a computer language and the specific nature of the Liverpool courses. While we accept the possibility of this, and are at present extending the research into

LEARNING A FIRST COMPUTER LANGUAGE 477

other areas of comput ing to test the generalizabil i ty of findings, we believe that the effects of opera t ion and comprehens ion styles are sufficiently distinct that similar differences will be found with o ther languages and comput ing situations. A l though there may be ditferences in detail, we would maintain that radical differences in at tent ion and representa t ion will p roduce simpler process ing characterist ics within a given content domain. So long as compute r p rog ramming requires the precise specification of code and languages fail to make macros t ruc ture perceptual ly evident to the novice and are implemented on machines which are not easily model led by the novice, we believe similar effect will be found.

We would like to thank Brian Walsh and Alan Dawson of the Computer Laboratory, University of Liverpool for preparing problems for the programming tesls. Research for this paper was supported by Social Science Research Council grant number I tR 4421.

References

ALTY, J. L. & COOMBS, M. J. (1981). Communicating with univeristy computer users: a case study. In COOMBS, M. J. & ALTY, J. L., Eds, Computing Skills and the User Interface. London: Acadcmic Press.

BEI_I., D. (1976). Programmer selection and programming errors. ('omputer Journal, 19, 202-206.

BRUNER, J. S., GOODNOW, J. J. & AUSTIN, G. A. (1956). A Study of Thinking. New York: Wiley.

COOMBS, M. J. (1977). Modality and order recall in learning from television, with reference to teaching medical procedures. Ph.D. Thesis, University of Liverpool.

DAHL, O. -J., DIJKSTRA, E. W. & HOARE, L. A. R. (1972). StructuralProgramming. London: Academic Press.

DE BONO, E. (1967). The Uw of Lateral Thinking. London: Jonathan Cape. DU BOUI_AY, B. & O'SHFA, T. (1981). Teaching novices programming. In COOMBS, M. J.

& AI,TY, J. L., Eds, Computing Skills and the User Interface. London: Academic Press. GOULD, J. D. (1975). Some psychological evidence on how people debug computer programs.

International Journal of Man-Machine Studies, 7, 151-182. GREEN, T. R. G., SIME, M. E. & FITTER, M. J. (1981). The art of notation. In COOMBS,

M. J. & ALTY, J. L., Eds, Computing Skills and the User Interface. I,ondon: Academic Press. H o c , J. -M. (1977). Role of representation in learning a programming language. International

Journal of Man -Machine Studies, 9, 87-105. HUDSON, L. (1966). Contrary Imaginations. London: Penguin. LANG, K., ALTY, J. L. & LANG, T. (1979). The Provision of Guidance to Computer Users in

Universities. Final Report to the SSRC, Grant No. HR4421. LEVY-AGRESTI, J. & SPERRY, R. (1968). Differential perceptual capacities in major and

minor hemispheres. Proceedings of the National Academy of Sciences, 61, 1151. LURIA, A. R. (1966). Higher Cortical Functions in Man. New York: Basic Books. MCNICOL, D. (1972). A Primer of Signal Detection Theory. London: Allen & Unwin. MAYER, R. E. (1976). Some conditions of meaningful learning for computer programming:

advance organizers and subject control of frame order. Journal of Educational Psychology, 68, 143-150.

NEWELL, m. & SIMON, H. A. (1972). Human Problem Solving. Englewood Cliffs, New Jersey: Prentice-Hall.

PAIVIO, A. ~1971). Imagery and Verbal Processes. New York: Holt, Rinehart & Winston. PASK, G. (1976). Conversation Theory: Application in Education and Epistemology. Amsterdam:

Elsevier.

478 M. J. COOMBS, R. GIBSON AND J. I,. ALTY

SIME, M. E., GREIZ.N, T. R. G. & GUEST, D. J. (1977). Scope marking in computer condi- t ionals-psychological evaluation, b~ternational Journal of Man-Machine Studies, 9, 107- 118.

VAN DtJK, T. A. (1977). Semantic macrostructures and knowledge frames in discourse comprehension. In JUST, M. A. & CARPENTER, P. A., Eds, Cognitive Processes in Comprehension. New York: Wiley.

WALL.ACI-I, M. A. (1962)~ Commentary: active-analytical vs. passive-global cognitive functioning. In MESSICK, S. & ROSS, J., Eds, Measurement in Personality and Cognition. London: Wiley.

WITKIN, H. A., DYK, R. B., FATERSON, H. F., GOODENOUGH, D. R. & KARP, S. A. (1962). Psychological Differentiation: Studies of Development. London: Wiley.

YOUNGS, E. A. (1974). Human errors in programming. International Journal o/Man-Machine Studies, 6, 361- 376.

L E A R N I N G A FIRST C O M P U T E R L A N G U A G E

Appendix A: Post-Registration Information Sheet

479

Native Language .........................

Department ..............................

**************

Course .................. ..... ..... ......

Experimental number .....................

Status (Lecturer/Postgraduate/etc.) ......................

Sex ..................................... Liverpool User Number ...................

Area of Research ...................

Have you used a computer of any kind?

If 'NO', turn to page 2.

Tick as appropriate

YES

NO

Please state briefly the type of computer(s) you have used and the extent of

your computing experience.

.............. ,.,.,o.~ ~176 ....... ,...,.........o...,~

.............. o.......~176 .... - .... �9 ..... .~176 ............... .~176

.............. o.. ..... ,.. ..... .......~176 ..... ,....~ ..... .....~

......,,. ................. ...., ..... . ..... .~176176 ~176176176 ~

Have you used a computer interactively?

(That is, from a terminal)

Have you used a minicomputer?

Have you used a microprocessor?

Have you prepared your own job control cards on

any computer system that you have used prior

to this course?

Have you written a computer program?

Frequently A few Never

times

480 M, J. C O O M B S , R . G I B S O N A N D J. L. A L T Y

Have you used any of the following programming languages

prior to this course?

Frequently

FORTRAN

ALC~L 60

ALGOL 68

COBOL

BASIC

Any other programming languages (please specify)

.. ....... , ..... ..~

........ ....~

......... . ....... ......

Have you used any of the following packages?

SPSS

SYMAP

GLIM

GHOST

GENESYS

NIMBUS

Any oth~ ~ckages (~ease specify~

........... ..... ..... ..

~ ..... --------o~176

.......... .~176 ..... ...

~ve you used a calcula~r?

Have you programmed a calcula~r?

Have you used a t~e~iter?

~ve you used a card punch or paper tape punch?

A few Never

times


Have you attended courses in any of the following general areas?

At At Little University A-level or none

Computer Science

Mathematics

Logic

Foreign Languages

Linguistics

Electronics

What are your reasons for attending this course?

a) You have an immediate need to write programs

b) You expect in future to have to write programs

c) General interest and information

How important is it to you that you learn FORTRAN?

a) Essential

b) Fairly important

c) Might be useful

Please assess your ability at each of the following activities.

Good Average

Chess

Draughts

Solving crossword puzzles

Building models from kits

Amateur electronics

Jigsaws

Remembering directions

Learning foreign languages

Poor


Please list below any other skills or courses or experiences that you think might help you in learning FORTRAN.

Below is a list of concepts which have a particular meaning in computing.

Please indicate how far you understand what each one means.

Good Some

Inderstanding Understandinc

Address

Assignment

Bit

Byte

Compiler

Debug

Filestore

Hardware

Iteration

Loop

Low-level language

Microprocessor

Mill

Monitoring file

Peripheral

Program

Register

Software

STACK

VDU

No

Idea

L E A R N I N G A F I R S T C O M P U T E R L A N G U A G E

Appendix B: Learning Activities Questionnaire

483

We are interested in the way you have used the lectures and practicals given over the past week in your learning of FORTRAN. For example, we would

like to know whether you took verbatim notes during the lectures and whether

these gave you adequate information for completing the problems during the

practicals. Your answers to such questions will enable us to design better

courses and so minimise the time you require to get started with FORTRAN.

I am going to ask you a number of questions on your learning activities

over the past week. These can mostly be answered by selecting a point on a

rating scale or by selecting one of a list of responses. In some cases you may find that you need to select several of the standard responses to a

question. After each of your answers I may ask you for a brief elaboration.

This will be recorded for later analysis. However, your answers will be

confidential and to ensure this you will be known by the experimental number

you were given on the first day.

A.

B.

PRELIMINARY QUESTIONS.

I) At the start of the course did you expect that:

i) you would get more out of the lectures than the

practicals?

ii) you would get more out of the practicals than

the lectures?

iii) you would find both to be equally useful?

iv) other? .......................................

2) Have your expectations been confirmed?

I I

I I

I ] I I Y ? N

I I I I LECTURES. Please would you allow us to have copies of your notes.

3) On average What % of your time during a lecture period was occupied

with note taking?

0 I-i0 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100

4) Was this% consistent over all lectures?

If not, what was the shortest time you spent taking notes

and what was the longest time.

Y N

Max Min


5) What was the main objective underlying your note taking:

i) to obtain as full and accurate a record of the

lecture as possible?

ii) to obtain a summary of the main FORTRAN facts?

iii) to record the main generalizations about FORTRAN?

iv) other (including no real objective)? ..............

........... ..... ...................................

6) Do you regard the lectures as primarily an opportunity to:

i) obtain specific facts about FORTRAN structures?

ii) obtain a general overview of the language?

iii) obtain information about potential uses of FORTRAN?

iv) other (incldding none of the above)? ..............

7) Did you often deliberately take "time-out" during the

lectures?

If yes, do you do so in order to:

i) think about some teaching point?

ii) work through examples?

iii) other? ....................................... ....

.......... o ....................... ~ ....... ..~.o.o.

8) Did you take a full and detailed copy of visuals?

9) Did queries often occur to you during the lectures?

I0) If a query occurred to you during the lecture, did you:

i) ask the lecturer at once to explain the point?

ii) note the point and ask for clarification at the

end of the lecture?

iii) note the point and expect its answer to become

clear during the course of the lecture?

iv) forget the point?

v) other? ............................................

.......... o....~ ...................................

If

I 1 i I I I

II I I I I I I Y N

FT]

I I I I I I

Y N

Y N

I I I I

I I

I I I I


Ii) Did you often need to refer to your subject area to help understand the lecture?

12] When learning about a particular language structure, do you :

i) concentrate on getting the coding details right? I I

ii) think about how it might work in the computer?

iii) think of possible uses for it in your subject area?

ivl oth ............................................... 11

Y N

12a) When you think of a possible use of a FORTRAN structure during a lecture, do you:

i) try to work it through during the lecture? [ I

ii) jot it down to be worked on later? I }

iii) forget about it?

iv) other ............................................

13) How difficult did you find the lectures?

Easy 1 2 3 4 5 Very difficult

C. PRACTICALS

14) What % of the exercises did you complete over the past week?

0 i-i0 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100

I I I I l L I I I 1 15) When you have completed a programming exercise, do you usually:

i) goontothenextproblem? II

ii) ~y a different solution? [ I

iii) modify the problem and attempt a new solution? I I l l

iv~other? ............................................ 11

. . . . . . . . . . . . . . o . ~ . . . . . . . . . , . , . . . . . . . . . . ~ . . . . ~ . . . . . .

16) Did you often need assis~nce during the practicals?

If so, did you usually ask for

i) information on FORTRAN facts?

ii) information on FORT~N concepts?

Y N

M


D)

iii) operational details of the Liverpool computer?

iv) other? .............................................

............. o ....... o ..... o~176176176176176 ...... ~ ....

17) How helpful did you find your lecture notes during the

practicals?

No use 1 2 3 4 5 Very helpful

18) Do you usually start a programming problem

i) from the input or output procedures?

ii) from the critical process operations of the program?

iii) other? ............................................

........... o.o~176176 ..... ~ ..... o ..... ~ ..... ~176

GENERAL

19) Were there any lectures or practicals you were not able to

attend?

Lectures

n

I I I I I I

Practicals

20) Would you recommend any changes to the Course?

21) How well do you think you will be able to solve programming

problems within your subject area at the end of the course?

Not at all 1 2 3 4 5 Very well

22) Have you found the course documents (handouts, booklets etc.)

useful?

If so, how do you use them?

23) How do you regard the practical sessions?

Waste of time 1 2 3 4 5 Very Useful

Y N

Documents

Learning a first computer language: strategies for making sense