Reading & understanding code experts are better at code comprehension because they focus on higher level patterns – patterns can be considered “discourse

1

reading & understanding code

• experts are better at code comprehension because they focus on higher level patterns– patterns can be considered “discourse rules”– naming conventions, design patterns, schemas

• experts work significantly better when reading & writing code according to these patterns

2

reading & understanding code

program comprehensionexpertise effectsmental models

tools

3

outline

• mental models– types– models

• conventions & “discourse rules”• expertise effects• tool implications• interesting tools

4

outline



5

mental model

• explanation of a someone’s thought process when carrying out a task– our someone: programmers– our task: program comprehension

• several models exist

6

mental model classes

• bottom-up– read code statement by statement then ascend for

a higher-level picture• top-down– start with a high-level picture of what the code is

doing then descend into code• mixed– incorporate elements from both, based on the

situation

7





situation

8

bottom-up mental models

• 1st: read code statements• 2nd: chunking: group statements as abstractions• 3rd: repeat

9

chunkingsequence

chunk 1 chunk 2 chunk n

element 1 element 2 element k

modified from wikipedia

10

chunking

• program model– reasoning about the order of computation, how

control moves throughout a program– “control flow”

• situation model– reason about how data moves through atomic

models– “data flow”

N. PenningtonStimulus Structures and Mental Representations in Expert Comprehension of Computer ProgramsCognitive Psychology, 1987

11

program & situation model studies

• participants first primed for either control flow or data flow– shown a piece of code, asked to recall another piece of

code which is related through either control flow or data flow

• participants then asked a question that relates to either control or data flow

• participants primed to think about control flow answered other control-flow questions faster, same with data flowN. Pennington

Stimulus Structures and Mental Representations in Expert Comprehension of Computer ProgramsCognitive Psychology, 1987

12

types of programmer knowledge

• semantic: general programming concepts– low-level knowledge, e.g. what a=1 means– high-level knowledge, e.g. sorting algorithms

• syntactic: language detail– overlaps between languages

• stylistic: programming conventions– “discourse rules”

B. Shneiderman and R. MayerSyntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental ResultsJournal of Computer & Information Sciences, 1979

E. Soloway, K. EhrlichEmpirical Studies of Programming KnowledgeIEEE Transactions of Software Engineering, 1984

13

problem statement program

short term memory

internal semantics (working memory)

knowledge (long term memory)

syntactic knowledge

COBOL

FORTRANPL/I

LISP

semantic knowledge

high level concepts

low level concepts


high level concepts

low level concepts

14

evidence forsemantic & syntactic knowledge

• lab studies using FORTRAN– participants: programmers and non-programmers– asked to perform tasks that used one type of

knowledge– six studies (will describe two)


program memorization

• study– two subject types: non-programmers & programmers– two program versions: normal & shuffled– participants asked to memorize a program

• results– non-programmers performed equally poorly with normal & shuffled

programs– programmers performed poorly with shuffled program, well with

normal• were able to remember semantic details with syntactic variations

• conclusion– programmers were not memorizing the program, but internal

semantics to represent its functionB. Shneiderman and R. MayerSyntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental ResultsJournal of Computer & Information Sciences, 1979

16

commenting• study

– two program versions• 5-line high-level block comment at top• numerous interspersed low-level comments

– participants asked to make modifications to program & memorize program

• result– high-level comment participants performed better– strong correlation between ability to make modifications and ability to

memorize• conclusion

– memorization is a strong correlate to comprehension– hierarchical chunking to organize statements into a unit facilitate

comprehension processB. Shneiderman and R. MayerSyntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental ResultsJournal of Computer & Information Sciences, 1979

17





situation

18





situation

19

top-down models

• 1st: develop hypotheses about the program• 2nd: evaluate and refine hypotheses– with the help of beacons

• 3rd: repeat

• a process of “reconstructing knowledge”

beacons

• “indexes into existing knowledge”• recognizable features in that are cues to the

presence of certain structures• e.g., looking for a listener pattern

M. StoreyTheories, Methods, and Tools in Program Comprehension: Past, Present, and FutureIEEE Workshop on Program Comprehension, 2005

R. BrooksTowards a theory of the comprehension of computer programsInternational J. on Man-Machine Studies, 1981

21

beacon types

• semantic knowledge “plans”– reusable generic program fragments– high-level or low-level

• programming discourse conventions– “rules” that make program comprehension easier– found across programmers


22

brooks’ model


modified from Jonathan I. Maletic’s slides:An Overview of Mental Models for Program Understanding

requirement documentation

internal representation –hypotheses and subgoals

design documentprogram

code

verify internal schema vs external representation

external representation

beaconsbeaconsbeacons

syntactic knowledge

problem

semantic knowledge

match

23





situation

24





situation

25

opportunistic & systematic strategies

• programmers enhancing existing program• two strategies:– systematically read code in detail, tracing through

control and data flow manually• developed control and data flow knowledge

– focus only on code relevant to a task• developed only control flow knowledge, resulted in a

weaker understanding

Margaret-Anne StoreyTheories, Methods, and Tools in Program Comprehension: Past, Present, and FutureInt. Workshop on Program Comprehension, 2005

integrated model

• maintainers switch between top-down and bottom-up comprehension– top-down if code or code type is familiar– program model (control-flow) when code is

completely unfamiliar– situation model (data-flow) after a partial data-flow

understanding is developed through top-down or program model methods

– knowledge base: information from previous three modelsA. von Mayrhauser and A.M. Vans

From Program Comprehension to Tool Requirements for an Industrial EnvironmentIEEE Workshop on Program Comprehension, 1993


28

validating the integrated model

• taped professional maintenance programmers– worked with a large code base– classified as domain and language experts

• tape transcriptions classified into model types• one of few studies with real world tasks

29

outline



30

outline



31

programming discourse rules

• specify the conventions of programming– e.g., a variable’s name should reflect its function– e.g., don’t include code that won’t be used

• similar to writing discourse rules, as outlined in books like Elements of Style– e.g., you expect to find the description for fig. 7

between those for fig. 6 and fig. 8


32

rules of programming discourse

1. variable names should reflect function2. don’t include code that won’t be used

a. if there is a test for a condition, then the condition must have the potential of being true

3. a variable that is initialized via an assignment statement should be updated via an assignment statement

4. don’t do double duty with code in a non-obvious way5. an if should be used when a statement body is

guaranteed to be executed only once, and a while used when a statement body may need to be repeatedly executed


33

testing discourse rules

• lab study with expert & novice programmers• two program types– α (plan-like): obeyed discourse rules– β (un-plan-like): disobeyed discourse rules

• participants given either α or β code, with one blank

• task: fill the blank with what seems “natural”– participants were not told about α or β code

• conclusion: experts fared best with α code

34

why have un-plan-like (β) code?

• machine limitations– limited memory, processing, bandwidth, etc.

• language limitations– less common. bugs, efficiency issues, etc.

• programmer limitations– does not have full mastery of discourse

• historical traces– resistance to changing legacy code, permanent

“temporary” codesource:The Psychology ofComputer Programming

35

XXX: PROCEDURE OPTIONS(MAIN);DECLARE B(1000) FIXED(7,2),

C FIXED(11,2),(I, J) FIXED

BINARY;C = 0;DO I = 1 TO 10;

GET LIST((B(J) DO J = 1 TO 1000));

DO J = 1 TO 1000;C = C + B(J);END;

END;PUT LIST(‘RESULT IS ’, C);END XXX;modified from The Psychology of

Computer Programming

36

XXX: PROCEDURE OPTIONS(MAIN);DECLARE A(1000) FIXED(7,2),

C FIXED(11,2),I FIXED BINARY;

C = 0;GET LIST((A(J) DO I = 1 TO

10000));DO I = 1 TO 10000;

C = C + B(I);END;

PUT LIST(‘RESULT IS ’, C);END XXX;

modified fromThe Psychology ofComputer Programming

37








38








39

naming conventions

• meaningful names– variable naming reflects cognitive structure

• grammatical sensibility– interact with language spec. to form expressions

• containers & paths– objects & pointers

• polysemy, homonymy, & overloading– operators, name sharing

B. Liblit, A. Begel, and E. SweetserCognitive Perspectives on the Role of Naming in Computer ProgramsPsychology of Programming Interest Group, 2006

40

naming conventions

• meaningful names– variable naming reflects cognitive structure

• grammatical sensibility– interact with language spec. to form expressions

• containers & paths– objects & pointers

• polysemy, homonymy, & overloading– operators, name sharing


41

meaningful names

• metaphors for domain tasks– e.g. pushing objects onto a stack

• keywords for grouping– e.g. common prefixes & suffixes

• informative names– balanced with name length

A. BlackwellMetaphor or analogy: how should we see programming abstractions?Psychology of Programming Interest Group, 1996

B. Liblit, A. Begel, and E. SweetserCognitive Perspectives on the Role of Naming inComputer ProgramsPsychology of Programming Interest Group, 2006

42

name length

• length harm readability and recall ability• idioms and memory ties improve readability

and recall ability

• takeaway: variable names with consistent and abbreviated vocabulary are optimal– (variable names that concisely express a metaphor)

D. Binkley, D. Lawrie, S. Maex, and C. MorrellIdentifier length and limited programmer memoryScience of Computer Programming, 2009

43

grammatical sensibility

• names as phrase fragments– methods as actions (change state of program)

• e.g. addElement, setSize, removeAll

– methods as mathematical functions (compute result, don’t alter state)• e.g. true/false: contains, equals, isEmpty• e.g. data: capacity, indexOf, size

• valence cues (phrase fragments w/ open slot)– e.g. roster.contains(player)– smalltalk makes use of this extensively:

• roster insert: player at: positionB. Liblit, A. Begel, and E. SweetserCognitive Perspectives on the Role of Naming in Computer ProgramsPsychology of Programming Interest Group, 2006

44

outline



45

outline



46

20:1 programmer performance

• Sackman et al.: best programmers are 20x better than worst programmers @ bug fixing– study originally meant to evaluate the

effectiveness of time-shared systems

H. Sackman, W. J. Erikson, and E. E. GrantExploratory experimental studies comparing online and offline programming performanceCommunications of the ACM, 1968

47

10:1 programmer performance

• there are substantial programmer efficiency differences, but not as dramatic as initially reported

• what makes experts so much better at understanding code?

48

testing discourse rules

• lab study with expert & novice programmers• two program types– α (plan-like): obeyed discourse rules– β (un-plan-like): disobeyed discourse rules

• participants given either α or β code, with one blank

• task: fill the blank with what seems “natural”– participants were not told about α or β code

49

α problem

PROGRAM Magenta(input, output)VAR Max, I, Num INTEGERBEGIN

Max = 0.FOR I = 1 TO 10 DO

BEGINREADLN(Num)If Num Max THEN Max = Num

ENDWRITELN(Max).

END

?


50

α solution


Max = 0.FOR I = 1 TO 10 DO

BEGINREADLN(Num)If Num > Max THEN Max = Num

ENDWRITELN(Max).

ENDE. Soloway, K. EhrlichEmpirical Studies of Programming KnowledgeIEEE Transactions of Software Engineering, 1984

51

β problem


Max = 999999.FOR I = 1 TO 10 DO

BEGINREADLN(Num)If Num Max THEN Max = Num

ENDWRITELN(Max).

END

?


52

β solution


Max = 999999.FOR I = 1 TO 10 DO

BEGINREADLN(Num)If Num < Max THEN Max = Num

ENDWRITELN(Max).

ENDE. Soloway, K. EhrlichEmpirical Studies of Programming KnowledgeIEEE Transactions of Software Engineering, 1984

53

percentage of correct responses

alpha

beta

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

advancednovice


54

debugging differences between novices and experts

• experts: situation-dependent problem solvers

• novices: situation-independent problem solvers

I. VesseyExpertise in Debugging Computer Programs: An analysis of the Content of Verbal ProtocolsIEEE Trans on Systems, Man, Cybernetics, 1986

55

outline



56

outline



57

tool implications

• browsing support– browse from high to low level and low to high level

• searching– looking for snippets by analogy

• multiple views– show orthogonal object relationships

• context-driven views– determine best view based on context

• additional cognitive support– external devices to support cognitive tasks neededMargaret-Anne Storey

Theories, Methods, and Tools in Program Comprehension: Past, Present, and FutureInt. Workshop on Program Comprehension, 2005

58

tool implications







59

browsing support

• traverse control and data flow paths• switching between top-down and bottom-up

models• breadth-first and depth-first

60

tool implications







61

tool implications







62

searching

• search for code snippets– not just by text

• example: query the role of a variable, when a function is called

• useful for top-down hypothesis testing

63

tool implications







64

tool implications







65

multiple views

• multiple ways of viewing programs– call graph– object hierarchy– etc.

• different views are applicable for different tasks

66

tool implications







67

tool implications







68

context-driven views

• alter views based on program metrics– size of program– interdependence of modules– flatness of hierarchy– etc.

69

tool implications







70

tool implications







71

additional cognitive support

• experts:– tools to support cognitive tasks• external devices• scratchpads

• novices– pedagogical support• programming language• task domain

72

outline



73

outline



structured editors

• reduce burden or memorizing syntax– focus on semantics

A. Ko and B. MyersCitrus: A Language and Toolkit for Simplifying the Creation of Structured Editors for Code and DataUIST, 2005

77

literate programming

• source code interwoven with exposition of logic, like an essay

• allows programmers to work top-down or bottom-up

D. KnuthLiterate ProgrammingJournal of Computer & Information Sciences, 1979

78

The purpose of wc is to count lines, words, and/or characters in a list of files. The number of lines in a file is ......../more explanations/ Here, then, is an overview of the file wc.c that is defined by the noweb program wc.nw: <<*>>= <<Header files to include>> <<Definitions>> <<Global variables>> <<Functions>> <<The main program>> @ We must include the standard I/O definitions, since we want to send formatted output to stdout and stderr. <<Header files to include>>= #include <stdio.h> @


79

conclusion

• beginners start off with an incomplete mental model for how code works



80

discussion

• what other discourse rules can you think of?• do these mental models resonate with your

style of understanding code?• what are some other tool implications of

these models?

81

references - 1H. Sackman, W. J. Erikson, and E. E. GrantExploratory experimental studies comparing online and offline programming performanceCommunications of the ACM, 1968


A. BlackwellMetaphor or analogy: how should we see programming abstractions?Psychology of Programming Interest Group, 1996




N. PenningtonStimulus Structures and Mental Representations in Expert Comprehension of Computer ProgramsCognitive Psychology, 1987


82

references - 2A. von Mayrhauser and A.M. VansFrom Program Comprehension to Tool Requirements for an Industrial EnvironmentIEEE Workshop on Program Comprehension, 1993


I. VesseyExpertise in Debugging Computer Programs: An analysis of the Content of Verbal ProtocolsIEEE Trans on Systems, Man, Cybernetics, 1986

A. Ko and B. MyersCitrus: A Language and Toolkit for Simplifying the Creation of Structured Editors for Code and DataUIST, 2005

83

does visual programming help?

non-significant result42%

significant result46%

significant result, but contribution of

AV uncertain8%

significant result in wrong direction

4%

C. Hundhausen, S. Douglas, J. StaskoA meta-study of algorithmvisualization effectivenessJournal of Visual Languages & Computing, 2002

84

underlying questions

• how do programmers read and come to understand unfamiliar code?

• what kinds of mental models to programmers create to think about code?

• why are experts significantly better than novices when looking at unfamiliar code?– hint: experts aren’t as good as you might expect!

85

why does it matter?

• reading code is done when:– searching for relevant code– re-acquainting oneself with a project– reading someone else’s code– refactoring– …

86

the gist of the talk

• beginners start off with an incomplete mental model for how code works



87

var Dict = function() {this.keys = [];this.values = [];

};

Dict.prototype.set = function(key, value) {var keyIndex = this.keys.indexOf(key);

if(keyIndex<0) {this.keys.push(key);this.values.push(value);

}else {

this.values[keyIndex] = value;}

};

Dict.prototype.get = function(key) {var keyIndex = this.keys.indexOf(key);if(keyIndex>=0) return this.values[keyIndex];return undefined;

};

88

mental models

top-down models• 1st: hypothesize about code• 2nd: check hypotheses• start on a high level, dig in

bottom-up models• 1st: read code statements• 2nd: mental chunking• start on a low level, ascend

hybrid models• incorporate elements from

both, based on the situation

89

shneiderman & mayer’s model

• semantic knowledge: general programming concepts– low-level knowledge, e.g. what assignments do– high-level knowledge, e.g. algorithms

• syntactic knowledge: programming language details– sometimes overlaps across programming langs.


90

brooks’ model

• “top-down”– analyze code on a high level, then look at specifics

• argues that programmers form a series of hypotheses

• beacons help verify or reject these hypotheses


91

containers & paths


92

polysemy, homonymy, & overloading


Documents

Reading & understanding code experts are better at code comprehension because they focus on higher level patterns – patterns can be considered “discourse