How do software engineers understand code changes?

How Do Software Engineers Understand Code Changes? An Exploratory Study in Industry

Yida Tao (HKUST), Yingong Dang (MSRA), Tao Xie (NCSU)

Dongmei Zhang (MSRA), Sunghun Kim (HKUST)

2

3

> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()

“Why this change here? This is the only one that doesn’t seem to make sense for

me…”

4

> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()

“Why this change here? This is the only one that doesn’t seem to make sense for

me…”

>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;

“What is this used for, I can’t spot it in use anywhere and every component and service

seems to have it set to false.”

5

> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()

“Why this change here? This is the only one that doesn’t seem to make sense for

me…”

>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;

“What is this used for, I can’t spot it in use anywhere and every component and service

seems to have it set to false.”

> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from

this patch, aren’t they?”

6

> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()

“Why this change here? This is the only one that doesn’t seem to make sense for

me…”

>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;

“What is this used for, I can’t spot it in use anywhere and every component and service

seems to have it set to false.”

> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from

this patch, aren’t they?”

>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;

“I’m not sure why you walk this char by char, javascript has awesome string

methods… ”

7

> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()

“Why this change here? This is the only one that doesn’t seem to make sense for

me…”

>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;

“What is this used for, I can’t spot it in use anywhere and every component and service

seems to have it set to false.”

> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from

this patch, aren’t they?”

>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;

“I’m not sure why you walk this char by char, javascript has awesome string

methods… ”

8

> if (hudRef && hud) {> if (hudRef.consolePanel) {> + hudRef.consolePanel.hidePopup()

“Why this change here? This is the only one that doesn’t seem to make sense for

me…”

>+ struct CIDEntry>+ {>+ const nsCID* cid;>+ bool service;

“What is this used for, I can’t spot it in use anywhere and every component and service

seems to have it set to false.”

> browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html“These files are missing from

this patch, aren’t they?”

>+ for (var i = aURL.length – 1; i >= 1; i--) {>+ var chPrev = aURL.charAt(i – 1) ;>+ var ch = aURL.charAt(i) ;

“I’m not sure why you walk this char by char, javascript has awesome string

methods… ”

“…”

“…”

“…”

9

“How do software engineers understand code changes?”

10

Research Questions RQ1: How frequent is code change

understanding practiced and in which development tasks it is required?

RQ2: What are engineers’ information needs and difficulty for understanding code changes?

RQ3: How to improve the effectiveness and efficiency of the practices in understanding code changes?

11

Study Methodology

Literature Review

• Potential information needs

Questionnaire Design

• Investigate RQ1, RQ2

Pilot Interview

• Question is relevant & clear

12

Study Methodology

Literature Review

• Potential information needs

Questionnaire Design

• Investigate RQ1, RQ2

Pilot Interview

• Question is relevant & clear

Online Survey

• 16% response rate (180 respondents)

Follow-up Interview

• Investigate RQ3

Analysis

• Answering RQs

13

Survey Participants

Dev55%

Test31%

PM14%

Role Distribution Product Team

OS

Desktop App

Web App

Mobile App

Service

Others

14

RQ1 • Frequency ?• Development tasks ?

RQ2 • Information needs ?• Difficulty ?

RQ3 • Improvement ?

RQs

15

RQ1: Frequency of Understanding Code Changes

How often do you need to understand code changes?o Several times each houro About once an houro Several times each dayo About once a dayo Several times each weeko About once a weeko Rarelyo Never

16

RQ1: Frequency of Understanding Code Changes

Several times each hour

About once an hour

Several times each day

About once a day

Several times each week

About once a week

RarelyNever

05

101520253035404550

DevTestPM

Abso

lute

# o

f re

sponse

s

17

RQ1: Tasks Requiring Code Change Understanding

“Select the top three tasks that most often require you to understand code changes” [Design/Planning] Refactoring [Implementation] Developing new feature [Implementation] Fixing bug [Integration] Resolving merge conflict [Verification] Reviewing others code changes [Verification] Reviewing my own code changes [Verification] Writing & updating test cases Other, please specify

18

RQ1: Tasks Requiring Code Change Understanding

Reviewing others' changes

Fixing bug

Developing new feature

Reviewing my own changes

Writing/updating test cases

Refactoring

Resolving merge conflict

0% 15% 30% 45% 60% 75%

121

100

89

73

48

34

30

Percentage of participants who select the task

19

RQ1• Frequently practiced• Major development

tasks

RQ2 • Information needs ?• Difficulty ?

RQ3 • Improvement ?

Answers to RQs

20

Potential Information Needs Literature review (code-change analysis and

management) 180 articles in 10 SE venues over the past decade

21

Potential Information Needs Literature review (code-change analysis and

management) 180 articles in 10 SE venues over the past decadeReasoning & assessing the

change• Completeness• Clones• Design• …Exploring the change’s context & impact• Risk• Consistency• Tests• ...

Evaluating the change history• Change-proneness• Defect-proneness

22

Survey Questions “Rate the importance & difficulty of each

information need (formulated as question) in a change understanding task”

Very Importa

nt

Important

Somewhat

ImportantNot

Important

3

2

1

0

23

Survey Questions “Rate the importance & difficulty of each

information need (formulated as question) in a change understanding task”

Very Importa

nt

Important

Somewhat

ImportantNot

Important

Very Difficult

Difficult

Relatively Easy

Straightfor-ward

3

2

1

0

24

Survey Questions “Rate the importance & difficulty of each

information need (formulated as question) in a change understanding task”

Does this change introduce code clones?

Does this change break any code elsewhere?

Which tests should be run to verify this change?

Is this changed location a hotspot for past fixes?……

Very Importa

nt

Important

Somewhat

ImportantNot

Important

Very Difficult

Difficult

Relatively Easy

Straightfor-ward

3

2

1

0

25

RQ2: Information Needs

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

26

RQ2: Information Needs

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

RiskConsiste

ncy

Completeness

Design

27

RQ2: Information Needs

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

RiskConsiste

ncy

Completeness

Design

Rationale

28

RQ2: Information Needs

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

RiskConsiste

ncy

Completeness

Design

Rationale

Change-proneness

Defect-proneness

29

RQ1• Frequently practiced• Major development

tasks

RQ2• Risk & Quality are

important but difficult to know

RQ3 • Improvement ?

Answers to RQs

30

RQ3: Interview Items

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

Risk

Rationale

Change-proneness

Defect-proneness

31

Determining a Change’s Risk

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

Risk

Current Practice on Determining a Change’s Risk

32

Manual Code Review• Error-prone• Cross-components• Unclear interface• Hidden assumptions• …

Unit & Regression Testing• Time consuming• Depends on how thorough the tests are• …

33

Support Determining a Change’s RiskM

anual co

de

revie

w • Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change

34

Support Determining a Change’s RiskM

anual co

de

revie

w • Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change

Diff

Code Analy

sis

Navigation in diff

“…miss a level of understanding object

relationships”

35

Support Determining a Change’s RiskTe

stin

g • which code must be retested as it is dependent upon the change?

• who owns testing that dependency?

• which tests must be run?

“An ‘Intelli-sense’ for updating these (affected) tests would be nice as well.”

36

Discussion

1 2 30

1

2

3

Importance

Diffi

cu

lty o

f acq

uir

ing

th

e

info

rmati

on

Rationale

Change-proneness

Defect-proneness

Discussion

37

• Availability & Quality of commit message• “It’s entirely up to the dev making the change as to

how hard or easy it is for someone else to figure out why the change was made.”

Why is understanding the rationale of a change easy?

• Developers• Here and now• Short-term issue• Own Knowledge

• Testers & PMs• Historical metrics might be good to reflect

bugginess and complexity of a specific area

Why are historical metrics not that important?

38

Other Information Needs

“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”

39

Other Information Needs

“Can this change be broken into smaller discreet

changes?”

“In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?”

40

Composite Code Change

41

Understanding a Composite Code Change

42

Decomposing a Composite Code Change

43

RQ1• Frequently practiced• Major development

tasks

RQ2• Risk & Quality are

important but difficult to know

RQ3• Determining a

change’s risk• Decomposing a

composite change

Answers to RQs

44

Summary Evidence

Understanding code changes is a fundamental practice that happens frequently in major development tasks

45

Summary Evidence

Understanding code changes is a fundamental practice that happens frequently in major development tasks

Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change

46

Summary Evidence

Understanding code changes is a fundamental practice that happens frequently in major development tasks

Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change

Practical Needs Navigation in diff Change decomposition Available & informative commit message

47

Acknowledgment

All participants of survey / interview

Miryung Kim, Robin Moeur, Thomas Zimmermann, Jacek Czerwonka, and Kathryn McKinley