Investigating Pair Programming in a Software Engineering Course in an Asian Setting

Raymund Sison College of Computer Studies

De La Salle University Manila, Philippines sisonr@dlsu.edu.ph

Abstract

Despite the growing attention given to agile methods and practices in developed countries, there is an exiguity of empirical studies showing their successful (or unsuccessful) use in the developing world. After reviewing the literature on agile practices in developing countries, especially in Asia, this paper adds to the scant literature by describing an experiment on the use of pair programming by students in an undergraduate software engineering course at a Philippine university. Results show that defect densities of programs written by pair programming teams were significantly lower than those written by teams that followed the traditional approach of individual coding of units followed by unit and integration testing. The results, which tend to support the positive outcomes of similar studies in the U.S. despite cultural differences, underscore the importance of seriously investigating the use of pair programming and other agile practices in academic as well as industrial settings in Asia. 1. Introduction

Agile methods (e.g., XP [3]) and practices (e.g., pair programming [26]) are increasingly gaining attention in the United States [2], Europe [1], Japan [10], and other developed countries. This is not surprising since agile methods are designed to produce quality software more rapidly and to accommodate change more easily compared to the more traditional “plan-driven” [5] methods, which are mostly based on the waterfall software process model that was predominant during the last three decades.

Meanwhile, the offshore outsourcing of software development and maintenance appears to have become a mainstream phenomenon [27, 16]. With companies in the U.S. (and other developed countries) “shifting tens

of thousands of IT jobs to countries such as India, China, Russia, and the Philippines” [27, p.2], it behooves us, therefore, to begin studying whether and how agile methods are being used as well as taught in these outsourcing destinations, most of which are in the developing world.

This paper first reviews studies on the use of agile methods or practices in developing countries, both in industry and in the academe. It then reports on the experiences of undergraduate students in the use of pair programming in a private university in the Philippines. A survey conducted by the Weissman Center for International Banking at Baruch College and the Paaras Group showed that the Philippines ranked second to India as the top IT outsourcing destination among North American and European companies in 2004 [11]. 2. Review of Agile Studies in Developing Countries

Not surprisingly, there is a dearth of research papers on the use of agile methods and practices in the developing world. Of the more than 200 papers presented in the past XP conferences, for example, there were only two papers that described agile software development experiences in developing countries. Again, not surprisingly, both were case studies. The first of these [13] described in 2004 how a particular agile practice, test-driven development, improved overall team performance in a software firm in China. Test-driven development, which is one of XP’s 12 practices [3], involves writing tests prior to coding, and repeatedly running the code on these tests until the code passes all of them.

The second case study [7] reported in 2005 how XP’s 12 practices were implemented in a software development project of a start-up firm in Brazil. The said project also used some other agile practices not

2008 15th Asia-Pacific Software Engineering Conference

1530-1362/08 $25.00 © 2008 IEEE

DOI 10.1109/APSEC.2008.61

325

part of XP, notably what the authors called retrospectives, which appear to be a cross between the scrum and sprint review meetings of Scrum [21], another agile software development method.

Outside of the XP conferences, [20] analyzed in 2006 the use of pair programming, another one of XP’s 12 practices, in a particular project of a multinational firm in Thailand. Pair programming involves two developers working together at one computer, with the one who has the keyboard and mouse serving as “driver”, writing and revising code, and the other serving as “navigator”, pointing out errors and thinking strategically.

Finally, [23], following one of the recommendations of [24], reported in 2007 on the use of two agile methods, XP and Scrum, in two software firms in the Philippines. Aside from confirming the results of many studies that indicate that agile methods can improve quality and productivity, [23] also demonstrated that agile methods could be successfully used for offshore outsourced software development. It also suggested that a level-2 Capability Maturity Model Integration (CMMI) maturity can be helpful in agile offshore development, but higher levels (CMMI levels 3-5) may actually lead to the death of some agile practices, including pair programming, in a software organization.

These four papers seem to indicate that the benefits observed in the use of agile methods and practices in developed countries, as reviewed for example in [22], might also be applicable in developing countries, including those in Asia. This is rather interesting given the cultural differences between programmers in Asian and western countries that Hofstede [8] detected, particularly in the area of individualism/collectivism. The U.S. and U.K., for example, are highly individualistic cultures, in which ties between individuals are loose, so individuals look only after themselves and their immediate families [8]. In contrast, Asian countries are generally collectivistic and so societies here have people who, from birth onwards, are integrated into strong, cohesive in-groups, often extended families, which continue protecting them in exchange for unquestioning loyalty [8].

3. Review of Pair Programming Studies in Developing Countries

Pair programming is an essential XP practice that

Williams and Kessler [26] define as follows: “Pair programming is a style of programming in

which two programmers work side by side at one computer, continually collaborating on the same

design, algorithm, code, or test. One of the pair, called the driver, is typing at the computer or writing down a design. The other partner, called the navigator, has many jobs, one of which is to observe the work of the driver, looking for tactical and strategic defects. Tactical defects are syntax errors, typos, calling the wrong method, and so on. Strategic defects occur when the driver is headed down the wrong path—what is implemented just won't accomplish what needs to be accomplished. The navigator is the strategic, long-range thinker… Another great thing is that the driver and the navigator can brainstorm on-demand at any time. An effective pair programming relationship is very active. The driver and the navigator communicate… at least every 45 to 60 seconds. It's also very important to switch roles periodically between the driver and the navigator.”

Several experimental and case studies conducted in

the late 1990s and early 2000s (e.g., [19], [6], [12]) have suggested several benefits of pair programming, including higher software quality, greater developer productivity, greater developer satisfaction and confidence, enhanced learning (particularly on the part of the novice developer), and reduced staffing risk (on the part of the software organization). There were to be sure dissenting opinions; in the experiment of [18], for example, pair programming did not lead to higher programmer productivity compared to the use of the Personal Software Process (PSP) [9] in the coding of the first four programming exercises of the PSP.

More recently, experimental results of [1] fine-tuned these earlier statements by showing that, although pair programming might not, in general, increase productivity (by reducing the time required to solve programming tasks correctly) or increase quality (by increasing the proportion of correct solutions), pair programming can increase software quality when the software being built is relatively complex and junior developers are on the team, and can increase developer productivity when the software being built is relatively simple and senior developers are on the team.

The growing attention being given to agile methods by software firms in both developed and developing countries has not escaped the academe. Several experimental and case studies (e.g., [14], [17]) suggested that, in addition to the benefits mentioned earlier, pair programming can also lead to higher rates of completion of programming and software engineering courses, and to reduced workload for teaching staff. However, most of the published studies on the subject refer to research conducted in the United States. In the literature review of [15], for example, of 17 studies from 2001 to 2005 on pair programming in the classroom, none could be found that was conducted

326

outside the U.S., though [15] itself was actually a report on a pair programming study in a New Zealand university. There is actually only one study [20] that we could find on the use of an agile practice in a university in a developing country. There is, therefore, an even greater scarcity of research on the use of agile practices in the academe than in the industry in developing countries.

[20] reported on a study of pair programming and software inspection conducted at a university in Thailand. In the said study, the seven student teams that used pair programming spent less time than the student teams that used software inspections in developing code of similar quality (i.e., code passing the same number of test cases). Moreover, the number of problems found (by a teaching assistant) in the artifacts submitted by the pair programming student teams was less than those found in the artifacts of the software inspection teams.

This paper describes in the next section a study conducted in a Philippine university on the use of pair programming in the development of relatively complex software.

4. Research Design

The purpose of this study was to obtain an

understanding of the impact of pair programming on third year Computer Science undergraduate students using the said practice for the first time. The subjects of the experiment were 24 students enrolled in a one-trimester Advanced Software Engineering course taught by the author from September to December 2006 to third year B.S. Computer Science students at the De La Salle University - Manila. The said course presented the various software methods on Boehm and Turner’s [5] spectrum of plan-driven and agile methods, including the Personal Software Process (PSP), Team Software Process (TSP), Capability Maturity Model Integration (CMMI), Unified Process (UP), XP, and Scrum. The said course also introduced students to important trends in software engineering, such as pattern, aspect, and component-oriented software development, and global software development.

The course had two programming requirements. The first was to do, individually, five (of the 10) carefully designed programming exercises of the PSP [9], a software development and process improvement method that is on the plan-driven end of Boehm and Turner’s [5] agility spectrum. Being on the said end of the spectrum, the PSP is very plan-oriented and documentation-intensive. Through this programming

requirement, the students were trained to log every defect in every phase of software development (i.e., in analysis, design, coding, and test), as well as to log the time – productive as well as interruption time – they spent in every phase. From their defect and time logs, they could compute their defect densities (number of defects detected during compile and test per thousand lines of code) and productivity (lines of code (LOC) per hour).

The second programming requirement of the course was to develop, as a class, a Unified Modeling Language (UML) Tutor. The class was divided into groups of 4, and each group had to develop within two months modules for the UML Tutor. A module could either: (1) allow users to draw a particular UML diagram (e.g., a use case diagram) or (2) check these diagrams against reference diagrams in a database and provide the user with feedback according to the differences detected between his/her diagram and the reference diagram. The basic screen layout of a module is shown in Figure 1. The resizable upper right window is where the user would draw a UML diagram using the buttons on the toolbar on the left. The resizable lower window is where the system’s feedback on the user’s diagram would be displayed. The resizable upper left window is where the problem specifications are displayed.

The students themselves formed their groups, similar to what was done in [14]. Each group chose a different module to develop. Each group also divided itself into two pairs. One pair developed the group’s module using pair programming. The other pair developed the same module using traditional programming, i.e., with the members coding and unit-testing individually and then integrating their code in the end. After all the modules were developed, each group decided which module (whether the module developed by the pair programmers or by the traditional programmers) to integrate with the modules of the other groups to form the UML Tutor. Each group was then asked to give its opinion, as a whole group, on pair programming, and present quantitative data (in the form of defect densities and man-hours) and qualitative data that supported its opinion.

The experimental design described above differs from all the 17 studies reviewed in [15] in one major respect. All the studies reviewed in the said paper that had experimental and control groups compared the output of pairs against those of individuals. In contrast, we compare the output of pairs that used pair programming against those of pairs that used traditional programming.

327

The study whose design ours would be closest to would be that of [25] in the sense that each experimental-control pair worked on the same programming project. However, those in [25] worked on small assignments, whereas the groups in our study worked on a relatively complex project. Moreover, the members of the control group in [25] actually worked individually, i.e., each individual wrote the complete code for an assignment, whereas the members of the control group in our study worked in pairs, albeit traditionally, i.e., with the members coding individually and then integrating their code in the end. Knowing that “two heads are (generally) better than one,” we decided in our study not to compare the program written by a pair versus the program written by a solo student, but the programs written by two pairs: one doing pair programming, the other doing traditional programming.

Another difference between [25] and the experiment reported in this study is that the measure of program quality used in [25] was the number of test cases passed, while here it is the total number of defects found in test. These defects were all logged by the students in this study in PSP style.

In this study we investigated the following two hypotheses:

H01: There is no difference in the defect densities of products produced by the pair programmers and the traditional programmers.

H02: There is no difference in the productivity (LOC/hour) of the pair programmers and the traditional programmers.

Following [20], since the population was small

(N=6 groups), the Student t-test was used. The significance level for rejecting the hypothesis was chosen at 0.05.

5. Results and Discussion

As shown in Table 1, the defect densities of the modules written by the pair programmers were consistently lower than those of the pairs that did not. Table 1 shows that the mean defect density of the pair programming teams was 28% less than that of the traditional programming teams. The table also shows that the p-value between the two groups is 0.013. Because the p-value is less than the chosen significance level (0.05), we can therefore reject the hypothesis (H01) that there is no difference between the defect densities of the products of the two groups, and accept the alternative hypothesis that the defect densities of the pair programming groups are significantly different from (i.e., significantly lower than) those of the traditional programming groups.

Menu

Toolbar

Show Hint/Answer Button

Drawing Area

Tips

Analysis Results

Analyze Button

Figure 1. Basic GUI of the UML Tutor

328

There was, however, no significant difference between the LOC/hour spent by the pair programming group compared to the traditional programming group (hypothesis H02). We will return to this shortly.

The high standard deviation in Table 1 would be due partly to the groups’ developing different modules and to a particular module’s being unusually more difficult than the others. If this particular module is removed from the data, the mean defect density of the pair programming teams would be 47% less than that of the traditional programming teams, with the standard deviation around 21, and a 0.02 p-value.

These results tend to support those of previous

studies, including the seminal studies of [19], [6], and [12], and those reviewed by [15], despite American and Filipino programmers being markedly different in the individualism and power-distance dimensions of Hofstede’s study of organizational cultures [8]. (The individualism dimension was described in section 2. The power-distance dimension refers to the degree of equality between power levels in a society, organization, group, or family. A low power-distance ranking, such as that of the U.S., indicates greater cooperative interaction across power levels.)

The results are also consistent with what XP’s inventor, Kent Beck, states in [3]: “Even if you weren't more productive, you would still want to pair, because the resulting code quality is so much higher.” It is also consistent with the result of [20] that the number of problems found (by a teaching assistant) in the artifacts submitted by the pair programming teams were less than those found in the artifacts of the non-pair programming teams.

Finally, the results confirm those of [1], the most recent pair programming study, in the sense that quality differences were observed – and no productivity differences were found – in systems that were complex and where junior developers were in the pair programming teams. Our notions of system complexity are not exactly the same, however. We view the UML Tutor modules as complex because of the number and complexity of classes, data structures,

and algorithms that they entail; in [1], however, complexity was viewed as the degree of decentralization of the architectural classes.

All six groups explained the quality result as being due to two heads being better than one, and the navigators’ performing static testing while the drivers were writing code. Because the products of the pair programming teams had invariably lower defect densities than those produced by the traditional programming teams, the modules written by the pair programming teams were the ones that were chosen unanimously by all the groups as the modules that they combined to form the UML Tutor.

In addition to providing numeric data on the quality

of their modules, the following advantages of pair programming were identified by all six groups based on their experiences:

1. There was a lot of learning. Drivers learned from navigators, and navigators learned from drivers.

2. It was enjoyable to work with someone. This was due to the sheer fun of social interaction, and to positive feelings that arose as a result of helping and being helped.

3. The resulting code had fewer errors to remove and debug during dynamic testing, because of the reasons identified earlier in relation to the quality result.

Furthermore, some (but not all) groups identified

the following additional advantages of pair programming based on their experiences:

4. The resulting code was shorter and more readable because of the collaborative effort.

5. Having someone observing one’s work prevented attention drift and boredom.

6. Less exhaustion due, in turn, to item 2 and the occasional switching of roles.

On the other hand, the following disadvantages (or

challenges) were identified by some but not all the groups:

Table 1. Mean, Standard Deviation, and P-value of Student T-Test of Defect Densities of the TwoTeams

Sample Mean Standard Deviation P-value

Pair Programming 58.66 70.44 0.013 Traditional

Programming 81.75 70.19

329

a. There can be difficulty in adjusting with someone whom one does not know so well. This would entail adjusting to someone else’s temperament, motivation, and time.

b. Having someone “looking over one’s shoulder” can be distracting for some.

c. Clashing ideas can take time to explain and resolve.

That the productivity of the pair programming

subgroups did not differ significantly from that of the traditional programming subgroups may be explained by items 1, 3, 4, 5, 6, a, b, and c, with items 3, 5, and 6 positively influencing short-term productivity (LOC/hour), and the rest negatively influencing short-term productivity. That is, any short-term productivity gains that were obtained through the shorter testing times (item 3), attention drift prevention (item 5), and fatigue reduction of occasional role-switching (item 6) in pair programming might have been “canceled out” by (possibly lengthy) driver-navigator discussions (items 1 and c), shorter code (item 4), driver-navigator personality adjustments (item a), and driver distraction (item b).

On the matter of enjoyment, it was pointed out in items 2 and 1 above that all six groups found the social interaction and mutual support inherent in pair programming to be enjoyable. That social interaction and mutual support was enjoyed by all six groups, despite personality adjustments (item a), distracting navigators (item b), and idea clashes (item c) that some of the groups experienced, could be explained by the collectivistic trait identified by Hofstede [8] in Filipinos in particular and Asians in general. That is, Filipino programmers, or at least Filipino student programmers, would probably risk personality adjustment problems and idea clashes for the enjoyment that the social interaction and mutual support in pair programming brings. 4. Concluding Remarks

Despite the growing attention given to agile software development methods and practices and to the offshore outsourcing of software development and maintenance, the literature is scant on the use of agile methods in developing countries where most offshore outsourcing takes place. Nevertheless, whatever scant literature exists suggests that the positive results reported in studies in developing countries are reproducible in Asia, despite cultural differences.

This paper describes an experiment on the use of pair programming, an XP practice, by undergraduate

students in a private university in the Philippines. Experimental results show that defect densities of programs written by teams using pair programming were significantly lower than those written by teams that used the traditional programming approach of individual coding of modules followed by the integration of the separately coded modules. However, there was no significant difference in the productivity of the pair programmers and the traditional programmers. Analysis of qualitative data indicate various factors that might have led to the above results, including collaborative learning, static testing, social interaction, attention drift prevention, fatigue reduction, and driver-navigator personality adjustments.

In light of these results and those of earlier studies on agile methods in Asia and elsewhere, it is recommended that Asian colleges and universities seriously study the incorporation of agile methods into their curricula. More studies are needed to see whether and how agile practices need to be modified to address any cultural issues in Asian and global software development.

5. References [1] E. Arisholm, H. Gallis, T. Dyba, and D. Sjoberg,,

“Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise,” IEEE Trans. on Software Engineering, vol. 33, no. 2, 2007.

[2] P. Armour, “Agile and Offshore,” Comm. ACM, vol. 50, no. 1, 2007.

[3] K. Beck, Extreme Programming Explained: Embrace Change, second ed., Addison Wesley, 2000.

[4] K. Beck and B. Boehm, “Agility Through Discipline: A Debate,” IEEE Computer, June 2003.

[5] B. Boehm and R. Turner, Balancing Agility and Discipline, Addison Wesley, 2004.

[6] A. Cockburn and L. Williams, “The Costs and Benefits of Pair Programming,” Proc. XP 2000.

[7] A. da Silva, F. Kon and C. Torteli, “XP South of the Equator: An Experience Implementing XP in Brazil,” Proc. XP 2005.

[8] G. Hofstede, Culture's Consequences: International Differences in Work-Related Values, Sage Publications, 1984.

[9] W. Humphrey, A Discipline for Software Engineering. Addison Wesley, 1995.

[10] O. Kobayashi, M. Kawabata, M. Sakai, and E. Parkinson, “Analysis of the Interaction between Practices of Introducing XP Effectively,” Proc. ICSE 2006.

[11] IT Facts: Top Outsourcing Destinations: India, Philippines, Central America. http://www.itfacts.biz/index.php?id=P3858 (2004)

[12] K. Lui and K. Chan, “When Does a Pair Outperform Two Individuals?” In Proc. XP 2003.

330

[13] K. Lui and K. Chan, “Test Driven Development and Software Process Improvement in China.” In Proc. XP 2004.

[14] C. McDowell, L. Werner, H. Bullock, and J. Fernald,, “The Effects of Pair Programming on Performance in an Introductory Programming Course.” Proc. SIGCSE Technical Symp. on Computer Science Education, 2002.

[15] E. Mendes, L. Al-Fakhri, and A. Luxton-Reilly, “Investigating Pair-Programming in a 2nd-year Software Development and Design Computer Science Course,” Proc. ITiCSE 2005.

[16] B. Meyer, “The Unspoken Revolution in Software Engineering,” IEEE Computer, January 2006.

[17] N. Nagappan, L. Williams, M. Ferzli, E. Wiebe, K. Yang, C. Miller, and S. Balik, “Improving the CS1 Experience with Pair Programming.” In Proc. SIGCSE 2003.

[18] J. Nawrocki and A. Wojciechowski, “Experimental Evaluation of Pair Programming,” In Proc. European Software Control and Metrics Conf. (ESCOM), 2001.

[19] J. Nosek, “The Case for Collaborative Programming,” Comm. ACM, vol. 41, no. 3, pp. 105-108, 1998.

[20] M. Phongpaibul and B. Boehm, “An Empirical Comparison between Pair Development and Software Inspection in Thailand,” Proc. Intl. Symp. on Empirical Software Engineering, 2006.

[21] K. Schwaber and M. Beedle, Agile Software Development with Scrum, Prentice Hall, 2002.

[22] P. Sfetsos, L. Angelis, and I. Stamelos, “Investigating the Extreme Programming System – An Empirical Study,” Empirical Software Engineering, vol. 11, pp. 269-301, 2006.

[23] R. Sison, “Use of Agile Methods and Practices in the Philippines,” Proc. APSEC 2007.

[24] R. Sison, S. Jarzabek, S.H. Ow, W. Rivepiboon, and N.H. Nguyen, “Software Practices in Five ASEAN Countries: An Exploratory Study,” Proc. ICSE 2006.

[25] L. Williams, R. Kessler, W. Cunningham, and R. Jeffries, “Strengthening the Case for Pair Programming,” IEEE Software, vol. 17, no. 4, 2000.

[26] L. Williams and R. Kessler, Pair Programming Illuminated, Addison Wesley, 2002.

[27] E. Yourdon, Outsource: Competing in the Global Productivity Race, Prentice Hall, 2005.

331