Forming Successful eXtreme Programming Teams

Alan Gray, Andrew Jackson, Ioanna Stamouli and Shiu Lun Tsang Computer Science Department,

Trinity College Dublin, Dublin 2, Ireland.

{firstname.lastname}@cs.tcd.ie

Abstract

XP is a lightweight process that provides principles for guiding projects and relies on the participants for its success. However, despite these guidelines, projects can be unsuccessful. We have previously identified that participants’ levels of “buy-in” into the ethos of the methodology is a significant determinant of project success. This paper investigates how team formation influences buy-in and how buy-in, in turn, effects success, learning and working attitude within an academic environment. The empirical study shows that teams comprised of people with similar understandings of, and attitudes towards, the process tend to have a higher level of buy-in and, as a result, exceed their expected performance potential.

1. Introduction

Although agile development processes such as eXtreme Programming (XP) [1] provide guidelines for achieving project success, many XP projects can be unsuccessful [6]. Agile processes are informal and more reliant on people and their interactions than formal processes, which focus more on regimented rules and structure. One measure of process maturity is repeatability with similar results [7]. XP processes have proven useful in dynamic environments where the participants involved in a project can change with minimised effect to the project. Change is also simplified as participants do not spend valuable time learning new rules and structures. However, this strength can also be a weakness of XP. Due to its reliance on people, the XP process may not achieve repeatability and as such is perhaps unreliable [4].

In previous work [6], we have identified that “buying into” the XP process can be a determinant for a successful project. “Buy-in”, as we define it, reflects the willingness of a project team to adapt the core XP

practices to their environment in order to make XP work for them. However, two key issues remained unclear: what effect buy-in actually has on success, learning and attitude; and how a culture of buy-in can be fostered within a team. It is our intuition that answering these questions is the first step toward being able to ensure repeatable success in XP projects.

Our hypothesis is that team formation, among other factors, is important in fostering buy-in. In order to validate our hypothesis, we carried out an empirical study in which we created teams whose members had similar development experience, programming ability, and working attitude. Our intuition was that such teams, given the sense of equality, can collectively drive the XP process, contributing to successful projects.

Success, in our context, is defined in terms of an academic environment, and is determined by the completion of customer stories (business value), and also by the students’ learning and adoption of, and participation in, the development process.

Our study was carried out in an academic environment. The teams in the study were comprised of final year undergraduate students studying Business Computing1 at the Dublin Institute of Technology. Each student had approximately six month’s industrial experience prior to the beginning of the academic year. The software development projects carried by the participants were part of a Distributed Systems Development module that spans an entire academic year, while the projects themselves were sixteen weeks in duration.

The subjects in this study were final year students and, as a result, the environment in which the projects were carried out was not one typically found in industry. Factors such as access to the customer, project timescale and process goals differ from those

1 http://www.dit.ie/DIT/study/undergraduate/courses/dt354.html

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

found in industry. A more detailed description of these factors can be found in [6]. However, steps were taken to mitigate this disparity and create a product-focused, customer-centric environment. Each team had a customer external to the teaching course, who was concerned with getting a working product. Therefore, there was pressure on the team to deliver functionality to specification, and in a timely fashion.

We believe that this closeness to the “real world” environment engendered a culture of professionalism within most of the teams. It is important that students not only learn the theory of software development, but also experience the process first-hand while at college. This fosters a positive attitude towards using development methodologies, and arms students with a realistic view of what to expect when they graduate and move to industry. This experience of the development cycle was a positive one for most teams, with the majority of them reporting an increase in their perception of the usefulness of the XP methodology. Having students who recognise the benefit of following software development methodologies is a great boon to industry. This is because students often complete an undergraduate course without any experience of coding other than the general “build and fix” process.

The remainder of the paper is structured as per the following. We present our hypothesis and the methodology we employed in our empirical study in Section 2, and describe the environmental background against which the study was undertaken in Section 3. In Section 4 we present the findings of our study, namely how team formation effects buy-in and how buy-in, in turn, affects project success, the learning experience, and attitude towards the development methodology. In Section 5 we present a discussion regarding interesting themes that were observed during our study. Finally, Section 6 concludes the paper, summarising our main observations.

XP XP UnderstandingUnderstanding

Academic Academic PerformancePerformance SelfSelf--EfficacyEfficacy

Team TeamFormationFormation PersonalityPersonality

BuyBuy--inin

Work Work ExperienceExperience

Project Project SuccessSuccess

Business Business ValueValue

Agile Agile MethodologyMethodology

LearningLearning

TechnologyTechnology ProcessProcess

AttitudeAttitude

SelfSelf--EfficacyEfficacy XP PerceptionXP Perceptionand Experienceand Experience

XP PerceptionXP Perceptionand Experienceand Experience

Figure 1. Buy-In: Cause and Effect

2. Empirical Study

This section describes the setup for our study. We present our hypothesis and the methodology that was followed to carry out the study. The questionnaires used to capture the students’ experiences and self-efficacy are also described in the methodology section.

2.1. Hypothesis

Our past work has shown us that differences in team members’ initial understanding of the development process can cause the team to under-perform. Following on from this, we have aimed to create teams with members who have roughly similar development experience, programming ability, and working attitude. Our hypothesis is that such teams, given a sense of equality, can collectively contribute and drive their learning of the process and remain motivated to making XP help them achieve the project goals. We anticipate that this comparability should help mitigate the effect of ego within the team, where “stronger” members disregard input from other, “weaker” members.

Figure 1 shows the various elements that contribute to our hypothesis. Teams were created based on four factors associated with students: XP understanding, work experience, academic performance, and self-efficacy. It is our hypothesis that teams with members of similar ranking in these categories are likely to exhibit a higher level of buy-in.

We postulate that buy-in has a distinct effect on the teams’ project success. In industry, a successful project is considered to be one that has fulfilled the customer requirements within the timeframe. As stated in the introduction, the measure of success in an academic setting is composed of two elements – business value and adoption of the XP methodology. We also hypothesise that a higher level of team buy-in results in a greater learning experience for students. Finally, it was our aim to investigate the effect of buy-in in relation to students’ self-efficacy, XP perception and experience. We hypothesise that students whose groups have a higher level of buy-in will have a greater increase in self-efficacy and XP perception and experience.

The primary goals of this study were to investigate the three aspects of our hypothesis:

1. How team formation affects buy-in (Section 4.2).

2. How buying into the XP process affects the success of the projects (Section 4.3).

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

3. How buying into the XP process enhances the students’ learning experiences with XP and attitude towards XP (Section 4.4).

Our hypothesis is concerned with how team equality influences buy-in. As a result, the personality types of the students were excluded from consideration when creating teams. How personalities effect team buy-in is an interesting topic, but lies beyond the scope of this study.

Due to the nature and timescale of the projects only the teams that followed the XP methodology achieved significant business value. Our findings support this argument.

2.2. Methodology

Our methodology consisted of both qualitative and quantitative techniques for data collection.

Team formation was essential for this experiment since the goal was to create teams with similar development experience, programming ability, and working attitude. We used four factors to assess these attributes. Experience was evaluated by:

1. XP understanding. 2. Previous work/programming experience.

Ability was evaluated by: 3. Programming-related academic performance.

Attitude was evaluated by: 4. Self-efficacy.

A questionnaire consisting of 40 elements was developed to evaluate XP understanding, perception and experience. These were designed to capture the students perception as to the meaning, relevance and importance of each of the 12 core XP principles and the XP methodology overall. The questions were graded using the Likert scaling method from 1 to 5 [3], 1 being “Not at all” and 5 being “A very great deal”. A sample of the questions about the individual XP principles is listed below:

1. To what extent do you understand the XP planning game principle?

2. How important do you consider the XP planning game principle to be?

3. To what extent do you believe you will apply the XP planning game principle to your project?

These questions were repeated for each one of the 12 XP principles. The last section of the questionnaire was designed to capture the students’ attitude towards teamwork. The questionnaire was pilot-tested before it was deployed in order to make sure that the students understood precisely what was asked of them.

Previous work and programming experience were evaluated by means of curriculum vitae (résumé). Programming-related academic performance was determined by past examination results.

Computer self-efficacy was used as an indicator for the student’s attitude toward programming and, more specifically, Java. Computer self-efficacy is an individual’s judgement on their capability to use and program a computer [8]. In the questionnaire, one’s belief of their efficacy is not concerned with past experience but is focused on an individual’s capability to perform programming tasks in the future. Generally, those with strong belief in their efficacy view difficult tasks as a challenge that they can master. By contrast, those with low self-efficacy tend to avoid difficult tasks, because they are viewed as a threat [8]. In relation to programming, those with a low sense of efficacy will find programming more complex than those with a high degree of confidence in their abilities. The computer self-efficacy questionnaire also uses the Likert Scale from 1 to 7, where 1 corresponds to “Not at all confident” and 7 to “Absolutely confident” [3, 8].

Furthermore, each student’s personality was identified using the Myers-Briggs Type Indicator test [2, 9]. However, this data was not used during team formation. It is known that conflicting personalities can affect team dynamics, and we wanted to observe these interplays within our teams. By excluding this data from the team formation phase, we did not prejudice these dynamics.

Using the data gleaned from our surveys, six teams were created – three with four members and three with five members. We created two teams with high performance potential (good understanding of the XP practices; good programming experience and ability; and high self-efficacy); two with moderate performance potential and two with poor performance potential. To ensure that we had a balanced mix within the teams, even the “poor” teams had one or more “moderate performance” people in.

Each team was assigned a project that was commensurate with their ability. Three of the authors each acted as customers for two teams, with the remaining author acting as lecturer and XP and technology coach. The customers were Ph.D. students who had significant experience of using XP, both in academia and (in most cases) professionally, and two had significant experience with the technologies the teams were to use. During the early iterations, the course lecturer acted as XP and technology coach to the teams. This ensured that teams fully understood the technologies and the requirements of the process, instilling a sense of confidence in them.

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

The projects ran for sixteen weeks. Each group consulted their customer regularly and presented their evolving system releases to their customer at the end of planed iterations. During this time the XP coach monitored the groups and offered advice when appropriate.

At the end of the project the students presented their projects to the academic staff and were questioned to ensure that the assignments were carried-out without any external help. This also served to gauge the level of buy-in and participation within the team.

We also aimed to investigate the educational effect that the XP process had on the students’ learning experience. This could be achieved by comparing the students’ understanding and perception of XP, and their self-efficacy before and after the projects. Therefore post examinations of students on their XP experience, and their attitude towards XP and programming were carried out and the students were asked to complete the same questionnaires after the completion of the projects.

Students were also asked to complete group reports and anonymous, individual reports designed to gauge their experience with the XP methodology and how much they bought into the process. This individual report was similar to the report we used in our previous work [6], but also included sections on how peer learning worked within the team and whether the whole process was enjoyed by the students. Additionally, post-test audio data from their final project presentations and interviews was captured and later used in the analysis. All the students that participated in the empirical study were guaranteed anonymity. Any names appearing in quotations in this paper are made up. Additionally, they were fully aware that their responses to the surveys would not affect their grade in the course.

3. Environmental Background

This section discusses the environmental background. We describe the university degree course in which the study took place, the participants involved in the study, and the projects that were developed.

3.1. The Course

This study was conducted on a final year module in Distributed Systems Development as part of a Business Computing degree at Dublin Institute of Technology. The course was suitable for this study because:

1. Prior to attending the final degree year, participants are placed in a software

development organization to gain work experience. Therefore, they should have observed a software development process in action and gained competency in software development.

2. The course is an (academic) year in duration, which provides enough time for students to be coached in the XP process, to learn new technologies, and to apply them to their projects.

3. The course emphasis is on practical development and teaches students how to use contemporary technologies and tools to develop enterprise-level distributed systems.

4. There were twenty-seven students in the class. This was a small enough number to ensure that we had the resources to conduct the experiment, but a large enough number for us to get valid results.

The course focused initially on technology and technological integration in the development of enterprise systems. Following this, the students were taught the XP principles and how to use XP in a project. The students were assigned to their projects and customers once they were comfortable with XP.

3.2. The Participants

The participants in the study were generally of a similar academic and professional profile. All of the students had been in the same class for the previous three years and had all taken the same courses.

All participants had been placed in software development positions for six months prior to the beginning of the course to gain real world software development experience. Some participants had other experience either directly or indirectly relevant to software development.

The typical age of the participants was 22 or 23 with some participants being as young as 20 and as old as 28. The gender balance in the group was approximately one quarter female and three quarters male.

3.3. The Projects

The projects were client/end-user focused. The academic environment still imposed some restrictions (as previously mentioned in the introduction), but the project goals aimed to provide high-quality, useful software to the end-user. Each team were given different projects, but were to use the same technologies to develop them. This meant that teams

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

could collaborate and share expertise with respect to the technologies without the danger of plagiarism.

In general, the projects conformed to enterprise-level, database-backed Enterprise JavaBeans (EJB) systems. The technologies used were a Tiles and Struts web-servlet presentation layer, with an EJB 2.1 bean layer sitting in front of a MySQL database server. JBOSS was used as an application server for both the web and bean modules. Where applicable, the teams were shown the legacy systems to which they were adding, so that they knew the constraints of their development scope. Brief descriptions of the six projects and their requirements follow.

(Team A) Auto-marker Online. This project was to develop an online multiple choice auto-marking system. Students from a class should be able to log on to the system and take online multiple choice tests, which should be automatically marked. Each student should be able to view their history of past examinations. Additionally, a teacher should be able to log on and view the progress of each student in the class, as well as class statistics, and to compose and upload new assessments.

(Team B) Research Paper Tracker. This project was to develop a web-based research paper tracker. The tracker should allow a researcher to upload and save research papers he/she has read, as well as information about the paper. Comments and opinions that the researcher has about the papers should be stored. A summary of each paper read should be available to the researcher, as well as the paper itself if he/she wishes to re-read it.

(Team C) Online Music Forum. This project was to develop a web-based music forum, where users can discuss bands and music genres they like. Artists should be able to create forums for publicising their work. Users should be able to create new discussions within general forums. Finally, administrators should have the ability to create, modify and delete forums and discussions, as well as individual posts that are deemed offensive.

(Team D) Music Site Manager. This project was to build a interface for managing the content of an existing music-downloading website. Administrators should have the ability to upload new artists, albums and tracks to be available for download. They should also be able to create and modify music genres, and control to which genres particular artists and tracks belong.

(Team E) Train Timetabler. This project was to develop an online train timetable information system. The regular timetabled information should be available for users to browse. A user should be able to put in two stations and be told the next train between. This time estimate should be fed by real-time information collected by track sensors. Finally, an administrator should be able to put updated information, such as delays or cancellations, into the system.

(Team F) Banking System. This project was to develop an online banking system. Customers should be allowed to make lodgements to and withdrawals from their accounts using their credit cards. Additionally, they should be allowed to transfer money to pay retailers by selecting them simply from a list. Finally, both customers and retailers should be able to browse statements of their accounts and search for specific transactions made in the past by amount, date or payer/payee.

4. Findings

This section describes our findings for team formation, how team formation effects buy-in, how buy-in effects project success, and how buy-in affects a student’s learning experience and attitude.

4.1. Team Formation

XP XP UnderstandingUnderstanding

Academic Academic PerformancePerformance SelfSelf--EfficacyEfficacy

Team Team FormationFormation

Work Work ExperienceExperience

Figure 2. Team Formation

As depicted in Figure 2, teams were formed based on their: understanding of XP; work experience; previous academic performance; and self-efficacy. Table 1 shows the average values of these factors for each team. The standard deviations were also included in order to show the magnitude of the variance within the teams. The values for XP understanding were drawn from the perception and experience questionnaire. Work experience was based on the individual curricula vitae, and was graded out of 5: up to two points for experience gained during their six month work placement; up to two points for other, extra-curricular work experience; and the final point for our perception of the relevance of the work experience to the project at hand. Academic

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

performance was measured based on the previous year academic results on a Software Engineering course, scaled to be out of five. Five corresponds to 80-100%, four to 70-79%, three to 60-69%, two to 50-59%, and one to 40-49%. Self-efficacy was measured out of 7 in the Likert scale as previously described in the methodology section.

Table 1. Team Formation XP

Understanding Work

Experience Academic

Performance Self-Efficacy Teams Avg. Std. Avg. Std. Avg. Std. Avg. Std.

Team A 3.58 0.46 3.75 1.26 3.75 1.26 5.12 0.23 Team B 3.42 0.43 4.25 0.95 4.25 0.96 5.1 0.99 Team C 3.19 0.23 3 0.8 3 0.82 4.2 1.81 Team D 3.44 0.08 3 1.16 2.75 0.96 5.12 0.89 Team E 3.14 0.74 2.25 1.26 2.25 1.26 4.85 1.22 Team F 3.14 0.74 2.25 1.26 2.25 1.26 4.85 1.22

Teams were formed as much as possible on the basis of similar understanding and capabilities. However, due to the variance of skills and understanding between the participants, it was not possible to have the teams to be of equal strength or performance potential. Equally, we could not group all strongest students nor all the weakest students together, since it would be unethical to construct a team designed to fail and unfair to the students within the team. As a result, the teams vary in strength and there is some variance within teams with regard to the individual members themselves and their capabilities. There were two teams of high performance potential (Teams A and B), two teams of medium performance potential (Teams C and D), and two teams of low potential (Teams E and F). To offset this disparity of performance potential, slightly more difficult projects were assigned to Teams A and B, whilst slightly easier projects were assigned to Teams E and F. Teams were not informed of their expected performance, nor indeed that there were different expectations of each group. This was because we did not want to prejudice how teams performed.

4.2. How Team Formation Effects Buy-In

We postulated that a significant factor of team formation that affects buy-in is the degree of disparity of the team, with regard to experience, ability and attitude (Figure 3). As discussed in the introduction, buy-in is a measure of the attitude of the team towards the developmental methodology. In particular, it is the belief and the drive to use the developmental methodology as a tool to complete a project successfully.

Team buy-in is difficult to quantitatively capture. Since buy in is a persons attitude towards the process,

it cannot be represented by a calculated number. In order to minimise the subjectivity of measuring this, we assessed buy-in combining quantitative (group, individual report) and qualitative (personal opinion of the XP coach and customers) data drawn from different sources. Therefore, we assessed team buy-in from:

1. The XP coach’s and the customers’ experiences of the teams, and their view of the team work and adherence to the XP methodology.

2. The project team’s group report. In the report, the team discussed how they used each XP principle and gave their opinion on the value of the principles.

3. Post-project presentations of the work, and post-test interviews with the students.

Team Team FormationFormation

BuyBuy--inin

Figure 3. Team Formation and Buy-In

Following these, we determined that one team bought very highly into the process, four teams bought moderately highly into the XP process, whilst one team had great difficulty adapting to the process and using it to advantage. Each teams’ level of buy-in is given in Table 2 below.

As asserted previously, we believe that team formation is a major determinant in the success of a project. This is because team formation affects buy-in which, in turn, affects project success. We contended in our hypothesis that teams comprised of members with similar skills and attitudes have good buy-in into the XP process. This is borne out by our findings.

As can be seen from Table 1 above, Team C (the team with the highest level of buy-in) did not have the best understanding of XP prior to the project, nor did they have the best work experience or academic performance. However, they did have the lowest standard deviations from the average for their team members. Note that we omit self-efficacy from this discussion, as it is a subjective measure of one’s own ability, and does not permit fair comparison of peoples’ actual ability. A member of Team C said the following, when referring to the team:

“W[w]e had the same level of understanding of the technologies that we were using and anything that one

didn’t understand the other could help.”

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

Team C took the XP principles to heart and enjoyed a very productive and fun work environment in which everyone felt confident in sharing opinions and completing work on any and all parts of their system.

“I was lucky to work with a great group of people where everyone pulled their weight and put the group project before anything else. The commitment made the project more enjoyable, easier to work on and

easier to deliver all functionality.”

Other teams enjoyed moderate buy-in, despite having members with wide differences of skills, and were willing to consider each other and help each other out. From members of Team E and Team F:

“I enjoyed the meetings with the group members and I feel that every persons input was valued and

considered.”

“I sometimes struggled to keep up with them, but I always felt comfortable asking questions.”

“By balancing the pairing of stronger developers with weaker developers, the weaker developers were able to increase the quality and volume of work that

they were able to do, plus it increased their confidence in working on their own. The stronger developer may have been slowed down by this, but it meant that the gap between stronger and weaker developers was

continuously balancing out.”

With the exception of Team A, all the other teams bought more or less equally into the XP methodology. The members of these teams were approximately equally disparate, as the standard deviations within each team are approximately comparable. Therefore, our hypothesis that team formation affects buy-in is borne out in all cases.

Why then, did Team A buck the trend? From our post-project interview data, it emerged that there were personality conflicts within the group. Using the Myers-Briggs Type Indicator test, we discovered that the team members all had very similar personality types, which is known to cause friction within teams [5]. This engendered a tense environment which reduced team spirit and buy-in, and detrimentally affected the team’s experience with XP. Quotes from various team members support this:

“To be honest it became very frustrating at the end, the lack of support from the other two […]. It really

became disheartening.”

“You can only delegate and tell somebody to do something only so many times, at the end of the day we are all mature final year students, and we all have our own work to be doing but there’s no excuse not to help

out.”

In addition, feedback offered by the students correlates with our hypothesis that team members of comparable skills buy more into the process:

“I strongly felt that myself and Alice would have benefited a lot more if we had been in a group who

had the same knowledge as us.”

4.3. How Buy-In Effects Project Success

We have stated that buy-in has a direct effect on the success of a project, with high levels of buy-in resulting in greater project success. We have previously stated that project success in our study is defined in terms of an academic environment and measured by the completion of customer stories (business value) and a student’s successful adoption of, and participation in the XP methodology in the development process (Figure 4).

BuyBuy--inin

Project Project SuccessSuccess

Business Business ValueValue

Agile Agile MethodologyMethodology

Figure 4. Buy-In and Project Success

Table 2 shows our findings in relation to buy-in and project success. The table shows each team and their levels of buy-in. It also shows the results of each team with grades given in relation to our two measures of project success: business value and XP methodology. The measure for business value is separated into two categories. The expected column gives the expected success of the team before the start of the project and was computed based on the group’s previous academic performance, previous development experience, and perspectives from the course staff (as per team formation). The actual column shows the grade given to each team after the project was completed based on the business value achieved. This grade was based on the number of customer stories that were completed and the overall quality of work done on the project.

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

Table 2. Buy-In and Project Success2

Project Success Business value Teams Level of

Buy-in Expected Actual

XP methodology

Team A Low 2.1 – 1.1 2.2 2.2 Team B Medium 2.1 – 1.1 2.1 2.2 Team C High 2.1 1.1 1.1 Team D Medium 2.1 2.1 2.1 Team E Medium 2.2-2.1 2.1 2.1 Team F Medium 2.2 2.1 1.1

The figures in the table support our hypothesis, showing that project success is correlated to buy-in. Team C, who had the high level of buy-in performed the best under both success categories. They successfully following the XP process and achieved more customer stories than was expected of them. This resulted in Team C exceeding their expected performance potential and producing the project with the highest degree of business value.

In contrast, Team A had a low level of buy-in and clearly underperformed with respect to their expected performance potential. Although they were expected to produce one of the best projects in the class, they did not buy into the XP ethos and failed to follow most of the XP guidelines. This meant that they had problems achieving customer stories and integrating functionality.

It is also interesting to note that Team F achieved more business value in their project than was expected of them. This can be attributed to their adoption and application of XP. Although they did not fully buy into the process, they managed to leverage it enough to slightly outperform what was expected of them. The remaining three teams performed generally as expected.

4.4. How Buy-In Effects Learning and Attitude

From our hypothesis, we expected that teams whose members had a high level of buy-in were more likely to exhibit a higher level of learning, both in terms of the technologies they used and also the XP principles (Figure 5). We also anticipated a more positive attitude towards their own programming skills and the use of XP as a development methodology.

Reports from customers and the XP coach support our hypothesis, as teams with greater buy-in seemed to grasp the technical aspects of the projects and the methodology much better than teams with lower levels of buy-in. Various members of different teams are quoted as saying:

2 Grades are given in the following ranges – 1.1 is 70-100%; 2.1 is 60-69%; 2.2 is 50-59%; 2.3 is 40-49%.

BuyBuy--inin

LearningLearning

TechnologyTechnology ProcessProcess

AttitudeAttitude

SelfSelf--EfficacyEfficacy XP PerceptionXP Perceptionand Experienceand Experience

Figure 5. Buy-In and Learning/Attitude

“XP made it easier while learning new technologies or when coding more complex parts of the system. It

gave us a better shared understanding of the project.”

“At the beginning I would not have felt comfortable using the new technologies but the fact that we

switched pairs meant that I got a full understanding of each technology.”

“We initially spent time together as a team learning the technologies. This gave me confidence in them

when we were pair programming.” Teams with lower levels of buy-in evidently learned

less effectively. This is due to the fact that members did not have good motivation to learn and were less inclined to share their knowledge with other members of their team:

“We really were just left watching them code and trying to grasp the concept of all these new

technologies.”

“Our group consisted solely of myself and one other member. The other two in our group did not even attempt to learn the new technologies and contribute.”

“Although the group had a collective idea of what was going on, needed to be done, and how it was to be done, it was basically Bob writing EJBs and Charlie

writing struts for most of the time.”

Table 3. Buy-In, Learning and Attitude

Teams Buy-In XP

Understanding Change in %

Self-Efficacy Change in %

XP Perception

Change in % Team A Low 12% 11% -2% Team B Medium 17% 22% 3% Team C High 41% 42% 15% Team D Medium 24% 17% 5% Team E Medium 37% 18% 1% Team F Medium 28% 17% 9%

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

Table 3 shows the results we obtained when investigating how buy-in affects students’ learning experiences in terms of the XP process. Students completed the same questionnaires regarding their understanding of the XP process, their perception of the importance of the XP process, and their programming self-efficacy both before and after their projects. The percentage change between the pre- and post-test data for each team is given.

From the table, Team C had the highest buy-in and also had the greatest increase in their: self-efficacy; understanding of XP; and perception of its usefulness. Teams B, D, E and F all had medium levels of buy-in, and had comparable increases in their figures. Finally, Team A had the lowest level of buy-in and had the least increase in their understanding. In fact, their collective impression of the XP methodology was a negative one, as evidenced by a drop in their perception of the value of the process. Thus, these figures validate our hypothesis that teams with higher level of buy-in had a better learning experience.

Section 4.3 discusses the success of the projects, supporting our intuition on buy-in and learning. Project results provide a measure of students’ progress and reflect what they have learned during their experience. The results given for these particular projects provide more evidence that a higher level of buy-in correlates to a higher increase in learning, with Team C performing the best and Team A performing the worst, with respect to the quality of work expected from them.

A similar finding can be shown for peer learning. Results relating to peer learning also indicate better peer learning in groups with higher buy-in. Examples showing their willingness to both learn and teach others within their group include:

“Starting out, none of us had any experience of any of the technologies so learning it was a shared group

experience.”

“The technology was completely new to us so it was a steep learning curve…[T]through teamwork we taught each other how to use the environment.”

“I found that I spent a lot of time explaining the technologies to him. I found this frustrating as it

slowed me down, but at the same time it meant that he improved and was able to get more work done, which

was better for the team.”

In contrast, Team A did not exhibit such willingness, with team members often working with little interaction:

“I just felt that working with him, it was very hard to get work done. He gets distracted easily and rarely

showed initiative.”

“It was frustrating sometimes when the other two did not contribute at all.”

The teams that had a positive experience with the process within their group seem to have increased their self-confidence towards programming and the methodology in general (Table 3). It is clear that when teams buy into the process the experience that they gain is more positive and this is manifested in the results and the feedback from the students in their reports.

“It has definitely improved my programming skills greatly and I would be a more confident programmer

now than at the start of the year.”

Our findings support our initial intuition in that greater buy-in will lead to greater learning experience and positive attitude towards the development methodology. It is evident that buying into the process affects how positive XP is perceived.

5. Reflection/Discussion

It may have been expected that teams comprised of people with high self-efficacy, good understanding of the XP process, and extended work/programming experience would be more successful than teams with combinations of “weaker” team members. However, analysis shows that members across the teams were successful in learning about the XP process, but also in developing their programming skills. The values for programming self-efficacy and XP understanding in Table 3 show a very strong upward trend in both scales. This highlights two interesting points: that team members, despite having completed many previous programming tasks are now more confident in their programming ability; and that a clear understanding of the XP practices is best obtained by “doing” in a project environment, rather than by just “learning” in a classroom environment. This reinforces the benefits of making students follow a realistic software development cycle.

Teams that were perceived to be particularly strong seem to have underperformed with respect to their own potential, when compared with teams who had a lower expected performance potential. We speculate that this is attributable to personality clashes within project teams. It has been documented [2, 5] that certain

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006

personality types do not work well with each other in tightly clustered team environments, such as XP presents. Although it has not been included in our analysis, all the team members of Team A had the same personality, according to the Myers Briggs test. There were no other conspicuous trends with regard to personality types in the other teams, but this may have been down to the scale of the study. A larger study, focusing more on personalities within teams may yield a deeper insight on this point.

Through working collaboratively on projects as part of XP teams, students had meaningful and practical learning experiences that they could apply in other scenarios. Students worked closely together within their project team, but also across teams, enhancing the learning experience further. Not only did students learn about the XP methodology and the technologies they used, but they also learned how to work together as part of a team – something that is often lacking in the university experience.

“Doing this project did bring the whole class together, there was always a great atmosphere in the

labs and if one group were really stuck with something, another group would always help out.”

“I’m definitely more tolerant and for future projects this was definitely a good overall experience, in terms of knowing how to react to certain situations

and respecting other people’s opinions.”

Finally, feedback received from course graduates indicates that this exercise has aided students to integrate smoothly into teams in an industrial setting. One graduate is quoted as saying that:

“I was much more competent and confident than other people in my graduate program due to my XP

experiences.”

6. Summary

In this paper we investigated our hypothesis that team equality positively effects buy-in and in turn, that buy-in effects project success, the learning experience, and attitude towards the development methodology. The factors that were taken into account in team formation were students’ understanding of the XP process; their previous work experience and academic performance; and their programming self-efficacy. It is clear from this study that our hypothesis holds true.

There are other interesting points that arise from our study. Firstly, there are other factors, such as

personality, that effect how much a given team will buy-into a software development methodology. Secondly, it is not clear as to whether our hypothesis holds for other agile development processes, such as SCRUM or PSP, or whether it holds for non-agile processes.

Although our study has been carried out in an academic environment, we believe that these results may be used to influence success in industrial environments. Through identifying the interrelationships between team formation, buy-in and success we believe that repeatable success of the XP process is more achievable.

7. Acknowledgements

We would like to thank the team members of all the projects that have been included in our experience.

8. References

[1] Beck K., “Extreme Programming Explained: Embrace Change”, 2nd Edition, Addison-Wesley Professional, 2000.

[2] Briggs M. I., “Gifts Differing: Understanding Personality Type”, Davies-Black Publishing, Palo Alto, USA, 1995.

[3] Cohen L., Manion L., Morrison K., “Research methods in education”, 5th edition, RoutledgeFalmer, New York, USA, 2000, pp. 252-5.

[4] Deming W.E., “Out of the crisis”, MIT Press International, 1982.

[5] Gorla N., Lam Y.W. “Who Should Work with Whom? Building Effective Software Project Teams”, Communications of the ACM Vol. 47, No. 6, 2004.

[6] Jackson A., Tsang S.L., Gray A., Driver C., Clarke S., “Behind the Rules: XP Experiences”, in Proceedings of the IEEE Agile Development Conference, Salt Lake City, 2004, pp. 87-94.

[7] Nawrocki J., Walter B., Wojciechowski A., “Toward Maturity Model for eXtreme Programming”, Euromicro, p. 0233, 27th Euromicro Conference 2001: A Net Odyssey (euromicro'01), 2001, pp. 0233.

[8] Ramalingam V., Wiedenbeck S., “Development and validation of scores on a computer programming self-efficacy scale and group analyses of novice programmer self-efficacy”. Journal of Educational Computing Research, 19(4), 1998, pp. 367-381.

[9] Wikipedia, Myers-Briggs Type Indicator Test, http://en.wikipedia.org/wiki/Myers-Briggs_Type_Indicator.

Proceedings of AGILE 2006 Conference (AGILE'06)0-7695-2562-8/06 $20.00 © 2006