Patterns for testing

in global software development

Anneke Pehmoller, Frank Salger

Capgemini, Carl-Wery-Strasse 42, 81739 Munchen

frank.salger@capgemini-sdm.com

Stefan Wagner

Technische Universitat Munchen, Boltzmannstr. 3, 85748 Garching

wagnerst@in.tum.de

Abstract

Although testing is critical in GSD, its application in this context has not been deeply in-vestigated so far. This work investigates testing in GSD. It provides support for test managersacting in a globally distributed environment. With this it closes a gap. The leading questionis ”What problems exist in testing in GSD and how can they be addressed in projects?”.Decomposing this question we a) identify problems of testing in GSD projects and b) providegood practices to support practitioners in testing in GSD projects. The research is realizedin the context of Capgemini sd&m, a German IT provider. Our contribution to solving thestated research problem is a collection of 16 patterns for testing in GSD projects. For prac-titioners the usage of the patterns is simplified by various views on the patterns. Herewithwe stipulate research and support project managers and test managers in the realization oftesting in GSD projects.

1 Introduction

Global Software Development (GSD) is a widely-used approach in software development as the industryreaches for benefits such as low labor costs, access to a large workforce, and time-zone effectiveness [1, 2].Compared to co-located software development projects, GSD projects face the additional characteristicsgeographic, temporal, cultural, and technological distance [1, 3, 4, 5, 6, 7]. If not addressed carefully,they often result in challenges regarding communication, coordination, processes and technology. Globaldistribution and its implications on software development has been investigated intensively in recent years.But researchers have not yet presented and evaluated solutions for all activities of the development lifecycle.

The impact of distribution on testing is one field that has not been in the focus yet. But the needto investigate test in the context of GSD challenges has been identified already by several researchers[8, 1, 2, 9, 10, 11]. Also the industrial community reveals the need for investigation and solutions inthis field. Experience at Capgemini sd&m shows that testing in GSD is a little investigated issue thatneeds practicable solutions in the context of custom software development. Many projects are realizedwith assignment of tasks to Polish and Indian team members in addition to German colleagues. Thecompany-own Offshore Custom Software Development (OCSD) method [12] does address issues such asproject management, handover points, and quality assessments (see, e.g., [12, 13, 14]). On testing little isprovided in this method.

The scientific as well as the industrial software engineering community has realized that testing activ-ities are an issue to be considered when looking at the impact of distribution on life cycle activities but ithas not been considered extensively and few solutions have been presented.

Research Problem It is not yet clear what impact the challenges of GSD have on single testingactivities and test management. Only selected aspects of testing in GSD have been investigated andsolutions proposed. With this work we contribute to closing the described gap regarding testing in GSDprojects. The leading research question we ask is: What problems exist in testing in GSD and how canthey be addressed in projects?

1

Research Objective By investigating resources from literature and industrial custom software de-velopment (CSD) projects we aim to answer the above mentioned research questions. We focus on twoobjectives: First, we want to present problems that result from the impact of challenges of GSD on test-ing. And second, we aim to extract practiced solutions to support test managers and project managersby providing reusable best practices for testing in GSD projects.

Contribution With this work we contribute to the research area of testing in GSD. We provide prac-ticable support for test managers and project managers who meet the challenge to set-up and successfullyrealize testing in a GSD and CSD context. We do so by providing patterns for testing in GSD projects.Each pattern describes a concrete problem and provides a solution to solve this problem.

Context This research work was done at the Technische Universitat Munchen (TUM) in cooperationwith Capgemini sd&m AG, a German software and consultancy company. The goal is to investigate thetopic of testing in GSD projects. Therefore, we explore relevant literature and interview test managersand project managers of real-life projects.

Outline The structure of this paper follows the guidelines for reporting case study research in softwareengineering proposed by Runeson and Host (2009) [15]. In section 2 we provide background informationon GSD in general and on related work on testing in GSD in specific. In section 3 we describe howwe approached the above defined research question, how we collected data, and the analysis procedure.Furthermore, we comment on validity procedures. In chapter 4 we present patterns for testing in GSD asour result for the research questions. Chapter 5 sums up the work and contains our conclusions includingfurther research options.

2 Related Work

Two major fields of related work can be distinguished. The first field includes information on problemsand proposed solutions for general challenges in distributed projects without addressing a specific activitywithin the software development life cycle.

The second field of relevant work contains reports that describe problems and solutions with a focus ontesting in GSD. Available GSD- and test-specific studies were investigated in a systematic literature reviewfollowing the guidelines for performing systematic literature reviews in software engineering presented byKitchenham and Charters (2007) [16]. The data source used for the review was Google Scholar [17]. Thismeta search engine searches well-known and accepted scientific databases including IEEE Xplore, ACMDigital Library, citeseer, Wiley Interscience, and Sciencedirect for references. We defined the search stringas ((test OR ”testing” OR ”Test management” OR ”test process”) AND (”Global software development”OR ”Distributed Software Development” OR ”Global Software Engineering”)). The search was executedusing the configuration ”at least summaries” (in contrast to ”include citations”) and not restricting theyears of publications. For the selection of studies, we define the following mandatory inclusion criteria:a) The resource is in English or German. b) The title contains ”test” (also accepted when in combinationwith other terms, e.g., test management) referring to software tests. c) The title hints to GSD, e.g.,one of the terms ”Global software development”, ”Distributed Software Development”, ”Global SoftwareEngineering” or similar is mentioned. Additionally, the following exclusion criteria are specified: d) Thestudy has the topic of ”remote testing”. Remote testing refers to testing of physically distant systems. Thissetting might occur in GSD projects but we consider it not directly connected to our research questions.e) The report deals with testing of ”distributed software” or ”distributed systems”. This is out of scopefor this work. f) The resource is a patent. We focus on report types that present scientific research, e.g.,conference and journal articles. g) The resource presents only statements of future research. Therewith wefocus on results of investigations. The quality of the selected resources were assessed using the followingquality criterion: h) The study has been accomplished by at least two researchers or/and has been subjectto a review, e.g., as practiced for the selection of conference contributions and journal articles. Resourcesthat did not fulfill the criteria were not further considered.

General GSD issues Sengupta et al. (2006) and Jimenez et al. (2009) [1, 2] present an extensiveoverview of literature available on issues in GSD. We briefly name some of the general GSD issues.

Missing informal communication [18, 19, 3, 1], control [20], trust [21, 1, 20], visibility [18] and knowl-edge management [22, 23] are issues resulting mainly from geographic distance. The necessity to relyon often inefficient asynchronous communication [3, 1, 19] is caused by temporal distance in distributedteams. Temporal distance can also result in reduced efficiency and speed [21, 24, 25, 4]. Cultural distancecomprises two aspects. Differing terminologies, roles and unsynchronized processes are issues of organiza-tional culture and have to be considered in GSD [19, 26, 5, 25]. Difficulties caused by different languages[21] and communication behavior [27, 21] are issues of national culture. Technological distance concernsdiffering technological equipment, format standards, and infrastructural issues as low band width and

2

connectivity problems [4, 5, 19, 1]. All these challenges (probably) somehow influence testing activities ina distributed setting. Nevertheless, they do not address the specific research problem.

Testing in GSD Casey (2009) [11] investigated in his research the question ”How can software testingin a globally distributed virtual team environment be effectively carried out?” on two projects with testingteams distributed to an Irish and an Indian site. However, besides the interesting result of interactionof distributed teams that consist in this case of testers, concrete consequences and solutions for the testorganization, test process, and artifacts are not presented.

Sengupta et al. (2004) [28] propose Test-driven Global Software Development as a solution to keepingknowledge on requirements and their changes consistent at all involved sites. This uses test artifacts asmedium for communication. It addresses a requirement engineering topic and does not specifically solvea test problem.

With their tool Global Access Testing Environment (GATE) Zage et al. (2005) [9] provide a solutionto manage test-related information and thus support effective testing in GSD. The tool provides supportbut it does not indicate solutions on how to establish a test organization in GSD and the process is notdescribed in detail. A drawback is that the authors only report of its usage in an university/researchenvironment, an industrial evaluation is missing.

Copstein and de Oliviera (2001) [8] investigate into a similar direction. They propose a workflow-basedtool for the management of the test process with the goal to support consistency among documents andto allow for distributed assignment and control of test activities and automating some of these. Thisfinding is relevant for the management of testing in GSD projects. Nevertheless, the work only takes intoaccount the activities test plan design and test plan execution, missing out other activities such as testspecification. Additionally, the project setting described in the paper only considers one option for taskdistribution: The development team is located at one site and the test team at another site. The optionof distributed test team members is not addressed.

Model-based testing as an approach to address the challenges domain knowledge transfer and com-munication overhead in a global environment has been investigated by Al-Jaridi and Bruegge (2006) [29].The pilot project compared four different realizations of a test process. The procedures varied in theassignment of tasks to the onshore and offshore team and in what test activities are realized traditional(meaning not model-based) or model-based. They evaluated the result using different criteria. Conclusionof the project was that using the offshore model and model-based approach, the activities test designand execution are highly offshorable. The described findings are very interesting and provide practicalinsight into the effectiveness of model-based testing in the offshore context. We see this as an valuablecontribution. Still, the need to get a broader overview on challenges and approaches of testing in GSDand especially intra-organizational CSD projects is not satisfied with this finding.

Focusing on offshoring test automation, Tervonen and Mustonen (2009) [30] investigated three casesof outsourcing activities of test automation to offshore providers. They identified two main classes andclassified risks: Insufficient quality is influenced by subcontractor is unfamiliar with test tools and software,language barrier, networking issues, and no required resources. In the class hidden costs the risks are lackof support for the offshore team and HW shipment and delivery problems. They also present correctiveactions to address these risks. The investigation focused on offshoring in an outsourcing context, whereasthe context of this research focuses on an inter-organizational setting. Thus this work does not directlyanswer my research question and especially it focuses on the one aspect test automation.

Bondi and Ros (2009) [31] report their experience on training a performance test team located inIndia by a performance engineer in New Jersey (USA). Main challenges were cultural differences, theteam members’ lack of knowledge and experience in performance tests and organizational issues. Theyovercame these by using frequent scrum meetings as well as on-site training, careful selection of teamleaders at the Indian site and usage of automated tools for execution and result tracking of performancetests. Their finding was that with these measures the remote performance test team increased theirproductivity. The investigation describes problems and solutions regarding testing in GSD. But it focuseson the training and does not directly describe the testing process and challenges that occur during testing.

The described findings and proposed solutions are helpful contributions and can be considered whensetting up testing in a GSD project. Still, very few relevant reports on testing in GSD have been publishedyet. Especially a structured, comprehensive presentation of problems and solutions is missing. We do seethe need for more investigation on how the challenges of GSD impact testing, what consequences this hasand what practicable solutions can be proposed.

3 Case study design

We use an exploratory case study approach with the goal to investigate contemporary phenomena in theircontext [15]. With our research we investigate qualitative data, i.e., data collected directly from softwaredevelopment projects. We base our research on the idea of grounded theory proposed by Glaser andStrauss [32] and thus use an hypothesis generating technique.

3

Research questions The leading research question we ask is What problems exist in testing in GSDand how can they be addressed in projects? Decomposing this question we investigate the followingproblems. a) GSD projects differ from co-located projects: What challenges exist? b) We expect thatthese challenges impact the development life cycle: What impact do these challenges have on testing?What implications do these impacts have for the test organization, the test activities, and test artifacts?c) Solutions might have been found already: How can the impact of challenges of GSD on testing beaddressed? What best practices regarding testing have been used in GSD projects?

Case and subjects selection Our research bases on qualitative data collected at the industrialpartner Capgemini sd&m (for more information see http://www.de.capgemini-sdm.com). For a couple ofyears, Capgemini sd&m has realized many projects with globally distributed teams. Besides Germany,sites in Poland and India are involved. Activities are distributed within the Capgemini group and thusan intra-organizational approach is realized. The company-own Offshore Custom Software Development(OCSD) method provides guidance and best practices for GSD projects. At Capgemini sd&m GSDprojects follow a so called ”one team approach”. That is the complete integration of all team membersirrespective of the site they are working at. All colleagues are full members of one team. Still the mainresponsibility of the project, namely the project management is kept in Germany.

Regarding the development process, projects at Capgemini sd&m have to be distinguished as variousprocess models from almost strict waterfall to iterative to agile methods are accomplished. Organizationand distribution in the GSD context is individual for all projects. This is characteristically of the domainof custom software development.

All study subjects and objects are from the described context. Study objects are four projects. Theprojects were selected by a GSD expert who knew that they are GSD projects. All projects were so calledfarshore projects with German and Indian team members. As study subjects we focused on test managersand project managers as interview partners. All of them are located at the German site. We took fivestudy subjects into account, two project managers (PM) and three test managers (TM) of which one TMalso took on the role of PM.

Data collection procedures The collection of practical data included mainly interviews with projectexperts and the realization of a workshop and expert discussion. The collection of this data was accom-plished in no defined order within a time frame of twelve weeks. The source for our qualitative data arethe above mentioned study subjects and objects.

As preparation of the interviews with project experts, in the first step the interviewee filled out ashort questionnaire of three pages. This collection of general and test-specific questions served as firstimpression of the project’s organization. In the second step the interview was held by telephone withone interviewee at a time. Each interview took approximately one hour. The conversation was recordedwhich all interviewees knew beforehand and agreed upon. Interviews were semi-structured using openquestions as guide for the interviewer. Information from the questionnaire was introduced in the guide.The interview was kept flexible to react to the flow of the conversation and information given by theinterviewee. Transcripts of all interviews were made for later analysis.

Expecting supplemental input from Capgemini sd&m colleagues, a workshop was held broaching theissue of testing in GSD. The workshop was organized within the scope of an in-house Farshore Preparationtraining. The participants were twelve colleagues of Capgemini sd&m. They are all experienced softwareengineers and/or project managers who know the business of CSD. Before the workshop they got generalinput on GSD. Each participant was randomly assigned to one of three groups where they discussed theimpacts of GSD challenges on testing. For the discussion the groups had one hour. Each group deliveredresults in a presentation and a document. Complementary input was received during discussion withexperts where valuable comments and reports of experience were given.

Analysis procedure For the analysis of practical data collected during project interviews, we useda grounded theory approach. We generated a theory from qualitative data. During the first analysisof the interviews’ transcripts, we developed the idea of extracting patterns to provide a solution for ourresearch problem. We have the opinion that patterns are an efficient way to present solutions for testingin GSD. Patterns are an established and well-known format to present knowledge of software engineering[33] (on patterns see also [34, 35]). Patterns serve the need for agile, flexible, practice-oriented approaches.Patterns have the benefits to be lightweight, easily extendable and focused [36]. They are briefly describedwith certain attributes, often following a structured template. They consolidate practical experience inthe field and are problem-solving and solution-oriented. Being aware of the possible existence of the prob-lems addressed by the patterns often is a first step to avoiding them. Flexibility is a further advantage:Each pattern addresses exactly one problem and presents an adequate, approved solution. In a specificproject setting relevant patterns can be applied, while ignoring others. Thus each pattern is adaptableindependent of the usage of other patterns. Additionally, the modular approach of patterns supports theusage in any life cycle model and method.

4

We used a structured process with the goal to extract patterns from the available data. For each projectwe analyzed the transcript of the interview. Test-specific and GSD-specific problems and in the projectrealized approaches and solutions are identified and excerpted. The found problems and approaches serveas basis to formulate ideas for patterns which are in the first step shortly described with a title. The resultis a list of data records of test-specific problems, approaches, and ideas for patterns for each project.

Analyzing all data records, patterns were identified and described in detail. If possible data records aregrouped to serve as input for one pattern. Documentation ensured traceability from original data source tothe patterns. For each pattern it is evaluated if it is a ”good” pattern. Based on [35, 37, 33, 38] we defineda list of criteria a good pattern has to fulfill. A good pattern is a) Relevant: Does the pattern addressimportant aspects of testing in GSD? b) Encapsulated: Is the problem/solution space well-defined? Is thepattern independent, specific and precisely formulated? c) Practiced: Does the pattern present a solutionthat has been practiced? d) Applicable/ practical: Does the pattern tell me what to do? e) Composable/open: Does the pattern fit with other patterns to solve a larger problem? f) Comprehensible: Does thepattern read well? Does it have a sense of tension (problem new or viewed from a new perspective toreader) and release (surprising solution)?

If the pattern does not fulfill the criteria, it is to decide if it can be re-fined. Otherwise it is not con-sidered further. The resulting patterns and its source (interview statements) are the base of an evaluationwith the interviewee. Additionally, we requested feedback from experts discussion. If necessary, changesare introduced in the patterns descriptions.

Finally, the patterns will be evaluated with further projects. Here it will be distinguished between therating ”we use the pattern in our project” and ”we do not use the pattern, but we think it is a good ideato use it”. As result we plan a statistic from the patterns’ evaluation. As the step of evaluation is notaccomplished yet, this result is not presented here.

4 Results

The analysis procedure resulted in 16 patterns. Table 1 shows a problem/solution summary (as proposedin the pattern Problem/Solution summary by Meszaros and Doble (1998) [39]) of all patterns.

Table 1: Problem/ Solution summary with highlighted patterns that are described in the following.

Pattern structure In the literature, patterns vary in structure, described elements (and how theyare titled), and writing style [40, 41, 42]. Examples for pattern formats are Alexandrian, GoF, Coplien,Portland, or POSA form [40, 41]. Further pattern forms are described in [43]. For our needs, we define a

5

pattern structure based on the Mandatory Pattern Elements pattern by Meszaros and Doble (1998) [39].Our pattern structure contains additional elements that serve our needs. Thus we will use the followingstructure to describe the patterns for testing in GSD: a) Name: A short, plain title. b) Problem: The issuethat is addressed by this pattern, in the form of a question. c) Context: The situation and constraintsto which the pattern applies. The role of potential users are named. d) Forces: The often contradictoryconcerns that must be considered when choosing a solution to a problem. e) Solution: The solution thathas been practiced and proved. f) Benefits: The improvement/profit the application of the pattern brings.Can be used as justification for usage of the pattern. g) Risks: Possible pitfalls that might occur whenthe pattern is applied. h) GSD specific characteristic: What is characteristic of this pattern that makesit GSD specific? i) Literature link: Hints to literature where the described problem and/or solution hasbeen described already. j) Sources: A description of the statements the pattern resulted from.

Pattern examples We deliver insight into our result by describing a selection of patterns and itsjustification, i.e., the data we based the extraction of this pattern on. The described patterns are high-lighted in table 1. The following abbreviations are used: On = onshore (in this case referring to a sitein Germany), Off = offshore (a site not in Germany, e.g., India), TM = test manager, BA = businessanalyst, PM = project manager. References to other patterns are formatted italic.

Name: Light-weight pre-acceptance test

Problem: How do you prove functioning of the system when you assign complete system test to Off-testteam but the final responsibility for the product is yours (On)?Context: a) The complete responsibility for the system test is with Off-TM and Off-test team. Thusthe On-PM is the client for the result of the system test (the tested system). b) You have to decide howyou check the quality of the (system-tested) product delivered by the Off-test team.Forces: a) When the system test is assigned to the Off-test team, the PM might not see necessity toinvest additional effort by expensive On-expert. b) Especially, when you have not worked with the Off-testteam before, a check of the delivered product is inevitable.Solution: a) Name an On-tester who knows the system and its critical points. This can be an On-TM ora BA who takes on the role of the On-TM. Consider the pattern Mirrored TM. b) After system test usinga structured approach the Off-test team delivers the tested system to the On-tester. c) The On-testerexecutes a light-weight pre-acceptance test with an exploratory/experience-based testing method focusingon high-priority functionality. d) The On-tester reports found bugs to the Off-TM for further dealing. Thepattern Use tool for bug tracking is recommended to simplify this process. Still, communication shouldnot only rely on the tool. e) The Off-TM takes the feedback from the pre-acceptance test as input for theapproach used during structured system test (e.g. to write further test cases). Therewith, a learning effectwill lead to an increase in efficiency of the system test. For this it is important to schedule needed feedbackrounds, especially during first releases. The goal is to give early feedback. f) If the pre-acceptance testhas reached test ending criteria (which have to be defined beforehand), the On-TM clears the system foracceptance test to the customer.Benefits: a) Light-weight, quick quality check of system. b) (Early) feedback to Off-TM, if Off-test isefficient.Risks: Pre-acceptance test might discover many or high-priority errors at a pretty late stage (shortbefore delivery). To avoid this, pre-acceptance test has to be scheduled considering this. If technicallypossible, a pre-acceptance test can already be executed before completeness of the system test.GSD specific characteristic: a) Off-team as additional provider that you might not be familiar withyet. b) Geographic distance leads to reduced informal ways of control which is necessary to monitor thetest status. c) Possibly, the test environment is closer to later productive environment at On-site than atOff-site because of availability.Literature link: In the literature a similar solution as the pre-acceptance test is described by Cusickand Prasad (2006) [44]. They propose Interim functional delivery by the Off-team with the goal that theOn-team (TA) can execute a quality check early. Their intention is mainly to address the risk that heavyquality issues occur late in the project, i.e., shortly before delivery. We describe this in the risk of thepattern.Sources: The pattern Light-weight pre-acceptance test was extracted from data of two projects thatapplied this approach.

The one project was mainly realized offshore in the second release, after onshore participation of off-colleagues in the first release. The team consists of eight Indian team members and one onshore PM whoalso took on the role of On-tester. The On-tester explained

The system test was in the responsibility of the Off-team. There the BA took on the role oftester. After system test the BA delivered the system to me for acceptance. Then I executedtest cases of which I thought, knowing the specification in detail, that it (the system) couldget stuck. [...] In the beginning this was for control. [...] When I cleared the system (after the

6

pre-acceptance test) the version was delivered to the customer. [...] I was the buffer betweenrealization and customer.

This helped to ensure the needed quality of the product delivered to the customer. Especially in thebeginning, when the On-tester did not know the system test’s quality realized by the Off-tester.

The other project’s team consisted of 10-15 members onshore and 15-20 team members offshore (teamsize varied in the course of the project). The test was mainly in the responsibility of the customer’s team.This resulted in reduced focus on the testing activities in the project team. Nevertheless, a dedicatedOff-test team with three testers was set-up as the system was to be tested before it was delivered to thecustomer. The PM reports:

Additionally our (onshore) BA tested much. Well, it was less structured testing than morelike trying out the system. He sat down at the system and just tried various things.

The exploratory testing was an approach with that many errors were found and still little effort had tobe put in.

Name: Extension of the day

Problem: How do you set-up the testing/bug fixing process efficiently when you have a time differencebetween teams?Context: a) Testers at one site, developers/bug-fixers at another site. b) The time difference is approx.4 hours (e.g. Germany - India). c) Use tool for bug tracking is in use. d) Light-weight pre-acceptance testmight be implemented. e) Now you have to set up the communication channels for the testing/bug-fixingprocess.Forces: a) Testing and bug fixing are alternating tasks that depend from one another. b) During bugfixing questions might occur that have to be answered by the tester.Solution: a) After development, the system is handed over to the On-tester. b) The On-tester executesthe test and documents found bugs in a bug tracking tool. Then he leaves off work. c) When the Off-developers start their working days they view (new or changed) bugs. Simple bugs are fixed during themorning. d) When the On-tester starts his day later, they discuss unclear bugs and questions. Synchronouscommunication is important especially to clear issues! e) At the end of their working day, the Off-developersupdate the bug descriptions of bugs they have worked on (status etc). f) Now the On-tester does a re-test.g) And so on.Benefit: A longer day, reduction in duration of test/bug fixing phase.Risk: When questions occur during bug fixing, the tester might not be available. This depend on thetime difference respectively overlapping time window.GSD specific characteristic: Time-differenceLiterature link: The solution of this pattern is similar to the idea of Follow-the-sun development, asdescribed by Carmel (2009) [45].Sources: The question if problems occurred caused by, i.e., time difference was answered by one TMwith the following statement:

Time difference was less of a problem. The Indians are ahead of us. Our procedure was thatthey [the Indian developers] worked [do bug fixing] in the morning, then I come into the office,we made an hand-over and then I worked [tested] and the next morning they have work [newbugs] again.

When asked for benefits of GSD he answered

Time difference was an issue. I am a late riser ;-) All together we had a longer day. In thedirection of India this is an advantage. In the morning, the Indians worked, we discussedmidday/afternoon, then I tested and then the next morning the Indians had new Bugzillaentries to work on. Thus a longer day - that is positive.

Name: Tester’s sparring partner

Problem: How do you ensure test coverage (coverage of specification) when Off-testers are very spe-cialized?Context: a) The test is assigned to an Off-Test team. b) You have to decide on mechanisms to ensurethat specialization does not result as barrier.Forces: a) Off-testers can lack creativity and view beyond ones own nose caused by strict specializationand clinging to this. b) The TM needs to be able to rely on completeness of the test accomplished atOff-site.Solution: a) When setting up the test process, the TM sets up a formal review process. In this process,an expert of a topic takes on the role of the sparring partner. This role can be staffed with, e.g., a Off-BAwho knows a certain part of the specification exactly because he was involved in the On-specification

7

phase. An Off-sparring partner has the advantage to be at the same site as the Off-tester and thus,communication is very probably more efficient. Nevertheless, an On-sparring partner can fulfill the taskas well. Consider the combination with Test cases as MoU of knowledge transfer. b) To provide earlysupport and to reduce risk, test ideas or test aspects are discussed early by the Off-tester and the sparringpartner. The sparring partner reviews the test cases as soon as they are fully specified. The sparringpartners task is to ensure that all aspects are covered by the test cases. His view on the system is broaderand thus he takes into account aspects that might not be in the testers focus. c) Ideally each tester shouldbe provided with one sparring partner (this depends on the size of the team).Benefits: a) Reduces the risk of low coverage. b) More confident tester as expert for system is availablefor questions.Risks: a) Cut back of role BA after specification as ”they are not needed anymore”. b) Communicationbetween BA and testers not efficient because of specialization (might cause misunderstanding, they mightnot be used to intensive collaboration between specialization fields). c) The pattern increases the effortfor support by the On-team.GSD specific characteristic: Strict specialization in other countries, e.g., very strict differentiationof different careers such as tester or developer in India.Literature link: Abraham (2009) [46] points out that cultural differences have an impact on how aperson tests software. Mainly they tend do leave out the domain perspective. Her proposal to reduce thisrisk is to have a BA or a domain specialist review test cases for completeness and correctness.Sources: On the challenge of specialization of team members in India, one PM explains

He [the developer] will seldom have an overall picture of the complete application. He alwayshas his small picture, but the rest is missing. You often get the problem that you think ’Youshould see that following the general train of thought’. That is an expectation that we oftenhave. But that is not how it works for them.

Although this statement is on developers, you can transfer the issue of specialization and missing creativityto testers. The PM also describes that Indian testers lacked getting ideas on things that are not concretelydescribed or explained. This missing creativity influences the quality of test cases. But he pointed out thatthe generalist understanding at Capgemini sd&m is in string contrast to the described strict specialization.This might have to be considered when applying the pattern in other organizations.

A statement of another TM confirmed this experience. He explained that at Capgemini India they”strictly separate functions” which results in the attitude that ”a developer is a developer and a functionaltester is a functional tester”. Thus they do not consider aspects that are not part of their role, they simplyexpect others to tell them what to do. Furthermore, he explained how they realized their review processfor test cases. Each tester had one special field to write test cases for. Then reviews were accomplished bylocal reviewers, i.e., Indian (German) colleagues reviewed test cases written by Indian (German) testers.He explains

The BAs were in the test team or, if not, they at least checked for their field of expertise ifthe test cases are ok.

The third source describes a negative experience:

The test team in India got his knowledge [of the system specification] from the set-up thatBAs were in the team and, if they had time, supported with the specification that tookplace onshore. The idea was that they [the Off-BAs] transferred their knowledge to the [Off-]test team and therewith enable them to specify and execute reasonable test cases. This didnot work well. The knowledge transfer was very poor and until the end you did have theimpression that the test team did not really understand what the system was supposed to do.

During verification, the PM confirmed the relevance of the pattern Tester’s sparring partner to avoid thedescribed problems they ran into.

These example were to show how we present our solution in the form of patterns. All patterns have aprofound basis in real-life projects which we show with the above project descriptions and citations.

Views on the patterns for testing in GSD To simplify the usage of the 16 patterns we presentthem in various views. The Relation view shows which patterns ”requires”, ”excludes”, ”recommends” or”complements” another pattern (see figure 2). This view shows relations of the patterns and supports theuser to identify groups of patterns that are likely to provide a comprehensive solution.

Additionally, a user of the patterns can easily identify which patterns are of main importance for hisrole using the Roles view. In the Test process view all patterns are assigned to the test activities (asdescribed in the basic test process of ISTQB [47]), the pattern is most relevant for and during whichactivity it should be considered.

8

Figure 2: Relation view of the patterns

Threats to validity We characterize the threats to validity of this work using a classification schemedescribed by Runeson and Host (2009) [15]. The following applies to the analysis of practical data.

A threat to construct validity definitely applies as it cannot be ensured that the interviewer understandsthings the same way the interviewee means them. We addressed this threat by providing backgroundinformation about the goal of the research. Therewith we achieve that the interviewee knows the contextof the questions. I also give examples when asking questions that seem to be unclear. Additionally, weaddress the threat to construct validity by verification of the analyzed results from the interviews.

All interviewees take part in the study voluntarily. Thus no pressure is applied. Also it is assuredby the researchers that the content of the interview will only be published anonymously and aggregated.Additionally, it is to mention that the authors at the company observed a very open culture regardingsharing of experiences, both negative and positive, in general at Capgemini sd&m. With all this, areasonably open atmosphere during the interview is provided. Thus from this point of view we do not seea threat to internal validity.

Nevertheless, to increase the precision of the research we use different types of triangulation. We useseveral data sources for data (source) triangulation. Only one qualitative method is used but methodolog-ical triangulation is ensured by usage of various data collection methods such as questionnaire, interview,expert discussion. Observer and theory triangulation could not be realized.

The research meets a threat to reliability as we will not be able to document every single one of ourideas, thoughts, actions, and rationales. Nevertheless, I describe the research approach as detailed aswe consider it to be necessary to make the report of results traceable. Especially during the analysisof practical data we used codes to document the trace from interview transcript to the data records ofproblem, solution, idea for pattern, and to resulting patterns. Thus the source of the result is traceable.

We see a heavy threat to external validity. Only projects of one company, namely Capgemini sd&mare investigated. As described above certain conditions apply to these projects. In this research weinvestigate four study objects. It is to be questioned if results from four projects that are above alldifferent in topic and project settings can be generalized. Additionally, it is to mention that only Germancolleagues and none from India or Poland take part in the interviews. As the main responsibility forthe projects is in Germany, main contacts are German colleagues. Taking these points into account, ourfindings might not be generalizable to other contexts. However we take various measures to reduce thethreat to external validity: We include literature as a complementary source for the research. Expertdiscussions and realization of an evaluation contribute further.

9

5 Conclusions

With our solution of a collection of patterns for testing in GSD we reached several goals.

1. We close a gap in the literature, where so far little has been reported on the challenges and solutionsfor testing in GSD.

2. We provide practitioners at Capgemini sd&m with easy to use solutions in the form of patterns whenrealizing test in GSD projects. With the problem/solution summary the user gets an overview.

3. Further views of the patterns support the selection of patterns relevant for a certain role and testactivity. The user benefits from the option to combine several patterns to set-up a tailored solution.

We provide a basis in the investigation on the topic of testing in GSD. The identified problems andproposed solutions are a first step. A first review of the patterns by experienced test managers in offshoreprojects indicate the relevance and applicability of the patterns. Nevertheless, more investigation is neededto comprehensively analyze problems and solutions. Our research approach can serve as procedure forfurther investigation. The solution presentation in the form of patterns can be used to present futurefindings.

Future work Future work contains the evaluation of the results in other projects within Capgeminisd&m. This step is part of the current research and will be realized soon. As extension of the present work,an evaluation of the usage of the patterns in other companies would provide information if the results canbe generalized. Findings from such evaluations should be introduced into the collection of patterns, e.g.,modifications and/or extension of patterns or the description of further patterns.

Acknowledgements We would like to thank all interview partners at Capgemini sd&m for supportingour work.

References

[1] B. Sengupta, S. Chandra, and V. Sinha. A research agenda for distributed software development. InICSE ’06: Proceedings of the 28th international conference on Software engineering, pages 731–740,New York, NY, USA, 2006. ACM.

[2] M. Jimenez, M. Piattini, and A. Vizcaıno. Challenges and improvements in distributed softwaredevelopment: A systematic review. Advances in Software Engineering, 2009.

[3] D.E. Damian and D. Zowghi. An insight into the interplay between culture, conflict and distance inglobally distributed requirements negotiations. In HICSS 36: Proceedings of the 36th Annual HawaiiInternational Conference on System Sciences, page 10pp., 2003.

[4] J. D. Herbsleb and D. Moitra. Global software development. IEEE Software, 18(2):16 – 20, 2001.

[5] J. Ralyte, X. Lamielle, N. Arni-Bloch, and M. Leonard. Distributed information systems develop-ment: A framework for understanding and managing. International Journal of Computer Science &Applications, Special issue New Trends in Information System Modelling, 5 (3b):1–24, 2008.

[6] M. Vanzin, M.B. Ribeiro, R. Prikladnicki, I. Ceccato, and D. Antunes. Global software processes def-inition in a distributed environment. In SEW 2005: 29th Annual IEEE/NASA Software EngineeringWorkshop, pages 57–65, 2005.

[7] R.E. Grinter, J.D. Herbsleb, and D.E. Perry. The geography of coordination: dealing with distancein r&d work. In GROUP ’99: Proceedings of the international ACM SIGGROUP conference onSupporting group work, pages 306–315, New York, NY, USA, 1999. ACM.

[8] B. Copstein and F.M. de Oliveira. Management of a Distributed Testing Process using Workflowtechnologies: A Case Study. In Seventh Workshop on Empirical Studies of Software Maintenance,pages 62–64, 2001.

[9] D. Zage, W. Zage, and C. Wilburn. Test management and process support for virtual teams. Technicalreport, Ball State University, April 2005.

[10] R.S. Sangwan and P.A. Laplante. Test-driven development in large projects. IT Professional, 8(5):25–29, 2006.

[11] V. Casey. Software Testing and Global Industry: Future Paradigms. Cambridge Scholars Publishing,United Kingdom, 2009.

[12] F. Salger. On the use of handover checkpoints to manage the global software development process.In On the Move to Meaningful Internet Systems: OTM 2009 Workshops, pages 267–276. Springer,2009.

10

[13] F. Salger and G. Engels. Knowledge transfer in global software development: leveraging acceptancetest case specifications. In ICSE ’10: Proceedings of the 32nd ACM/IEEE International Conferenceon Software Engineering, pages 211–214, New York, NY, USA, 2010. ACM.

[14] F. Salger, G. Engels, and A. Hofmann. Assessments in global software development: a tailorableframework for industrial projects. In ICSE ’10: Proceedings of the 32nd ACM/IEEE InternationalConference on Software Engineering, pages 29–38, New York, NY, USA, 2010. ACM.

[15] P. Runeson and M. Host. Guidelines for conducting and reporting case study research in softwareengineering. Empirical Software Engineering, 14(2):131–164, 2009.

[16] B. Kitchenham and S. Charters. Guidelines for performing systematic literature reviews in softwareengineering. Technical Report EBSE-2007-01, Keele University and University of Durham, July 2007.

[17] Google schoolar. http://scholar.google.de/.

[18] J.D. Herbsleb and R.E. Grinter. Splitting the organization and integrating the code: Conway’s lawrevisited. In ICSE ’99: Proceedings of the 21st international conference on Software engineering,pages 85–95, New York, NY, USA, 1999. ACM.

[19] J. D. Herbsleb, D. J. Paulish, and M.Base. Global software development at siemens: experience fromnine projects. In ICSE ’05: Proceedings of the 27th international conference on Software engineering,pages 524–533, New York, NY, USA, 2005. ACM.

[20] E. Carmel and P. Tjia. Offshoring Information Technology. Sourcing and Outsourcing to a GlobalWorkforce. Cambridge University Press, 3 edition, 2007.

[21] H. Holmstrom, E.O. Conchuir, P.J. Agerfalk, and B. Fitzgerald. Global software development chal-lenges: A case study on temporal, geographical and socio-cultural distance. In ICGSE ’06: Interna-tional Conference on Global Software Engineering, pages 3–11, 2006.

[22] K.C. Desouza, Y. Awazu, and P. Baloh. Managing knowledge in global software development efforts:Issues and practices. IEEE Software, 23(5):30–37, 2006.

[23] J.D. Herbsleb, A. Mockus, T. A. Finholt, and R.E. Grinter. An empirical study of global softwaredevelopment: distance and speed. In ICSE ’01: Proceedings of the 23rd International Conference onSoftware Engineering, pages 81–90, Washington, DC, USA, 2001. IEEE Computer Society.

[24] E. Carmel and R. Agarwal. Tactical approaches for alleviating distance in global software develop-ment. IEEE Software, 18(2):22–29, 2001.

[25] C. Ebert and P. DeNeve. Surviving global software development. IEEE Software, 18(2):62–69, 2001.

[26] R. Kommeren and P. Parviainen. Philips experience in global distributed software development.Empirical Software Engineering, 12:647–660, September 2007.

[27] S. Krishna, S. Sahay, and G.Walsham. Managing cross-cultural issues in global software outsourcing.Communications of the ACM, 47(4):62–66, 2004.

[28] B. Sengupta, S. Chandra, and V. Sinha. Test-driven global software development. In Proceedings ofThe 3rd International Workshop on Global Software Development, co-located with ICSE 2004, 2004.

[29] B. Bruegge, A. H. Dutoit, and T. Wolf. Sysiphus: Enabling informal collaboration in global softwaredevelopment. In IEEE International Conference on Global Software Engineering (ICGSE 2006), 2006.

[30] I. Tervonen and T. Mustonen. Offshoring Test Automation: Observations and Lessons Learned. InICGSE 2009: Fourth IEEE International Conference on Global Software Engineering, pages 226–235.IEEE Computer Society, 2009.

[31] A.B. Bondi and J.P. Ros. Experience with training a remotely located performance test team in aquasi-agile global environment. In ICGSE 2009: Fourth IEEE International Conference on GlobalSoftware Engineering, pages 254–261. IEEE Computer Society, 2009.

[32] B.G. Glaser and A.L. Strauss. The discovery of grounded theory: Strategies for qualitative research.Aldine, 1967.

[33] L. Hagge, K. Lappe, D. Elektronen-Synchrotron, and G. Hamburg. Sharing requirements engineeringexperience using patterns. IEEE software, 22(1):24–31, 2005.

[34] M. Kircher and M. Volter. Guest Editors’ Introduction: Software Patterns. IEEE software, 24(4):28–30, 2007.

[35] J.O. Coplien. Design pattern definition.

[36] F. Salger, J. Englert, and G. Engels. Towards specifications patterns for global software developmentprojects - experiences from the industry. In QUATIC 2010: 7th International Conference on theQuality of Information and Communications Technology, 2010.

[37] L. Doug. Christopher Alexander: An introduction for object-oriented designers. SIGSOFT SoftwareEngineering Notes, 19(1):39–46, 1994.

11

[38] K. Beck. Patterns and software development. http://www.drdobbs.com/mobility/184409176, Febru-ary 1994. viewed 5th April 2010.

[39] G. Meszaros and J. Doble. A pattern language for pattern writing. Pattern languages of programdesign, 3:529–574, 1998.

[40] T. Tsumaki. Requirements engineering pattern structure. 2004.

[41] M. Fowler. Writing software patterns. http://martinfowler.com/articles/writingPatterns.html, Au-gust 2006. viewed 14th April 2010.

[42] S. Henninger and V. Correa. Software pattern communities: Current practices and challenges. InPLoP 07: 14th Conference on Pattern Languages of Programs, 2007.

[43] S. Fincher. The pattern gallery. http://www.cs.kent.ac.uk/people/staff/saf/patterns/gallery.html,2009. date referres to last update, viewed 14th April 2010.

[44] J. Cusick and A. Prasad. A practical management and engineering approach to offshore collaboration.IEEE Software, 23(5):20–29, 2006.

[45] E. Carmel, Y. Dubinsky, and A. Espinosa. Follow the sun software development: New perspectives,conceptual foundation, and exploratory field study. In HICSS 2009: 42nd Hawaii InternationalConference on System Sciences, pages 1–9, 2009.

[46] L.R. Abraham. Cultural differences in software engineering. In ISEC ’09: Proceedings of the 2ndIndia software engineering conference, pages 95–100, New York, NY, USA, 2009. ACM.

[47] A. Spillner, T. Linz, and H. Schaefer. Software Testing Foundations. Rocky Nook Inc, 2nd edition,2007.

12