Embed Size (px)
Citation preview
Infosys Defect Fixing Process Improvement Plan
INF 5181 – Final Project Report
Yannick Lew Yaw Fung [email protected]
15 November 2012
Page 1 of 19
Table of Contents Introduction ............................................................................................................................................. 3
Purpose of this document ........................................................................................................................ 3
1.0 Improvement Context .................................................................................................................. 3
1.1 People Overview ..................................................................................................................... 3
1.2 Defect fixing Process Overview .............................................................................................. 4
2.0 Process Improvement Goals (PI Goals) ...................................................................................... 6
2.1 Project Scope ........................................................................................................................... 6
2.2 Process Improvement Goals .................................................................................................... 6
2.2.1 Objective: Reduction in defect fixing turnaround time for new and re-opened defects
from two and a half days to no more than a day. ............................................................................. 6
2.2.2 Process Improvement Goal 1: Faster defect clarification responses ............................... 7
2.2.3 Process Improvement Goal 2: Reduction in the number of re-opened defects due to
delivering the wrong fix .................................................................................................................. 7
2.2.4 Process Improvement Goal 3: Reduction in the number of re-opened defects due to
poor implementation testing ............................................................................................................ 8
2.2.5 Process Improvement Goal 4: Improved Defect fixing success rate ............................... 8
3.0 Process Improvement Changes .................................................................................................... 9
3.1 Process Improvement Goal 1: Faster defect clarification responses ....................................... 9
3.1.1 SG 1 ................................................................................................................................. 9
3.2 Process Improvement Goal 2: Reduction in the number of re-opened defects due to
delivering the wrong fix .................................................................................................................... 10
3.2.1 SG 2 ............................................................................................................................... 10
3.2.2 SG 3 ............................................................................................................................... 10
3.3 Process Improvement Goal 3: Reduction in the number of re-opened defects due to poor
implementation testing .......................................................................................................................11
3.3.1 SG 4 ................................................................................................................................11
3.3.2 SG 5 ................................................................................................................................11
3.4 Process Improvement Goal 4: Improved Defect fixing success rate ..................................... 12
3.4.1 SG 6 ............................................................................................................................... 12
3.4.2 SG 7 ............................................................................................................................... 12
4.0 Implementation of Process Changes ......................................................................................... 12
4.1 Key Stakeholders and their responsibilities........................................................................... 12
4.2 Critical success factors .......................................................................................................... 13
4.3 Expected roll-out and completion time schedule .................................................................. 13
5.0 Monitoring and Control Processes ............................................................................................ 15
5.1 Time schedule ........................................................................................................................ 15
Page 2 of 19
5.2 Process measure data collection ............................................................................................ 15
5.3 Process monitoring and control ............................................................................................. 15
6.0 Discussion ................................................................................................................................. 16
6.1 Process Improvement Goals .................................................................................................. 16
6.1.1 PI Goal 1: Faster defect clarification responses ............................................................ 17
6.1.2 PI Goal 2: Reduction in the number of re-opened defects due to delivering the wrong
fix 17
6.1.3 PI Goal 3: Reduction in the number of re-opened defects due to poor implementation
testing 17
6.1.4 PI Goal 4: Improved Defect fixing success rate ............................................................ 17
6.2 Suggested changes ................................................................................................................. 17
6.2.1 SG 1: .............................................................................................................................. 17
6.2.2 SG 2: .............................................................................................................................. 18
6.2.3 SG 3: .............................................................................................................................. 18
6.2.4 SG 4: .............................................................................................................................. 18
6.2.5 SG 5 ............................................................................................................................... 18
6.2.6 SG 6: .............................................................................................................................. 18
6.2.7 SG 7: .............................................................................................................................. 19
6.3 Risk ........................................................................................................................................ 19
Page 3 of 19
Introduction
In the beginning of 2012, Infosys Limited, a large software outsourcing company headquartered in
Bangalore, India, was contacted by a major European Insurance company providing both life and non-
life insurance products to its customers. The insurance company relies heavily on software for its day-
to-day operations and has a large portfolio of web applications to manage its various activities. One of
such web applications was the company’s internal web-based insurance management application that
was in the process of being migrated from VB6 to a VB.NET platform which was an in-house product
developed by the company’s own IT department. It was agreed that Infosys would provide defect
fixing support to the insurance company’s existing development team in fixing the software bugs or
defects that were identified by the company’s application test users as part of the overall test migration
strategy elaborated by the company’s software engineers.
An Infosys software development team, comprising of people which would be located in Europe at the
client site, Mauritius, and India, was formed. The Infosys development teams were given remote
access to the client’s source code and fixing of defects began at the beginning of September 2012 with
an estimated completion period of eight months based on the project’s size and complexity.
Two months after the start of the project, a preliminary assessment was carried out to evaluate the
project status and propose solutions to problems or issues being faced by the team. Some of these
issues involved, for example, difficulties in proper understanding defect descriptions, re-opening of
defects due to poor or incomplete fixes, and utilization rates of the various team members. As a result,
the main conclusion of the assessment was that the average defect fixing turnaround time was getting
higher due to these problems and an immediate remedy had to be found.
Purpose of this document
The purpose of this process improvement document plan is to address the various objectives of the
assessment study by proposing changes to the current defect fixing process. Each proposed change is
explained with respect to its impact on one or more desired goals of the improvement plan.
Moreover, since the code base of the new insurance management web application is also going to be
used in the insurance company’s other software migration projects, it is clear that Infosys has to
provide entire satisfaction to the insurance company in order to be considered for subsequent defect
fixing support contracts in the near future – further highlighting the importance of this process
improvement document.
1.0 Improvement Context The improvement context for the defect fixing project is explained in this section.
1.1 People Overview One of the client requirements for the project was for Infosys to provide development personnel with
the ability to speak English and French because defect descriptions were mostly written in French
based on the application test users’ preferences. Consequently, Infosys decided to leverage the French-
Page 4 of 19
speaking abilities of its workforce based in Mauritius. A team was thus set up comprising of nine
Infosys employees with the following role and geographical configuration: two software engineers
based in India (one in Bangalore and the other in Pune); one project manager, one project leader, one
technical analyst, and three software engineers based in Mauritius; and one technical analyst based in
Europe at the client location. The main responsibilities of each team member are defined as follows:
Project manager (PM): provides planning and decisional support to team members at a high-
level and ensures the smooth running of the project.
Project leader (PL): supervises the development teams and resolves problems that occur
within his job capacity.
Technical analyst (Mauritius) or TAM: helps the development team with the resolution of
specific technical problems.
Technical analyst (Europe): Also referred to as point-of-contact or POC, answers questions
that need to be asked to the client.
Software engineers (Dev. team): are responsible for the actual fixing of bugs or defects that
are identified and signaled by clients. Their responsibilities include the reproduction, testing,
debugging, and fixing of defects raised by application test users at the client location.
1.2 Defect fixing Process Overview The approach, which was adopted for the defect fixing activities, follows an ad-hoc process based on
the nature and specific requirements of the project. As the latter progressed, the ad-hoc process would
then be continuously refined and improved based on feedback given during weekly team discussions
comprising of the complete Infosys team for the project. The diagram in figure 1 illustrates the usual
defect fixing process being undertaken after a two-month period.
Each defect fixing scenario is started by a test user at the client location as shown in figure 1. The test
user is generally an active employee having extensive knowledge of the application functionalities and
executes a series of test cases previously identified. Whenever an unwanted behavior or software error
occurs resulting in a bug in the insurance management application, a defect fixing request is raised in
the client defect tracker system or DTS. The latter is a local software program that is used to collect,
track, and manage defect fixing requests raised by test users. As soon as a new defect is registered in
the client DTS, a notification mail from the latter is sent to the client’s development team who then
evaluates the nature of the defect and assigns it to either another team member or Infosys for fixing.
This previous step is done to ensure that the time to fix a defect or defect turnaround time is kept to a
minimum as some defects can be more quickly corrected by the client’s Dev. team than Infosys’.
Page 5 of 19
After the client DTS is updated with the new defect fixing request attributed to Infosys, the Infosys
POC forwards the defect fixing request to the Infosys project leader who assigns the defect request to
any available team member based on whether this person is free or does not have many defects already
allocated to him already, i.e., his defect fixing queue list is relatively short compared to others. It is
worth mentioning that defect request allocation is also prioritized based on the defect importance
which ranges from ‘Low’, ‘Medium’, ‘High’, and ‘Critical’ such that a critical defect is allocated first
than a medium one to an available team member. Upon successful defect fixing assignment, the
Infosys DTS, which is basically a spreadsheet document stored in a shared network folder that any
member of the Infosys Dev. team can have access to, is updated accordingly. A mail is also sent by the
Infosys PL to the Dev. team member responsible for the defect request.
When the Infosys Dev. team member is informed of the bug request, he opens both the Infosys and
client DTSs to validate the defect and understand the corresponding fix that needs to be implemented.
If the defect description is not clear and a technical clarification is needed, the Infosys Dev. team
member contacts the TAM for help. If the defect query is of a non-technical nature, the POC contacts
the client at the onsite location for clarification. On the other hand, if the Infosys Dev. team member
fully knows how to correct the defect, he starts the defect analysis before making the required changes
to the client source code. After the fix has been written, the team member tests the defect fix to ensure
that the fix is working properly.
When the Infosys Dev. team member is satisfied with his defect correction changes, the fix is
committed to the client source code repository and a mail is sent to the POC for a final testing activity
on the committed code modifications. If the fix is approved by the POC, the client test user is
informed for inspection and the defect fixing request is considered to have been fulfilled and the client
Start
Test target system and raise defect
Evaluate and assign defect
Forward defect
request
Assign defect to Dev. team member
Client DTS
Validate defect requirements
Is defect clarification
needed?
Provide defect query resolution
Code and test defect fix
Is defect fix ok?
Verify defect fix
Is defect fix ok?
End
YES
NO
YES
YES
NO
NO
Client Test User
Client Dev. Team
Infosys POC
Infosys PL
Infosys Dev. Team
Infosys POC
Infosys DTS
Client Source Code
Client DTS
Role
Process
ArtifactLegend
Start Decision End
uses
produces
consumes
Infosys TAM
Figure 1 – Defect fixing process model
Page 6 of 19
DTS is updated accordingly. However, if there is any issue with the code changes that require
remediation, the defect request is re-opened and sent back to the Infosys PL along with the problem
encountered by the POC. The Infosys PL then re-assigns the defect to another available team member.
Likewise, a defect can also be re-opened by the client test users if the defect fixing changes do not
fully conform to their expectations. The process of handling re-opening of defects follows a similar
path as the one shown in figure 1.
2.0 Process Improvement Goals (PI Goals) The issues that have been identified in the assessment report are addressed in this section as part of the
process improvement planning undertaken by Infosys for the defect fixing project.
2.1 Project Scope The scope of the analysis and its subsequent recommendations as presented in this document are
limited only to Infosys’ practices as essentially described in the process model in figure 1 and bringing
improvements to client processes is beyond the intent of this document.
2.2 Process Improvement Goals The required process improvement goals are discussed in this section.
2.2.1 Objective: Reduction in defect fixing turnaround time for new and re-opened defects
from two and a half days to no more than a day.
The primary objective of this process improvement plan is to lower the average time taken to fix
defects raised by client test users from the current situation of two and a half days to no more than one
day.
Reason: Fixing of defects is taking too much time and this delay is being seen negatively by the
insurance client.
Benefits: Improved client satisfaction as well as renewed trust and reputation towards Infosys.
Reaching this goal can have a very positive effect on the allocation of future project contracts by the
client to Infosys.
Process measure: Average defect fixing turnaround time
Entity Defect (New or re-opened)
Attribute Fix duration (Average number of days taken by the whole team to fix a new or
re-opened defect per week)
Unit Calendar day (Number of days taken to fix a new or re-opened defect / Number
of defects fixed per week)
Range (0, ∞)
Page 7 of 19
Improvement target:
Process measure Average defect fixing turnaround time
Before 2.5 days After 1 day
Time period: Depending on the required time frame for the successful implementation of the other
secondary improvement goals, we expect to see a positive effect in the defect fixing turnaround time in
two to three weeks’ time.
Achieving a reduction in defect fixing turnaround time is equivalent to raising the work productivity
of each member of the Infosys team such that defects are fixed in the shortest time possible on a day-
to-day basis. In order to do so, a list of improvement goals have been identified to address the issues or
problems that are hampering the ability of the Infosys team members to fix defects using the least
amount of time.
2.2.2 Process Improvement Goal 1: Faster defect clarification responses
This improvement goal is concerned with decreasing the amount of time it takes to reply to defect
clarification queries which are currently being handled by the TAM for technical ones and POC for
non-technical ones.
Process measure: Average technical defect clarification response time
Entity Technical defect clarification request
Attribute Response time (Average time taken in hours to respond to a technical query per week)
Unit Clock hour (Total time taken (in hours) to respond to queries / Number of queries
received per week)
Range (0, ∞)
Improvement target:
Process measure Time taken in hours to respond to a query
Before 2 hrs. After 1 hr.
Time target: Three weeks including a transition phase period.
2.2.3 Process Improvement Goal 2: Reduction in the number of re-opened defects due to
delivering the wrong fix
This improvement goal is concerned with properly validating defect descriptions before these are sent
to the Dev. team so that the number of re-opened defects due to delivering the wrong fixes is reduced.
Currently, this validation is not being executed. Moreover, the Infosys POC does not currently check
all defect fixes sent to him because of his time constraints. Therefore, improper fixes are being sent to
the client for testing without a secondary round of verification.
Page 8 of 19
Process measure: Defect fixing re-open rate due to delivering the wrong fix
Entity Defect
Attribute Re-opened for fixing (for the whole team per week)
Unit Percentage [(Number of re-opened defect requests due to delivering the wrong fix
/ Total number of defect fixes committed per week) * 100]
Range (0, 100)
Improvement target:
Process measure Defect fixing re-open rate due to delivering the wrong fix
Before 60 % After 30%
Time target: Three weeks including a transition phase period.
2.2.4 Process Improvement Goal 3: Reduction in the number of re-opened defects due to
poor implementation testing
This improvement goal is concerned with reducing the number of defects that are re-opened due to
poor implementation testing. This is, in contrast, to improvement goal 1 where the defect description is
not clear. In this case, the expected outcome of a defect fix is known, but the latter’s implementation is
incorrect (mainly due to poor testing of the defect fixes). Similarly, as for process improvement goal 2,
the Infosys POC does not currently check all defect fixes sent to him because of his time constraints.
Process measure: Defect fixing re-open rate due to poor implementation
Entity Defect
Attribute Re-opened for fixing (per team member per week)
Unit Percentage [(Number of re-opened defect requests due to poor implementation
testing / Total number of defect fixes committed per week) * 100]
Range (0, 100)
Improvement target:
Process measure Defect fixing re-open rate due to poor implementation testing
Before 60 % After 10 %
Time target: Four weeks including a transition phase period and two weeks for creating test procedure.
2.2.5 Process Improvement Goal 4: Improved Defect fixing success rate
This improvement goal deals with the way defect fixing requests (new and re-opened) are currently
being allocated to the Infosys Dev. team by the project leader. It is important that defect allocation is
done based not only on member availability, but also on the defect fixing complexity and the technical
experience of team members so as to improve the defect fixing success rate of each Dev. team
member.
Page 9 of 19
Process measure: Defect fixing success rate
Entity Defect
Attribute Fixing rate (per team member per week)
Unit Percentage [1 - (Number of defects that are re-assigned* / Total number of defect
requests per week) * 100]
Range (0, 100)
Improvement target:
Process measure Defect fixing success rate
Before 50% After 80%
Time target: Two weeks including a transition phase period.
3.0 Process Improvement Changes
This section lists the problems encountered with respect to each process improvement goal, proposes
changes (also known as suggested changes or SG) to be made to the existing defect fixing activities,
and mentions the affected stakeholders during process improvement initiative.
3.1 Process Improvement Goal 1: Faster defect clarification responses Currently, the Infosys POC is the only one in charge of replying to defect explanation requests while
technical queries are handled exclusively by the TAM.
3.1.1 SG 1
Create a defect support unit (Infosys DSU) comprising of both the POC and TAM to handle defect
clarification requests. Since both of them have the abilities to respond to technical and non-technical
requests, their tasks, which were previously the responsibility of either the POC or TAM, can now be
shared between both of them. The new process of handling defect clarification requests is shown in
figure 2.
Page 10 of 19
Affected stakeholders: Infosys POC and TAM.
3.2 Process Improvement Goal 2: Reduction in the number of re-opened defects
due to delivering the wrong fix Defect fixing descriptions are currently not being validated before being sent to the Infosys PL for
defect assignment. Furthermore, the Infosys POC does not currently have the resources to test all
defect fixes based on his time constraints and available resources.
3.2.1 SG 2
Have the defect support unit (Infosys DSU) validate the defect requirements before forwarding to the
PL. If a defect description is not clear or understandable, the defect support unit should query the
client test user for further clarification. The defect support unit can also try to reproduce the defect by
running the web application if they feel that the defect is missing vital information. Once the defect is
properly described, the latter can then be forwarded for fixing. This is shown in figure 3.
3.2.2 SG 3
Have the Infosys POC share the workload of the verification process with the Infosys TA as part of the
tasks required by the new defect support unit. This process change is shown in figure 3.
Affected stakeholders: Infosys POC and TAM.
Provide defect query resolution
Is defect clarification
needed?
Is defect clarification technical?
YES
Provide technical
assistance
NOAsk client for clarification
Code and test defect fix
Infosys DSU
Client Test User
Figure 2 - New process of handling defect clarification requests
Page 11 of 19
3.3 Process Improvement Goal 3: Reduction in the number of re-opened defects
due to poor implementation testing Currently, there is no formal method specified for the verification of defect fixes created by the Infosys
Dev. team member. Moreover, a systematic testing culture of the defect fixes has been found to be
lacking according to the assessment report. As such, some Infosys Dev. team members do not always
verify their defect fixes. Furthermore, the Infosys POC does not currently have the resources to test all
defect fixes based on his time constraints and available resources.
3.3.1 SG 4
Educate team members regarding the need for proper, thorough testing of defect fixes and establish a
complete test procedure on how to effectively perform defect fixing verification based on the
particular defect scenario. The test procedure should be created in collaboration with the client.
3.3.2 SG 5
Same as SG3.
Affected stakeholders: Infosys POC, TAM, and Dev. team.
Start
Test target system and raise defect
Evaluate and assign defect
Validate defect
requirements
Assign defect to Dev. team member
Client DTS
Validate defect requirements
Is defect clarification
needed?
Provide defect query resolution
Code and test defect fix
Is defect fix ok?
Verify defect fix
Is defect fix ok?
End
YES
YES
NO
YES
NO
Client Test User
Client Dev. Team
Infosys POC
Infosys DSU
Infosys Dev. Team
Infosys DTS
Client Source Code
Client DTS
Role
Process
ArtifactLegend
Start Decision End
uses
produces
consumes
Infosys DSUInfosys PL
Is defect clear?
YES
Infosys Dev. Team
Ask client for clarification
NO
Is fixing completed
?
YESHas 2 days elapsed?
NO
YES
NO
Client Source Code
NO
Figure 3 – New defect fixing process model
Page 12 of 19
3.4 Process Improvement Goal 4: Improved Defect fixing success rate Currently, defect fixing requests are being assigned to Dev. team members based only on their
availability by the Infosys PL. Another issue is that fixing a defect can take three or more days to be
completed by a team member.
3.4.1 SG 6
The Infosys PL should consult the defect support unit to obtain more information regarding a defect’s
complexity and the technical experience required to do so from team members before assigning
defects to them. If necessary, any Dev. team member can be consulted to assess his technical know-
how for a particular defect fixing scenario. This process change is shown in figure 3.
3.4.2 SG 7
Check the status of any defect fixing activity that is taking more than two days. If a team member is
not able to fix a defect within a two-day period, the defect’s complexity is re-evaluated based on inputs
from the Dev. team member and a decision is taken by the Infosys PL and defect support unit members
to either give more time for the team member to complete the defect fixing or assign it to a new
person. This process change is shown in figure 3.
Affected stakeholders: Infosys POC, PL, TAM, and Dev. team.
4.0 Implementation of Process Changes The implementation details of this process improvement plan is presented in this section.
4.1 Key Stakeholders and their responsibilities The Infosys project leader will be the process owner and will be tasked to ensure that the process
improvement recommendations are being correctly followed and executed by all the team members.
Monitoring and control activities of the various process measures are also part of his responsibilities.
Any deviation or problem encountered during the course of the process improvement activities should
be dealt with swiftly and reported to the project manager for further assistance if required.
The Infosys PL will also serve as process anchor in answering any process improvement-related
queries that team members may have during the course of the process improvement initiative.
The process anchor will also be assisted by other Infosys team members in making sure the PI Goals
are reached within the prescribed time limit as shown in table 1.
Table 1 – Responsibilities of each stakeholder
Task Responsible
person Responsibilities
SG 1 Robert, POC
and Stina, TAM
Robert and Stina will ensure that both receives any future defect
clarification requests and will coordinate on who actually responds to
each query. They need to inform the Dev. team of the changes. If any
Dev. team member forgets to inform both members of the Infosys
DSU, he or she must be reminded in doing so next time by either one
Page 13 of 19
of them. Both will have to create a proper planning regarding how
defect queries are replied based on each other’s availability.
SG 2 Robert and Stina
Robert and Stina will establish a proper checklist to make sure that
defect descriptions follow a specific pattern such that they are clear
and understandable in terms of what needs to be fixed. They need to
discuss with the Dev. team on the creation of this checklist. During
the transition phase, this checklist can be updated and refined.
SG 3 Robert and Stina Robert and Stina will have to create a proper planning regarding how
defects are verified based on each other’s availability.
SG 4 Jona, PL
Jona will drive the creation of the test procedure and will conduct
several meetings to educate the Dev. team in adhering to the
guidelines specified in the test process.
SG 5 Robert and Stina Same as for SG 3.
SG 6 Jona, Robert and
Stina
The three will set up a proper procedure for identifying complex
defects and establish the technical background of each team member
before creating a method to properly allocate defects based on these
parameters.
SG 7 Jona
Jona will set up the Infosys DTS so that fixes that are taking more
than two days can be easily identified. He will also inform the Dev.
team regarding how the new defect allocation process will take place.
4.2 Critical success factors Three critical success factors have been identified for the success of this process improvement
initiative, namely:
Client understanding
The client will be informed about the process improvement changes being performed by
Infosys and the resulting delays in defect fixing that may occur during the transition period
for each PI Goal. It is important that the client appreciates and understands the temporary
disruptions that will be caused during the process improvement activities.
Team member cooperation and collaboration
The success of the process improvement initiative heavily relies on the participation of
each Infosys team member in properly executing the tasks required to achieve each PI
Goal. Each member must also be an active participant who provides feedback, proposes
changes, and reports any problem that may arise during the implementation of the process
improvement activities.
Stable team size
It is important that the team size remains stable or at least is not weakened. Since the team
unit is already small, removing resources will have an ultimate penalty in Infosys’ ability
to properly execute defect fixing activities.
4.3 Expected roll-out and completion time schedule It is expected that the implementation phase of this process improvement initiative to be rather short
since the changes to be accomplished are rather straight-forward and can be put into practice fairly
Page 14 of 19
quickly. Furthermore, it is important to show the immediate benefits of this process improvement plan
to the client owing to the relatively short time frame remaining for the completion of the defect fixing
project.
The process improvement initiative is set to start on the 3rd of December 2012 and end on the 28th of
December 2012. The benefits of the process improvement activities are expected to be recorded after
this time period. The start date of the 3rd of December 2012 has been selected so as to give team
members time to understand their role and new responsibilities in fulfilling all the imperatives
identified in this process improvement plan.
The detailed implementation plan for the various process improvement (PI) goals is shown in table 2.
The key milestones for this process improvement plan are defined as follows.
Initial Milestone 1 reached on 14.12.2012 for PI Goal 4.
Milestone 2 reached on 22.12.2012 for PI Goals 1 and 2.
Final Milestone 3 reached on 28.12.2012 for PI Goal 3.
Table 2 – Detailed implementation schedule
Task Start End Duration Remarks
PI Goal 1: Faster defect clarification responses
SG 1 03.12.2012 15.12.2012 2 working
weeks
Transition phase. Any issues that crop up
are resolved.
SG 1 17.12.2012 22.12.2012 -
(Milestone 2)
1 working
week
Fully implemented changes and process
measures are being actively controlled
and monitored.
PI Goal 2: Reduction in the number of re-opened defects due to delivering the wrong fix
SG 2 and
SG3 03.12.2012 15.12.2012
2 working
weeks
Transition phase. Any issues that crop up
are resolved.
SG 2 and
SG3 17.12.2012
22.12.2012 -
(Milestone 2)
1 working
week
Fully implemented changes and process
measures are being actively controlled
and monitored.
PI Goal 3: Reduction in the number of re-opened defects due to poor implementation testing
SG 4 and
SG 5 03.12.2012 15.12.2012
2 working
weeks
Define test procedure in collaboration
with the client.
SG 4 and
SG 5 17.12.2012 22.12.2012
1 working
week
Transition phase. Any issues that crop up
are resolved.
SG 4 and
SG 5 24.12.2012
28.12.2012 -
(Milestone 3)
1 working
week
Fully implemented changes and process
measures are being actively controlled
and monitored.
PI Goal 4: Improved Defect fixing success rate
SG 6 and
SG7 03.12.2012 07.12.2012
1 working
weeks
Transition phase. Any issues that crop up
are resolved.
SG 6 and
SG7 10.12.2012
14.12.2012 -
(Milestone 1)
1 working
week
Fully implemented changes and process
measures are being actively controlled
and monitored.
Page 15 of 19
5.0 Monitoring and Control Processes The process owner will be take charge of handling all monitoring and control activities of the various
process measures.
5.1 Time schedule The process improvement activities are expected to be carried out continuously throughout the entire
duration of the project and the process owner will be in charge of monitoring and controlling its
overall progress.
5.2 Process measure data collection Each Infosys team member has the responsibility to provide inputs or base measures needed for
properly recording process measures defined for each PI Goal. These inputs must be provided by the
Dev. team on a weekly basis to the process owner before the weekly process review meetings. Some
information can also be obtained from the Infosys defect tracker system. Table 3 describes the process
base measures, their data sources, and frequencies at which they are recorded.
Table 3 – Process measure collection
Base measure Data source Frequency
PI Goal 1: Faster defect clarification responses
Total time taken (in hours) to respond
to queries
Dev. team
members
Each time a defect clarification request
is created.
Number of queries received per week
Dev. team
members and
DSU
Each time a defect clarification request
is created and received respectively.
PI Goal 2: Reduction in the number of re-opened defects due to delivering the wrong fix
Number of re-opened defect requests
due to delivering the wrong fix Infosys DTS Each time a defect request is re-opened.
Total number of defect fixes
committed per week Infosys DTS
Each time a defect fixing request is
successfully completed.
PI Goal 3: Reduction in the number of re-opened defects due to poor implementation testing
Number of re-opened defect requests
due to poor implementation testing Infosys DTS Each time a defect request is re-opened.
Total number of defect fixes
committed per week Infosys DTS
Each time a defect fixing request is
successfully completed.
PI Goal 4: Improved Defect fixing success rate
Number of defects that are re-assigned Infosys DTS Each time a defect request is re-
assigned.
Total number of defect requests per
week Infosys DTS
Each time a defect fixing request is
obtained from the client.
5.3 Process monitoring and control At the end of each milestone, the corresponding improvement goal changes would have been
implemented and the process owner can then start formally assessing the process measures defined for
each particular improvement goal.
Page 16 of 19
Process review meetings can be scheduled to take place on a weekly basis such as every Friday of the
week. The entire Infosys team is expected to attend the meeting which can also take place via video-
conferencing with the POC in Europe and the Dev. team in India. The process owner should drive the
meeting agenda and present the results of the process improvement activities and their effects on
defect fixing tasks undertaken for the current week.
Based on the process measures defined previously in section 3 for each improvement goal, a set of key
performance metrics or indicators (KPI) can be derived to aid the process owner to quickly gauge the
health of the process improvement activities. These key performance metrics along with their
suggested health levels are described in table 4 below.
Table 4 – KPI process measure
KPI Health level
Optimal Warning Dangerous
Average defect fixing turnaround time (days) <= 1 > 1 and <=3 > 3
Average defect clarification response time (hours) <= 1 > 1 and <=2 > 2
Average defect re-open rate (%) <= 20 > 20 and <=60 > 60
Defect fixing success rate (%) >= 80 > 50 and < 80 <= 50
Note:
Average defect re-open rate = (Defect fixing re-open rate due to delivering the wrong fix + Defect
fixing re-open rate due to poor implementation testing) / 2
Based on table 4 above, process measures that are in the ‘Warning’ zone for more than two weeks and
those that are in the ‘Dangerous’ zone for more than a week should be analyzed and the root causes for
the deviations must be found. Brainstorming and root-cause analysis techniques can be employed to
pinpoint any problems that are preventing the expected performance metrics to be reached. For
instance, if, for two consecutive weeks, the average time taken to reply to a technical query is still
more than an hour by a large margin even though milestone 2 has been reached, all the stakeholders
involved in implementing the improvement goal 1 must provide inputs regarding the process measure
deviation. Any problem discussed should be resolved and a new process benchmark should be started
in the following week.
6.0 Discussion
6.1 Process Improvement Goals Each process improvement goal has been created based on the SMART - Smart, Measurable,
Attainable, Realistic, Time-bound paradigm. As such, each PI Goal contains a precise description of
what process measures need to be collected, a time schedule for its implementation, measurable
outcomes, and their importance to Infosys. The reasons why they were chosen can be explained based
on the benefits that they bring to Infosys.
Page 17 of 19
6.1.1 PI Goal 1: Faster defect clarification responses
Reason: The length of time to reply to queries can cause the resolution of defects to be delayed. This
issue is having a definite impact on the Dev. team’s defect fixing productivity on a daily basis and also
needs to be remedied.
Benefits: If defect clarifications are sent back in a timely manner, the time wasted in waiting for replies
can be minimized which can improve defect fixing productivity.
6.1.2 PI Goal 2: Reduction in the number of re-opened defects due to delivering the wrong
fix
Reason: If defect descriptions are not clear at the very start, the Dev. team may commit the wrong fix
for the defect such that the latter will not fulfill the original client’s defect requirements. As such, the
defect request will be re-opened later, thereby, causing more effort and time spent in fixing the same
defect again.
Benefits: By having a better understanding of what to fix at the very start, the Dev. team can prevent
defects from being re-opened which can lead to effort and time gained from having to fix the same
defects again.
6.1.3 PI Goal 3: Reduction in the number of re-opened defects due to poor implementation
testing
Reason: Owing to this lack of proper verification routines, additional effort and time must be devoted
by the Dev. team to work on fixing any re-opened defect.
Benefits: The effort and time saved on code rewrites for re-opened defects can be utilized in active
defect fixing activities. Better defect testing practices can also be indirectly translated into reduced
client frustration during verification of defect fixes and, thus, improved confidence in the defect fixing
capabilities of Infosys.
6.1.4 PI Goal 4: Improved Defect fixing success rate
Reason: Defects with a high degree of complexity can be assigned to less technically-savvy Dev. team
members and vice-versa. As a consequence, a “complex” defect that would normally have been
resolved in one day or so by a more experienced team member, would currently be resolved in three or
more days by a less experienced Dev. team member as pointed out in the assessment report.
Benefits: A more efficient defect allocation procedure, in terms of defect fixing complexity and
technical experience of team members, means that defects can be resolved more quickly. Moreover,
the number of technical defect clarification requests is expected to be lowered.
6.2 Suggested changes
6.2.1 SG 1:
Having each defect clarification type to be handled individually by either the POC or TAM can cause
the following situations:
Page 18 of 19
Either the POC is overwhelmed with the number of non-technical requests or the TAM is so
with the amount of technical queries that are sent by the Dev. team members
The POC may sometimes consult the TAM for additional advice on how to reply to a
particular defect clarification query and vice-versa.
Therefore, it was a natural conclusion to merge these two functions into one to share the workload of
responding to defect clarification requests and improve upon the quality of the sent replies.
6.2.2 SG 2:
Since it has been established that a growing number of defect re-openings was caused by incorrect
understanding of the bug descriptions, it is clear that such misconceptions need to be resolved before
being read by the Dev. team.
The basic premise of SG 2 is that it will not only save effort required by the Dev. team to understand
the incorrect bug descriptions, but will also prevent the Infosys DSU from receiving too many non-
technical defect clarifications afterwards.
6.2.3 SG 3:
Based on a similar reasoning for pairing up the POC and TAM in handling defect clarification
requests, having both of them to share the workload of the defect verification process will prevent
either one from being overloaded with work and to better coordinate with each other in testing if a
defect has been fixed correctly.
6.2.4 SG 4:
As is often the case in software testing, not knowing what and how to test a particular code function
can leave software errors to be uncorrected. Presently, the Infosys Dev. team did not have any prior
formal knowledge of the specific testing process required to test the client application. The intended
objective of SG 4 is to introduce a testing culture to all team members. In consultation with the client,
knowing and how to test the application with respect to specific types of defects will be easier.
It is also important that the testing process follows the principles of software agile testing so as for the
team to remain flexible and not be stuck in following a rigid testing process.
6.2.5 SG 5
Same as SG 3.
6.2.6 SG 6:
Since the Infosys DSU members will have a better, more thorough understanding of a defect's
complexity based on their own experience in handling defects, the Infosys PL should therefore consult
them so as to make an informed decision on which Dev. team member will be more suited to resolve a
particular defect in the least amount of time.
Moreover, a Dev. team member can also provide information about his technical abilities in solving
defects of a particular type. If he feels that the defect is going to be too complex for him to resolve in
less than the one-day defect turnaround time objective, another capable member can be allocated the
defect.
Page 19 of 19
6.2.7 SG 7:
While SG 7 may seem to have a punitive intent in removing a defect from a team member, its purpose
is to ensure defects are fixed in less than the one-day defect turnaround time objective. The idea
behind it is that either a defect is too hard for a member to fix or the latter is not sure regarding what to
fix. In both scenarios, the most sensible thing to do is to re-allocate the defect to another bug for a
fresh perspective on how to resolve it. However, a defect is never forcefully removed from a team
member as the latter is allowed to justify the need for extra time to fix the defect. SG 7 is, thus, quite
comprehensive in making sure defects are fixed by the right person in the right amount of time.
6.3 Risk To understand the business risks associated with each process improvement goal, a risk-opportunity
matrix is proposed to map each PI goal into one of the quadrants of the matrix as shown in figure 4.
As shown in figure 4 above, all of our PI Goals carry a high opportunity with them meaning that if
they were not implemented, there will be a significant value loss to the defect fixing project. However,
the amount of risks vary from one PI Goal to the next. PI Goals 1 and 2 carry the greatest risks since
their success depends on the client’s involvement. PI Goal 1 relies on the client’s quick response to be
able to achieve the one-hour technical defect clarification target imposed in this process improvement
plan. This is, unfortunately, not within Infosys’ control.
PI Goal 2 carries a similar risk because there is always the chance that the client test user wrongly
interprets the expected defect fix outcome with what he has written as the defect fix scenario. In this
case, the Infosys’ team will be applying the wrong fix for the defect. Both PI Goals 3 and 4 carry a low
risk because they are not dependent on external factors and their implementation phases are rather
short. Despite carrying a high risk, both PI Goals 1 and 2 are important in the overall success of the
process improvement initiative.
Opportunity (Low to High)
PI Goal 1
PI Goal 2
PI Goal 3
PI Goal 4
Risk (Low
to High)
Figure 4 – Risk-opportunity matrix
