Quality Analysis ReportQuality Assurance Activities Summary

http://code.google.com/p/isec-deis-mis-ds-ps-group1/2008/2009 IPC ISEC-DEIS

Summary

• Trinity

• Quality Roadmap

– Quality Vision

– Quality Actions

– Quality Metrics

• Quality Assurance Activities

• Change Control Activities

• Network Knowledge

• Survey

• Lessons Learned

• Conclusions

3 Master PlanTrinity

Quality VisionQuality Roadmap

Qualitative

Software is develped right the first time as much as possible

Quantitative

User Satisfaction

Conformance to correct requirements

Internal Value

Quality ActionsQuality Roadmap

Walkthrough Activities

Review Activities

Inspection Activities

Templates; Guidelines

Test Campaign

Metrics

Quality MetricsQuality Roadmap

Number of quality activities conducted Number of changes to project schedule and

milestones Number of tests by requirement Number of tests failed and passed Change Request Impact Network Knowledge Lessons Learned Self Evaluation

Quality Assurance ActivitiesQuality Analysis Report

http://code.google.com/p/isec-deis-mis-ds-ps-group1/2008/2009 IPC ISEC-DEIS

QAA CoverageQuality Assurance Activities

Document Quality Activity Notes

RA N.A. Due to its specificity it was included in the global results

PI walkthrough -

SDP review After a review, it was decided by the author to conduce a walkthrough

SDP walkthrough -

PAR N.A. This document will be closed after QAR. So, it was excluded from the metrics collects. Although, it will be under walkthrough

QAP walkthrough Walkthrough, but was conduced before the processes and the document it self are agreed.

QAP review

QAR N.A. It refers to this document. Because this document refers to the metrics it self, it was excluded from statistics and analysis. Although, it will be under walkthrough

CCP review -

CSD walkthrough -

UIP walkthrough -

UR review -

SRS inspection-doc -

SA review -

TCP review -

ITS review -

ATS review -

UIR walkthrough -

ATR walkthrough -

UM walkthrough -

Code1 N.A. This code was under test campaign

Code2 inspection-code -

General StatisticsQuality Assurance Activities

General Statistics #

Total minor defects found in eighteen (18) documents 896

Total major defects found in eighteen (18) documents 12

Average of Nº defects found by document 49.8

Nº of pages under review 306

Nº of defects found by page 2.9

Work time in documents 267 hours

Work time in document reviews 128 hours

Nº of Walkthroughs 8

Nº of Reviews 8

Nº of Inspections 2

Quality Assurance MeetingsQuality Assurance Activities

25%

75%

Cost in hours of QAAQuality Assurance Activities

90%

10%

Total nº defects by doc & nº defects by documentQuality Assurance Activities

193.00

560.00

138.00

5.0024.1370.00

138.00

5.00

walkthrough review inspection-doc inspection-code

Nº Defects by doc & by hour reviewQuality Assurance Activities

24.13

70.00

138.00

5.007.67 12.814.38 0.18

walkthrough review inspection-doc inspection-code

Time spent by page & nº of defects by pageQuality Assurance Activities

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

PI SDP SDP QAP CCP CSD UIP UR SRS SA TCP ITS ATS UIR ATR UM

Nº of effort hours by page & nº of defects by pageWalkthrough Activities

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

PI SDP CSD UIP UIR ATR UM

Nº of effort hours by page & nº of defects by pageReview Activities

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

SDP QAP CCP UR SA TCP ITS ATS

Combined ResultsWalkthrough Activities

0

5

10

15

20

25

30

35

40

PI SDP CSD UIP UIR ATR UM

Author Effort

Aut. Eff by Pag

Avg Effort by Rev

Avg Def by Rev

Avg Def by Pag

Combined ResultsReview Activities

0

10

20

30

40

50

60

70

SDP QAP CCP UR SA TCP ITS ATS

Author Effort

Aut. Eff by Pag

Avg Effort by Rev

Avg Def by Rev

Avg Def by Pag

Global StatisticsCode Inspection

#

Java 5874JSP 2444VB.net 1263Total 9581% insp 3.49%Minor Def 5Major Def 7

Potential Hiden Defects

Minor 142

Major 200

61%26%

13%

Java

JSP

VB.net

3%

97%

Change Control ActivitiesQuality Analysis Report

http://code.google.com/p/isec-deis-mis-ds-ps-group1/2008/2009 IPC ISEC-DEIS

General StatisticsChange Control Activities

Data Start Data End Days Impact Hours Reason

QAP 04/06/2009 29/06/2009 25 5 Environment Changes

UR 27/05/2009 31/05/2009 4 7 Remove Defects

UR 12/06/2009 13/06/2009 1 1 Environment Changes

SA 10/06/2009 12/06/2009 2 23 Remove Defects

SA 04/07/2009 06/07/2009 2 6 Environment Changes

TCP 01/07/2009 02/07/2009 1 0.5 Environment Changes

ITS 30/06/2009 02/07/2009 2 0.5 Environment Changes

General StatisticsChange Control Activities

25

4

1 2 2 1 2

5

0

5

10

15

20

25

30

QAP UR UR SA SA TCP ITS Average

Impact of CR & development timeChange Control Activities

0

5

10

15

20

25

30

35

QAP UR UR SA SA TCP ITS

Templates, Guidelines, RepositoryConfiguration Management

• Templates

• Guidelines

• Checklists

• Convention for filesname

• Google Docs + Codes + Groups

• Other Tools

– MS Project is a complex tool ( Openproject should be an option next years )

– MS Office fulfill its expectations, but OpenOffice3.0 can replace it completely

Network KnowledgeQuality Analysis Report

http://code.google.com/p/isec-deis-mis-ds-ps-group1/2008/2009 IPC ISEC-DEIS

AreasNetwork Knowledge

Area DescriptionPM & RA Project Management and Risk Analysis

A & R & I Architecture, Requirements and Implementation

QA & CC Quality Assurance and Change ControlTC Test Campaign

Forum Network Knowledge

Nº Posts 975

Nº Topics 136

Avg ~7.2Apr110

May332

Jun324

Jul209

PM & RA40

QA & CC48

A & R & I34

TC14

Wiki Pages Network Knowledge

PM & RA1

QA & CC7

A & R & I2

TC1

SurveyQuality Analysis Report

http://code.google.com/p/isec-deis-mis-ds-ps-group1/2008/2009 IPC ISEC-DEIS

Configuration Management &Quality ManagementSurvey

Quality Assurance Questioner Score [0-10]

The QAP was clear and detailed enough. 8.4

The QAP contributed to the successful execution of my tasks. 8.5

The QAP contributed to the successful execution of the project. 8.3

The Google Mailing List (Forum) contributed to share my knowledge with my colleagues. 8.9

The Google Code (SVN & Wiki) was demonstrated being a properly tool to this project. 9.3

The Google Docs (Weakly Report) was demonstrated being an adequate tool to be used in this project. 9.2

The Quality Manager knew his responsibilities. 9.3

Lessons LearnedQuality Analysis Report

http://code.google.com/p/isec-deis-mis-ds-ps-group1/2008/2009 IPC ISEC-DEIS

Summary Lessons Learned

• Start meetings at time

• Communication is important

• Simulate inspections (trainning sessions)

• Motivation, comprehensive and calm

• Right choice of development tools

• Time invested in the project planning

• Testers should follow the SA/DD development

• A software project should have a ratio of 3 developers to 1 tester

• It should exist control over software versions that are available for testing

Final Conclusion, RecomendationsQuality Analysis Report

• The Quality Assurance Activities are conducted with rigor and accuracy. The team members followed the guidelines provided by the three plan documents: QAP, CCP and TCP.

• The number of change request (five (5) out seven (7) - only two correcting defects) shows that mainly the documents are developed right the first time.

• The final product conform its requirements and requirements are fully accomplished.

• The Quality Team presents a several numbers and evidences that proofs the high quality achieved.

• Those Quality Assurance Activities gave visibility to the team members in their weaknesses and confidence on how to overpass them.

• Fix a maximum number of defects to impose a new document review

• Record the unique number of defects found

Thank you !