Copyright Feature Creep 2008 Metrics Josephine Soap Freda Bloggs Hugh Jarse

Copyright Feature Creep 2008Copyright Feature Creep 2008

MetricsMetrics

Josephine SoapFreda BloggsHugh Jarse

Copyright Feature Creep 2008

Agenda

Development Metrics

Verification Metrics

Quality Metrics

Project Metrics

Field Metrics


Development Work Outstanding

Shows latest forecast for total effort per release and outstanding work per release

Project A & Project B Progress

0

500

1000

1500

2000

2500

04

-Se

p

11

-Se

p

18

-Se

p

25

-Se

p

02

-Oct

09

-Oct

16

-Oct

23

-Oct

30

-Oct

06

-No

v

13

-No

v

20

-No

v

27

-No

v

04

-De

c

11

-De

c

18

-De

c

25

-De

c

01

-Ja

n

08

-Ja

n

15

-Ja

n

22

-Ja

n

29

-Ja

n

05

-Fe

b

12

-Fe

b

19

-Fe

b

Da

ys

Total Work A

Total Work B

Work Left A

Work Left B


Original versus Current EstimatesOriginal vs Current Estimates

0

10

20

30

40

50

60

Wee

ks

Original

Current

Shows which tasks or components were the source of the biggest estimation errors


Monthly Development MetricsProduct X Development Metrics

Trend Metric Target Actual CommentP1 & P2 Pre GA (in QA) 30 n/a still gathering

P1 & P2 Post GA 5 n/a will gather post 5.1.1

Number of cores during PAC testing 0 0 No cores on GA load

Peak TPS/CPU 35 37 using WTP-SAR

Number supported codelines 4 TBDTotal Number Tests n/a 3629Functional spec coverage 100% TBD project for intern

Percentage of Tests Automated 90% 83% 3800

Number Planned Releases 1 1 EA of 5.2 for Huawei

Number Unplanned Releases 1 2Box Colors

Cyan means that the Target has been met

Red means that the Target has been missed

Arrow Colors

Green means that the Target has been met

Red means that the Target has not been met

Means that the value has gone up since last month

Means no change

Means that the value has gone down since last month


Agenda

Development Metrics


Quality Metrics

Project Metrics

Field Metrics


Test Case Covered and Passed

Shows progress on test execution and/or test passing against theoretical S curve


Incident Count

Shows bug trend towards zero release criteria


Defect Cumulative Count

Shows decrease in rate of defect discovery and severity as release date approaches


Feature Test Status

Shows progress on feature test execution and passing in each test run


Agenda

Development Metrics


Quality Metrics

Project Metrics

Field Metrics


Root Cause

Shows phase in lifecycle where defects of each severity arise


Component RCA

Shows components where defects of each severity arise


Bug Catch Net

Shows phase in lifecycle where defects of each severity should have been caught


Agenda

Development Metrics


Quality Metrics

Project Metrics

Field Metrics


Product Resource Profile

Shows allocation of resources across products


Project Resource Profile

Shows allocation of resources across tasks – highlights excess bursts


Days Work Outstanding

Shows forecast number of days to reduce bug count to zero against planned release date


Milestone Trends

Shows movement in project milestones over time


Agenda

Development Metrics


Quality Metrics

Project Metrics

Field Metrics


Deployment DashboardBugs Trends

0

10

20

30

40

50

60

70

80

90

Mar

chApr

ilM

ayJu

ne July

Augus

t

Septe

mbe

r

Octobe

r

Novem

ber

Decem

ber

Janu

ary

Febru

ary

Mar

chApr

ilM

ayJu

ne July

Augus

t

Septe

mbe

r

Octobe

r

Novem

ber

Months

No

. Is

sues

0

5

10

15

20

25

30

35

40

No

. C

ust

om

ers

Live Systems Criticals Total Closed

Closed P1 Closed P2 Customer reported issues backlog

Engineering Assists Total Open P1 Open

P2 Open

Shows number live deployments, open defects and critical accounts


Release Adoption


Maintenance Metrics

Target performance for individual product

Actual performance for this product meets target

Actual performance for this product does not meet target

Key

Declining – performance has deteriorated since previous monthImproving – performance has improved since previous monthNo Change – performance is the same as previous month

Product A Product B Product C Product D Product E57 2 8 5 3379 5 9 11 36

89 10 36 6 78178 10 37 30 90

161 7 21 28 106

202 10 33 30 106

175 10 50 40 105

Declining No Change Improving No Change No Change

75 9 12 1 89

98 8 30 2 85

120 12 30 10 100

Declining Improving Improving No Change DecliningTrend

Trend

Closed issues on Codeline

6-Month Rolling Average

Engineering Assist Backlog


Target (Maximum)

Defect Backlog


Target (Maximum)


Opened issues on Codeline

Metric

Shows performance of maintenance team/activity on each product


Release Frequency

Shows frequency and trend of releases


Customer Loyalty Matrix

Want to Use;Will Continue to Use

PositiveImpressions;May Not Use

Will Continue to Use; Don’t Want to Use

Don’t Want to Use; Probably Won’t

Fix Concerns to Retain or Enhance

Intervene & Re-direct

If Relationship Profitable, Save!

Truly Loyal Accessible

Trapped High Risk

Maintain & Grow Relationship

16%22%

9%6%

27%28%

48%44%

HIGH

LOW

Com

mitm

ent

STRONG WEAKIntent to Continue

Using

Beacon %All %

28%32%

6%5%

30%30%

35%32%

Our Company

53% 6%

24% 17%

Industry Average


Monthly Operational MetricsProduct X Operational Metrics

Trend Metric Target Actual CommentTotal Service Availability 99.99% 100.00% exc planned outages

Best Service Availability 99.99% 100.00% exc planned outages

Worst Service Availability 98.00% 100.00% exc planned outages

Number (unplanned) outages 0 0Duration of longest outage 0 0 minutes

Average Cores Per Day in Beacons 2 6 Customer A

Peak Cores Per Day in a Beacon 4 12 Sprint

Number P1 bugs reported per week 0.3 3 normalised over period

Number P1 bugs fixed per week n/a 10 normalised over period

Current P1 bugs outstanding in 5.x 0 59 51 AQA

Current P2 bugs outstanding in 5.x 20 147 55 AQA

Upgrade satisfaction 4 n/a Avg on scale 1-5

Release Predictability (avg delta days) 5 2.5 3 CE + 1 Dev release

Box Colors

Cyan means that the Target has been met

Red means that the Target has been missed

Arrow Colors

Green means that the Target has been met

Red means that the Target has not been met

Means that the value has gone up since last month

Means no change

Means that the value has gone down since last month


Backlog of Customer issues Customer Reported Backlog Per Priority

0

100

200

300

400

500

600

700

800

Months

Ba

ck

log

BackLog P1 Backlog P2 BackLog P3 BackLog P4

Shows age of open defects – highlights any “falling behind” in maintenance activity


Questions?

Documents

Copyright Feature Creep 2008 Metrics Josephine Soap Freda Bloggs Hugh Jarse