C O N F I D E N T I A L Apr 22, 2023 1

StarCite Engineering Weekly Meeting

StarCite Engineering

Feb 9, 2009

C O N F I D E N T I A L Apr 22, 2023 2

Agenda

• Quarterly Engineering top priorities

• Shanghai Engineering team weekly update – 5 minutes– Releases schedule overview– Unit Test weekly progress chart– Rally Iteration Weekly status– Engineering Operation Metrics– QA metrics

• Special Topics – 10 minutes – Complete Agile progress in SM – Leslie– Code review process - Justin

• Q&A

C O N F I D E N T I A L Apr 22, 2023 3

Engineering Q1/09 Top 3 Priorities (aligned)

Priority Description Business Benefit Projects Involved

1Performance, Stability and Quality

-Customer Satisfaction and Adoption

-Improved performance under increased load from grow growth and migration

-Improved quality with increased usage from growth and migration

-Reduce long term operation cost

-SM

-MarketView

-CBS – Search/UAS

-Mpoint.com

-Reporting

2 Migration-Customer retention

-Reduced complexity

-Strategic advantage to single suite

-Spend Mgmt

-Attendee Mgmt

-MarketView

-Reporting

3 New Revenue

- By new features or new projects

-By enhanced features

-New product / services

-Attendee Mgmt (Event Marketing)

-Reporting (Buyer, Supplier)

-SM

-StarCite Express

-MKV

-More …

Absolutely has to happen – put best effort and team to meet the metrics and dates

C O N F I D E N T I A L Apr 22, 2023 4

Performance, Stability and Quality – Why, What and How

Why? • To improve internal quality to enhance both the functional and non-

functional quality of our products to satisfy customers• To gain business benefits

What to do?1. Effective code review including design review2. Unit test coverage and effectiveness 3. SQL profiling4. Code re-factoring

How?• To re-enforce code review, unit test and SQL profiling; to make them as

part of the development work• To provide guideline and good examples, to build good practice, • To be measurable; to know where we are today; to set near- and long-

term target; to track progress;

C O N F I D E N T I A L Apr 22, 2023 5

Release Schedule in Q1/2009Project Release Date Version

SM Feb 18, 2009

March 29, 2009

12.15.002

12.16

Reporting Feb 20, 2009 1.7

G2 AM Feb 17, 2009

Feb 24, 2009

March 28, 2009

14.6.2

14.6.2 hotfix

14.6.4

MKV Feb 25, 2009

March 25, 2009

MKV86

MKV87

Mpoint.com March 9, 2009 (Target date) Mpoint.com 1.0.0.1

CBS – Search Feb 2, 2009 Search1.0.0.1

C O N F I D E N T I A L Apr 22, 2023 6

Unit Test ObjectivesPriority Description Business Benefit

1 Quality improvementImproved quality level for all products. Reduction in regressions and customer issues

2 Predictable, aggressive, re-factoring

Reduces risk and potential impact in aggressive performance and maintainability improvements

3 Overhead reduction for development and testing

Shorter release cycles, more stories per sprint

4 Fail fast developmentFailures found sooner in development cycle. Allows faster course correction and more predictability in product delivery

C O N F I D E N T I A L Apr 22, 2023 7

Current Unit Test Coverage Numbers – by last Friday

Project Classes Line Coverage

Branch Coverage

Complexity

Mpoint.com 340 87% 90%MarketView 613 49% 54% 1.709

Spend Management

2144 16% 11%

CBS – UAS* 0* 0* (92%) 0* (81%)CBS - Search* 105 75% 65%

* Newly started project, has no unit test coverage yet

C O N F I D E N T I A L Apr 22, 2023 8

QA Automation – Accomplished in 2008

Product CategoryQ4 2008

QTP QTP QTP %

SM

smoke 0 98 91.60%

SM UAT 78 413 64.10%

Maintenance 40 149 99.30%

         

MKV

smoke 0 267 100.00%

UAT - RFP WF 0 1583 79.70%

UAT - non RFP WF 252 755 56.40%

Maintenance 588 588 100.00%

         

G2AM

smoke 0 29 100.00%

UAT 46 46 43.40%

Configuration Test 0 9 45.00%

         

ReportingSmoke 0 18 100.00%

UAT 73 245 29.80%

C O N F I D E N T I A L Apr 22, 2023 9

QA Automation – Q1 2009 Target

Product CategoryQ1 2009

QTPSM UAT 77%

MKV

UAT RFP WF 80%UAT - non RFP WF 80%

G2 AM

UAT 100%Configuration Test 100%

Reporting

smoke 100%UAT 58%

100% Smoke test script get updated

C O N F I D E N T I A L Apr 22, 2023 10

Engineering Operation metrics

*3 customer discovered S0 – due to server out of memory.

Total Open/Closed Level1 (Last 7 days)

C O N F I D E N T I A L Apr 22, 2023 11

Engineering Operation metrics (cont.)

Level1 Status and Assignment (Last 7 days)

C O N F I D E N T I A L Apr 22, 2023 12

QA metrics – Q4 2008 release reviewDefect Resolution Efficiency - SM

0%20%40%60%80%

100%

SM release

Reso

lutio

n %

Defect Resolution Efficiency - MKV

0%20%40%60%80%

100%

MKV(831

)

MKV(832

)

MKV(833

)

MKV(84)

MKV(840

1)

MKV(840

2)

MKV(841

)

MKV(842

)

MKV(842

1)

MKV(85)

MKV release

Def

ect R

esol

utio

n %

C O N F I D E N T I A L Apr 22, 2023 13

QA metrics – Q4 2008 release reviewTesting Efficiency - SM

0%20%40%60%80%

100%

MV (12.1

2.1)

MV (12.1

3)

MV (12.13

.1)

MV (12.1

3.2)

MV (12.1

4)

MV (12.14

.1)

MV (12.1

5)

MV (12.1

5.1)

SM release

% o

f def

ects

in Q

A c

ycle

ov

er to

tal d

efec

ts

Testing Efficiency - MKV

0%20%40%60%80%

100%120%

MKV release

% o

f def

ects

of Q

A c

ycle

ov

er to

tal d

efec

ts

C O N F I D E N T I A L Apr 22, 2023 14

QA metrics – Q4 2008 release reviewRegression bug ratio - SM

0%10%20%30%40%50%60%

MV(12.12.1)

MV (12.13) MV(12.13.1)

SM release

% o

f reg

ress

ion

over

tota

l de

fect

s

Regression bug Ratio - MKV

0%10%20%30%40%50%60%70%

MKV(831)

MKV(832)

MKV(833)

MKV(84)

MKV(8401)

MKV release

% o

f reg

ress

ion

over

tota

l de

fect

s

C O N F I D E N T I A L Apr 22, 2023 15

QA metrics – Q4 2008 release review

Defects/Test case Ratio - MKV

0%2%4%6%8%

10%12%14%

MKV(831

)

MKV(832

)

MKV(833

)

MKV(84)

MKV(840

1)

MKV(840

2)

MKV(841

)

MKV(842

)

MKV(842

1)

MKV(85)

MKV release

Def

ect p

er T

C

C O N F I D E N T I A L Apr 22, 2023 16

Special Topics – Complete Agile

SM.12.15.2 – sprint 1

SM.12.15.2 – sprint 2

Big improvement is evident by comparing the iteration burn down charts

-- SM12.15.2 – sprint 2 shows healthier burn down chart than the SM12.15.2 – sprint 1

-- SM12.15.2 – sprint 2 accomplished much more story points by last day of

the iteration

C O N F I D E N T I A L Apr 22, 2023 17

Special Topics – Code Review processCross Code Review

Purpose: promote a culture “team excellent” of 2009 goals from Yang, and an attitude “I DO welcome anyone reviewing my codes and it can help me grow faster“.

Focus: most important technique hotspots:1. Out-of-memory issue identified by operation team2. Unexpected server restart identified by operation team3. Very long DB operations identified by SQL profiling log and app Log4. Legacy codes need re-factor and the new codes5. Other important issues identified by Team

Peer Code ReviewPurpose: improve code quality by regular code review activities  

Focus: 1. All new codes have to be reviewed and the activities must be tracked on

Rally. 2. Legacy codes are encouraged to have more code reviews and related

action plans

C O N F I D E N T I A L Apr 22, 2023 18

Special Topics – Code Review process

Encourage Plan

1. Performance bonus: code review efficiency (including frequency and quality) will be treated as one very important performance metric of team member.

2. Recognition: we will recognize the best engineers for the contribution on code review activities and the detailed plan will be decided later.

3. Leaning Credits: the engineers with good contribution on code review activities will get extra leaning credits to get encouraged.  

Introduce a web based tool for code review:

Review Board which developed by Vmvare(http://www.review-board.org/)

Evaluation is underway. If anyone is interested, please come forward and talk to Justin

C O N F I D E N T I A L Apr 22, 2023 19

Mr. Q&A