QA EVALUATION

TOOL- A KPIs based evaluation

- An Excel/Spreadsheets based tool

Overview

Problems of QA Evaluation

New Evaluation Pros vs. Cons

Criteria and Calculation

Using the tool

Questions and Answer

Problems of QA Evaluation

The evaluation criteria are unspecific.

QA Managers’ evaluations are based on

emotion.

QA Managers evaluate in short term instead of

the whole year of good performance.

Testers complain that their evaluations are

not clear and unfair.

New Evaluation Pros vs. Cons

Pros

Automated KPIs tool.

Emotionless evaluation.

Easy to follow-up.

Easy to compare the real

performance.

Fair for every tester, no

more demotivation.

Easy to double check.

Cons

More “paper work” for QA Managers.

Must follow and update regularly.

Must organize massive contests

Evaluate up to QA Senior.

Criteria and

Calculation

Late at Work – 5%

QA Tester is paid by real working hours.

If a tester go to work late: 0%

If a tester never go to work late: 5%

“Late at Work” definition:

Present in the office after 8:10 AM without notice.

Take day/half-day off without notice or

explanation.

Support Others – 5%

Based on 360 degree Feedback.

The final score will be the average of all Feedbacks.

Formula:

Support Others 360 Feedback Average x 5%

Working Attitude – 15%

Based on 360 degree Feedback.

The final score will be the average of all Feedbacks.

Formula:

Working Attitude 360 Feedback Average x 15%

Need Review Rate – 10%

Apply for QADB Junior only.

For QADB Senior, use Team’s Bug Quality Average instead.

Need Review formula:

[100% - (No. of Need Review/No. of Total Bug x 100%)] x 10%

Team’s Bug Quality formula:

Team’s Bug Quality Average x 10%

Personal Bug Quality – 15%

Based on the result of Bug Scanning Contest.

This is a cross teams contest.

The contest must be arranged each 02 months.

Formula:

Bug Scanning Contest Result x 15%

Personal Bug Quantity – 15%

This is a two-step calculation because:

Each team has its own project/game and number of bugs.

When tester takes a day off, another one has to support and do his/her task.

Duplicated bugs are not counted.

Testing Time = Normal Working Time + Overtime Working Time

Bug Per Hour formula:

Tester’s Total Bug Caught/Team’s Testing Time Average

Personal Bug Quantity formula:

Tester’s Bug Rate/Team’s Highest Bug Rate x 15%

Checklist Contest – 20%

Based on the result of Android Checklist Contest.

This is a cross teams contest.

The contest must be arranged each 02 months.

Formula:

Android Checklist Contest Result x 20%

Report Skill – 5%

Give the score base on the ranges (Good, Above

Average, Average, Below Average, Bad)

Using 360 Degree Feedback method.

QA Supervisor/Lead evaluate the reports of each

tester.

Formula:

Report Score x 5%

Workshop Quantity – 5%

Workshop must be validated by Trainer Team.

Encourage sharing knowledge/experience.

Formula:

𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐞𝐫′𝐬 𝐍𝐨. 𝐨𝐟 𝐖𝐨𝐫𝐤𝐬𝐡𝐨𝐩𝐬

𝐓𝐞𝐚𝐦′𝐬 𝐇𝐢𝐠𝐡𝐞𝐬𝐭 𝐍𝐨. 𝐨𝐟 𝐖𝐨𝐫𝐤𝐬𝐡𝐨𝐩𝐬× 𝟓%

Workshop Quality – 5%

Trainer must join the workshop and give the Presenter a

score for:

Presentation skills

Quality of the workshop

If Presenter organize more than 01 workshop, the final

score will be average of his scores.

Formula:

Presenter’s Workshop Scores Average x 5%

Management Skills – 5%

This score is a bonus based on 360 Degree Feedback.

Only the main key get this bonus.

QA Supervisor/Lead must register the name of the main

key at the beginning of Evaluation term.

Formula:

Management Skill 360 Feedback Average x 5%

Day Off & Comments

Day Off is not counted in Total Score of a tester.

Comments is not counted in Total Score but it’s

mandatory.

Comments should be in compact form.

QA Supervisor/Lead can collect comments from team

members (via 360 Feedback emails).

Using The Tool

Enter Necessary Information

Teamlist Sheet:

1. Replace <YourName> part with your real name.

2. Change the Start and Finish time of Evaluation

period.

3. Enter the correct information of each tester.

Position means the real position of the tester.

QADB Position means the position of the tester

on QADB

Enter Necessary Information

Input Sheet:

Enter necessary information follow instruction.

Do not change the format of data.

Remember that blank and 0 are very different.

360 Degree Feedbacks

1. Feedback request should be given from QA

Supervisor/Lead to tester.

2. Each tester can feedback about all members in his

team (include himself).

3. Tester will give feedback follow the template.

4. QA Supervisor/Lead will sum-up and use the result for

the evaluation.

Rank AverageGood 81 - 100

Above Average 61 - 80

Average 41 - 60

Below Average 21 - 40

Bad 0 - 20

360 Degree Feedbacks

360 Feedback Template:

Use “Team 360 Feedback” tool to sum-up.

360 Degree Feedbacks

QA Supervisor/Lead can have his/her own evaluation for

each tester.

The evaluation of QA Supervisor/Lead must be another

360 Feedback email or in excel file.

QA Supervisor/Lead point of view is treated equally with

tester’s.

The Evaluation Tool and all 360 Feedback emails

(compressed in a zip file) must be send to all QAPMs as

attachment.

Understand The Result

The results of each 02 months are automatically

calculated in Monthly-Points sheet.

Final results of Evaluation Term (06 months) are calculated

in SUM-UP sheet.

Total Sum-up Chart compare all the testers in team.

To see how a tester improves his/her performance though

Evaluation Term, enter the name of the tester into cell

C58.

Questions &

Answers