Service Team Organizational Chart – 1 Year Ago, <20 FTEs
Now with > 200+ FTEs, Hieararchies and Divisions are Inevitable
Exponential Complexities in Operations Division
Case Study : Service team Promotions Problem
Attribute agreement analysis can be utilized to assess the consistency of QA scoring
Based on the latest excercise of QA team, the value of coefficient is far from satisfying at 0.66
There are three possible causes on why this issue occured that we may test one by one
Let’s deep dive on the first cause: Experience of the team
We tested the relatively more experienced QA members and it’s found that the coefficient remains low at 0.68
We then conducted exercise to two different groups of the most experienced members vs. the least experienced members
Let’s take a look at the second cause: Standardized matrix
Some questions deserve attention to ensure better standardized variables matrix
Let’s take a look at the third cause: Scoring logic
We found the calculation can be improved to make it less bias
Steps to improve QA agreement and assessment
STEP 1: FGD to determine categories & attributes
STEP 2: Determine weighting score for more reliable calculation logic
STEP 3: Make new scoring template
STEP 4: Sampling and Making Detail Guideline
STEP 4: Sampling and Making Detail Guideline
New Guideline is developed with clear scoring description and examples
Other People Operation Projects...
Semangat Gan! Makasih