BenchmarkQA Software Quality Forum on Retrospectives, March 2011

Presented by:Betty Schaar, CSQA, CSMBenchmarkQA

Date:March 03, 2011

BenchmarkQA helps project teams deliver higher quality software through:

QA IS ALL WE DO!

Quality Assurance Consulting

Contract & Permanent Staffing

QA/Test Training & Seminars

Local Outsourcing Test Lab

© 2011 BenchmarkQA

Agenda• Introductions

• Retrospective Defined

• Planning for a Retrospective

• At the Retrospective

• After the Retrospective

• Lessons Learned

• Let’s Try One!

• Q & A

• Closing Remarks

3/3/2011 Slide 3© 2011 BenchmarkQA

Top 10 Fashion Trends We Wish Had a Retrospective (So They Won’t Be Repeated)


3/3/2011 © 2011 BenchmarkQA Slide 5

Retrospective Defined

Retrospective Defined

retrospective (rèt re-spèk-tîv) a ritual held at the end of a project to learn from the experience and to plan changes for the next effort.

• “Inspect and Adapt” opportunity for process and team

• “What went well?”

• “What could be better?”

• “What do we want to change?”


Retrospective Defined cont’d

• Attendees and Roles“No one knows the whole story of the project. Each person has a piece of the story. The retrospective ritual is the collective telling of the story and mining the experience for wisdom.” - -Norman Kerth

• Include everyone who has a piece of the story to tell; all disciplines on the project should be represented

• Make sure there is a facilitator, preferably someone who was not involved with the project, to lead the retrospective

• A designated scribe is helpful



• When to hold a retrospective?– At periodic intervals such as at the end of every delivery cycle

– When a significant change or disruption to the project occurs

– At the conclusion of the project



• Why have a retrospective?“It’s something that every true learning organization has as part of its culture, and it’s one of the best ways to grow—project by project—into a smarter and increasingly successful organization.” - -Norman Kerth

• To consider ways to improve overall performance• To address ongoing obstacles to productivity• For continuous improvement

Also…• To acknowledge and CELEBRATE successes


Planning For A Retrospective

3/3/2011 © 2011 BenchmarkQA, Inc. Slide 10

Planning For A Retrospective

• Invite right players; make sure adequate representation

• Decide who will facilitate • Best if someone outside the team so entire team can participate• If from within the team, rotate so not always the same person• Want someone with good facilitation skills

• Facilitator determines questions to be asked and if will solicit input/feedback at the session or ahead of time

– If ahead of time, decide input mechanism–electronic or hard copy


Planning For A Retrospective cont’d

• Determine how much time for the session

• Collect defect info for top 1 or 2 classes of defects• For review and correction of assigned root cause• Team to consider if root cause assignments indicate trends to

address

• Organize supplies needed – white board, large paper, sticky notes, markers, tape, treats!


At The Retrospective


At The Retrospective

• Review the objective; suggest:– Reflect on last delivery cycle to identify what went well, what didn’t

go so well in order to determine what can be done in future delivery cycles to improve on project execution and teamwork.

• Set the stage– “Regardless of what we discover, we understand and truly believe that

everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.”- - The Prime Directive (Norman Kerth)


At The Retrospective cont’d

• We know more at the end than when we started• This is wisdom to be celebrated!

• Session rules: • Respectful communications, single speaker, all opinions count

• Esther Derby and Diana Larsen approach:


Focus On Focus Off

Inquiry Rather Than Advocacy

Dialogue Rather Than Debate

Conversation Rather Than Argument

Understanding Rather Than Defending


• Review top defects for proper root cause classification and for problematic process trends to address

• Identify and group into categories what went well and what could be done better– Or review pre-submitted items grouped by facilitator

• Vote or rate; approaches:• 1 to 3 or 1 to 5 rating – item of highest significance gets highest

value• 3 voting dots – each member can place their 3 votes on different

items or all on one item3/3/2011 Slide 16© 2011 BenchmarkQA


• For the top vote getters or highest rated items (recommend no more than top 2 or 3): – Decide who will “own” the top item(s)– Team identifies possible root causes; use 5 why’s– Team brainstorms ideas to experiment with that they think will

improve the issue– Team selects one or two ideas to try during next delivery cycle– Define action steps for the chosen solutions


After The Retrospective


After The Retrospective

• Keep the results of the retrospective in a prominent spot for the team

• Plan the selected action steps into the next delivery iteration along with product work

• Follow up on the experimented actions at the next retrospective to see if had desired affect

• Use the previous retrospective results as a starting point for the next retrospective process


Lessons Learned


Contributors

• Sheila Conway

• Denise Fitzgerald

• Pat Stearns

• Nirmal Chaudhari

• Mark Nelson

• Susan Chapman

• Steve Anderson

• Warren McLeod

• Aimée Willoz

• Shankar Jayavelu

• Santosh George

• Betty Schaar

3/3/2011 © 2011 BenchmarkQA Slide 21

Lessons Learned: General Thoughts

• Try something different to achieve a different result.

• A retrospective allows aggrieved team members to let off steam and not carry it over to the next round of work.

• It also helps make the case for new ideas with past experiences to justify it rather than have someone from high above bringing about a change for the sake of change.

• I have seen too many retrospectives talk about what went well and what did not go well; but never end up with implementable process improvements that individuals have responsibility for.


Lessons Learned: General Thoughts

• Problems observed during initial retrospectives:– 1. No one talks, big silence.– 2. Sometimes too much discussion on one issue.– 3. QA folks are very nervous to talk about the issues.– 4. Shutting down another person's views before coming to conclusion.– 5. Bringing up the same issue again and again.– 6. Very short retrospective.

• Retrospectives can be very political where no one wants to be really honest because they don’t want to single any one person out regarding their bad behavior, code quality, etc. So, a lot can go unsaid yet needs to be brought out somehow by the facilitator.


Lessons Learned – Planning

• It can take time to get the thoughts out of everyone in the group. • Solicit the input ahead of the group session; this can also help with

anonymity.

• Retrospectives held too far after the work period are less effective; people forget.• Hold the Retrospective no more than 1 week after the delivery cycle

• Holding two levels of retrospective helps. First on sprint level and second after release/phase.

• If a lot of issues occurred between two teams, split up their retrospectives.

• Retrospectives have been a formal 2 hour meeting with minutes.



• It really helps to have an agenda and rules of conduct.

• Make sure ground rules are set beforehand. No personal attacks, attack the process, etc.

• If people think they can’t speak to the group, send in the comments to the Project Manager ahead of time.

• If people cannot attend the meeting at all, ask for their comments beforehand and share in the meeting.

• I have done surveys as well but I don’t think they work as well. It’s more difficult to get peoples’ perspectives and it’s also harder to get responses.



• Ask each team member to bring one thing that went particularly well and one thing that went poorly.

• Often only the recent pain or joy is remembered or only the “Big Stuff” gets remembered.

– Have a Project Notebook (lessons learned journal) for every member of the team from the beginning, and capture the good, bad, and ugly along the way.

– Note the resolution for any challenges encountered. – It can be a great way to track what the team does well, even though

they may not realize it. It can be very enlightening to the rest of the team when sharing individual entries.


Lessons Learned – At/During

• Items might come out that the group feels are important to their success, but not in their control to resolve. • Still capture these, someone should own, but the group should also take

on one or more other items that are in their control.

• Without an effort to make process changes, the results of the post-

mortems just don’t go anywhere.• An action plan coming out of the effort can help to actually get some of

the suggestions implemented.

• Bring in a moderator who knows nothing about the project. – Having a neutral moderator helps keep things on track and not get

steered in a particular direction.



• Bring in a scribe not involved in the project. – If the scribe knows nothing of the project, they can focus on taking

notes.

• Have food.

• It can be nice just to look back and appreciate the amount of effort that went into a project even if you do not get to fix anything that gets complained about.

• A lot of the issues were discussed in hallway conversations during the project. As a result, the effort can just turn into a complaint session with little or no effort to fixing the problems.



• Put the one thing that went particularly well for the Sprint and one thing that went poorly from each team member on a white board. Get consensus from the team on two of the most critical things that went well and two of the most critical items that went poorly.

• Create a standard checklist or questionnaire which helps team to stick to the point during retrospective. (e.g. requirements/user stories, communications, development, unit test, database, QA testing, data preparation, inter team dependencies, test environment etc. – focused topics).

• Ask team members individually if feedback is not received.



• Make sure the questions are open ended.

• To get people to open up, bring up specific things that happened during the process. Specific meetings, documents due, process of getting through the phase, etc.

• Any effort to include metrics is helpful as they provide a somewhat objective perspective on a given aspect of the project. Although, they can also be political if bug rates, etc. are brought up.



• Review defects and root cause and use that for opportunities to improve the process that results in defects.

• I've seen fishbone diagrams and mind maps used to organize the input, which seems to work well.

• Capture action steps to get the most value out of the session.

• Action plans should be created for the top items to improve those that went poorly and standardize the items that went well.


Lessons Learned – After

• Find a way to keep the team accountable for trying changes. Checkpoints mid-release to see if on track with the changes we said we would make/do.

• Without an effort to make process changes, the results of the post-mortems just don’t go anywhere. An action plan coming out of the effort can help to actually get some of the suggestions implemented.

• The most critical aspect of doing a retrospective or postmortem is having management motivated to actually “do” something with the data that's gathered (otherwise it's a waste of effort to gather it).


Lessons Learned – After

• Compare retrospective notes from different teams (if multiple teams) and share that information.

• Compare current retrospective to previous retrospective to understand problems and improvements.

• Always share notes immediately after retrospective. Do not wait for another day.

• Document all findings, send them out to the team for review.

• Action plans should be assigned as tasks to individuals for the following Sprint just like all the other tasks required to generate software Sprint deliverables.


Let’s Try One!


A Retrospective About Retrospectives

• The subject of our group retrospective is retrospectives

• We’ll use collected “Lessons Learned” for this exercise

• Each attendee gets 3 votes to place on any of the items on the wall – place your votes on 1-3 items

• We’ll take the top vote-getter and work out some possible root causes, possible solutions, action steps


References

Project Retrospectives: A Handbook for Team ReviewsNorman L. Kerth Dorset House Publishing ISBN 978-0-932633-44-6 2001

Agile Retrospectives: Making Good Teams GreatEsther Derby, Diana Larsen Pragmatic Bookshelf ISBN 978-0977616640 2006

Agile Testing – A Practical Guide For Testers and Agile TeamsLisa Crispin, Janet Gregory Addison Wesley ISBN 978-0-321-53446-0 2009

http://www.scrumalliance.org/

Slide 36© 2011 BenchmarkQA

Questions?


In Closing…


Thank You!

For additional information on how BenchmarkQA can be of assistance, please contact:

Molly Doyle Decklever952.392.2384

[email protected]

© 2011 BenchmarkQA

Technology

BenchmarkQA Software Quality Forum on Retrospectives, March 2011