CS4723 Software

Validation and Quality Assurance

Lecture 9Bug Report Management

2

Issue Tracking System

After software testing Testers find a lot of bugs They need to tell the developers about the bugs They usually put them into a so-called issue

tracking system

Compared with Mailing list Hard to manage, come with all other mails Not categorized by topic No features for describing specific aspects in SE

(versions, components, commits, etc)

3

Issue Tracking System

A platform for developers to communicate with each other

Like a forum

People can register and raise a issue

The one who raise the issue will describe the issue in details

Others can comment

4

Issue Tracking System

What is special

More structured for describing issues Component, assignees, schedules, status,

resolution Customizable Change the contents while progress is made

(status, resolution)

Sometimes allow anonymous issue raising Users of software are involved They use the software and raise bugs

5

Issue: an example

6

Issues – Life Cycle

Submission

Triaging

Inspecting

Closed

Fixing

Verifying

Reporter

Triager

Developer

Developer

Duplicate/Invalid

Not reproducible

Retriage

Refix

Assigned

Fixed

Re-open

7

Issues

Type of issues Process of issues Resolution of issues

8

Type of issues

Bug reports Feature Requests Patches

9

Type of Issues

Bug report (e.g., system shows wrong message) About a bug of the software

Raised by a user/developer

Should include: Step to reproduce

Expected behavior

Actual behavior

Stack trace or error message if any

10

Example of a bug reporthttps://bugzilla.mozilla.org/show_bug.cgi?id=876495

11

Type of Issues

New feature (e.g., add sorting to results) About add a new feature, e.g., add sorting by

modify time

Raised by a user/developer

12

Example of a feature requesthttps://bugzilla.mozilla.org/show_bug.cgi?id=382491

13

Type of Issues

Patch (e.g., add checking for input validity) A fix to the software

By a professional user or a developer on a important fix

Should include: Version to patch

Patch code: basically a code diff, Diff (buggy, correct)

14

Example of a patchhttps://bugzilla.mozilla.org/show_bug.cgi?id=657763

15

Process of an issue

Open/New The issue is raised

Nobody in the project has looked at it yet

Assigned A person called triager assign issue to a developer

Bug report: the developer will first reproduce the bug, and then try to fix

Feature request: discuss with colleagues on whether to accommodate the request, and implement the feature

Patch: Validate the patch Milestone: may be assigned to a sub-group

16

Process of an issue

Closed When the decision on the issue is made in any way

Fixed, usually accompany with code commits Reject Later

Reopened After the issues is closed, something happens and the

issue become active again Incomplete fix

Start to implement a postponed feature Or revoke any wrong decision before

17

Resolution of an issue

Fixed A bug is fixed

A feature is added

A patch is applied

Invalid Bug cannot be reproduce

Feature does not make sense (request is not understandable)

Patch is not correct

18

Example of invalid bug reportshttps://bugzilla.mozilla.org/show_bug.cgi?id=198057

19

Resolution of an issue

Duplicate The issue is a duplicate of another existing issue Often happens for user raised issues Usually bug report / feature request Some issue tracking system allows merge of

duplicate issues

Won’t fix The developers decide to not fix the bug or

accommodate the new feature Limited human resource, new version is about to

released

20

Example of duplicate bug reportshttps://bugzilla.mozilla.org/show_bug.cgi?id=137325

21

Example of won’t fix bug reportshttps://bugzilla.mozilla.org/show_bug.cgi?id=143329

https://bugzilla.mozilla.org/show_bug.cgi?id=290131

22

Existing Issue Tracking Systems Many project hosting websites provide issue

tracking systems also Google Code: so you will also have a issue tracking

system GIT Hub Source Forge

Issue tracking service provider: Bugzilla

Companies often has their own issue tracking system Users can submit issues, but the tracking system is

not public

23

Tips on submitting an issue report Search for existing reports Specify: product, platform/OS, version Describe

Input / Steps to reproduce Expected results / actual results Bug: Always reproducible or happen randomly

(maybe related to concurrency, system resource, environ, etc.)

Provide Snapshot / bug: error message Bug: Stack trace

24

Submit a bug report Demo on sourceforge, bugzilla, and github

https://bugzilla.mozilla.org/enter_bug.cgi

https://sourceforge.net/p/sweethome3d/bugs/new/

https://github.com

25

What are the most important part in a bug report? Step to reproduce?

Test cases?

Screenshots?

Error Messages?

Stack Traces?

Version?

…

26

A survey did by a number of German Scientists With 130 developers and 215 reporters

Ask about what are the information they provided / used and expect to be useful

Information types include: steps to reproduce, stack traces, test cases,

observed behavior, screenshots, code examples, summary, version,…

27

Answer of the developers Step to reproduce: 83%

Stack trace: 57%

Test cases: 51%

Observed Behavior: 33%

Screenshots: 26%

Expected behavior: 22%

Code examples: 14%

Summary: 13%

Version: 12%

Error report: 12%

28

Mismatch of understanding

29

Tips on triaging and handling issue reports Triaging:

Search for existing reports

Talk with the developers who are familiar with the area

Decide the severity and who should handle the report (according to the expertise and workload)

Handling an issues report (e.g., fix, add feature) Reproduce

For features: communicate on the issue tracking system about whether and how to support it

If reproducible Always report progress as comments in the issue tracking

system: others may help or change their work accordingly

30

Review of issue tracking system

Where the bugs go to after you find them

Type of issues

Resolution of issues

How to write a good bug report?

How bug reports are handled?