Bugs are part of software. Reporting bugs well is a necessary skill in an ever-growing number of job titles. In this post, I’d like to explain how to write a perfect bug report and offer some bug-reporting best practices.
What do I mean by ‘perfect’? A perfect bug report allows the receiver to start effectively solving the problem without additional conversation. I’ve found that it’s the standard on great teams.
My Bug Report Template
Teams that are great at this tend to have one of these:
- A bug report template that’s pre-populated
- A form or wizard to collect this information before submitting a bug
Here’s a markdown template I’ve used for years:
### Steps to reproduce
How did you get to the bug? What did you do?
### Expected behavior
Tell us what should happen
### Actual behavior
Tell us what happens instead
### System configuration
**Operating System**:
**Browser**:
Template Parts
Let’s look at each part of this template.
Steps to reproduce are vital. What’s the setup, and what actions did you take?
Expected versus actual behavior comes next. The expected behavior is often skipped, yet it’s crucial. If I’m not a consumer of the part of the application you frequent, I might not know what you expect to happen. Your expectation could be wrong, contain insider knowledge to your team, or expose other bugs that you’ve found ways to work around.
System configuration helps me start thinking about what the issue might be. If you’re experiencing zooming in on a form input, I expect that to be happening in Safari mobile. If you aren’t using Safari mobile, that’s useful information.
Attachments like images, screen recordings, and logs are a major plus.
Here’s one of these forms filled out, in the Given-When-Then format:
White Screen on Cart
### Steps to reproduce
Given I am not logged in
And I add a sweatshirt to my cart
When I click on my cart icon in the navbar
### Expected behavior
Then I see my cart with my sweatshirt
### Actual behavior
Then I see a white screen (screenshot attached)
### System configuration
**Operating System**: Windows 10
**Browser**: Chrome Version 102.0.500
Note: Seems to happen more during busy times.
Automating the Template with Bitbucket or GitHub
You can take this template and apply it to your repo, so that every pull request opened by your team uses it as a placeholder in the description.
For Bitbucket, add the template here:
https://bitbucket.org/<your-org>/<your-repo>/admin/pullrequests/default-pull-request-description/
For GitHub, you can add the file to the repo in a few places. A simple one is pull_request_template.md in the root of the project.
Bug Reporting Recommendation: Leave “Severity” Off
Some bug reports have a field to indicate the severity of the bug. Does this affect one person (low severity), a team (medium), or all customers (high)? What is the importance of the bug? I suggest you omit this field from the form.
In my experience, the person creating the report usually doesn’t know the actual severity. How could they? Even if the site appears completely down at your location, without communication, it’s hard to know that that’s true for all users. Which has an impact on the severity.
What happens when the severity is wrong? When it’s overestimated, it can create a sky-is-falling overreaction. When it’s underestimated, it can lead teams to de-prioritize severe issues.
When a severity is found to be inaccurate, we rarely update it, even when that mistake is widely understood. This leads people to ignore it and that can send mixed messages to stakeholders.
If severity is an important metric for your team, I suggest you have the person who first responds to the ticket estimate the severity. They’ll at least know a little bit about the bug and be more likely to get it right.
Bug Reporting Recommendation: Report What You See, Not What You Think
A suggestion I’d offer to anyone opening a bug report: report what you see, not what you think.
A practice that hinders a bugfix is latching onto the wrong solution early in the process. The guidance “I think that the email jobs are failing”, especially from a respected engineer, sends one down a particular road. It may be the wrong road, wasting time and making it hard to generate other solutions. “This customer didn’t receive this email” is better because that is a fact.
If you’re a programmer who’s certain you know what’s going on, jump on a pair programming session with the bug fixer and confirm that you’re right from first principles.
Conclusion
This has been my attempt to demonstrate a perfect bug report. In summary, if you follow a good template, delay estimating severity, and report what you see, you’ll produce a bug report that’s communicative, facts-based, and unburdened by premature assumptions.