Tips for Writing Better Bug Reports

Though we might wish otherwise, bugs (or, more generally, defects) are a fact of life in software development. Unfortunately for developers, so are unhelpful and infuriating bug reports.

Bad bug reports waste time and money while everyone goes back and forth to clarify the necessary details. Perhaps worse, bad bug reports tug at developers' sanity and can put strain on the team dynamic.

I know this from experience -- not as a developer, but as a bug reporter who has infuriated plenty of developers in my time as a product/project manager. Because I love my dev colleagues, I've tried hard to improve my bug reporting. Here are some tips I've come up with along the way for writing better bug reports:

• The first step is changing how you think about experiencing bugs, so you can report them in sufficient detail.
• Next, be as specific as possible in providing that detail.
• Finally, always try to reproduce a bug before you report it.

Let's look at each of those tips in depth.

Tip No. 1: Change Your Mental Process

The two most common, but least helpful, bug reports are "X is broken" and "X is not working." These are unhelpful because they don't provide enough detail to understand what the problem is -- "broken" and "not working" could mean any number of things -- and don't explain how the user arrived at this unspecified problem.

This type of bad bug report is unfortunately common because our default mental process when encountering a bug -- or any life experience, for that matter -- is different from the mental process needed to write a good bug report.

The act of experiencing a bug can be broken down into three parts:

• Steps: The specific series of steps or actions that the user took up until encountering the problem.
• Expected result: What the user expected would occur after the step that immediately preceded the problem.
• Actual result: What actually occurred after that final step/action.

Any good bug report should provide details on all three parts. The problem for bug report recipients is that people don't mentally experience bugs in those terms.

Mentally, we elide the multiple discrete steps that make up any experience, whether in the real world or on a computer. We think, "I turned on the car," rather than, "I inserted the key into the ignition, depressed the brake, and turned the key three clicks to start the engine." We say, "I checked my email," rather than, "I typed 'www.gmail.com' into the web address bar; then I entered my username and password, checked 'Stay signed in', and clicked the 'Sign in' button; then I clicked the 'Important' subnav link in the left sidebar."

Describing experiences in the latter way sounds weird at first, like you're being overly literal or robotic. But if you want to write better bug reports, you must train yourself to think in this way. Without all of these discrete, specific details, it's difficult to accurately diagnose and fix a bug.

The steps are important to document because a problem may be different depending on the steps a user took before encountering it, and because the details allow the bug fixer to try to reproduce the problem exactly as the reporter experienced it. For example, a client recently reported that a particular URL was redirecting users to the site homepage when viewed on a mobile device. I tested by entering the URL in mobile browsers' address bars but could not reproduce the redirect. It turned out the client experienced the bug not after entering the URL in the mobile browser address bar, but rather after scanning a QR code. Knowing those details about the steps taken made it easy to figure out that the issue was with the QR code, not with the underlying URL.

The expected result is important to document so the bug fixer can determine whether the bug is in fact occurring as the reporter described and can tell when the bug is fixed. And the actual result is important to document so the bug fixer knows what problem to look for, and because the actual result may be an error message that includes key information for diagnosing the bug.

Tip No. 2: Be Specific

Once you've started to think of a bug encounter in terms of these three distinct parts, the most important thing you can do is be specific when reporting each part.

To illustrate this, consider a sort of non-software bug report that you've probably encountered in some fashion: your grandparents call and tell you, "I turned on the TV and it didn't work."

Here are some possible scenarios this vague report could be describing:

• Grandpa pressed the TV's "on" button and the TV did not turn on. Problem: TV is not plugged in.
• Grandma pressed the TV remote control's "on" button and the TV did not turn on. Problem: the remote's batteries are out of juice.
• Grandma pressed the DVD player remote's "on" button and the TV did not turn on. Problem: Grandma's using the wrong remote.
• Grandpa successfully turned on the TV but the screen is black. Problem: cable box is not turned on.
• Grandpa successfully turned on the TV but there's no sound. Possible problems: TV is on mute; sound system is not plugged in or not turned on; Grandpa is not watching a "talkie" like he thought.

From the original report, it's impossible to tell which scenario your grandparents have encountered. If Grandpa were more specific about the three parts of the "bug" experience, you would have a much easier time diagnosing the problem.

Turning back to software, here are some ways to be specific in bug reports to avoid the grandparents scenario:

• Break down the experience into granular steps
  • Again, you need to retrain your mental process to identify the discrete steps you took before encountering the bug. Be reasonable -- you don't have to start with "I woke up this morning and had coffee" -- but thorough.
• List URLs
  • Because different people refer to pages in different ways -- sometimes by the page's title, sometimes by an idiosyncratic personal nickname for the page -- it's always best to include the URL of any page you refer to in a bug report. It's okay to also include the page name (or whatever you call the page), but always couple that with the page's URL.
• Provide user account details, if any
  • If a site has multiple user roles, a bug may be specific to a particular user role. So if you were logged in when you experienced the bug, provide login details. If you don't want to share your password, at least provide the account email/username.
• Provide screenshots
  • Screenshots are a super-helpful supplement to written details. Sometimes it's extra helpful to annotate the screenshot (or just circle the relevant portion, if it's not obvious) using Skitch or a similar tool.
• Provide error details
  • If the bug included an error message, provide as much of the error text as possible. You can provide a screenshot, cut-and-paste text, or a link to the relevant page on a bug-tracking site, if your site is using Airbrake or a similar tool.
• Provide browser and operating system details

You don't have to provide all of these details for every bug report; browser and OS can be more important to provide for front-end issues, for example, and screenshots aren't always needed if your other details are sufficient. But the more details you routinely provide, the more your developers will thank you.

Tip No. 3: Try to Reproduce

It's also helpful to try to reproduce a bug before you submit a bug report. For one thing, sometimes random glitches just happen -- maybe an update was being deployed right as you were doing something; maybe your browser crapped out on you. Sometimes you may actually be doing it wrong.

When you do come across a persistent bug, reproducing it will help you be specific about the three steps. When you experience a bug the first time, chances are you don't remember exactly what you did leading up to the bug. Trying to reproduce the bug lets you take solid notes on the exact steps and grab screenshots if needed.

Anatomy of a Bug Report

Let's look at a sample (made up) bug report. First, the bad versions: "The Viget.com Extend blog is not working," or "The links on Extend blog posts are broken."

Now here's a more helpful version of these bug reports:

Overview: The headlines on all posts on the Viget.com Extend blog index page appear to be linked to www.viget.com rather than to the individual blog post page.

Tested browser and OS: Chrome 16.0.912.63, Mac OSX

Steps to reproduce:

• Went to www.viget.com/extend
• Clicked the "Preventing Broken Windows" headline
• Expected result: I am taken to the "Preventing Broken Windows" blog post page (www.viget.com/extend/preventing-broken-windows/)
• Actual result: I was taken to www.viget.com

I usually add an overview at the top of my bug reports, to provide some immediate context and a summary of what I think the issue is. Sometimes the overview is a little vague and other times it repeats part of the steps to reproduce, but that's okay as long as the steps are sufficiently detailed. (I tend to leave out the overview if the bug is discrete and obvious.)

If the report includes user roles, I list those details either separately above the steps to reproduce, or as the first step.

Sweat the Details, Not the Format

The exact format of your bug reports isn't important; you can figure out what works best for you and your devs. The important thing is that you start thinking about experiencing bugs in a different way, and that you respect your developers' time enough to try to reproduce a bug and collect sufficient details yourself before sending over the bug report.

Now let's hear your tips for writing better bug reports!