The Neo Fallacy: How Our Mental Model of Developers Leads to Bad Bug Reports

In two previous posts about writing better bug reports, I've detailed the information developers need to be able to reproduce — and hence diagnose and fix — bugs. Since then, an unstated assumption in both posts has been nagging at me: that developers must be able to reproduce a bug in order to diagnose and fix it.

To a developer, this is not an assumption — it's truth, and obvious. But to non-developers, the necessity of reproducibility is not obvious. If it were, devs wouldn't see so many bug reports that say little more than "A vague thing vaguely happened — fix it!!!"

I think this disconnect is the primary cause of frustrating-to-devs bug reports. If we can better understand the reason for the disconnect, maybe it'll be easier to convince non-devs of the necessity of reproducibility. 

So what causes this disconnect? Here's my theory:

I believe that most non-developers think developers are like Neo in The Matrix: that devs can see all parts of an app simultaneously at all times, like Neo's living-ASCII-art view of the Matrix itself. That devs are omniscient, omnipotent beings in regard to an app. I've come to think of this mental model as the Neo Fallacy.

If pressed, I don't think we could say how we think developers do this. (Optic fiber running from servers to devs' brain stems? Magic emanating from the bowels of 4chan? Google Glass?) But I can't think of a better explanation for why so many of us assume devs can figure out a bug from so little information.

Returning to theory No. 1, I asked some Viget developers why reproducibility is so important. Some of their answers:

• "So we can hopefully write a failing test first, and then have one in place to prevent future regressions"
• "To confirm it's an actual bug"
• "To know when we've fixed it"
• "To be able to investigate the issue deeply"
• "So we don't waste our super valuable time"

All make sense. But those answers are beside the point to someone operating under the Neo Fallacy: "Fine, but why do you need to reproduce the bug to achieve all those things? Can't you just 'see' what's happening?"

Here's an answer that I hope helps reframe the way we think about developers:

Developers are not Neo. Yes, they have superpowers when it comes to writing code. But they're mere mortals when it comes to knowing what's happening in an app in real time. They're not omniscient.

Occasionally, they may be able to quickly ascertain a problem based on an intuitive sense of an app's behavior and an auto-generated error report from Airbrake. But most of the time, a context-free, vague bug report will mean no more to a developer than it would to you. Literally: without the details I outlined in my other posts — URLs, login creds, specific steps — you might as well submit the bug report to yourself. 

And having to spend time trying to glean meaning from such a vague report? For devs, that's about as fun as hanging out with these guys.