We've all been there, a new bug pops up, it's the highest priority and needs to be fixed yesterday... along with the 3 other highest priority bugs already on the list!
There's a saying we often repeat here:
if everything is important, then nothing is important
The difficulty in correctly prioritising bugs lies in the challenge of actually being able to quantify the detriment caused by that bug relative to other bugs.
As developers, bug prioritisation is an important part of the development process because quite often, it's a developer who will be the one to identify the issue's likelihood of occurrence as they would be the ones to understand or investigate the cause, and advise the PMs & BAs about it.
Priority
Assigning a priority seems like it should be easy enough thing to do - a priority tells us how important it is to resolve. Or to look at it another way, how detrimentally impactful the issue is to the application.
Explaining that importance can in itself be difficult, so a scale is needed. There's so many different ways to try and describe or label them. Some common scales used are numeric - P1 for the highest priority, through to P4 for lowest. Another choice is the simple word labels; critical, high, medium & low.
We like to use the word labels, so for this article we'll assume that as the scale.
Often however, you'll find a scale doesn't do the job on its own, because then you have another problem - if you have 2 different bugs, both of which impact your software's ability to function, and you class them as critical priority, how do you determine which of them is more critical than the other?
We quickly fall into a sort of Buridan's donkey type situation.
That's often why we start to use the companion scale of severity, alongside priority.
Severity
Severity's job is to try and quantify the bug's detrimental impact, rather than how important it is for it to be resolved. A subtle but important difference.
So where you have a critical priority bug of high severity and another bug of critical priority but low severity, it tells us that while it is just as important to resolve the latter quickly, we need to address the former first, because the impact it is having to the application is much worse.
To contextualise this, imagine our critical priority, severe bug is causing the application to crash and blank screen, while our critical priority, low severity bug is a missing button to continue the purchase user journey.
But now, what happens with a critical priority, low severity bug vs a high priority, high severity bug? 🤔 Yep, starting to feel like that donkey again right?
Objective Criteria
Evidently, the problem with prioritising is that priorities are often assigned by the intuition and perspective of the one raising/logging it.
To make priorities meaningful we need to assign them objectively.
By coming up with a set of criteria for what constitutes each priority in our scales, we can more confidently and objectively choose our priorities.
To do this, we suggest the following qualifiers as a minimum:
Level Of Functionality Loss
Making a distinction between different levels of functionality in your site or application makes it easier to identifier issues that are more damaging.
We like to use 3 categories of functionality:
Core functionality
The basic functionality that your site or app exists to performs, and without which it might as well be offline.
Auxiliary functionality
These are the features which support, extend or enhance the core functionality and contributes to the success of the site or app.
UX/aesthetic functionality
Features which deal with user interface or content in a way that does not impact the core or auxiliary functionality of the app in any way - such as layout, colours of buttons, buttons that perhaps are the wrong shape or style, or missing confirmation/information states etc.
Frequency Of Occurrence
Bugs tend to occur under certain circumstances, which means once discovered they can be predicted and replicated. Once we know how, we can estimate how often the issue is likely to be seen on the production site/app. It stands to reason that the higher the likelihood of a bug being seen by users, the higher it would be in priority.
Recoverability
Sometimes bugs cause complete failures that make the site or app become completely unusable (often referred to as fatal errors), in which case the only way to return to any semblance of a working app is to reload it. (But the bug will still occur under the same circumstances).
Other times, bugs may completely prevent usage of only a particular part of or function of the app.
These types of bugs are unrecoverable. Which means that once the bug is encountered, there's nothing the user can do from inside the site or app, to try and get it running correctly again.
If a user can take some action that allows the app or functionality in question to return to a state where it can be used again (also known as a workaround), then the issue is recoverable.
Unrecoverable bugs should hold higher priority than recoverable ones, because the latter means that even though the bug is serious, there is a chance of the user being able to continue using the affected functionality - however, we must also consider how likely it is that the user would be able to discover the workaround when deciding the priority of a recoverable bug.
Audience Impacted Or Reach
If a bug is only present to a minority of your audience, and you find that audience is non-converting, then you probably don't need it fixed yesterday!
It's important to remember that even though you have a myriad of different technology factors that make up the audience, in the world of web a user ultimately boils down to a combination of operating system & browser software.
Where your main converting audience is impacted, the priority should be higher, where it is only a small percentage, you can safely de-prioritise because the reach will be much lower.
Expected Outcome
With a criteria such as this in place, the expected outcome is that less bugs will get classified as critical, this is because the criteria will make it much more difficult for bugs to fall into that category. But this is a good thing, it means that the criteria is in fact being objective, because we should expect critical bugs to be very rare. If we had lots of critcial bugs, it would basically imply that our site or application was hardly functional.
Always Critical
Any bug that has a security impact should immediately be critical priority. This should be the case even without a criteria in place, because security vulnerabilities can have far reaching, highly damaging effects on our users, as well as potentially having legal ramifications (for example GDPR).
Set, Revise, Tweak
One size doesn't fit all. Be sure that you consider the specific cases that apply to your application when coming up with your criteria. Always start with the statistics and data available to look at what criteria is available to you in the first place. Also, continuousy revise the criteria you set, because many of them may change over time, such as your audience impacted changing as browsers and operating systems update.
Happy bug squashing! The sofware kind, not the creatures!
