Project Management

Project Management Central

Please login or join to subscribe to this thread

Topics: Agile, Quality
Bugs that nobody has noticed (yet)
Hi all, I develop software for corporate clients. In our current project, we have many bug reports from our clients and spend a lot of time fixing them. At the same time, we find many bugs by ourselves. We prioritize bugs found by clients higher than the bugs found by us, therefore bugs found by clients we call major bugs, and the bugs found by us we call minor bugs. In most cases, minor bugs just reside in the backlog and probably will never be fixed, because all the time there is something more important than a bug that no client has noticed, often in a very long time.

Most likely it means that our clients don't use everything that we have, or maybe they don't want to report every issue they see.

Recently I tried to get the actual numbers and found that it was even less than 5% of those minor bugs that would be reported by a client later.

Relying on that number I can simply throw all the backlog of minor bugs away because there is no need to work on something that is not noticed by clients anyway.

Nevertheless, my gut feeling is that this situation might be a symptom of something bad. And why throw away the actual information that something doesn't work correctly.

I wonder what other people's opinions on this. How important is it to fix all the known bugs? And what may be the percentage of the issues that customers don't report?
Sort By:
Dmitry -

In general, defects identified and resolved internally will cost less than those which a customer reports. Even if a customer doesn't report a "minor" bug, they might be aware of it, and it might impact their perception of the product's quality.

Rather than spending effort fixing bugs, I'd suggest shifting quality left and focus on preventing them using quality design & development practices.

Kiron
...
1 reply by Sarmad Azhar
Oct 06, 2022 5:42 AM
Sarmad Azhar
...
I would hire a Certified Lean Six Sigma Black Belt in order to determine the cost savings regarding efficiencies and VOC.
Dmitry,

some thoughts, in addition to Kiron's.

- reporting bugs the developers find, is a honest thing to do and may increase the trust of the client in you - ethics on display - or not. That's why I like to work with independent testers (not the users)

- what you call minor bug might turn out to be a relevant bug that brings the system to a breakdown, you might want to add another category like severity of the bug. Calling it minor may be seen as an attempt to hide it

- depending on the usage of the system (e.g. a marketing app vs a space craft code), and therefor the potential impact (death of people or loss of some money) and the ability to correct a bug when found later (on a space craft near Mars?, developers fired) the severity of the same bug may change

- yes, for your company it is a balance of cost and risk, but can management judge the risk?

- if you share minor bugs with the client, they may be able to improve their testing, looking at corners they did not before

- no software is without bugs
When you find bugs that are not reported by the customer, are you using the software the same way the customer does? Are you finding bugs in the course of the normal flow of use, or edge cases?

Do you have a mechanism for the customer to prioritize the backlog of bugs?
...
1 reply by Sarmad Azhar
Oct 06, 2022 5:42 AM
Sarmad Azhar
...
I believe the Voice of the Customer plays a pivotal role.
Maybe bugs found by your development team, which you call "minor" bugs, are severe and affect in a great manner the quality of the software, in case of being found by your client. You should do the classification of the risks according to their probability and impact and then perform a quantitative risk analysis. Do not classify according to "found by your client" or "found by your development team".
I agree with assessing the potential severity of the bug.

Sometimes you can backlog them forever with no impact such as a function of process not applicable to the customer.

On the other hand, it could be a function your supplier has not used yet but will discover eventually and will be upset when they do. Other bugs may cause erroneous output that might not be noticed but can have very adverse effects. A bad calculation passed to another algorithm can create real world problems where the root cause is difficult to find. Yet others could cause a system failure if/when the right conditions occur such as something you thought the customer wouldn't do, but they did and reached the blue screen of death.
Oct 05, 2022 8:53 AM
Replying to Kiron Bondale
...
Dmitry -

In general, defects identified and resolved internally will cost less than those which a customer reports. Even if a customer doesn't report a "minor" bug, they might be aware of it, and it might impact their perception of the product's quality.

Rather than spending effort fixing bugs, I'd suggest shifting quality left and focus on preventing them using quality design & development practices.

Kiron
I would hire a Certified Lean Six Sigma Black Belt in order to determine the cost savings regarding efficiencies and VOC.
Oct 05, 2022 10:37 AM
Replying to Aaron Porter
...
When you find bugs that are not reported by the customer, are you using the software the same way the customer does? Are you finding bugs in the course of the normal flow of use, or edge cases?

Do you have a mechanism for the customer to prioritize the backlog of bugs?
I believe the Voice of the Customer plays a pivotal role.
First of all call it problems or incidents instead of bugs. Why? Because then you will have the possibility to understand if it is a matter of perception (eg: the functionality exists and works well but the client do not know it then you have to work in training the client), it is a matter of desire (then you can start a change on the solution) or it is a bug. But to do that you have to stay clear about all your solution components and this is a matter of configuration management. If after that you still have lot of you can call bugs then is time to review all your solution in advance which is making quality assurance instead of making quality control (testing and discover bugs). Behing all this I am assuming your company has defined what quality means for it because it is a matter to define the balance between quality and grade (eg: Microsoft realeased products with high level of grade and low level of quality).
It sounds like your customers are not aware of the problems found internally. Do you want to take the chance and wait for a customer to find the same problem, only to tell the customer you didn't do anything about it?

While software licenses usually abdicate any liability for software problems, it doesn't make for a great product or reputation.

I suggest you share your findings with the customers and let them prioritize the list. Then, talk to your management about a one-time push to reduce the problem list until it is manageable on an ongoing basis.
In my mind, there are very few things that are noticed. There is a difference between being unnoticed and complicit.

Please login or join to reply

Content ID:
ADVERTISEMENTS

"Opera is where a guy gets stabbed in the back, and instead of dying, he sings."

- Robert Benchley

ADVERTISEMENT

Sponsors