Brigitte FortinDeputy Director for Business Systems| ICFMyersville, Md, United States
Hi Friends, I've just been challenged by colleagues who think that testing database scripts is optional and unnecessary. In my career, the only time testing is skipped is when the system is down and there's an emergency change request to document the change.
I'm looking for a simple, non-technical, but also authoritative document that I can share with them to explain why testing must never be skipped, even for simple scripts.
Brigitte FortinDeputy Director for Business Systems| ICFMyersville, Md, United States
Jul 28, 2022 1:52 PM
Replying to Keith Novak
...
Sometimes it may be OK to not test, but first you need to ask what is the consequence of finding an issue.
If a change doesn't affect the customer's functionality such as it is limited to your own internal data collection on the SW performance, then it might be acceptable to not test, but then you also don't know if your internal data is any good. Another example is if the functionality changed isn't needed now but we are inserting it early to gain practical experience, and it is isolated from other systems
Otherwise, the change may adversely affect the intended function, or inadvertently impact functions either within the changed module, or other modules with different owners who interface with the prime group.
In a safety intensive industry, if someone wishes to avoid functional or regression testing, then they must justify their rationale. That is treated on a case by case basis and reviewed with cross-functional team of equipment owners to ensure potential adverse affects were not overlooked.
Good point, we can ask test resisters to define the consequences when something goes wrong, that may get them thinking differently. Usually, the consequence is "we did it wrong" and "they have been jilted" by our failure. So you can see, this gets pretty touchy. So we end up in a bit of a tug of war, with customers saying, just put the data in Prod and no we don't want to test, but if anything goes wrong, it's because you failed. Not that we as a wider team failed to take the few minutes that is required to run a quick test.
...
1 reply by Keith Novak
Jul 28, 2022 3:14 PM
Keith Novak
...
Yes, unfortunately I know that situation well. "We don't have time to do it right the first time, but we always find time to fix the defects later."
I would personally try to explain the Cost of Poor Quality (COPQ) to the team. Most of my references are very technical, so I try to explain it more layman's terms. Some sources put the typical cost of waste as 20% of total revenue, but that can range from less than 1% to over 40% depending on the error rate. This article is a bit dry for most people, but might provide ideas:
Good point, we can ask test resisters to define the consequences when something goes wrong, that may get them thinking differently. Usually, the consequence is "we did it wrong" and "they have been jilted" by our failure. So you can see, this gets pretty touchy. So we end up in a bit of a tug of war, with customers saying, just put the data in Prod and no we don't want to test, but if anything goes wrong, it's because you failed. Not that we as a wider team failed to take the few minutes that is required to run a quick test.
Yes, unfortunately I know that situation well. "We don't have time to do it right the first time, but we always find time to fix the defects later."
I would personally try to explain the Cost of Poor Quality (COPQ) to the team. Most of my references are very technical, so I try to explain it more layman's terms. Some sources put the typical cost of waste as 20% of total revenue, but that can range from less than 1% to over 40% depending on the error rate. This article is a bit dry for most people, but might provide ideas:
I like some of the graphs in the link... thanks, what keeps coming up is the cost benefit balance. That will definitely be a touch point for the conversation when it comes up again.
Saving Changes...
Peter RapinSubject Matter Expect; Project Delivery| Independent ConsultantOntario, Canada
Is this not a risk-benefit consideration? Benefit of testing versus risk of not. If so, maybe it is not a one-answer question. If the cost of testing is $50 and the cost of failure is $1000 with a 10% probability thus $100 exposure - then spend the $50 and test. If the exposure is equal to or less than the cost of testing, don't test.
...
1 reply by Brigitte Fortin
Jul 28, 2022 4:41 PM
Brigitte Fortin
...
I'll buy that, thanks for keeping it simple.
Saving Changes...
Brigitte FortinDeputy Director for Business Systems| ICFMyersville, Md, United States
Jul 28, 2022 3:14 PM
Replying to Keith Novak
...
Yes, unfortunately I know that situation well. "We don't have time to do it right the first time, but we always find time to fix the defects later."
I would personally try to explain the Cost of Poor Quality (COPQ) to the team. Most of my references are very technical, so I try to explain it more layman's terms. Some sources put the typical cost of waste as 20% of total revenue, but that can range from less than 1% to over 40% depending on the error rate. This article is a bit dry for most people, but might provide ideas:
I like some of the graphs in the link... thanks, what keeps coming up is the cost benefit balance. That will definitely be a touch point for the conversation when it comes up again. Saving Changes...
Brigitte FortinDeputy Director for Business Systems| ICFMyersville, Md, United States
Jul 28, 2022 4:29 PM
Replying to Peter Rapin
...
Is this not a risk-benefit consideration? Benefit of testing versus risk of not. If so, maybe it is not a one-answer question. If the cost of testing is $50 and the cost of failure is $1000 with a 10% probability thus $100 exposure - then spend the $50 and test. If the exposure is equal to or less than the cost of testing, don't test.
I'll buy that, thanks for keeping it simple. Saving Changes...
Sergio Luis ConteHelping to create solutions for everyone| Worldwide based OrganizationsBuenos Aires, Argentina
Is not about testing. Is about quality. And in the context of quality testing must be in the second priority. The first one is assurance activities. For example, code inspection of scripts. I am not saying that testing has to be avoided. But there is a lot of literature outside there about software testing starting for "The art of test software". Returning to quality point when you work with this type of topics there is a balance between quality and grade that has to be defined in advance. If you are the project manager then what you have to do in situations like that is to clear state the risks and assign it a person that will be responsible for the consecuencies. Saving Changes...
Mark WarnerProject Manager| AURATucson, Az, United States
While it's not software-related, our data on my last project (a major 11-year-long, $362M design-build project) showed that more than 75% of our commissioning problems came on systems that didn't under-go pre-ship testing. We would have saved a lot of money, time, and hassle had we spent the money, time, and hassle up front with testing. One of our biggest lessons learned, imo. Saving Changes...
Stéphane ParentSelf Employed / Semi-retired| Leader MakerPrince Edward Island, Canada
As a former database administrator, I cannot imagine foregoing testing any script that will change the database or it's data!
At the very least, the scripts should be executed in a non-production environment. Ideally, you want an environment that is as similar as the production environment as possible.
You'd be surprised how a database script can work like a charm in a development and test environements but bomb in production!
If you can't convince them to test, make sure they are aware that they will be on standby for call-back when everything goes South! Saving Changes...
Brigitte FortinDeputy Director for Business Systems| ICFMyersville, Md, United States
Merci Stéphane, at last someone who gets me ;) I'm afraid I will have to be that blunt. Saving Changes...