When Testing Becomes A Dog And Pony Show
From the pmStudent Blog
by Josh Nankivel
Ranting and raving about project management and systems engineering.
Recent Posts
The Problem with Project Management
The Problem with Project Management
The Problem with Project Management
LinkedIn Recommendations Are Easy
The Catch-22 of Project Management Certification and Experience
Categories
Agile,
Career Development,
Certification,
Change Management,
Communications Management,
Cost Management,
Documentation,
Earned Value Management,
Education,
Integration and Test,
Kanban,
Leadership,
Lean,
Lessons Learned,
Methodology,
Misc,
Multitasking,
New Project,
Operations,
Planning,
PMP,
Productivity,
Professional Development,
Project Estimation,
Project Leadership,
Quality,
Requirements Management,
Risk Management,
Schedule Management,
Scope Management,
Software,
Systems Thinking,
Tools,
Video,
Work Breakdown Structures (WBS)
Date
I'm going to ask you a question.
It's a simple question.
What is testing for?
Some people seem to think that testing is for display purposes only. These people would expect your testing phase to consist of showcasing your product instead of actually testing it.
I don't think so I think the best definition of true testing is this: Try to break the product.
That's right, the only value in testing your product is to try to poke as many holes in it as you can.
If you're testing phase consists of trying to look good in front of your customer, your stakeholders, your sponsors, then you are doing it wrong.
Get this. Earlier this week I was actually asked this question. "Next time, could we do integration testing before the official integration testing?"
That's right folks. We are talking about testing before we do testing. How sad is that?
Now let me make sure I'm clear.
We did dry run testing before the official testing. So it's not that we didn't test fully, or have ample preparation before the official testing. No, this is about looking good politically in a testing cycle.
This type of showmanship has absolutely no place in effective project management.
Period.
Posted on: August 25, 2011 11:48 PM |
Permalink
Comments (9)
Please login or join to subscribe to this item
Wai Mun Koo
PMO Director| Intergraph PP&M
Singapore, Singapore
I think the whole issue is lack of trust. The person that told you to do integration testing before official integration testing probably does not trust that you have done your due diligence part in the testing and that he is a non-believer of 'it is impossible to eliminate all bugs completely through testing'. Maybe he is also a die-hard fan of six sigma. Perhaps he is also looking for a TESTimony at the end of the so called 'official test'.
"I don't think so I think the best definition of true testing is this: Try to break the product."
I totally agree. But really, people don't like the idea of anyone poking around in their stuff. What if you found something was wrong? Someone might be blamed for not having built the product right! Oh, the horror!
In general though, people are receptive to imagery and metaphor - if only because they can repeat them at social events and sound clever. Regarding the particular issue of testing, when a random executive or other starts to look at this phase in my project schedules with the intention of giving it the ax, I usually ask them: "Would you buy a car that hasn't been thrown at full speed against a wall hundreds of times to make sure it's safe? Well, same goes for your product." It forces them to pause and consider. Because they understand this. They have cars (usually provided by the company). They know people wouldn't buy an unsafe car, and they know they want people to buy their precious product. So, they pause, and they think. And then they still give testing the ax - but definitely not as much as they had set out to.
Sometimes, a small victory makes a big difference.
Josh Nankivel
Engineering Project Manager| Apple
Sioux Falls, Sd, United States
Thanks Wai. That shouldn't be the issue in this case. In this case, the major interfaces between systems have not been tested yet, that is what this level of testing is actually designed for. So 'due dilligence' doesn't enter into it...this is the first time the full systems are integrated and tested.
Josh Nankivel
Engineering Project Manager| Apple
Sioux Falls, Sd, United States
Thanks Julien.
It's odd, but in the example I gave here, they may actually be spending TOO MUCH on testing - because of the focus on process that doesn't add value.
Testing isn't to show how good the developers are, it's to check the product before real customers use it. The business people shouldn't see testing as the opportunity to demo the software, it's their chance to spot bugs. They should try all the obscure functionality and aim to break it, just as you say. Otherwise, what's the point? It sounds like the wrong people are doing the testing if they feel that they shouldn't find any errors at all.
Josh Nankivel
Engineering Project Manager| Apple
Sioux Falls, Sd, United States
Wai Mun Koo
PMO Director| Intergraph PP&M
Singapore, Singapore
I believe there are different types of testing each comes with a different objective and target audience (e.g. unit testing, system testing, integration testing, regression testing, acceptance testing, alpha and beta testing etc.). The key is then to communicate the objective of the testing clearly to the target audience and set the right expectation. Things may turn out bad if the expectation does not meet with the objective, and this is where we need to cope with.
However, I have not seen a testing that is meant for a demo as you have pointed out.
Sam Motes
Manager II Business Sys, Operational Excellence| BA Systems Inc.
Ellenton, Fl, United States
In a perfect world testing is just window dressing so you can tell the customer that we tested our product. It is a complete waste of time other the PR.
Of course this assumes:
- 100% accurate and clear requirements before coding begins.
- Expectations of the end users are 100% understood and we know we are delivering exactly what they need even if they aren''t fully up front about what they need.
- We thought about the 60 ways our product wasn''t designed to be used that our users will be forcing it do even though it wasn''t in scope for the project.
- No scope change in what the program is supposed to do since the beginning of the project.
- All the developers fully understand the integration points with all of the other modules
- Their are no typos in the 1 Billion lines of code written by 20 programers in isolation over the last year of development.
- Our product we have built over the last year will interface perfectly with the 402 security patches and fixes released just last week for our OS platform.
- We are 100% reliant that the 12 layers of hardware and software we built on are relable and setup with the utmost performance tunning in place already.
OK, now if we can stand there with a straight face and say we are 100% sure of all of the above assumptions (not to mention the 50 others we missed during scope planning) then yes, testing is just window dressing and a waste of time. Then again, I wouldn''t stake my reputation on a single one of those assumptions being 100% true. I guess we better get to testing.
Josh Nankivel
Engineering Project Manager| Apple
Sioux Falls, Sd, United States
Amen Wai and Sam, well said indeed.
Please Login/Register to leave a comment.
|
"Smoking is one of the leading causes of statistics."
- Fletcher Knebel
|
