I have lost count of the number of times my project testing phase has been squeezed. When you are up against a deadline, your carefully planned 3-rounds of testing with time for bug fixes and validation suddenly slide out the door.
And yet, no one wants to put out a product that hasn’t been robustly tested. That’s just asking for trouble from the customer. I know iterative methods allow for rounds of improvements, but they should be functional, incremental improvements, not fixing bugs you let slip through because the testing wasn’t good, or long, enough. That’s my view, anyway.
It is tricky to schedule time for testing, because you don’t know what you’ll find. Perhaps the solution is so brilliantly coded that nothing buggy will be found. (Ha!) I think this is where some of the pressure comes from: sometimes managers disconnected from the process of creating something new feel that as each component of the software works fine, together the whole will work just perfectly. “If the build has been good enough, there won’t be much to fix.” If only.
Testing goes beyond simply making sure the code is good enough. We test the processes, integration, training materials, communication approach and more. Here are 5 quick tips that I’ve picked up over the years for testing. Perhaps they will be helpful to you too.
1. Keep notes
I know, keeping a note of exactly what you pressed and what happened is boring. Users don’t always understand the rationale of following the script and noting down what happened. Detailed notes help other people replicate the error so they can see it, understand it and fix it.
Get into the habit of documenting everything. You’ll be grateful later.
2. Try to break it
This is the part of testing I enjoy the most! And it kind of contradicts with following the script – except you should have test scripts for trying to break the product too.
Encourage testers to do things in the wrong order, use the product incorrectly and see what happens. You need to make sure it is adequately developed to deal with those use cases ‘in the wild’ as well as for the users who have read the instruction manual!
3. Test with real users
Talking of users reading the instruction manual, ideally testing should be done with the involvement of users. I have been on projects where testing has been done by a professional test team, in our test lab. That was great. The level of detail and accuracy and information provided was awesome and elevated our confidence in the product. Testers are brilliant.
But you should also involve some end users. They may find the test process regimented and a bit difficult to get on with, but a little training should help with that. That community will provide a different insight into how the product works and can give you feedback on usability and process that a testing professional might not know, not being an expert in their job function with the wider business context around that.
4. Test what you can’t see
A lot of testing in my experience has been around what buttons do on the screen, functionality, do you get the data you expect. But when working with professional testers (as opposed to subject matter experts and team members we’ve drafted in to help check the software meets their requirements) they have always focused on what you can’t see as well.
Those are the non-functional requirements. Does it meet the company security guidelines? Is it fast? Can we back it up and do those back ups work? You should have non-functional requirements in the product spec or as use cases, so make sure the testing doesn’t overlook those.
5. Plan the testing
Finally, and I know it sounds obvious, but all of those things above should be in a test plan. Sometimes the test team has written this on my projects, sometimes I’ve had to write it (and probably didn’t do that great a job). But whatever you do, to whatever level of quality, the important thing is that a test plan exists so you have some idea of what is expected, who is going to do it, what you are looking for and how the test results will be fed back to the people who can make the improvements.
Make sure a test plan is included in your overall project plan, and if you aren’t sure how to put one together, get some support from people who have done it before.
I’m not a tester, so I’m sure there is more to it than what I’ve written above from the point of view of a project manager. What would you add? Leave a comment below to tell me!
Do you have a formal Change Control Board (CCB)? If not, this is the perfect time of year to be thinking about levelling up your processes and putting new ways of working in place to formalise the way change management is done across projects, programmes and the portfolio.
A Change Control Board is simply a group of experts that represent different organisational departments and who oversee both the process of change management and the different changes being put forward.
At a project level, your CCB is a group of people who know the project well and who can assess project-related changes, but at some point if your project is making changes to the live environment, like most IT and business change projects do, the change will need to be submitted to the wider, department or organisational CCB.
The role of the CCB is to:
How it works
In our CCB, the functional lead or the project manager had to present the change. We had to talk about what it was, why it was useful and what it solved, and then make the case for whether it was a priority fix or not. If the change was considered a priority, it could go in the same day (mostly). If it wasn’t, it could be packaged up with a bunch of other small changes and go in the next release.
That made it easier to communicate changes to the end users.
Discussing the change
First, the change should be analyzed and discussed to see whether it has impacts beyond what the project team can comment on. The Change Control Board is convened to do that. I think the CCB is a really useful group and we relied on it in my last job. Our CCB looked at operational and project changes so the team could see the impact of ‘normal’ changes as well as the project-related ones.
I think it’s important that the CCB is made up of a cross-organisation group. It’s too common for changes (especially IT changes) to go in and for there not be a full understanding of the business impact somewhere else down the line. Complex ERP systems like SAP make that more likely, so a group of functional consultants getting together to discuss changes before they happen is a good thing.
I’ve had some changes rejected by the CCB because they didn’t have enough information to make an informed decision, or because something else was going on and they needed to wait on that, or because there was a change freeze. There might be many reasons why your change doesn’t go through.
The CCB can also schedule changes. There are normally scheduled windows to put changes in, especially in the live IT environment. That helps the support teams and the users know what is going to be different and what to look out for when they next log in.
Scheduling as a team also ensures that conflicting changes don’t get put in on top of each other. For example, if my project is updating the list of available categories in one part of the system, and another team is also updating that part of the system but taking away the category feature (that’s a bit extreme, but you see what I mean) then those conflicting changes can be discussed and overseen in an appropriate way.
It might involve putting them live in a particular order, or prioritising the changes so that one piece goes in this time and the additional change is put in next time.
I remember being told a story of a change in a data centre where engineers were working on cabling and flooring on both sides of a server stack. Without the support of flooring on both sides, the server stack toppled over! That’s the importance of making sure that changes are managed in a scheduled and sensible way.
We also had an emergency change procedure for anything that could not wait until the next release. On the SAP projects, for example, mostly things could be scheduled in a batch and changes pushed through on a fortnightly basis. But sometimes it was important to fix an issue straightaway without waiting until the next release. For example:
All of these are emergency fixes to live systems that wouldn’t be appropriate to delay, and they are all issue-related, not nice-to-have features.
How does your CCB work?
How do you actually go about mitigating risk? We talk about the need to mitigate all the time, but what kinds of things can you do to ensure that risks really are managed appropriately and mitigated to avoid the impact you think might be on the horizon? In this video, I talk about 5 different things you can try to mitigate risks. They are simple and practical, and you can easily turn them into solid actions for your risk log.
If you want a couple of extra suggestions, and some more detail (or you just prefer to read rather than watch), then the original article that prompted this video is available here: 7 Ways to Mitigate Risk.
Here are 5 ways to see the bigger picture in your work in order to make a bigger impact with what you do.
There is more about each of these in the infographic below.
Recently I spoke to Nick Nuss, data manager, Excel expert and blogger. I like to think that I’m OK at using Excel, but one of the things I don’t understand is pivot tables. So I figured it was about time I learned how to use them.
Luckily, Nick was on hand to explain all.
If you’ve ever struggled with pivot tables, you’ll want to read this interview.
Nick, how can I use Excel to better report on data?
Excel is very versatile. You already know this. As a project manager, you probably have Gantt charts and templates saved out. Excel solves many of your data problems. As a data manager myself, I use Excel in a variety of ways but the functionality I use most is the pivot table.
OK, basics please. What does a pivot table do for me?
Pivot tables allow a user to report on a large data set in a table format. For instance, if you have to track anything in excel, you can report on it. Financials, time, counts of sick days, anything that you can lay out in a table, you can report on it.
So what do I need to know?
First, I want to tell you about table design. Then we’ll move on to creating your pivot table. Lastly, I’m going to dive into how to best use the pivot tables.
Great. Tell me about tables.
Good table design is essential for good reporting. Garbage in is garbage out in the data world, so if you create your table in a poor way, your reports will not be accurate.
Let’s talk lingo for a second. I will be referring to rows as “records” and columns as “fields” from now on. When we have a record, it will be a unique instance that we want to track. This can be per person, per date, per person per date, etc.
The lower down you go, the more information you can get later on. By tracking at a department level, you may not get the results you want later had you tracked things at an individual level.
What about fields?
Fields are what we will use in the pivot table to describe the records. These can be dates, IDs, financials and numbers to tell us information about what has or is to happen.
It’s important that fields have the correct formatting. For instance, financials should always be listed with the financial signs, and dates should always be listed as a date. Without the correct formatting, when you go to use the data, your dates may not calculate correctly (i.e. the difference between 11/30/2017 and 12/1/2017 is one day, but if you do not have it in date format, excel may calculate it as 1212017 - 11302017 = -10,090,000!).
Got it! Anything else I should know about fields?
Fields can be further broken down into two levels; descriptors and metrics. The descriptors are ways to split the data, such as department and job role. The metrics will be the things you want to control down the road, like your financials or hours.
You can certainly use metrics as descriptors, such as people who work more than 40 hours per week, against people who work fewer than 40 hours.
How do I do that?
It is usually easier to create the data table before creating the pivot table. You can create these metrics with functions like “=if()” which will take on one value when logic is true and another when logic is false. You can even have the function return “true” or “false” if you don’t enter what to do.
To do this, set up this equation =if(CELL >= 40, “what does it do when true”, “what does it do when false”). Be sure to replace “CELL” with the correct cell for your records! You can then click and drag this down your table to auto populate.
This image shows the beginnings of a data table with the IF formula for working out whether someone works more than 40 hours a week.
OK. Any other tips for the data table?
A good rule of thumb is to also have one field set as “count” where you enter a 1 for each record. This allows you to report on the total number of records later that meet a certain criteria.
Keep your records clean as they are created. Fill in each field for the record and keep them accurate. If you can track numbers as a decimal, do so. The more exact you are, the more exact future reports will be.
Each field needs a label name in record one for a pivot table to work. Be sure to name these with descriptive names that you can use later. These labels should be unique for later on. It’s also good practice to label these with the field type in case you forget. Keep “date” in the name for any dates and “amount” for any financial values.
Now I’ve got a good quality data table. Are we on to the pivot table?
Yes, creating the pivot table comes next. Go to a new tab within Excel to keep your table data separate from your pivot table. Select cell D6, which will give ample space for your pivot table to populate. This is will be the upper left of your new report. When we discuss how a pivot table works, selecting down and over will make more sense.
Now, in the ribbon at the top, navigate to the “Insert” tab and select “Pivot table” from the left.
This will open the data selection. Click where it asks you to enter the range for the data.
Now, click on the worksheet tab where you entered your data and select the upper left cell of your data (this will be field A and row 1). Next, press ctrl+shift+end on the keyboard (PC shortcut). This shortcut will select all records and fields in the table. Press enter and you have successfully created your pivot table!
This image shows selecting a portion of the data source for creating your pivot table. The full data table is shown below.
Pivot tables work best with lots of data. In this table you can see resource names, hours worked per week, whether or not the hours are above 40, a field for 'count' and the week number. That gives lots of metrics to analyse and report on.
Excellent! Is that it?
Not quite. The third step is reporting from your pivot table. By selecting the area of the pivot table that we had before, a wizard will appear at the right. The top section will show all of your fields from the data table and the bottom will contain four quadrants. The lower left quadrant will be where you place what descriptors you want for your data. In the lower right, you will place the metrics you want to display. You can still use your metrics in the lower left quadrant, but you cannot report on descriptors in the lower right.
In this pivot table, we've calculated sum of hours per person worked on the project overall during the first 3 weeks.
My favourite part about metrics is you can use the drop down to select properties, and display an average, sum, or percent of the total.
The top two quadrants are used for additional analysis. If you wanted to cross your descriptors by ANOTHER descriptor, you would want to place the second criteria in the upper right. The upper left quadrant is used as a filter, which tells your report “only by these records.” This limits the records to certain selected values, but they do not necessarily show in the report.
This pivot table shows average hours worked by employees who work over or under 40 hours. We can also add in more columns and see hours worked per week per employee as in the screenshot below.
That’s really helpful! I think I get it. Anything else I should know?
You can create any number of robust reports using a pivot table. The value of using pivot tables grows with the size of your data table. The larger the table, the better analysis you’ll get. Pivot tables are fast and easy to create. You will be amazed at how quickly you can report on things in Excel using pivot tables.
For more advanced pivot tables, you could also get data out of a database or Access file as long as you select the data as an import. You can do this from the data tab to allow you to select the information in a pivot table! Excel is amazing.
About my interviewee: