I have lost count of the number of times my project testing phase has been squeezed. When you are up against a deadline, your carefully planned 3-rounds of testing with time for bug fixes and validation suddenly slide out the door.
And yet, no one wants to put out a product that hasn’t been robustly tested. That’s just asking for trouble from the customer. I know iterative methods allow for rounds of improvements, but they should be functional, incremental improvements, not fixing bugs you let slip through because the testing wasn’t good, or long, enough. That’s my view, anyway.
It is tricky to schedule time for testing, because you don’t know what you’ll find. Perhaps the solution is so brilliantly coded that nothing buggy will be found. (Ha!) I think this is where some of the pressure comes from: sometimes managers disconnected from the process of creating something new feel that as each component of the software works fine, together the whole will work just perfectly. “If the build has been good enough, there won’t be much to fix.” If only.
Testing goes beyond simply making sure the code is good enough. We test the processes, integration, training materials, communication approach and more. Here are 5 quick tips that I’ve picked up over the years for testing. Perhaps they will be helpful to you too.
1. Keep notes
I know, keeping a note of exactly what you pressed and what happened is boring. Users don’t always understand the rationale of following the script and noting down what happened. Detailed notes help other people replicate the error so they can see it, understand it and fix it.
Get into the habit of documenting everything. You’ll be grateful later.
2. Try to break it
This is the part of testing I enjoy the most! And it kind of contradicts with following the script – except you should have test scripts for trying to break the product too.
Encourage testers to do things in the wrong order, use the product incorrectly and see what happens. You need to make sure it is adequately developed to deal with those use cases ‘in the wild’ as well as for the users who have read the instruction manual!
3. Test with real users
Talking of users reading the instruction manual, ideally testing should be done with the involvement of users. I have been on projects where testing has been done by a professional test team, in our test lab. That was great. The level of detail and accuracy and information provided was awesome and elevated our confidence in the product. Testers are brilliant.
But you should also involve some end users. They may find the test process regimented and a bit difficult to get on with, but a little training should help with that. That community will provide a different insight into how the product works and can give you feedback on usability and process that a testing professional might not know, not being an expert in their job function with the wider business context around that.
4. Test what you can’t see
A lot of testing in my experience has been around what buttons do on the screen, functionality, do you get the data you expect. But when working with professional testers (as opposed to subject matter experts and team members we’ve drafted in to help check the software meets their requirements) they have always focused on what you can’t see as well.
Those are the non-functional requirements. Does it meet the company security guidelines? Is it fast? Can we back it up and do those back ups work? You should have non-functional requirements in the product spec or as use cases, so make sure the testing doesn’t overlook those.
5. Plan the testing
Finally, and I know it sounds obvious, but all of those things above should be in a test plan. Sometimes the test team has written this on my projects, sometimes I’ve had to write it (and probably didn’t do that great a job). But whatever you do, to whatever level of quality, the important thing is that a test plan exists so you have some idea of what is expected, who is going to do it, what you are looking for and how the test results will be fed back to the people who can make the improvements.
Make sure a test plan is included in your overall project plan, and if you aren’t sure how to put one together, get some support from people who have done it before.
I’m not a tester, so I’m sure there is more to it than what I’ve written above from the point of view of a project manager. What would you add? Leave a comment below to tell me!
A recent survey by GetApp, an online research company, sheds a bit of light on what’s really going on with project management software tools – which regular readers will know is a topic close to my heart.
I found the results interesting and here I’ve picked out a few to comment on. Starting with…
More than half of respondents (57 percent) work at organisations with annual budgets of $2k or more for project management software.
12 percent of respondents said their company had $25k or more to spend on software annually. My thoughts here are that the survey doesn’t correlate this to company size. You’d imagine that the cost of cloud software goes up with the number of users, so knowing the budget without knowing what they get for it in terms of project management population signed up isn’t very helpful.
However, this next point is interesting…
91 percent of project managers are willing to spend more for project management software that includes the missing features they need.
I’ve looked at a lot of tools and I have to say that there are similarities between what project management software does. After all, there are only so many ways to record tasks and mark them as complete. What extra bells and whistles you get for dashboards, resource management and so on are all good, but a Gantt chart is a Gantt chart at the end of the day.
So paying more for software that differentiates itself by having something “extra” – well, for many companies I think that is a hard sell.
However, this survey looks like customers will pay for what’s missing. The things they report as missing from the products they are using currently are:
No individual feature was reported missing by more than 12 percent of people, so there wasn’t a clear winner – if that’s the right term – for what is lacking in PM software. Rather, it’s a broad range of features that might not be in the tool you happen to be using.
The researchers draw an interesting conclusion from this, and I tend to agree: software hasn’t caught up with what project managers are actually expected to do these days.
It has been a while since the formal project manager was handed a fully-specified requirements document, and spent their days doing project scheduling, monitoring and control. Today we’re expected to handle a wide range of tasks from budgets to change management and everything in between.
And we need the tools that can deliver.
The amount people are prepared to spend on getting the features that matter to them is set out in the chart below. Most people don’t want to go more than 15 percent more and nearly 10 percent wouldn’t pay any more even if they got the extra features.
The challenge for software vendors is that every PM seems to have a slightly different set of requirements for what they need, at least in my experience. Even within my own business we use multiple tools because it’s easier and there isn’t one integrated product that does everything.
I’m used to it now, so I don’t see it as a problem. In itself that’s a problem! You can’t improve or look for efficiencies if you aren’t open to the idea that there might be some somewhere.
I’m not alone in using more than one tool, as the next survey result shows.
Almost three quarters (74 percent) of project managers surveyed use between two and five total tools for project management.
5 percent of survey respondents say they use over 10! I can’t imagine what they would be unless they have included software like Microsoft Word and email in their list. Perhaps they work with clients who use different products and as freelancers or contractors have to use multiple tools by default as every client has their own in-house specification.
This study seems to show that the situation for project managers today is that they don’t seem to be happy with the tools that are out there. With over 90 percent saying they’d pay for missing features and struggling along using multiple tools there sounds like plenty of demand for project management software vendors to step up.
However, I think that’s a sweeping conclusion to come to from this set of data points. I agree with the researchers that unless project managers have the right tools that help them do the job they are paid to do, we risk project failure (or at least not 100 percent success). But in my experience people are happy to talk about the things that aren’t going so great – and without access to the questions it’s hard to see if they were asked if there was nothing they felt was missing.
I’m naturally sceptical about surveys and I do think the data here is interesting. What do you think?
The survey was conducted with around 200 project managers in the US who use project management software for their day jobs. They came from a wide spread of industries but with a focus on IT and 49 percent of them had the job title Project Manager, with the others being slightly more junior or senior but still in PM-related roles. You can read the whole survey and see all the results here.
Do you manage your projects ‘in the cloud’? It’s the buzzword that has stopped being ‘buzz’ and is now a critical part of being able to operate successfully as a project manager – even if you don’t manage software projects.
Growing up in cold and rainy Hamburg, Mauricio Prinzlau had a semi-legitimate excuse for passing his time in front of the computer as a kid. After a degree in Business Communication Management, he joined the Cloudwards.net team as a managing editor and backup expert, where he's in charge of the cloud backup and storage reviews section.
I asked him what project managers need to know about cloud computing, starting with the really, really basic stuff.
Mauricio, what is 'the cloud'?
The cloud is obviously a very trending topic with a variety of definitions. The cloud leverages computing power and makes it available via the Internet so that businesses, organizations, and even individuals can use the cloud to solve business problems, upload photos, or host a website.
What else do people use it for?
Project managers use it for hosted project management collaboration tools but also other, non-project management apps that they are using in their day jobs or deploying to others.
For example, businesses use the cloud to make operations more efficient and leverage the computing power of servers to store and archive files cheaper and more secure. What used to be a complete on-site infrastructure that needed maintenance, security patches, and staff is now outsourced to cloud data centers that are off-site, thus reducing cost and risk for businesses and organizations.
Outside of work, you probably use the cloud on a daily basis without really noticing it. Facebook relies on cloud infrastructure, iCloud uses the cloud to backup people’s photos, Google Now helps us with day-to-day tasks. And there are many more examples we could come up with where the cloud plays a central role in people’s lives now.
So let’s say I need to back up all my project files. How do you choose the right cloud storage solution for your project?
How do you choose the right car? Just kidding, of course, it is very hard to give a general answer. What do you need the cloud for? Do you have a team of 5 people who need to collaborate and share files? Then you need to look at solutions such as Dropbox or Google Drive.
But what if security is a major concern? Then Dropbox and Google Drive are not an option. Project managers need to look at solutions such as Sync.com, who put security at the forefront of their business model.
Think about what your project teams and organization needs regarding features because it’s easy to overpay for a cloud service that you don’t need.
And what about the projects I might be delivering? Outside of file sharing and back up, what sort of things am I going to be hearing my tech architects talk about?
If you need virtual computers in the cloud to do calculations, then you you might hear them talk about Amazon Web Services because they offer great flexibility and scalability of computing power.
There are other options, of course, and what I would say is, there is no one-size-fits-all cloud. You need to compare features carefully, decide on your budget and then make an informed decision.
Can the cloud or online solutions be a cost-effective way to back up files?
In one word: yes. Online and cloud backup are terms used synonymously because there is not a real difference. Cloud (online) backups are one of the cheapest ways for businesses to get files off-site and protect them from disasters (theft, fires, flood etc.).
There are even services such as CrashPlan or Backblaze, who offer unlimited cloud backup, so they put no limit as to the amount of storage you can send.
Sounds great. Surely there are disadvantages we should know about?
There can be risks associated with cloud storage. That’s why I recommend my clients not to rely on one single cloud with all their data. Data loss can happen even to the best companies, so a fallback (ideally with a local backup) is always best.
Security is a concern if a cloud backup or storage company does not encrypt files before they are sent to the cloud (sometimes called end-to-end encryption). Always make sure you choose a service that supports this feature.
If your company has certain limitations as to the location of where the data is hosted then most companies have to be aware that the majority of cloud storage companies are located in the US. This is not ideal for most European businesses.
Yikes. So much of my project data is confidential. Is it safe to put stuff in the cloud?
It is safe if a) you encrypt files yourself before you send them, or b) a cloud storage service offers local encryption (or zero-knowledge privacy). I would personally recommend Sync.com, which is a Canada-based service, they work in the same way Dropbox does, but with zero-knowledge privacy.
I don’t even know what zero-knowledge means!
Oh, sorry! There’s a detailed explanation in this article but it’s too much to cover in this interview today, but it’s a way to store your data so that even the storage company can’t get into it.
OK, I’ll look at that later, thanks. How do you balance accessing work files on your work cloud and your personal stuff in your personal cloud, like Dropbox, for example?
Well, for one you could create two different accounts, one that you use for personal and one for business.
Dropbox and other cloud storage service offer business versions with enhanced privacy and collaborative features. For business collaboration, I tend to use Google Drive because my team and I can work together in real time on documents and spreadsheets, but I do not use the public clouds (Google, Dropbox) for very sensitive files like contracts. That’s where I would personally choose a zero-knowledge cloud.
Can you give me a short answer to what do managers need to know about the private/public cloud overlap and how best to manage it so that employees don't put confidential business records at risk?
That’s one of the major problems larger organizations face: the consumerization of the cloud. Employees use Dropbox, OneDrive or Google Drive to share business files. That’s why enterprises need to offer their employees a solution that is as easy to use as the public clouds. We have been working with Autotask Workplace as a solid solution for enterprises because it offers the same flexibility and ease-of-use as Dropbox, but without the security holes. Users can synchronize files (personal or business files) and managers can keep an eye on what gets in and out.
It sounds like an industry that is moving a lot. Where is next for cloud storage - what are the trends that you are seeing?
Cloud storage services do not focus on storage anymore, because, well, some argue that there is no money to be made with storage.
We clearly see a development into intelligent solutions for project and document management. Dropbox has hundreds of millions of users, all using their service in different ways, so in a couple of years, I believe we’ll be seeing a lot of artificial intelligence added to it, which helps us organize information automatically based on our specific business.
Thanks, Mauricio. Any final words?
I would say, don’t put all your eggs in one basket, especially when using the cloud for backup. But also, think about what your project teams and organization needs regarding features because it’s easy to overpay for a cloud service that you don’t need.
Overall, I’m excited to be close to seeing the developments in this industry and am looking forward to more innovative features and services based on the cloud.
Make Interoperability Part of Your Project
With CRM being this month’s focus area, making all your systems interact with each other is a key part of being able to get the data you need to feed your CRM system. Your project, therefore, if it has anything to do with IT, systems or data, probably needs some kind of interoperability approach in order to make it possible to pull relevant bits of information out when they are needed.
Think about it for a moment. If you design and build something that is completely standalone that’s good for a short while. But when you need to create a single view of the truth – a single customer record or report showing various data points – you will have to merge data from that system with data held in other systems. Which one is the master data? Even something simple like a customer informing you of a change of address becomes a problem if the systems aren’t linked.
What is interoperability?
Interoperability is that link between systems. This includes things like programming languages. When you are designing a product, think about the environment in which it will operate, who needs to use it and how it will link to, provide data for or send data to other systems. It’s often easier to think about information or data flow rather than the systems themselves, as these can be mapped on afterwards.
Using common standards and programming languages for system builds can save money and make it easier to find technical team members to work on your projects. Using open source tools, for example, is one way to build interoperability into your systems, but of course this will only work if your other systems use the same protocols.
Why is it important to your project?
No project is really standalone. Including interoperability in your design specifications as a non-functional requirement builds in future proofing. It also simplifies making links with other projects and systems which is especially important if your project is being carried out as part of a programme.
How do you get it?
You can’t design in interoperability yourself, although it doesn’t hurt to know that you need it! Involve your firm’s technical architect or, if you are using commercial off the shelf packages, talk to the supplier. A business analyst can help map processes and explain how the business users will actually use the system, so they can be really helpful when it comes to showing where the data comes from and how individual records are used and updated.
The best advice is to look at the big picture from the start of the project. Consider how things connect and consider what might be asked of the system or the project in the future. Of course, you can’t always predict how your new IT package will be used, but you can have a good guess. And you may find that users are already suggesting technical and functional changes that you are having to put into a bucket called ‘Phase 2’ for assessment and analysis later. These features may give you a good idea about the sort of things users will be asking for in the future so you can build in (or at least not shut any doors) for them.
Interoperability is probably already in your company’s technical strategy, so that’s a good place to start if you are building up your project requirements or including constraints in your project initiation document.
It isn’t the most glamorous of project requirements, but if you want your product – be that CRM or something else – to be useful and to be used, then it is worth considering from the outset.
Microsoft Project 2013 gives you a number of ways for handling resource costs including standard rate, overtime rate, cost per use and another field to let you accrue resources at a cost. Let’s take a look at these and see when you might want to use them.
If you use resource costs in Microsoft Project at all, this is the cost field that you will find the most useful. It’s, as you would expect, the normal pay rate for someone or something over a time period. For people, Project will default to a per hourly rate but you can use a different unit of time if you want – just change the setting. Or you can work out their hourly rate based on their monthly or weekly rate and enter that if you want to keep everyone standard by calculating hourly rates for all your resources.
Non-people resources are not calculated on an hourly rate. Instead, they are worked out by price per unit. You’ll have to work out what units you want to use. Use the Material Label field to record what unit you have set. For example, if you are hiring a software testing lab at $1500 per day, you can use the Material Label field to record ‘daily lab fees’. Then Project does the calculation for you – quantity multiplied by standard rate.
Cost per use
Only use this if you have certain resources where you only pay each time you use the resource, and it’s a flat fee. An example would be a call out fee for a plumber for your new office conversion. Each time you call on the plumber, Project will calculate the hourly rate plus the call out fee. You can also use this for delivery charges.
Another field that does exactly what it says, but it doesn’t apply to non-people resources. Leave it blank unless you pay your team members overtime for hours worked above and beyond their contracted hours. If you do want to use the overtime function, you’ll have to assign overtime hours to the resource, otherwise Project will assume they are either salaried and don’t get any extra payments or that they earn the same amount regardless of how many hours they work.
Of course, you don’t have to use Project to calculate overtime payments for your team, and it can get quite complicated to keep on top of what’s an overtime hour assignment and what’s normal working time. But if you are expecting Project to calculate your total project budget for you, you’ll need to make time to record all this data otherwise your expense figures will be out.
This field is only useful if you are bothered about when the money is spent. On many projects, this won’t make any difference at all, as your project sponsor will only be interested in the overall budget and estimate to complete. But there might be times when you need to know if you’re paying out the money in advance or after the job is done.
You’ve got three choices here:
They are pretty self-explanatory. ‘Start’ means the cost goes at the start of the task, so you’ve paid upfront. ‘Pro-rated’ means it is spread out across the duration of the task. And ‘End’ means you pay when the work is completed.
It’s fine to record this level of detail in your project plan for expense tracking purposes, but you’ll need to know how to read the reports to interpret it! That’s beyond the scope of this article. But hopefully this has given you a flavour of how to use the different resource cost settings in Microsoft Project and you can choose which ones (if any) are useful to you on your project.