The Business Driven PMO

Spreadsheets are ubiquitous, heavily relied on by organizations to manage data and make critical business decisions. Spreadsheets are an excellent tool for independent analysis. The problem is that they are often stretched far beyond their home territory. Furthermore, spreadsheets tend to have limited scalability beyond the desk and the fidelity is constrained by the organization’s ability to invest in additional technology capability to manage its reliability.

It’s imperative to note that the goal of inputting data in a spreadsheet is not to get an answer, but to get the correct answer. Often a wrong answer is much worse than no answer at all. There are a number of features of spreadsheets that present a challenge to error-free analysis. It is extremely common for data to be added to a spreadsheet after it has been created. The augmentation of data can go wrong, rendering a correct spreadsheet incorrect. Even the most experienced practitioners, using all the armory at their disposal to prevent mistakes from creeping in as they work, will make common errors from time to time. The requirement in organizations to work under huge pressure to achieve deadlines makes the probability of error even higher. Some of these errors will be caught by the fail-safe mechanisms built in. But some will not.

Additionally, many spreadsheets take all night to compute. These computations can be complicated and commonly fail. However, when such spreadsheets are replaced by capabilities more suited to the task, it is not unusual for the computation time to be cut to a few minutes and the process much easier to understand.

Why Do Organizations Continue to Use spreadsheets?

The technology acceptance model holds that there are two main factors that determine the adoption of a technology: the perceived usefulness and the perceived ease-of-use. Perception need not correspond to reality. The perception of the ease-of-use of spreadsheets is to some extent an illusion. It is easy to get an answer from a spreadsheet, however, it is not necessarily easy to get the right answer. Thus the distorted view. The difficulty of using alternatives to spreadsheets is overestimated by many people. Safety features can give the appearance of difficulty when in fact these are an aid.

The hard way looks easy, and the easy way looks hard.

When Do Spreadsheets Go Wrong?

In a recent survey conducted by Daptiv, it was revealed that 76 percent of IT organizations still rely on spreadsheets for critical decision-making purposes. Spreadsheets are good when small amount of data needs to be managed. However, the "spreadsheet as database" is not always easy to maintain. At some point of enterprises will need specialized application capability to manage their database for managerial purposes. Research, such as that reported by Raymond Panko in "What We Know About Spreadsheet Errors", has found that most of the spreadsheets used by organizations contain errors—and that a considerable number of those errors are serious. In one case reported in Panko's research, the error would have caused a discrepancy of more than a billion dollars! Finally, academics have published studies of the prevalence of spreadsheet error and have sought to identify circumstances dangerous in the context of error and other circumstances that are regarded as safer. Therefore, unless spreadsheets are being used for single functionality, it must not be overburdened with complex calculations and codes. 
 

One of the most widely used tools for project management in software teams today is the spreadsheet. Although fairly cheap and easy to use, spreadsheets are extremely vulnerable to errors. They hide problems that can hinder the success and create more costs than one has planned for. Here are some of spreadsheets’ inherent limitations:

• Consistency is hard to maintain when storing data in spreadsheets. A particularly pernicious problem with using spreadsheets is its tendency to auto format fields. In addition, there's a significant risk of messing up data when it's held in a spreadsheet.

• It's extremely difficult to develop accurate spreadsheet models. The software development community has invested extensively in developing agile methodologies, frameworks and tools to improve the quality of software. In comparison, relatively little work has been done to improve the quality of spreadsheet modelling -- and the work that has been done has had minimal impact on spreadsheet users.

• It's also all too easy to enter incorrect data into spreadsheets. A wrong keystroke and a formula is replaced with a static value, rendering the calculations meaningless. Another (ab)use of spreadsheets is its use in business models -- strategic planning models, forecasts, simulations, what-if analyses, etc. With limited insight into data and superficial view of results, spreadsheets should not be made the foundation for significant corporate decisions.

• Transparency: Spreadsheets are disconnected. All feedback, ideas and requirements are stored in separate files and there is no easy way to relate them to each other. This way the initial intent behind some features can get lost and the flow of context disrupted.

• Traceability: It’s almost impossible to manage something that you cannot track. With spreadsheets information is usually scattered across multiple files and folders. There is no easy way to get a complete view of the task in progress and/or its status. This can delay decision making and increase time to market.

• Planning: Spreadsheets are very one-dimensional in nature. However, seeing the interconnectedness and hierarchical arrangement of all requirements and tasks is a crucial element of the planning process. It allows you to properly allocate resources, create schedule and prepare for impediments. Using spreadsheets is a tedious and error-prone task as in most cases one will need to update information across multiple files even for the smallest adjustment in plans. In addition, there is no easy way to make sure that everyone on the team/organization has the latest version of the plan and is working on the things with highest priority.

• Collaboration: Team collaboration is the key for every organizations success, and this is especially true for geographically dispersed teams. Usually collaboration occurs over some kind of document. The most common way to share such information with other members of the team, when you use spreadsheets, is to email it. This can lead to problems like multiple versions of the same file being updated simultaneously, delays in decisions, and miscommunication. This lack of a common environment where a team can collaborate is one of the biggest limitations of spreadsheets.

• Salability: Spreadsheets are really intended to do the analysis not to the “database”. Although we try, the more data that is in this “database” increases the likelihood that a simple error made with a typo will result in these aforementioned errors.

• Referential Integrity: Well we have already established that spreadsheets should not be considered as a “database”. But maintaining the integrity of data relationships is fundamentally not the role of the spreadsheets.

To be fair, spreadsheets aren't the only models that contain errors. We all know that software has its fair share of bugs. But the sheer number of spreadsheets, coupled with "homespun" development, and the difficult of reviewing their logic, makes spreadsheet development the Wild West of the modeling community. If you are using spreadsheets for anything more than individual prototyping in your organization, I would recommend you to seriously consider replacing them with models that are more suitable.

In our day-to-day project management and PMO activities, the easiest and the most important thing to miss is plan for ahead what happens AFTER we cross the finish line. So technically speaking, once project managers hand over the reins of the completed project to the business owner, their job is just half done. For a project to be considered complete, project managers must focus on the other half, which is “Benefits Realization”.

Benefit realization is the confirmation that the value a project was expected to generate really does get delivered.  In our everyday project management lives it is easy to get buried in details around task management, risk mitigation, resource capacity, balancing budgets and all the other moving parts.  We often forget why we set out to do the project in the first place:  the delivery of a product or service, an enhancement or improvement, or a capability.  For example to meet some new regulation, standard or market demand.  But what if, after we deliver the goods, and did exactly what the customer asked for, we realize that all the effort and resources we used to deliver the project don’t amount to what they were supposed to?  That’s exactly what benefits realization is all about.

We’ve all heard of ROI – return on investment.  It is the concept of an investment of some combination of resources (people, money, equipment, etc.) yielding a benefit to the investor.  A high ROI means the investment gains compare favorably to investment cost. As a performance measure, it is one of the best methods to evaluate the efficiency of an investment.  ROI does not exclusively have to be in financial terms.  It can easily be an operational advantage, an improvement in position, or other positive change.  In order to compare the efficiency of a number of different investments we need to compare like measures, which is why a financial ROI is one of the most commonly used.  Unfortunately, without benefits realization, our ROI is simply a guess.  And that is why benefits realization is so important.

I’ve discussed with   many of our clients about this and have found out that there is a need for a wide degree of maturity around the realization process.  This is an indication that while the concept of realization is gaining interest, it is still far from a mature practice.  Which presents a great opportunity for those organizations that are not doing it – now is a great time to implement this practice.

How to launch a benefits realization initiative?

One of the best approaches involves setting goals, tracking against those goals and including a ‘hand-off’ step, similar to the passing of the baton in an Olympic relay race.  Tactical steps you can take include:

• Set your sights:  using whatever calculation available, combined with experience and validated by results from similar projects, come up with a target for what the value you think the project will deliver.  Set that as the initial estimate.  Enlist the help of a financial leader or controller to help set the original estimates.
• Continual monitoring:  Using the initial estimates or targets as a first guess, continue to refine the success factors throughout the lifecycle of the project.  These are often called forecasts or committed values and are more accurate than the original estimates.  It is best to continue revising these figures throughout the lifecycle of the project.  The objective here is to have these forecasted numbers eventually match the actuals.
• Start tracking actuals now:  some project can actually generate benefits even before the project is complete.  What a great win for the project team to be able to report these.
• Put a plan in place: Add a milestone or stage beyond the Complete step called Close or ‘Realization’ and set a validation step 3, 6 or 12 months after the project is complete.  It sets the expectation that the work is not over at Complete.
• It is outside of the project manager’s responsibility:  As the project comes closer to its Complete or End date, engage the financial sponsor and the process owner (the person who is benefiting or owning the project’s or product’s outcome, improvement or change) and have them start validating and “owning” the numbers.
• Go back to the beginning:  how accurate were your original estimates compared with your forecasts and your actuals?  Take those learning and apply them to future estimates.  This is called continual improvement – applying lessons learned and best practices to improve the entire PMO.

One last point is that it isn’t always about the money.  Sometimes projects generate other value, such as an improvement in customer satisfaction, or increase market share by launching a game changing product.  It is important to be able to quantify the value of these types of projects even if they do not generate direct revenue or cost improvements.  Many organizations call these ‘Level 2” or “Indirect Benefits”.

Finally, is a project complete without benefits realization?  To the project manager who’s already run their marathon and marked the project as complete, I expect their answer to be ‘yes’, but common sense tells us otherwise.  As a best practice, one of the most important factors in a projects success isn’t “how we did it” – coming in under schedule or under budget – but “what we did” – that the project delivered what it set out to do.

When I was running the IT-PMO at PeopleSoft we faced an interesting dilemma. As we finished work on the integration of JD Edwards there was a ton of unmet demand for IT work from all corners of the enterprise. This ranged from tweaks to the purchasing system to an all-new global training environment. We quickly realized even our ability to analyze the demand would be swamped by the incoming flood of work.

So, we devised a scoring system. Why? There were three main reasons, all of which really comprise some fundamental principles when creating a scoring model.

First was the need to analyze and separate the wheat from the chaff quickly. Our primary driver was to be able to make an initial cut from 120+ requests to something more manageable for more in-depth analysis. So we needed a way to make quick judgment calls to find the top 20-30 project requests with the most merit.

We further realized that any analysis that came up with a specific number (like $300K for changing the purchasing program), even with a caveat of +/- 100%, would become sticky. That is to say, if the $300K estimate was later revised to $400K - well within the +/- 100-% - the executives would still want to hold us to the $300K! "I thought you said $300K 2 months ago - what changed!" was a familiar refrain. Scoring models, on the other hand, place estimates in ranges. So as long as you don't exceed the top range it’s all good.

Many project-driven organizations today face this same dilemma on an ongoing basis. Scoring models meet this challenge well. So, to create a scoring model that will quickly find the projects with the most merit without being nailed down to estimates too early, keep these key principles in mind:

1. Group your scoring criteria into around three buckets - these will be used as axis on a bubble chart later. My favorites are benefits, cost/size, and risk. Others include impact, and for product development groups may include market share, technical feasibility, and margin.
2.  Scoring criteria should comprise ranges.  An example would be a 1-5 rating of potential revenue increase, with 0 = none, 1 = less than $1 million, 2= 1-5 million, 3 = 5-10 million, etc. Same goes for project cost or other financial metric. For criteria like risk, an example would be a rating on project familiarity with 1 = very familiar with this type of project and 5 = never done this kind of work before. Make sure all the criteria produce the same range of scores (e.g. 0 - 5) so you can create weighted averages for each group and a weighted average total project score.
3.  Scoring criteria should fit the company's strategic direction and business needs. A retailer will be concerned about increasing market share, while a SaaS company like Daptiv is concerned with customer satisfaction.
4.  Bubble charts are a great tool for graphically envisioning which projects will produce the most bang for the buck. While the simplicity of a single chart is more efficient, I have seen new product development organizations with up to 6 criteria groupings used on 2-3 bubble charts.
5.  Back test the model. Take the scoring model produced and score the current slate of active projects. When I did this with a major retailer a couple of years ago, we knew we had it right when the only current projects that wouldn't have made the cut turned out to be problem children that should never have been launched.
6.  Always analyze requests in cycles. Applying a scoring model to each request as it comes in negates the comparative process. It also leads to new priorities interrupting live projects, which results in project and resource churn. We typically recommend quarterly cycles. Monthly can work in an environment with larger quantities of shorter lifespan projects. Generally annual cycles are too long as too much work comes up in the interim. However, an annual planning process for the larger, more strategic work can be coupled with a quarterly cycle for the smaller work.
7.  Scoring models work best when there is a cross-functional team empowered with the ability to make decisions. This means they will be high enough level in the company to not be second-guessed by colleagues or superiors.

Once requests are reviewed and sorted using a scoring model, decisions can be made about which should proceed for further analysis. Those that pass muster then pass into the more traditional initiation process for projects, ensuring that valuable analysis time is not wasted while allowing the focus necessary to properly present the best projects for funding.