Turning Estimating on its Head

From the Project Management 2.0 Blog
by
New technologies, concepts, and Web 2.0 tools are popping up everywhere. How can you use them to help your project team collaborate, communicate - or just give your project an extra boost? [Contact Dave]

About this Blog

RSS

Recent Posts

Are You Prepping For The PMP 24/7?

Are You Just Too Darn Busy?

Eliciting Requirements... Creatively!

What To Expect When Your Stakeholders Are Expecting

8 More Templates to Save You Time



Situation: You need to take a fresh approach to estimating.

Estimating tools are always interesting to understand because they reflect what their makers feel are the key inputs to the estimation process.  Many of us go tool-free and estimate based on person experience and the experience of SMEs around us.  We recently spoke with J. Chris White of SimBLOX about the pmBLOX product™.  Whether or not you are interested in a new tool to estimate with, some of the approaches he describes are pretty interesting.


Q.  Can you give us a quick overview on pmBLOX™ and simulation-based project management software in general?  Is the function of the software similar to Monte Carlo simulation?  (What is it? How does it work? Why is it better than traditional estimating?)

A.  What makes pmBLOX truly unique and revolutionary in the field of project management is its underlying model is completely different from anything that’s been done before with simulation-based project management (PM).

Unlike current methods like Monte Carlo (which are based on the CPM/PERT approach requiring task duration as an input), pmBLOX produces task duration and resource utilization as OUTPUTS.  Work backlog, resource availability, productivity, and several other fundamental factors are used as INPUTS.  In effect, pmBLOX turns the traditional CPM/PERT method on its head.

(Note that this is not far off what is already done with current planning tools.  When a user makes an estimate for the duration for a task (an input with the CPM/PERT approach), he/she typically has some assumptions about using particular people for particular amounts of time so that the estimated duration is not a complete guess.  pmBLOX simply starts with these assumptions and makes them explicit so that they can be challenged/defended to ensure a more realistic project plan.)

In the underlying pmBLOX model, a task is represented with a backlog of “work to do”.  For example, a task may require 40 hours of work.  If a single person works on this task for 8 hours/day with 100% productivity (i.e., each hour the person is paid results in an actual hour of task work), then the task will be completed in 5 days (40 hours / 8 hrs/day = 5 days).  (If you are familiar with any simulation techniques, we use system dynamics – a continuous simulation methodology invented at MIT in the late 1950’s that is based on engineering feedback control theory.)

This is a very simple example and is straightforward.  In fact, this is actually what current planning tools like MS Project do behind the scenes.  When a user says a task has a 5-day duration and a single resource is assigned to the task at 100% and that resource works 8 hours/day, behind the scenes the software converts that task to 40 hours.  So, when the user adds a second resource, the duration is cut in half to 2.5 days because now 16 hours of work are being done each day.

The difference with pmBLOX is that the user would designate the task as a 40-hour task instead of inputting a 5-day duration.  Assignment of the single resource is the same.  Very little difference in the inputs, but the underlying approach is fundamentally different.  As long as the single resource is available as expected, the final result (i.e., task duration) is the same for both pmBLOX and the CPM/PERT approach:  task duration is 5 days.

However, once we changed the underlying approach from CPM/PERT to hours-based simulation, we found that the door was open to making many more substantial enhancements to bring the simulations even closer to the real-world activities that the simulations are trying to mimic.  For instance, since we moved to an hours-based approach for completing work, we now had access to variables such as resource productivity.  If conditions ever changed in the simulation so that the same situation in the real world would result in productivity losses for a resource, we could now incorporate that effect.

As an example, imagine working overtime.  As you work more and more overtime over an extended period of time, you get more fatigued and “burned out”.  This is common knowledge.  If you work 8 regular hours for a day, you come back fresh and productive the next day.  If you work a few hours of overtime for a few days, you come back that next day a little drained and less productive, but in a day or two you are back to normal.  If you work 6 hours of overtime for several weeks in a row, your productivity greatly decreases during that time (which impacts that amount of “good work” you can do) and when you come back the next day you still have not fully recovered.  It takes several days or even weeks to get back to your original level of productivity, even when you are only working 8 hours a day regular time.  pmBLOX incorporates this burnout effect.  So, as tasks fall behind schedule in the simulation and resources are tapped to work overtime, there may be productivity losses, which makes work take longer than expected.

As another example, take a “senior” level designer and a “junior” level designer.  In the real world, these two people will not have the same level of productivity.  For a given amount of time, the senior level designer will produce more work than the junior level designer.  This cannot be accommodated in current PM tools without some manipulation or “gaming” of the software.

At this point, one of two things can happen with current planning tools.  With the most commonly used approach, the user does nothing to account for these productivity differences.  A designer is a designer is a designer.  They are all considered equal.  When the project plan is implemented in the real world, the differences in productivity will change the duration of the task depending on which designer is used.  Typically, because the junior designer is cheaper than the senior designer, the junior designer will be used in the real world and will be asked to keep to the schedule that was estimated based on the allocation of a senior designer (because it made the estimated schedule look better).  This is a recipe for disaster.

With the less-used alternative approach, the user can “game” the planning software and say that the junior designer only works 6 hours/day compared to the senior designer working 8 hours/day (to represent that the junior designer is only 75% as productive as the senior designer).  This may work for the execution of tasks based on hours, but now the user must change the hourly wage rate for the junior designer to reflect that each hour (based on a 6 hour day) is more expensive than the junior designer’s actual salary (based on a regular 8 hour day).  While it is possible to do all this with current tools, it is cumbersome and does not reflect reality.  pmBLOX makes this process of assigning productivity levels much easier.

Since you mention Monte Carlo simulation specifically, let me make a few comments about the Monte Carlo approach.  In reality, the Monte Carlo approach is not a simulation, but it is an analytical approach applied to some form of simulation.  So, for instance, MS Project (or Primavera or any other PM tool on the market) “simulates” a project through the use of a database/spreadsheet methodology.  The Monte Carlo approach simply changes a few input variables and re-runs the “simulation” to get a different set of results.

All of this could be done manually by a user, but a Monte Carlo tool automates this process and allows the simulations to occur thousands of times very rapidly.  The Monte Carlo approach is typically used when there is a fair amount of uncertainty in input parameters (e.g., task duration).  The output of a Monte Carlo analysis is a range of results (e.g., a range of project timelines) with a level of confidence for the most probable results (e.g., the most likely project timeline).

Thus, the Monte Carlo approach could be used with pmBLOX, too.  In fact, we already have that feature scheduled for a future version of pmBLOX.  As stated, current Monte Carlo approaches to CPM/PERT vary the inputs on task durations, which are actually an output of all of the variables accounted for in the pmBLOX model.  Therefore, you are varying real-world outputs, not inputs.

With the pmBLOX approach, Monte Carlo analysis will entail varying true inputs:  resource availability, hours backlog, productivity, etc.  This makes the pmBLOX Monte Carlo analysis far more valuable to the user, because it will show the user which of these input variables have the most impact on a project, allowing the user to foresee pitfalls and construct management policies to minimize them.


Q.  You talk about having “every task embedded with a complete set of logic and rules that define how it is performed”.  That sounds like a lot of overhead – kind of like figuring out what every possible variable might be up front.  How do you make sure you create all of the right rules within a reasonable period of time? 

A.  Actually, the pmBLOX simulation runs extremely fast and has a very small memory footprint.  It’s not a lot of overhead because the underlying simulation models are quite basic, and all variables are not figured out up front.  Variables are calculated only when the user runs a simulation.

When we say that every task is embedded with a complete set of logic and rules, we mean that each task has an operational simulation model that mimics activities and decisions that would typically occur in the real world.  In the opening paragraphs, I mentioned that the underlying approach is hours-based and uses the system dynamics simulation methodology.  The amount of work completed for a particular task in any time step of the simulation (e.g., each hour of the day) is basically the product of the number of people assigned to the task, the number of hours those people work in a day, and the productivity of those people during that time.  Three people working on a task for 6 hours/day at a productivity level of 50% would complete 9 hours of task work (3 * 6 * 0.50 = 9).  This is what we call the task execution simulation.

A key factor here is that the underlying system dynamics methodology is very “structural” in nature and not incredibly data-intensive.  As a result, the task model in pmBLOX uses data that a good PM should already be using to make an estimate, no matter what tool they are using.  pmBLOX just makes those fundamental inputs explicit and open to discussion.

On top of the task execution simulation, we add real-world management decision making through several “management policies”.  These management policies are a way for the user to incorporate their management style or decision preferences.  For example, we give the user the ability to change the number of hours any resource works on a task and the ability to change the number of resources assigned to a task, or a combination of both, based on “schedule pressure” or “cost pressure” experienced on the project.

This is another area where the system dynamics methodology provides power.  The system dynamics methodology can incorporate feedback and non-linear relationships among variables.  In the real world, these are what drive changes.  A small change in X may lead to a small change in Y.  But a slightly larger change in X may result in a huge change in Y (i.e., non-linearity.  And, the change in Y may come back around again to influence X (i.e., feedback).

pmBLOX uses the traditional Earned Value calculations to determine the status of a task.  With EV, the two key parameters are the Schedule Performance Index (SPI, schedule pressure) and the Cost Performance Index (CPI, cost pressure).  The pmBLOX user can set a management policy that says that as a task falls behind schedule (i.e., SPI drops below 1.0), assigned resources will work 1 hour of overtime.  As the task falls very far behind schedule (i.e., SPI drops way below 1.0), the policy may state that the assigned resources will work 3 hours of overtime.

Now, as conditions are experienced in the simulation of the project, whether or not the assigned resources work overtime will depend on how far behind schedule a particular task falls.  If conditions end up being such that the task is on time with assigned resources working regular hours, then no overtime is allocated.  However, if for some reason the task falls behind schedule (perhaps due to certain resources being allocated to other tasks for a portion of the time), then the management policy will “kick in” and the assigned resources will work overtime to try to get the task back on schedule.

Each user can set his/her own policies.  One user may have an approach in which overtime is assigned in proportion to the lateness of a task:  the further behind a task is, the more hours of overtime are assigned.  Another user may have an aggressive approach that immediately works resources several hours of overtime as soon as a task is even just slightly behind schedule.  It is completely the user’s choice.

In addition to these simulation methods, pmBLOX incorporates some basic “common sense” approaches that have been lost in some of the current planning tools.  For instance, in MS Project, suppose a task specifies a resource to be used at 50% level of effort.  If that resource works 8 hours/day, this equates to 4 hours/day.  If that resource is only available 3 hours on a given day, MS Project will not use that resource on the task and the resource will sit idle, which delays the task completion.  In the real world, that resource would be used for the 3 hours that it is available.


Q.  What do you mean when you say that “individual tasks actually manage themselves in relation to the entire project”?

A.  In PM tools on the market today, any calculations are “static”.  That is, they do not change as the PM tool generates a project plan.  With pmBLOX, the activities of the first time step influence the activities of the second time step, which influence the activities of the third time step, and so on.  As a result, activities can change throughout the simulation.  In other words, the simulation is “dynamic”.

For example, a specific task may not have a required resource available initially.  In the first few time steps, this may not change the priority or schedule pressure for the task because the due date may be several weeks away.  However, as the simulation progresses through more time steps and the required resource is still not available, the priority and schedule pressure for the task increase to the point where the required resource may be pulled off another activity to come work the high priority task to get it back on schedule.

This dynamic re-allocation of resources occurs in real life, but cannot be simulated with current planning tools.  In current planning tools, the user specifies that the resource is either available or unavailable; the resource cannot dynamically change availability.  So, in the case of the simulation in pmBLOX, the task has “managed itself” to try to achieve its scheduled end date by changing its priority level and pulling resources from other lower-priority tasks.  Note that this is an example of feedback:  the task has a goal and makes changes as necessary to achieve the goal.


Q.  What skills and knowledge enable a user to effectively use pmBLOX?  Do you need a lot of experience with the particular type of project you are facing?  Do you need to be a statistician or math guru?

A.  No special computer skills are needed.  As with current planning tools, only basic PC/Windows skills are necessary.  Users do not have to be statisticians or math gurus.  In fact, the user does not even have to know about the simulation.  We have done our best to make pmBLOX look like any traditional PM software with a Gantt chart, start/end dates, etc.

Because it is a simulation-based tool and there are additional inputs to use the power of the simulation, the interface is slightly different, but not much.  Since our focus is to extend the capabilities of MS Project, any MS Project user should have no trouble navigating and using pmBLOX.  Users can import MS Project files directly and immediately run a simulation.

It should be noted that, as with any project planning tool, actual PM expertise helps.  If someone is new to the field of project management and is not familiar with the estimation process, resource allocation, task dependencies, etc., that person will have a difficult time using pmBLOX to its fullest potential, just as that person would have trouble with any planning tool.  Seasoned PM’s who already know how to build a reasonable project plan and manage a project will appreciate the new power and capabilities available to them with pmBLOX.


Q.  Which types of projects are appropriate for this software?  Do they have to be highly iterative so that the modeling can improve over time – or does it help with one-offs as well?

A.  pmBLOX is appropriate for all types of projects, whether they are repetitive or one-time.  The appropriateness of pmBLOX is more tied to the size and complexity of the project.  For small projects, pmBLOX may be a bit much.  pmBLOX can definitely be used on small projects, but the simulation approach will yield no better results than a spreadsheet.

pmBLOX’s “sweet spot” is medium to large projects where traditional PM tools tend to be limited due to overwhelming complexity.  With these larger projects, someone using a traditional PM planning tool would need to “work around” some of the constraints of the tool or even employ a few “tricks” to fool the software into providing a certain result.  The simulation approach in pmBLOX can handle the complexity and provide much more realistic projections of task completion, resource allocation, etc. without the user having to “game” the software.


Q.  Are there particular industries where this software is more effective?  Are there industries where this software doesn’t work well at all?

A.  Most of the attention pmBLOX has received so far has been from the construction/infrastructure industry, but pmBLOX is not limited to that industry at all.  As an example, one of the industries in which pmBLOX has huge potential value is the IT/software development industry.

With software-related projects, there is a common phenomenon of the “mythical man-month” that cannot be accounted for with current planning tools.  PM’s who have managed software-related projects can tell the stories of how adding people to late projects only makes the projects even later because experienced people have to bring new people up to speed, new people make mistakes that require rework, etc.  With the management policies mentioned previously, the user can designate when to add people to a task due to schedule pressure, as well as the productivity losses due to “overmanning”.  This is much closer to the real world.  There is no such thing as a free lunch.

Just like making people work large amounts of extended overtime can have productivity losses due to fatigue and burnout, adding people to a task can also have productivity losses as everyone “steps on each others’ toes”.  pmBLOX is the only tool available that incorporates these effects.  And, because of pmBLOX’s ability to include non-linear relationships, these types of productivity losses can mirror reality.  When one person is added to a task, the productivity loss may not be much.  However, by adding just another one or two people, the productivity for the whole group typically may suffer a precipitous drop.


Q.  Can you give me an interesting example of a time when this approach was particularly effective?

A.  We took an example engineering project plan right off of the MS Project website.  It was fairly complex with several hundreds of tasks, so it was a good test for pmBLOX.  The total project time was estimated at about 3 years in MS Project.  We imported the file and the pmBLOX simulation showed the project to take about 5 years.  In MS Project, we level-loaded the resources for the engineering project and the timeline pushed out to about 8 years in MS Project after the level-loading.  A very big difference compared to the original file.  We took the level-loaded project and imported it into pmBLOX.  The pmBLOX simulation of the level-loaded project gave exactly the same timeline as the previously imported file:  5 years.

Current PM tools are either overly “optimistic” and provide a short timeline that is completely unattainable but looks good on paper, or current PM tools are overly “pessimistic” and show long timelines that no one would ever accept and in real life would never occur because resources would be shifted along the way.  We like to say that pmBLOX is “realistic”.

A final question that you may still have is:  so what makes the SimBLOX company uniquely qualified to challenge the status quo with project management tools with pmBLOX?

The pmBLOX product that is currently available for community preview is the maturation of a concept that two of the SimBLOX partners originally worked on over 10 years ago for the Department of Defense. At that time, we helped create an advanced project management simulation, using system dynamics as the foundation, for DoD aircraft manufacturers and shipyards.  Thus, it was geared toward very experienced PM’s in the defense industry.  That tool required a higher level of expertise and honestly was priced well beyond what most companies could afford to pay.

After seeing no fundamental changes in PM tools for the last decade, we applied for and received an SBIR (Small Business Innovative Research) grant from DARPA to create a next-generation simulation-based project management tool for a broader commercial market.  Our DARPA customer is actually an ex-Microsoft executive and knows that that field of PM has not advanced much in the last few decades.  pmBLOX is the result of our Phase I SBIR effort and we have been awarded a Phase II SBIR contract to continue development of pmBLOX.
Posted on: January 06, 2009 05:19 PM | Permalink

Comments (3)

Please login or join to subscribe to this item
Very interesting, it looks very similar to Critical Chain Method. The biggest weakness of CCM was the estimation of the buffers, it seems that this software could help with it. I think the difference is that in CCM, the complexity of statistical fluctuations - whatever they are - are implement through final project buffer rather then on the task level. But I think it is worthwhile reviewing and testing - especially on software projects where statistical fluctuations are enormous.

Good insight into usage of simulation in PM tool to make plans more realistic. I have two questions
[1] Can I use pmBLOX to manage a program (inter-dependant multiple projects)? Typically when we manage a mid-size program, the data we use are: resources available, possible locatioms, hourly rates, work schedules, reporting relationships, basic work process, product vision, and deliverables.
[2] We work as management consultants and the effort required for similar type of work vaires from client to client, culture to culture, so we are working on to create a standard multi-variable estimation process. It is not like typical software estimations. Can pmBPLOX simulate & help in validating effort estimations?

Regards,
Dipanker

Dipanker,

The short answer to both of your questions is Yes. I am managing the development effort on this tool for ViaSim Solutions right now and I would welcome the chance to talk with you about your questions.

Regards,

Curt

Please Login/Register to leave a comment.

ADVERTISEMENTS

I'm lactose intolerent. I have no patience for lactose and I won't stand for it.

- Jerry Seinfeld

ADVERTISEMENT

Sponsors