I would like to know how you go about validating some technical estimate/quotation in a technology that you don't have so much technical knowledge of. Do you follow any methodology? Do you use any basis to validate the pointed data? Do you use historical data?
Practical example: I have a quote for developing a new module on an "Internal Site". I have no technical development experience and my IT team does not have a developer.
As I did in some situations: I have a compilation of all the improvements already applied previously and I make an association by similarity. Example: At the beginning of the year, a similar module was implemented for x hours, so the estimate has to fluctuate in these x hours.
This format gives a good view but I wanted to find some way to improve the accuracy of this validation.
Any tips, experiences, and contributions will be VERY welcome :) Saving Changes...
in the absence of expert judgment (related to the specific context of the project in question) or sufficiently similar historical data, you won't be able to improve estimation accuracy so a better approach might be to make small bets by using an adaptive approach where you try to surface and tackle the areas of greatest uncertainty early on to increase confidence in the team's approach.
Kiron
...
2 replies by Jose Germano
Aug 30, 2022 8:43 PM
Jose Germano
...
Thank you very much for the feedback and considerations, Sergio.
Aug 30, 2022 9:06 PM
Jose Germano
...
Thanks so much for the feedback, Kiron. I will try to delve deeper into this adaptive approach.
I usually ask the estimator to provide assumptions, risks, contingencies that are built in or associated with the estimate.Then I ask questions until I am satisfied that the estimate is reasonable. You do ask for an estimate range, right?
...
1 reply by Jose Germano
Aug 30, 2022 9:03 PM
Jose Germano
...
Thank you very much for your feedback, Stephane. And yes, I always consider a margin in the estimate I get based on scope, as as we evolve with alignments, scope is refined and can fluctuate.
If you have historical data, you may as you stated use it as a basis of comparison. In that case, I tend to find a typical range (mean value and standard deviation). If an estimate is within that range, about all the comparison can tell you is it is consistent with previous performance.
The other thing you might consider is a non-advocate review. This is where you bring in some independent reviewers with more technical knowledge to ask the more difficult questions. If they don't have an existing bias such as "I made this estimate myself so I'm pretty sure it's accurate." then they are more likely to evaluate the inputs objectively.
...
1 reply by Jose Germano
Aug 30, 2022 9:00 PM
Jose Germano
...
Thank you very much for the feedback and details on the use of comparisons in the estimate. I will also apply the independent reviewers
TLDR - the primary factors to improve the accuracy of your estimates, especially on unique project work, are time and effort. I would call out "understanding the work" in that statement, but understanding the work SHOULD follow time and effort.
I'm going to sound redundant, but make sure your estimates include a range. I'd go so far as to say that if you're not providing a range, you're not providing an estimate; your audience will most likely assume the same. I'm not saying you shouldn't calculate a deterministic estimate, I'm saying you should communicate a probabilistic estimate.
Here are some (oversimplified) approaches to estimating:
- Top Down estimating is helpful when you're doing something similar to something you've done before. That is what you're describing in your scenario.
- You can use Bottom Up estimating once the tasks have been identified. The sum of the tasks gives you the overall estimate.
- Three-point estimating (PERT) also requires that you know the tasks and then calculate an estimate based on the pessimistic, optimistic, and most likely estimates.
I will typically use more than one approach to estimating, on a project. If you're not familiar with the Cone of Uncertainty, you should look into it.
At the beginning of a project, you may not know much about it. If it's truly a unique endeavor, you don't know much about the tasks, either. Unfortunately, leadership wants to know how long the project will take and how much it will cost before you start. This is where Top Down estimating CAN help. But, this is a ROM (rough order of magnitude) estimate. On average, ROM estimates are thought to have the potential for 100% variance: -25% to +75%. So, the range of variance for a ROM estimate for a $10,000 project would be $7,500 to $17,500. You improve the estimate, and the variance, by learning more about the scope and tasks of the project, reducing uncertainty over time.
Key points for improving your estimates are:
- once requirements are done
- once the WBS is complete and tasks have been estimated (Bottom Up)
- as you get closer to finishing testing (on a software project - defects can really blow up your schedule)
If you're using Scrum or Rolling Wave Planning/Progressive Elaboration, you are effectively stretching out the cone of uncertainty. You can estimate a few sprints or your current wave to a low degree of uncertainty, but the rest is still a ROM estimate, at best.
...
1 reply by Jose Germano
Aug 30, 2022 8:58 PM
Jose Germano
...
Thank you so much for all the detail and guidance. It was practically an estimation class. Excellent insights.
Saving Changes...
Thomas WalentaGlobal Project Economy ExpertHackenheim, Germany
Jose,
you might have a look at the nine estimating methods the PMBoK Guide ed7 lists. I found it always enlightening to use different methods for estimating.
Also, make sure you can reassess estimates as you gain more information during the project.
Thomas
...
1 reply by Jose Germano
Aug 30, 2022 8:46 PM
Jose Germano
...
Thank you so much for the feedback and interaction, Thomas. I noted here in my PMBOK studies to validate the estimation methods.
Saving Changes...
Sergio Luis ConteHelping to create solutions for everyone| Worldwide based OrganizationsBuenos Aires, Argentina
If you do not have somebody technical that help on that you are in trouble. Find somebody including it if it has an associated cost. Then evaluate the cost/benefit relation and go ahead. If it is not possible then you have a risk. Just let me say that is not evaluate the cost. Is about evaluate the whole solution then the cost. Saving Changes...
in the absence of expert judgment (related to the specific context of the project in question) or sufficiently similar historical data, you won't be able to improve estimation accuracy so a better approach might be to make small bets by using an adaptive approach where you try to surface and tackle the areas of greatest uncertainty early on to increase confidence in the team's approach.
Kiron
Thank you very much for the feedback and considerations, Sergio. Saving Changes...
you might have a look at the nine estimating methods the PMBoK Guide ed7 lists. I found it always enlightening to use different methods for estimating.
Also, make sure you can reassess estimates as you gain more information during the project.
Thomas
Thank you so much for the feedback and interaction, Thomas. I noted here in my PMBOK studies to validate the estimation methods. Saving Changes...
TLDR - the primary factors to improve the accuracy of your estimates, especially on unique project work, are time and effort. I would call out "understanding the work" in that statement, but understanding the work SHOULD follow time and effort.
I'm going to sound redundant, but make sure your estimates include a range. I'd go so far as to say that if you're not providing a range, you're not providing an estimate; your audience will most likely assume the same. I'm not saying you shouldn't calculate a deterministic estimate, I'm saying you should communicate a probabilistic estimate.
Here are some (oversimplified) approaches to estimating:
- Top Down estimating is helpful when you're doing something similar to something you've done before. That is what you're describing in your scenario.
- You can use Bottom Up estimating once the tasks have been identified. The sum of the tasks gives you the overall estimate.
- Three-point estimating (PERT) also requires that you know the tasks and then calculate an estimate based on the pessimistic, optimistic, and most likely estimates.
I will typically use more than one approach to estimating, on a project. If you're not familiar with the Cone of Uncertainty, you should look into it.
At the beginning of a project, you may not know much about it. If it's truly a unique endeavor, you don't know much about the tasks, either. Unfortunately, leadership wants to know how long the project will take and how much it will cost before you start. This is where Top Down estimating CAN help. But, this is a ROM (rough order of magnitude) estimate. On average, ROM estimates are thought to have the potential for 100% variance: -25% to +75%. So, the range of variance for a ROM estimate for a $10,000 project would be $7,500 to $17,500. You improve the estimate, and the variance, by learning more about the scope and tasks of the project, reducing uncertainty over time.
Key points for improving your estimates are:
- once requirements are done
- once the WBS is complete and tasks have been estimated (Bottom Up)
- as you get closer to finishing testing (on a software project - defects can really blow up your schedule)
If you're using Scrum or Rolling Wave Planning/Progressive Elaboration, you are effectively stretching out the cone of uncertainty. You can estimate a few sprints or your current wave to a low degree of uncertainty, but the rest is still a ROM estimate, at best.
Thank you so much for all the detail and guidance. It was practically an estimation class. Excellent insights. Saving Changes...