In Software Project Estimation: The Budget, Estimate, Plan Continuum we defined a numerical continuum that makes up estimation. There are numerous specific techniques for generating budgets, estimates and plans. The techniques can be sorted into three basic categories. Hybrids exist that the leverage components of each.
Expert techniques use the judgment, generally based on experience, of an individual to determine the cost, duration or effort of a project. The primary strengths of an expert approach is that it can be developed relatively quickly and is championed by a person who has developed a high level of organizational trust. The obvious weakness of these techniques is the reliance on an individual with all of his or her biases. Dr. Ricarado Valerdi in SPaMCAST 84 noted his research has found that IT personnel are notoriously poor estimators. One of the reasons cited in the interview was that IT personnel are generally overly optimistic of their problem solving ability. Techniques such Delphi and Planning Poker use multiple experts as a technique to fight individual bias by using collaboration in an attempt to triangulate on the better answer. Developing an estimate or budget leverages past performance on a specific project to anchor the estimator(s) memory, and then uses judgment to determine how much one project will be like another. Expert techniques make the most sense when there is little or no data to base the prediction, for instance when a budget is being developed.
The second technique is parametric estimation. Parametric estimation is generally an estimation technique (as opposed to a budget or a plan, although many commercial products also include planning features) that generates an estimate based on historical data of productivity, staffing and quality that is used to create a set of equations. These equations are then fed information about the size of the project (IFPUG Function Points for example), project complexity and the predicted capabilities of the team. Tools like SEER-SIM and COCOMO II are parametric estimation tool. The strengths of parametric estimates are derived from the historical performance data they use to generate the estimates and the enforced rigorous estimation process. The weakness of any parametric based estimation model is that they require the estimator generate, or have access to, a numerical size which can add overhead to the project or take time that would be better spent building the software. We have discussed the fallacy of these issues in the discussion of IFPUG function points. A bigger issue exists when there is no historical data that can be used to generate the productivity equations. When no data exists I would recommend seeking external data (many firms, including the David Consulting Group – my day job – and ISBSG can sell or help you with this issue). When no trustworthy data exists, parametric estimation does not make sense.
Work breakdown structures are the third category. This category is generally used for planning, and some cases, as means of building a bottom-up estimate. In this category a planner or team will generate a list of tasks need to complete the job. The level of granularity of the tasks can vary greatly – I had a colleague that planned tasks at an hourly increment. Constraints, staffing and sequence can be added to the plan to generate schedule. The sprint backlog used in Scrum is a form of this technique. The power of the techniques are derived from a focus on what is to be done, by whom and when at an actionable level of detail. The problem is that you need an incredible amount of information about the project and project team to be able to generate an accurate task list let alone an accurate project schedule. It is well known that the amount of data needed for this technique is generally only accurately known over short time horizons. However, I have seen processes that require detailed schedules for long projects up to a year before they are scheduled to start. These techniques are best use for deriving short term plans.
Most IT organizations tend to fixate on one of these categories of techniques however organizations that understand differences between a budget, an estimate and a plan will use techniques from all three categories. Using the data and knowledge gained from using each tool or technique as a feedback loop to improve the performance of all the techniques the use. For example, an organization I recently spoke with uses both parametric and expert techniques to generate estimates on critical projects. Both techniques cause the estimation team to surface different assumptions that need to be understood when deciding which work can be done and how much money to ask for from the business.
Tcagley Blog
