If you are reading this, then it is very likely that you have already been estimating software projects or you are about to estimate one. Making an estimation is a complex undertaking. In many cases, this is because we misunderstand what the executive (managers and stakeholders) making the requirement is looking for.
While we are probably trying to provide a list of features with their approximate delivery times, the executive is looking for a commitment or a plan to meet a specific target.
In this context, usually we end up providing a fixed time, which the executive takes as the date that the project will be finished (i.e. a commitment). Such situations may result in a possible failed project or in a budget overrun. I’m pretty sure that most of you are already thinking: “that’s exactly what happened!”. Well, let me tell you that you are not alone and we are here to share some of our experiences, which may give you a couple of tips on how to provide good estimates.
During our time as estimators we found Steve McConnell’s book, “Software Estimation: Demystifying the Black Art” invaluable. Thus, this article uses the learnings from the book to explain some important aspects of estimation, and summarizes some of the key points that McConnell makes.
Let’s start by defining the word “estimate”. According to the dictionary Merriam-Webster, it is “to determine roughly the size, extent, or nature of”. Therefore, in software, an estimate is a prediction of how long a project will take or cost. It serves as an input to build a plan to reach a target that may lead to a commitment. A commitment is a promise to deliver a specific group of features at a predefined level of quality by a certain date.
Never confuse an estimate with a commitment or a target. A target is an objective, something we want to achieve, for example if you are asked to estimate a project that must be presented by a certain date. Then in such a case your target is: have the project ready for a certain date. Your estimation must help to prepare a plan to achieve the target. A plan can start by asking the client, if it is possible to remove some features, or if they can prepare a list of priorities, so we can deliver the most important features on time. Once your plan is ready, then you can make a commitment with the things that can be achieved until the specified date. It is very important to recognize the mismatch between a project’s business target and a project’s estimate, as this may define whether the project is successful or not.
“An estimate is the most optimistic prediction that has non-zero probability of coming true”
— Juan Urrego
One of the most valuable things (if not the most) that we have learned, is how important it is to always keep a range or a probability for the estimations. For example:
This is because if you provide a single-point estimate, you are assuming that there is a 100% probability that the actual outcome is equal to the planned outcome, which is not realistic. In general, a single-point estimate is actually a target masquerading as an estimation. Such estimates usually lead to failure projects, because executives will make a commitment using that input as a basis.
Moreover, at the moment of estimating, never forget that you should include time for requirements and design. In fact, you should remember that in most cases, we as developers (who estimate), tend to underestimate between 20% to 30%. So the first thing to do is to start overestimating a little bit.
Most executives fear that if the project is overestimated, developers will find something to do with the extra time (Parkinson’s law) or will procrastinate a lot and rush at the end (Student’s syndrome). Such fear makes them narrow the estimate even more to avoid such situations.
However, as mentioned before, developers almost always generate a too-optimistic schedule (in particular this was emphasized by Chris Peters, Microsoft’s 105th employee). So if you are an executive, don’t “fear” the estimate you are receiving from the development team, and definitely don’t reduce it further.
Besides that, you have to remember, that whenever your project is late, some activities will appear that will drain more time – such as more meetings with upper management, apologizing to customers, preparing more internal releases to support customer demos, fixing problems arising from improvised hotfixes. These activities are not supposed to appear if the project is on time or meeting its goal. Furthermore, with large projects there is less chance of completing on time, because they include more activities, which we will mention in detail later on.
To conclude, try to always avoid underestimating or overestimating. However if you cannot estimate with complete accuracy, make sure your process leans towards overestimating, as the penalties (i.e. Parkinson’s law, Student syndrome) are not as severe as with underestimation (i.e. budget overrun).
As individuals take decisions during the execution of a project, the uncertainty of the estimation starts to decline. Such behaviour led researchers to discover certain stages where there are predictable amounts of uncertainty. Hence, the cone of uncertainty represents such stages as the best-case accuracy. It is possible to do worse, but it is most likely you won’t be more accurate. What you can do is to force the cone to narrow by removing sources of variability from your project (what the project will deliver, and what it won’t).
The cone of uncertainty
Software organizations commonly sabotage their own projects by making commitments too early in the cone of uncertainty. Let’s say the company commits at the product definition stage, that would mean an error factor of 2x which no project manager would be able to handle. Therefore, one must delay commitments as far as possible to the wide part of the cone.
Moreover, you have to remember that for every new requirement, cost and schedule will be estimated from that baselined set of requirements. As new requirements are added or old requirements are revised, cost and schedule estimates will be modified to reflect those changes.
The following are key sources of estimation errors:
We already mentioned a couple of times that studies have found that developers tend to underestimate most of the time. One of the important factors of such studies is that developers tend to forget 20% to 30% of necessary tasks (software and non-software).
Among such tasks, Steve McConnell highlights the following in particular:
Non-software activities can be excluded for small projects, but are important to keep in mind for longer ones. Forgetting such activities may lead to a tight schedule which will generate deadline pressure. Furthermore, developers tend to commit more errors when they are under stress, which is normally caused by a tight schedule. This last aspect, is caused by quick-and-dirty fixes deployed to avoid spending time in thinking in correct solutions. Deadline pressure is the single greatest enemy of software engineering. Excessive or irrational schedules are probably the single most destructive influence in all of the software.
As we have pointed out, executives want to make decisions about the project scope and this usually happens during the first stages of the Cone of Uncertainty. A good estimator will refuse to provide a highly accurate estimate and will clearly state that a high-level estimate is the only way to proceed. In this context, executives or stakeholders are actually asking for a classification of the features to sort them into a rough cost/benefit order.
A very useful approach, called T-Shirt sizing, comes handy in these kind of situations. Developers have to classify each feature in T-shirt sizes (i.e. small, medium, large and extra large). And at the same time, stakeholders have to classify features by business value on the same scale. Such a process will lead to entries combined in both aspects.
Example of T-shirt sizing estimation
A T-shirt classification is incredibly helpful, because it makes clear which set of features can provide more value to the product with the least cost. Look at the table for example, at first sight a stakeholder may think that Feature 2 is very easy because it has small business value.
However, when developers add the development cost, it is easier to find out that it isn’t cost justified and thus can be moved to the end of the backlog. This a very helpful technique that can be used at the wide part of the Cone of Uncertainty in order to provide valuable information for management to make decisions.
Some of the most common estimation techniques include intuition, guessing, and memory based on past projects. These techniques usually produce poor results. Moreover, they are correlated with cost and schedule overruns. On the contrary, some of the best results are obtained when you base your estimations on historical information. So, if you haven’t been recording information from past projects, you should start now, even if you just start with lines of code per project. Among the multiple benefits that you will find, the following are the most relevant:
For a long time, there has been a major problem with communication between those estimating (developers) and executives. Executives usually ask for estimations, but what they really want is a plan to achieve certain targets. This is something estimators don’t usually consider when preparing the estimation. Thus, the first step to reduce the gap in communication, is to differentiate between estimations, targets and commitments.
An estimation is a tool that helps leaders prepare a plan in order to achieve a target. With the plan ready we can present it to the executives, so they can make a commitment. You can negotiate the commitment, but don’t negotiate the estimate.
Moreover, there are always a lot of error sources in estimations, like omitted activities, lack of information (or inaccurate information) about the features, the team, that we should always remember and avoid. Continuing, we should be always aware of the project stage in the cone of uncertainty, to realize if it is possible to make a commitment or not.
A final important aspect is that estimates should be based on historical data in order to be more accurate. For that, during a project, make sure your underlying assumptions remain true (or have a low percentage risk of changing).
For example, if you start a mobile app project assuming users will login via Facebook, but in the end you use email/password, this will have knock-on effects. In terms of estimating, if you have estimated the project with an accuracy of +-5%, but it changed by 100%, then it is not going to provide you much value.
We are just touching the surface of all the topics related to estimation. There is much more to learn, experience and apply. But we hope this article provides you with some tips, best practices, and explains some of the benefits of good estimates.
Finally, always remember, don’t use precision and accuracy as synonyms in estimates. It is always better to provide accurate dates rather than precise ones.
“Software Estimation: Demystifying the Black Art” by Steve McConnell
February 13 / 2020
December 17 / 2019
Digital technology enables organizations to boost productivity and better engage with customers. However, many technology initiatives continue to see cost overrun, damaging the bottom line and undermining the value...Read post