Subscribe to our Newsletter

Receive our news and insights

Blog / Software development  

Solving some of the enigmas in software estimation

A Picture of Luis Talavera
By:
January 08, 2020 | Topic: Software development  
Solving some of the enigmas in software estimation

Introduction

If you are reading this, then it is very likely that you have already been estimating software projects or you are about to estimate one. Making an estimation is a complex undertaking. In many cases, this is because we misunderstand what the executive (managers and stakeholders) making the requirement is looking for.

While we are probably trying to provide a list of features with their approximate delivery times, the executive is looking for a commitment or a plan to meet a specific target.

In this context, usually we end up providing a fixed time, which the executive takes as the date that the project will be finished (i.e. a commitment). Such situations may result in a possible failed project or in a budget overrun. I’m pretty sure that most of you are already thinking: “that’s exactly what happened!”. Well, let me tell you that you are not alone and we are here to share some of our experiences, which may give you a couple of tips on how to provide good estimates.

During our time as estimators we found Steve McConnell’s book, “Software Estimation: Demystifying the Black Art” invaluable. Thus, this article uses the learnings from the book to explain some important aspects of estimation, and summarizes some of the key points that McConnell makes.

Estimations, commitments and targets

Let’s start by defining the word “estimate”. According to the dictionary Merriam-Webster, it is “to determine roughly the size, extent, or nature of”. Therefore, in software, an estimate is a prediction of how long a project will take or cost. It serves as an input to build a plan to reach a target that may lead to a commitment. A commitment is a promise to deliver a specific group of features at a predefined level of quality by a certain date.

Never confuse an estimate with a commitment or a target. A target is an objective, something we want to achieve, for example if you are asked to estimate a project that must be presented by a certain date. Then in such a case your target is: have the project ready for a certain date. Your estimation must help to prepare a plan to achieve the target. A plan can start by asking the client, if it is possible to remove some features, or if they can prepare a list of priorities, so we can deliver the most important features on time. Once your plan is ready, then you can make a commitment with the things that can be achieved until the specified date. It is very important to recognize the mismatch between a project’s business target and a project’s estimate, as this may define whether the project is successful or not.

“An estimate is the most optimistic prediction that has non-zero probability of coming true”

— Juan Urrego

Overestimation vs underestimation

One of the most valuable things (if not the most) that we have learned, is how important it is to always keep a range or a probability for the estimations. For example:

  • We are 90% sure that we can complete this project in 26 weeks
  • We estimate that the best case is 17 weeks and the worst case is 25 weeks
  • We estimate between 18 to 24 weeks

This is because if you provide a single-point estimate, you are assuming that there is a 100% probability that the actual outcome is equal to the planned outcome, which is not realistic. In general, a single-point estimate is actually a target masquerading as an estimation. Such estimates usually lead to failure projects, because executives will make a commitment using that input as a basis.

Moreover, at the moment of estimating, never forget that you should include time for requirements and design. In fact, you should remember that in most cases, we as developers (who estimate), tend to underestimate between 20% to 30%. So the first thing to do is to start overestimating a little bit.

Most executives fear that if the project is overestimated, developers will find something to do with the extra time (Parkinson’s law) or will procrastinate a lot and rush at the end (Student’s syndrome). Such fear makes them narrow the estimate even more to avoid such situations.

However, as mentioned before, developers almost always generate a too-optimistic schedule (in particular this was emphasized by Chris Peters, Microsoft’s 105th employee). So if you are an executive, don’t “fear” the estimate you are receiving from the development team, and definitely don’t reduce it further.

Besides that, you have to remember, that whenever your project is late, some activities will appear that will drain more time – such as more meetings with upper management, apologizing to customers, preparing more internal releases to support customer demos, fixing problems arising from improvised hotfixes. These activities are not supposed to appear if the project is on time or meeting its goal. Furthermore, with large projects there is less chance of completing on time, because they include more activities, which we will mention in detail later on.

To conclude, try to always avoid underestimating or overestimating. However if you cannot estimate with complete accuracy, make sure your process leans towards overestimating, as the penalties (i.e. Parkinson’s law, Student syndrome) are not as severe as with underestimation (i.e. budget overrun).

The Cone of Uncertainty

As individuals take decisions during the execution of a project, the uncertainty of the estimation starts to decline. Such behaviour led researchers to discover certain stages where there are predictable amounts of uncertainty. Hence, the cone of uncertainty represents such stages as the best-case accuracy. It is possible to do worse, but it is most likely you won’t be more accurate. What you can do is to force the cone to narrow by removing sources of variability from your project (what the project will deliver, and what it won’t).

The cone of uncertainty

Source: http://www.agilenutshell.com/cone_of_uncertainty

  1. Initial Concept (0.25x – 4x) or (-75% to +300%)
  2. Approved Product definition (0.5x – 2x) or (-50% to +100%)
  3. Requirements Complete (0.67x – 1.5x) or (-33% to +50%)
  4. User Interface Design Complete (0.8x – 1.25x) or (-20% to +25%)
  5. Detailed Design Complete (0.9x – 1.10x) or (-10% to +10%)
  6. Software Complete

Software organizations commonly sabotage their own projects by making commitments too early in the cone of uncertainty. Let’s say the company commits at the product definition stage, that would mean an error factor of 2x which no project manager would be able to handle. Therefore, one must delay commitments as far as possible to the wide part of the cone.

Moreover, you have to remember that for every new requirement, cost and schedule will be estimated from that baselined set of requirements. As new requirements are added or old requirements are revised, cost and schedule estimates will be modified to reflect those changes.

Estimation errors sources

The following are key sources of estimation errors:

  • Inaccurate information about the project. Until you understand each specific feature in detail, you won’t be able to estimate accurately the cost of a project. It is not possible to estimate the amount of work required by something that has not been defined. Thus, you should try to get as much information as possible from the client and stakeholders, prepare a list of questions for them to solve in the next meeting, or in the worst case by email.
  • Inaccurate information about the capabilities of the team. This includes poor coding practices that give rise to extensive bug fixing, poor design that lead to numerous errors in the code, requirements that weren’t investigated very well in the first place, abandoning planning under pressure. In short, if you do not have (enough) information about the team, it is very hard to provide an accurate estimate because you are not aware if there is technical debt.
  • Too much chaos in the project. The uncertainty only narrows as you make decisions to eliminate variability. Designing the user interface, a complete flow of the application, defining the backlog, all help to reduce the risk of variability arising from misunderstood requirements. If the project is not well controlled, or if estimators aren’t very skilled, estimates can fail to improve. You have to remember that estimates have to narrow during the development process of the project.
  • Inaccuracies arising from the estimation process. The accuracy of the software estimate depends on the level of refinement of the software’s definition. The only way to reduce the variability in the estimate is to reduce the variability in the project. There are a lot of variables that we usually forget when we estimate, for example developers leaving or joining the team, vacations, sick days. With larger projects, such variables can have a big impact, so you need to take them into account.
  • Unwarranted precision. Most of the time people tend to treat accuracy and precision as synonyms. However for estimation purposes that’s not the case. While the former refers to how close a number is to the real value, the latter refers to how exact a number is. For example, an estimate that states you will require 341.8 days is precise but not very accurate. However, an estimate stating you will require 3 staff years is not very precise but could be accurate.

Omitted activities

We already mentioned a couple of times that studies have found that developers tend to underestimate most of the time. One of the important factors of such studies is that developers tend to forget 20% to 30% of necessary tasks (software and non-software).

Among such tasks, Steve McConnell highlights the following in particular:

  • Requirements. This can include setup/installation program, glue code needed to use third-party or open-source software, deployment modes, interfaces with external systems, portability, security, usability, responsiveness, performance, modifiability.
  • Software activities. This can include time to get new team members, mentoring new team members, management and meetings, deployment, installation, customization, requirements clarifications, maintaining the revision control system, supporting the build, maintenance required to run daily builds and smoke tests, creation of test data, management of beta test program, participation in technical reviews, integration work, processing and estimating change requests, coordination with subcontractors, technical support, performance tuning, learning new development tools, coordination between testers and developers, create and review user documentation, demos for customers or users, interaction with stakeholders, reviewing plans, estimates, architecture and designs.
  • Non-software activities: Vacations, holidays, sick days, training, weekends, company meetings, department meetings, installation/maintenance/problems with hardware or software.

Non-software activities can be excluded for small projects, but are important to keep in mind for longer ones. Forgetting such activities may lead to a tight schedule which will generate deadline pressure. Furthermore, developers tend to commit more errors when they are under stress, which is normally caused by a tight schedule. This last aspect, is caused by quick-and-dirty fixes deployed to avoid spending time in thinking in correct solutions. Deadline pressure is the single greatest enemy of software engineering. Excessive or irrational schedules are probably the single most destructive influence in all of the software.

T-shirt sizing

As we have pointed out, executives want to make decisions about the project scope and this usually happens during the first stages of the Cone of Uncertainty. A good estimator will refuse to provide a highly accurate estimate and will clearly state that a high-level estimate is the only way to proceed. In this context, executives or stakeholders are actually asking for a classification of the features to sort them into a rough cost/benefit order.

A very useful approach, called T-Shirt sizing, comes handy in these kind of situations. Developers have to classify each feature in T-shirt sizes (i.e. small, medium, large and extra large). And at the same time, stakeholders have to classify features by business value on the same scale. Such a process will lead to entries combined in both aspects.

Example of T-shirt sizing estimation

A T-shirt classification is incredibly helpful, because it makes clear which set of features can provide more value to the product with the least cost. Look at the table for example, at first sight a stakeholder may think that Feature 2 is very easy because it has small business value.

However, when developers add the development cost, it is easier to find out that it isn’t cost justified and thus can be moved to the end of the backlog. This a very helpful technique that can be used at the wide part of the Cone of Uncertainty in order to provide valuable information for management to make decisions.

Benefits of accurate estimates

Some of the most common estimation techniques include intuition, guessing, and memory based on past projects. These techniques usually produce poor results. Moreover, they are correlated with cost and schedule overruns. On the contrary, some of the best results are obtained when you base your estimations on historical information. So, if you haven’t been recording information from past projects, you should start now, even if you just start with lines of code per project. Among the multiple benefits that you will find, the following are the most relevant:

  • Improved status visibility: It is a great way to track your process, to see if everything is going as planned. Comparing the estimate versus the actual progress is very important to see how realistic the planning was. For example, if you are aware of how many user stories are finished, and the effort in each one of them, then you can predict how much work is left simply by looking at the backlog.
  • Higher quality: About 40% of all software errors are caused by stress. When developers are suffering from stress, approximately 4 times more issues are reported in a release in comparison to a project under less extreme pressure.
  • Better budgeting: It is very hard to forecast the cost of projects that don’t have accurate estimates. Most software projects tend to fail because in order to finish on time they start adding more resources to the project (exceed budget). This is not a great way of reducing time, because new communication channels become introduced into the project which leads to very little time gained and can even generate more delays .
  • Increased credibility for the development team: A project team that has good estimations improve their credibility within the organization. However, when things go wrong they are the first to be blamed by the organization.

Conclusions

For a long time, there has been a major problem with communication between those estimating (developers) and executives. Executives usually ask for estimations, but what they really want is a plan to achieve certain targets. This is something estimators don’t usually consider when preparing the estimation. Thus, the first step to reduce the gap in communication, is to differentiate between estimations, targets and commitments.

An estimation is a tool that helps leaders prepare a plan in order to achieve a target. With the plan ready we can present it to the executives, so they can make a commitment. You can negotiate the commitment, but don’t negotiate the estimate.

Moreover, there are always a lot of error sources in estimations, like omitted activities, lack of information (or inaccurate information) about the features, the team, that we should always remember and avoid. Continuing, we should be always aware of the project stage in the cone of uncertainty, to realize if it is possible to make a commitment or not.

A final important aspect is that estimates should be based on historical data in order to be more accurate. For that, during a project, make sure your underlying assumptions remain true (or have a low percentage risk of changing).

For example, if you start a mobile app project assuming users will login via Facebook, but in the end you use email/password, this will have knock-on effects. In terms of estimating, if you have estimated the project with an accuracy of +-5%, but it changed by 100%, then it is not going to provide you much value.

We are just touching the surface of all the topics related to estimation. There is much more to learn, experience and apply. But we hope this article provides you with some tips, best practices, and explains some of the benefits of good estimates.

Finally, always remember, don’t use precision and accuracy as synonyms in estimates. It is always better to provide accurate dates rather than precise ones.

References

“Software Estimation: Demystifying the Black Art” by Steve McConnell

The chatbot revolution

Related Services

EXECUTIVE INSIGHTS

Business  

The leaders we need to navigate the COVID-19 storm

By

April 23 / 2020

1 Stars2 Stars3 Stars4 Stars5 Stars
Loading...

As we gradually get used to our new COVID-19 reality, daily life from just a few weeks ago now feels like a lifetime away. For businesses this has created,...

Read post

HOT
TOPIC