Workload Estimation

Workload Estimation

Why Estimation is Important

To get a working software system in active use as quickly as possible requires not only planning the development, but also planning the documentation, deployment, and marketing. In a commercial project it also requires sales and finance. Without predictability of the development time, it is impossible to plan these effectively.

Good estimation provides predictability. Managers love it, as well they should. The fact that it is impossible, both theoretically and practically, to predict accurately how long it will take to develop software is often lost on managers. We are asked to do this impossible thing all the time, and we must face up to it honestly. However, it would be dishonest not to admit the impossibility of this task, and when necessary, explain it. There is a lot of room for miscommunication about estimates, as people have a startling tendency to think wishfully that the sentence:

I estimate that, if I really understand the problem, it is about 50% likely that we will be done in five weeks (if no one bothers us during that time).

really means:

I promise to have it all done five weeks from now.

This common interpretation problem requires that you explicitly discuss what the estimate means with your boss or customer as if they were a simpleton. Restate your assumptions, no matter how obvious they seem to you.

How to Estimate Programming Time

Estimation takes practice. It also takes labour. It takes so much labour it may be a good idea to estimate the time it will take to make the estimate, especially if you are asked to estimate something big.

When asked to provide an estimate of something big, the most honest thing to do is to stall. Most engineers are enthusiastic and eager to please, and stalling certainly will displease the stalled. But an on-the-spot estimate probably won't be accurate and honest.

While stalling, it may be possible to consider doing or prototyping the task. If political pressure permits, this is the most accurate way of producing the estimate, and it makes real progress.

When not possible to take the time for some investigation, you should first establish the meaning of the estimate very clearly. Restate that meaning as the first and last part of your written estimate. Prepare a written estimate by de-constructing the task into progressively smaller subtasks until each small task is no more than a day; ideally at most in length. The most important thing is not to leave anything out. For instance, documentation, testing, time for planning, time for communicating with other groups, and vacation time are all very important. If you spend part of each day dealing with knuckleheads, put a line item for that in the estimate. This gives your boss visibility into what is using up your time at a minimum, and might get you more time.

I know good engineers who pad estimates implicitly, but I recommend that you do not. One of the results of padding is trust in you may be depleted. For instance, an engineer might estimate three days for a task that she truly thinks will take one day. The engineer may plan to spend two days documenting it, or two days working on some other useful project. But it will be detectable that the task was done in only one day (if it turns out that way), and the appearance of slacking or overestimating is born. It's far better to give proper visibility into what you are actually doing. If documentation takes twice as long as coding and the estimate says so, tremendous advantage is gained by making this visible to the manager.

Pad explicitly instead. If a task will probably take one day - but might take ten days if your approach doesn't work - note this somehow in the estimate if you can; if not, at least do an average weighted by your estimates of the probabilities. Any risk factor that you can identify and assign an estimate to, should go into the schedule. One person is unlikely to be sick in any given week. But a large project with many engineers will have some sick time; likewise vacation time. And what is the probability of a mandatory company-wide training seminar? If it can be estimated, stick it in. There are of course, unknown unknowns, or unk-unks. Unk-unks by definition cannot be estimated individually. You can try to create a global line item for all unk-unks, or handle them in some other way that you communicate to your boss. You cannot, however, let your boss forget that they exist, and it is devilishly easy for an estimate to become a schedule without the unk-unks considered.

In a team environment, you should try to have the people who will do the work do the estimate, and you should try to have team-wide consensus on estimates. People vary widely in skill, experience, preparedness, and confidence. Calamity strikes when a strong programmer estimates for herself and then weak programmers are held to this estimate. The act of having the whole team agree on a line-by-line basis to the estimate clarifies the team understanding, as well as allowing the opportunity for tactical reassignment of resources (for instance, shifting burden away from weaker team members to stronger).

If there are big risks that cannot be evaluated, it is your duty to state so forcefully enough that your manager does not commit to them and then become embarrassed when the risk occurs. Hopefully in such a case whatever is needed will be done to decrease the risk.

If you can convince your company to use Extreme Programming, you will only have to estimate relatively small things, and this is both more fun and more productive.

How to Manage Development Time

To manage development time, maintain a concise and up-to-date project plan. A project plan is an estimate, a schedule, a set of milestones for marking progress, and an assignment of your team or your own time to each task on the estimate. It should also include other things you have to remember to do, such as meeting with the quality assurance people, preparing documentation, or ordering equipment. If you are on a team, the project plan should be a consensual agreement, both at the start and as you go.

The project plan exists to help make decisions, not to show how organized you are. If the project plan is either too long or not up-to-date, it will be useless for making decisions. In reality, these decisions are about individual persons. The plan and your judgement let you decide if you should shift tasks from one person to another. The milestones mark your progress. If you use a fancy project planning tool, do not be seduced into creating a Big Design Up Front (BDUF) for the project, but use it to maintain concision and up-to-dateness.

If you miss a milestone, you should take immediate action such as informing your boss that the scheduled completion of that project has slipped by that amount. The estimate and schedule could never have been perfect to begin with; this creates the illusion that you might be able to make up the days you missed in the latter part of the project. You might. But it is just as likely that you have underestimated that part as that you have overestimated it. Therefore the scheduled completion of the project has already slipped, whether you like it or not.

Make sure your plan includes time for: internal team meetings, demos, documentation, scheduled periodic activities, integration testing, dealing with outsiders, sickness, vacations, maintenance of existing products, and maintenance of the development environment. The project plan can serve as a way to give outsiders or your boss a view into what you or your team is doing. For this reason it should be short and up-to-date.