How do you predict a release date? In a traditional waterfall project the project manager probably maintains an unwieldy Gantt project plan that tells him the exact date when the system will be finished, but we all know that doesn’t work in real life. Agile teams realise this, and usually conclude that a ship date cannot be predicted, because it depends on the customer’s continuously changing requirements and priorities. That isn’t very helpful, however, if your customers demand a prediction or if your organisation needs to plan a roadmap of future projects and releases.
Joel Spolsky proposes “Evidence Based Scheduling,” a technique that combines detailed estimates and historical estimation precision with Monte Carlo simulations in order to output a confidence distribution of ship dates. While I’m sure this works perfectly well, it isn’t exactly agile, as you have to identify and estimate all tasks in painstaking detail. That means you have to design the entire system down to a certain level up-front. He actually says this himself. Why is that so bad? Because you are bound to spend time designing and estimating features that won’t make it to the end product. And of course you don’t get the benefit of designing features after the requirements have matured from several demos and a lot of communication.
You can predict ship dates in agile projects too. If you use a product backlog and plot a product burndown chart, you can extrapolate a trend line after a few sprints and see when you’ll be finished. You need a reasonably complete backlog to do this, however, and all items must have been estimated. But you won’t spend much time estimating them, and you don’t have to worry about the estimates being very accurate. Actually, you don’t predict the ship date by summing up all the estimates. That would be as inaccurate as the Gantt method mentioned above. You predict using your team’s velocity, i.e. your team’s ability to reduce the backlog per sprint. That is why it takes a couple of sprints before you can start predicting.
Now, there are still a few problems with this approach. First, the burndown goes down as you implement features, but it also goes up and down as the product owner revises the backlog, creating havoc with your trend lines. That could be rectified by using a burn-up instead, where the ceiling changes with the backlog scope. Second, the trend line may fluctuate a lot from sprint to sprint, and there may not even be a trend line if the burndown is curved or erratic. It may be useful to bring in some more advanced metrics, like AgileEVM. Third, there is no confidence distribution as with the Evidence Based Scheduling. Hmm, why don’t we just take advantage of the same technique using historical data and Monte Carlo simulations?
What I’m proposing, then, is this. Build a product backlog. Make it as complete as you can. The backlog will evolve over the course of the project, of course, but be aware that the release prediction will assume that the product backlog represents what you actually intend to release. Now estimate all items on the backlog. How you estimate them doesn’t matter. You could use hours, function points or T-shirt sizes. Next, run a few sprints. If you have estimated using T-shirt sizes (S, M, L, XL) or something else discrete, try to implement some features of all sizes. Track the hours spent on each of the items. You can now calculate a set of evidence (hours spent per estimated hour or function point), or several sets (hours spent on XL items, hours spent on L items, etc.). Now do the following a hundred times:
- For each unfinished item on the backlog, select a random evidence element and use it to predict how many hours needs to be spent on the item.
- Divide the sum of hours remaining on the average number of hours the team is able to work during a sprint.
- The result is the number of sprints remaining.
You will end up with a hundred predictions. Sort them and plot them. There is your confidence distribution.