Showing posts with label Estimating. Show all posts
Showing posts with label Estimating. Show all posts

Friday, November 27, 2009

What unit to use in burndown?

What unit do you use for plotting your Burndown Chart? Do you use “Remaining Hours” of the tasks? Or perhaps “Story Points” of the stories in the sprint backlog? Or something else?

Up until recently I’ve taught Scrum teams to just sum up the number of remaining hours for all tasks on their Scrum Board, and plot that sum on the burndown chart. That has been my default approach and I haven’t really experimented with it.

Why even bother breaking down a Story into Tasks?
The immediate answer to this question is of course that we do it to be able to sum up estimated Remaining Time. Duh :-). But I think there’s more to it, because a Story is more than just the sum of its Tasks. The real reason, I believe, is to get the team thinking; to get the team to reflect over what challenges the Story hides so that they maintain an as clear picture as possible of where the implementing is heading and how the sprint is progressing. This implies that breakdown of a story shouldn’t be done just once (e.g. at sprint planning). The view of what tasks a story contains changes as the implementation progresses and things are discovered. It’s natural that the contents of the story vary over time. Nothing of what I have written here is controversial. It’s all part of Scrum.

What happens in practice?
Plotting a burndown based on the sum of Remaining Time obviously requires that the team first breaks down the Stories into Tasks. This, in my experience, has always been somewhat of a challenge; for the team to actually end up with useful tasks. A good breakdown of a scope into User Stories is hard enough, and then to also successfully break a Story down into good Tasks … Well, it is hard and it requires lots of practice. I’ve always struggled getting teams to continuously reflect over their breakdown and about how to improve it. But in the end, tasks often seem to end up saying more or less meaningless things like “Implementation”, “Testing” and “Review”. Fail.

Failure in Task breakdown like this effectively prohibits the team from having any use of it. A breakdown like this doesn’t say much about the Story and consequently there is little or no use in continuously reviewing the breakdown to try to learn more about the Story as implementation progresses. The only remaining purpose of the breakdown, then, is to force a daily re-estimation, so that numbers can be summed up and plotted on the chart. The burndown and the estimation of Remaining Time becomes an end in itself.

We started counting "Number of remaining tasks"
My team, being in the situation described above, felt that the daily task re-estimation was (more or less) meaningless, and proposed that they should try and completely skip the estimation all together. If nothing else, it would save them that estimation-time that was borderline waste anyway. But they also realized that the burndown has an important purpose (yesterday’s weather, indicating progress, early indication of problems, etc) so they felt that they still needed burndown. But if they don’t estimate Remaining Time, then what do they plot on the burndown chart? Well, they agreed to try to plot the number of remaining (not done) Tasks. Simple! And not completely new; it’s a fairly common practice, if I’m not mistaken.

Interesting consequences of this change
This change in the process had a very interesting side-effect: all of a sudden the Task breakdown produced a large number of highly meaningful tasks. Instead of just saying “Implementation” the tasks that the team now produce actually describe what needs to be done, in pretty deep detail. The number of tasks produced both at the sprint planning and on the Standups is significantly higher now than before, and they say a lot more about what needs to be done.
It just seem to come more naturally now. The only difference is that we no longer estimate the tasks afterwards. That’s it. I think this is very interesting. It’s like if there was a mental block before, that has now been removed once the team doesnt have to bother estimating the tasks. I don’t know, but whatever the reason, I think it's great. The team is satisfied with the change. I feel that the quality of the task breakdown of the Sprint backlog is much higher now, and consequently the understanding of the team’s progress is much better (both for themselves and for me as a stakeholder). The project is more transparent.

The drawback of this change might be that the burndown probably is a little less accurate, because the size of tasks vary a lot. On the other hand, how accurate was the burndowns before, when we were estimating remaining time almost as an end in itself? Hopefully the accuracy now is not much worse. And if it is, I think it is outweighed by the gain in insight and increased feeling of progress.

Let me know how you work with tasks, and why… As always I’m curious to get some feedback.

Monday, July 20, 2009

Estimating & prioritizing the product backlog

One of the things that I’ve been struggling with getting my head around since I started using Scrum is; how much effort do I put into estimations of the product backlog, and do I or don’t I estimate all at once? And do I re-estimate stories as I go along and learn more?

Originally I’ve been under the impression that I shouldn’t attempt to estimate it all up front. It's easy to argue that it is rather un-agile to do so: because I would be spending effort on low priority stuff, which is a big risk for waste. On the other hand, if I don’t estimate the backlog up front, I can’t tell its entire size and consequently I can’t (potentially) tell what I will have ready by when.

So far, I’ve been trying a couple of different approaches. One of my favorites has been to estimate in priority order up to a few months worth of stories (a "handful" of sprints). The remaining unestimated stories I apply an average size to based on the estimated ones. I know, it’s risky, but it’s at least something. And obviously, as time passes, I estimate more and more; trying to keep about 2-3 months worth of stories estimated at all times. For some time this has been my approach to release planning without estimating everything up front.

A problem I only recently realized I’ve had is that this approach requires me to prioritize before estimating; and consequently my prioritizations can’t take size into consideration. My prioritization has only been made based on “value” (whatever that is) without regard to cost. Is that optimal? I think the answer is No. Given I have a choice prioritization should take size into account. Or at least, if I don’t take it into account it needs to be a conscious decision not to. Which brings me back to my original question; how do I do it, do I estimate everything up front or not?

Recently, I’ve decided to try changing my approach (inspect and adapt, right?) and actually estimate the entire backlog up front, before prioritization. The good thing is that estimating in story points is very quick (the team usually doesn’t have to spend more than a few minutes per item) so the time spent on (potentially) low-priority items is minimal.

Once I have the story point estimates of all backlog items I use a prioritization technique called “Theme Scoring”. Let me try to describe it with a simple example;

The first step is to decide on a couple of aspects ("selection criteria") to judge stories on; for example “Improves user interface”, “Simplifies maintenance”, “Brings revenue in Q3” and so on. The combined criteria will represent what you call “business value” so whatever you put in that term should in one way or another be reflected by those criteria that you pick. Try, however, to keep it rather simple e.g. up to a maximum of 4-6 criteria or otherwise your scoring will become very tedious and the effort you spend on scoring (and consequently the risk of waste) I think will be too great compared to the benefit.

Once you’ve decided on your criteria you need to weigh them relative to each other. Maybe “Brings revenue in Q3” is the most important criteria to you right now, so let’s give it the weight of (for example) 5. “Simplifies maintenance” is not that important right now so you give it the weight 2 (it’s less than half as important as the revenue criteria). “Improves user interface” is almost as important as the revenue aspect, so you assign it the weight 4. Here are your criteria and their weights so far; “Brings revenue in Q3” = 5, “Improves user interface” = 4 and “Simplifies maintenance” = 2.

The next step is to go through all of your backlog items, one by one, and score them on a scale from 1-5 in terms of how they affect each of your selected criteria. For each criteria you can usually identify a “baseline story”; a story that gives you a fair amount of value in terms of that criteria, yielding in a 3 point score on that 1-5 scale. There should be stories that yield in a higher score as well as lower. Stories should be compared to that baseline story for each criteria to determine if it should get a higher or a lower score than the baseline.

This could be your result after going over all criterias for all stories;


The numbers marked red are the baseline stories for each criteria. The black numbers in the criteria columns are grades I assign each story on a scale from 1-5 relative to the baseline. 1 and 2 means much lower and lower than the baseline, and 4 and 5 means higher and much higher than the baseline. A grade of 0 means that the criteria doesn’t apply to the story. The Score is simply your grade multiplied by the weight, for each criteria.

Given your selected criteria and their weights, in the example here Story 3 is the one giving you the most value (37), followed by Story 2 (18) and then Story 1 (17), Story 5 (8) and Story 4 (6).

Now, it is time to compare value to cost (estimated size in Story Points);


So far this technique has been possible for me to do all along, even while I have been lacking estimates of parts of the backlog. But the following part – taking cost into the equation – obviously requires me to have the estimates. Putting the Benefit in perspective to Cost is a great idea; it gives me the opportunity to maximize benefit while minimizing cost.

So, divide Value (your total score for each story) with Cost (measured in story points) and sort your backlog by the result: the higher the value the more value I get back on the effort put in:


The above table is the resulting suggested prioritization based on the Theme Scoring technique and on calculating value/cost. Obviously it is worth noting that this is a suggestion only. It is an aid that I can use when prioritizing my backlog, it’s not something that replaces my thought-process – I still need to think and consider every item and e.g. be careful that I don’t miss any dependencies between stories (yes I know, we shouldn’t have any dependencies between stories, but sometimes we do anyway! :-)).

As always, I’m interested in input on how other people do things…!

Wednesday, June 17, 2009

Why estimate using Story points?

The other day I was talking about why bothering using Story Points instead of just estimating everything in hours. Here's a link to the slides.

Traditionally, the aim has been to try to estimate time by looking intensely at a problem (or a task, activity, requirement,…) – so traditionally there’s a lot of effort spent on breakdown, analysis & estimation. But at the same time, we all know that estimating time is very hard and often pretty inaccurate – because software development is research-oriented and there are things that surface only once we start digging in, which then change the picture. We all know this – yet those time estimates tend to become commitments down the line.

I argue that it’s not only the problem (task, activity, requirement,…) that is hard to foretell; the environment where we solve the problem also has a significant effect on the amount of time it will take to complete. And often the environment is much harder to foretell than the problem itself. So, in the end it doesn’t matter how long time you spend analyzing, you will still have a large unknown in the shape of the environment.

The point I want to make is that estimating in Story Points is about recognizing that Time is a result of the Size of a problem solved within a certain Context.
Time = Size x Context

Story Points is a unit for representing Size. This way Context becomes something we can grasp by comparing Size to Time – which is why we measure Velocity.

Thursday, September 18, 2008

Estimating at Sprint Planning

When do you estimate? And what?

We estimate stories using Gnome Days, which becomes the story's initial "Story Point" value. I know Scrum teaches to estimate the story points as a relative complexity or size, not taking time into account.

So you have a bunch of stories in your backlog, all estimated using Gnome Days. Meaning, I have knowledge about the size of the stories in the backlog, and with my knowledge about previous actual velocity I can derive an estimate of how long those stories will actually take.

So I go to the Sprint Planning meeting with a bunch of stories, the top priority stories in the backlog. The team picks a bunch that they feel comfortable with.

Now, how much do they look at the story point values when they pick what they commit to? Do they even care about it? At this point? Or do they adjust completely to it in relation to knowledge about past velocity? We do something in between. The team picks a set of stories they feel comfortable with but use the story point sum to compare against past velocity, as a sanity check.

Next question: Do you estimate the un-estimated stories at the sprint planning or are you supposed to have that ready when you enter? If so; at what point do you estimate stories? Does the Product Owner estimate the stories himself? Or is it done by the whole team at some other point (other than the sprint planning)? Or do you have an "estimation team" that meet regularly to estimate the most important currently unestimated stories in the backlog?

The latter is my preferred approach, until someone gives me a good argument as to why not. I like the idea of a relatively stable team of mixed competences who estimate stories ahead of time, in priority order. That way, that is done when the team enters the sprint planning, and the team can focus on picking stories and instead revise the estimates, breakdown into tasks where needed and/or sanity-check the commitment. How do other organization do it?

Tuesday, September 16, 2008

Tool for recording Planning Poker games

When we play Planning Poker I wanted a way of recording the results of each story in a simple way. I also wanted to be able to accept the fact that we won't always reach consensus. So if we have three people playing a 3, person playing a 2 and one person playing a 5, and all of them refuse to change their minds, what do you do? Do you count it as 3? Or 5? Or 2? After you've tried letting the extremes argue (defend) their choices and then had a couple of replays, it starts to feel silly. So being a fan of mathematics and formulas I wanted to be able to live with that difference in opinion and still have a clear way of counting the result. The obvious approach would be to count the average; (2+3+3+3+5)/5 = 3,2 (or rounded to nearest half; 3.0). It works. Do it if you want. I on the other hand wanted the formula to tilt towards "worst case" rather than letting the lower estimates affect it equally. So I used a formula I picked up in (I think) "Agile Estmiating and Planning" by Mike Cohn; Estimate = (1*Min + 2*Avg + 3*Max)/6. The result in the example above would be: (1*2+2*3.2+3*5)/6=3.9. Or rounded to nearest half (which we always do); 4.0.


So for every play we do we record the final estimates from each player. We enter them in the formula above and that is what we the put as the estimate or story point value.


I have a tool I made in Excel, which we now use. Feel free to download and use if you like:
EstimationSheetTemplate-v4.xls

Planning Poker and tasks vs stories

We've used Planning Poker for over a year now. Our experience is that it is a great and easy way of coming up with estimates. If you don't know what it is, you can read more about it here.

When we started it seemed natural to us then that the values of the Planning Poker deck represented hours; 1,2,3,5,8 hours and so on. We had a rule of thumb that said that no task/requirement/story should be larger than e.g. 18-24 hours. If estimators felt like playing a higher card than that then it indicated that the story needed to be split up into smaller bits.

So, this seemed fine, especially on paper. The problem was that many sessions became extremely lengthy and we often got stuck in discussions about details and solutions. I know this is to be avoided in Planning Poker, but that's what I wanted to point out here: we made that mistake. It wasn't as easy as just saying "No let's not talk about design details now"... We said that. Over and over :-). People felt they needed to discuss the details, in order to decide between e.g. a 5 or an 8, or between 8 and 13... It was often possible to force a play and ask people to follow gut-feeling and intuition first, but the discussions kept popping up over and over again.

Our approach on avoiding this and speeding up the sessions was to simplify: forget hours. We estimate days instead. Our smallest estimate now is 0.5 (days) and then we follow the typical Planning Poker deck series; 1,2,3,5,8 and so on. A typical task/story takes between 0.5 and 5 days, I would say.

In my experience estimating in hours causes a false sense of security and leads to over-confidence in the plans. One of the arguments against using days as the estimation unit may be that we then miss the opportunity of really thinking through what parts/activities/tasks a certain story/task consists of and thereby miss possibly important aspects in the estimate... Maybe it is so. I dont know yet... So far I think it's a good tradeoff; I trade waste of time and false sense of security for a possible decreased accuracy... Well, I've already accepted to sacrifice accuracy given the 2-3 minutes per story and the whole idea behind Poker Planning to begin with: a fast and simlpe way of estimating.

My understanding of the common practice in Scrum is that you estimate two sorts of things;
1. Estimate stories in Story Points, which is a relative estimate of complexity (regardless of time)
2. Break stories down into tasks and estimate tasks in hours.

Our implementation (so far :-)) is to simply estimate stories using Gnome Days. Sometimes we have to break stories down into tasks during a sprint planning session, but certainly not always. If we do, we obviously estimate the tasks individually, but we still use days as the unit, not hours.

I'm curious to hear about other people's experience in estimating days vs hours in Scrum.

Monday, September 15, 2008

Gnome Days

So how does other people estimate?

I know, estimate relative size or complexity. That's all good. But how do you do it? In my experience it's not really natural. There's always a discussion first about neglecting (or the impossibility of neglecting) the time factor in the estimation process.

The idea with relative complexity estimation is - as I've understood it - to ignore time and instead focus on (in lack of a better word) the size of the task/feature/requirement/story. This means that you should ignore thinking in hours or days and instead just order the stories relative to eachother; this is twice as hard as this and this is about the same, and this is half as hard, and so on. Henrik Kniberg's approach of Turbo Estimation is perhaps the simplest form of complexity estimation; you just divide each story into simple, medium and hard.

This is all good. My problem with the above mentioned method is that it doesnt "feel" natural.
Every time I've initiated an estimation session with the intent of estimating complexity acc to the above, it has sparked (sometimes) lengthy discussions about why it works or not.

My usual argument is that the estimators should think of it as time being a result of the complexity and the term complexity should be regarded more like "size". E.g. the time is a result of the "size" of the work of completing a certain task/requirement/story/feature.

Another complication here is the lack of continuity in the estimation process. As you know, measuring velocity (actual vs estimated) is a key point in Scrum in order to manage sprint intake, so you will want the complexity estimates to mean the same thing from one estimation session to another in order for velocity to actually have a meaning. And since we usually dont have estimation sessions more than maybe once every two or three weeks people tend to forget the relative size of stories across sessions.

My conclusion from the above is that sure, relative estimation of complexity is great in the absence of any other method, but it really leaves many things to be desired.

Our solution now is to estimate in ideal days, but we call it "Gnome Days" - or Tomtedagar in Swedish. The term was inspired by Henrik Knibergs book "Scrum and XP from the trenches" (if you haven't read it yet: do! It's free and can be found here, although I suggest you buy it to support Henrik) where he describes that time estimation should be done considering how long a task will take to complete for an optimal number of resources locked in an room and left completely undisturbed. I thought of that as a gnome (swedish "Tomte") in a basement, who you lock up and don't let out again until the task/story/requirement is completed.
Anyway, we estimate the initial "story point" value of each story using gnome days. This is in our experience much more natural and still lets estimators estimate stories relative to eachother. A challenge here is to avoid micromanagement and microestimation, so we forbid the use of hours. Instead, our smallest estimate (ever) is 0.5 gnome day. If a story is estimated to more than e.g. 3-4 gnome days then it could probably use breakdown into smaller stories.