The velocity log - another Scrum tool

Do you measure velocity? If so, how do you record it? (And if not: start!)

You measure (or should be measuring) velocity in order to be able to adjust the intake of future sprints, allowing the team to optimize the size of its commitment, facilitating for keeping a constant pace. What you should be measuring, at the very least, is the number of story points completed according to the Definition of Done in each sprint. This is, as you probably know, called Actual Velocity. You should also measure how many story points the team originally thought they would complete when they started the sprint (on the sprint planning meeting). This is called the Estimated Velocity.

I like to do the counting as part of the Retrospection. The team review what they have delivered and we do the math and record it.

The actual (and estimated) velocity are dependent on the context in which the team work; plenty of disturbances, impediments, new people etc will yield in a lower actual velocity. The velocity is also dependent on the sprint duration and on the team size. You want to keep both as constant as possible, so that the team really has a continuous pace – a “beat”. However, there’s a reality, where people have vacations and there are parental leave and educations, and there are holidays and day-before-holidays and what not. Despite best efforts, the sprint duration and the team size will vary over time – a lot, sometimes. This makes it hard to compare the velocities of different sprints; one sprint resulted in 25 SP and another 40. Was that because the environment was such that it allowed the team to complete almost double the amount, or was it because the sprint had a few extra days and a couple of people less on vacation?

I like to remove sprint duration and team size from the equation so I get something comparable over time (across sprints); because variances in velocity should point out differences in the context, and not differences in the sprint duration or team size.

For this reason, I record also sprint duration (in number of working days) and team size (in number of people), together with every pair of velocity (actual + estimated). I try to keep the numbers to integers, but sometimes I have to count “half” people (because of e.g. parental part-time leave, educations, or similar). I try not to go into more details than halfs – if I’m just consistent in the way I measure and in the level of detail, it won’t matter in the end – so why waste time and effort trying to figure out if I have 5.18 persons or 5. Anyway, with this information (Sprint Duration and Team Size) I can calculate the number of available man-days in the sprint (number of days x number of people), and then I can use that to calculate the “Estimated Story Points per Man-day” and “Actual Story Points per Man-day”. It will say something like “1.5”. It means that for every man-day the team completes 1.5 Story Points.

I’ve made an Excel (I always do ;-)) for recording velocity like this. I call it “Velocity Log”. It also has some nifty charts to plot velocity and trends over time. It’s available for download here:
VelocityLog-v1.xls

As an example, here’s the velocity of my current team over a couple of sprints:

Let me know your thoughts.

Why estimate using Story points?

The other day I was talking about why bothering using Story Points instead of just estimating everything in hours. Here's a link to the slides.

Traditionally, the aim has been to try to estimate time by looking intensely at a problem (or a task, activity, requirement,…) – so traditionally there’s a lot of effort spent on breakdown, analysis & estimation. But at the same time, we all know that estimating time is very hard and often pretty inaccurate – because software development is research-oriented and there are things that surface only once we start digging in, which then change the picture. We all know this – yet those time estimates tend to become commitments down the line.

I argue that it’s not only the problem (task, activity, requirement,…) that is hard to foretell; the environment where we solve the problem also has a significant effect on the amount of time it will take to complete. And often the environment is much harder to foretell than the problem itself. So, in the end it doesn’t matter how long time you spend analyzing, you will still have a large unknown in the shape of the environment.

The point I want to make is that estimating in Story Points is about recognizing that Time is a result of the Size of a problem solved within a certain Context.
Time = Size x Context

Story Points is a unit for representing Size. This way Context becomes something we can grasp by comparing Size to Time – which is why we measure Velocity.

Estimating at Sprint Planning

When do you estimate? And what?

We estimate stories using Gnome Days, which becomes the story's initial "Story Point" value. I know Scrum teaches to estimate the story points as a relative complexity or size, not taking time into account.

So you have a bunch of stories in your backlog, all estimated using Gnome Days. Meaning, I have knowledge about the size of the stories in the backlog, and with my knowledge about previous actual velocity I can derive an estimate of how long those stories will actually take.

So I go to the Sprint Planning meeting with a bunch of stories, the top priority stories in the backlog. The team picks a bunch that they feel comfortable with.

Now, how much do they look at the story point values when they pick what they commit to? Do they even care about it? At this point? Or do they adjust completely to it in relation to knowledge about past velocity? We do something in between. The team picks a set of stories they feel comfortable with but use the story point sum to compare against past velocity, as a sanity check.

Next question: Do you estimate the un-estimated stories at the sprint planning or are you supposed to have that ready when you enter? If so; at what point do you estimate stories? Does the Product Owner estimate the stories himself? Or is it done by the whole team at some other point (other than the sprint planning)? Or do you have an "estimation team" that meet regularly to estimate the most important currently unestimated stories in the backlog?

The latter is my preferred approach, until someone gives me a good argument as to why not. I like the idea of a relatively stable team of mixed competences who estimate stories ahead of time, in priority order. That way, that is done when the team enters the sprint planning, and the team can focus on picking stories and instead revise the estimates, breakdown into tasks where needed and/or sanity-check the commitment. How do other organization do it?

Tool for recording Planning Poker games

When we play Planning Poker I wanted a way of recording the results of each story in a simple way. I also wanted to be able to accept the fact that we won't always reach consensus. So if we have three people playing a 3, person playing a 2 and one person playing a 5, and all of them refuse to change their minds, what do you do? Do you count it as 3? Or 5? Or 2? After you've tried letting the extremes argue (defend) their choices and then had a couple of replays, it starts to feel silly. So being a fan of mathematics and formulas I wanted to be able to live with that difference in opinion and still have a clear way of counting the result. The obvious approach would be to count the average; (2+3+3+3+5)/5 = 3,2 (or rounded to nearest half; 3.0). It works. Do it if you want. I on the other hand wanted the formula to tilt towards "worst case" rather than letting the lower estimates affect it equally. So I used a formula I picked up in (I think) "Agile Estmiating and Planning" by Mike Cohn; Estimate = (1*Min + 2*Avg + 3*Max)/6. The result in the example above would be: (1*2+2*3.2+3*5)/6=3.9. Or rounded to nearest half (which we always do); 4.0.


So for every play we do we record the final estimates from each player. We enter them in the formula above and that is what we the put as the estimate or story point value.


I have a tool I made in Excel, which we now use. Feel free to download and use if you like:
EstimationSheetTemplate-v4.xls

Planning Poker and tasks vs stories

We've used Planning Poker for over a year now. Our experience is that it is a great and easy way of coming up with estimates. If you don't know what it is, you can read more about it here.

When we started it seemed natural to us then that the values of the Planning Poker deck represented hours; 1,2,3,5,8 hours and so on. We had a rule of thumb that said that no task/requirement/story should be larger than e.g. 18-24 hours. If estimators felt like playing a higher card than that then it indicated that the story needed to be split up into smaller bits.

So, this seemed fine, especially on paper. The problem was that many sessions became extremely lengthy and we often got stuck in discussions about details and solutions. I know this is to be avoided in Planning Poker, but that's what I wanted to point out here: we made that mistake. It wasn't as easy as just saying "No let's not talk about design details now"... We said that. Over and over :-). People felt they needed to discuss the details, in order to decide between e.g. a 5 or an 8, or between 8 and 13... It was often possible to force a play and ask people to follow gut-feeling and intuition first, but the discussions kept popping up over and over again.

Our approach on avoiding this and speeding up the sessions was to simplify: forget hours. We estimate days instead. Our smallest estimate now is 0.5 (days) and then we follow the typical Planning Poker deck series; 1,2,3,5,8 and so on. A typical task/story takes between 0.5 and 5 days, I would say.

In my experience estimating in hours causes a false sense of security and leads to over-confidence in the plans. One of the arguments against using days as the estimation unit may be that we then miss the opportunity of really thinking through what parts/activities/tasks a certain story/task consists of and thereby miss possibly important aspects in the estimate... Maybe it is so. I dont know yet... So far I think it's a good tradeoff; I trade waste of time and false sense of security for a possible decreased accuracy... Well, I've already accepted to sacrifice accuracy given the 2-3 minutes per story and the whole idea behind Poker Planning to begin with: a fast and simlpe way of estimating.

My understanding of the common practice in Scrum is that you estimate two sorts of things;
1. Estimate stories in Story Points, which is a relative estimate of complexity (regardless of time)
2. Break stories down into tasks and estimate tasks in hours.

Our implementation (so far :-)) is to simply estimate stories using Gnome Days. Sometimes we have to break stories down into tasks during a sprint planning session, but certainly not always. If we do, we obviously estimate the tasks individually, but we still use days as the unit, not hours.

I'm curious to hear about other people's experience in estimating days vs hours in Scrum.