Should the Actual Velocity measurement include “Unplanned stuff”?

A question that keeps popping up here is whether or not the Actual Velocity (i.e. the sum of the Story Point values of all Done-Done stories from a sprint) should include “Unplanned stuff”. I.e. things that were not expected and which are not really part of the Sprint Backlog. The question assumes that the team somehow estimate (in Story Points) that “unplanned stuff” as it appears, and put it into the Sprint Backlog, and consequently track it to Done-Done state and as a consequence include it in the Actual Velocity measurement.

First of all, I think the way to go is to minimize the amount of unplanned stuff. Period. If you have lots of unplanned stuff then you should consider shortening the sprint length, or possibly look at how well the Product Backlog is managed and prioritized.

But there may still be unplanned stuff that appears. There’s no way around it. It’s a reality and we have to be agile enough to be able to handle it.

Let me just emphasize that if you were to estimate the unplanned stuff, you would have to estimate it using the same unit and estimation reference as the regular User Stories; otherwise the Story Point values would not be comparable or possible to sum up. In our case we estimate all our User Stories using “Gnome days” (se blog posts below), so the Story Point value of the unplanned stuff would have to be estimated using that unit too.

So in every sprint we have two alternatives, right: either you do include unplanned stuff (in Actual Velocity) or you don’t. Let’s take an example where we have a team doing a sprint starting out with 30 SP of Estimated Velocity. Let’s also assume that the team so far has had no unplanned stuff appearing in previous sprints and the Estimated Velocity of 30 is pretty accurate and stable.

Case 1: the team decides to include “unplanned stuff” in the Actual Velocity measurement
The team works for a sprint and there’s a bunch of “unplanned stuff” that appears. The team estimates it and includes it in the Actual Velocity measurement at the end of the sprint. The team’s Actual Velocity will probably be close to the Estimated Velocity, even though parts of the work Done-Done is actually “unplanned stuff” that has replaced User Stories that were originally part of the sprint backlog. As a consequence the team will have completed less User Stories than expected, because it had to deal with that unplanned stuff. So the understanding of the team’s velocity at the beginning of the sprint has proven incorrect, in the sense that it incorrectly indicates that the team can perform 30 SP of User Stories from the Product Backlog. That is true only if there’s no new “unplanned stuff”.

Case 2: The team does not include “unplanned stuff”
The team works for a sprint and the unplanned stuff that appears during the sprint is not included in the velocity measurement, so when the sprint ends the team’s Actual Velocity (25 SP for example) will be lower than the Estimated Velocity (30 SP in the example above) – because the team was distracted from the original sprint backlog doing that unplanned stuff. As a consequence, on the next sprint planning the team will observe its past Actual Velocity (25 Story Points in this example) and they will thereby commit to less. This way, they automatically adjust for a “similar amount” of unplanned stuff to appear in future sprints.

Conclusions
Given that you have a relatively steady inflow of unplanned stuff (as opposed to very rare occurrences, or spikes) in your sprints then you do not want to include Unplanned Stuff in your Actual Velocity, because it decreases the quality/reliability of the measurements. As a consequence you risk constantly taking in more User Stories than you’re actually able to complete.

Note that I’m not saying that you shouldn’t estimate or do the “unplanned stuff”. If there’s unplanned stuff that appears that has to be done, then sure you can include it in your scrum board, estimate the remaining time and track it in the burndown. But do not give it a Story Point value and do not include it in the measurement of Actual Velocity.

What metrics should be collected and how to measure and use "Velocity"?

What should be measured during a sprint? Obviously, you can measure a million different things. My suggestion below is tilted towards the simplistic approach. I like to keep measurements and maths to a minimum. Don’t get me wrong; I love Excel, metrics and complicated statistical formulas... But as far as the team goes, and what they need to care about and focus on, I suggest sparing them the nittybitty details and just collect the bare essentials.

• Estimated Velocity: The sum of the story point value of all those User Stories taken into the sprint backlog (at start of sprint). You can read some more about the concept of "Velocity" here, on the Scrum Alliance site.
• Actual Velocity: The sum of the story point value of all those User Stories that are determined to be Done-Done at end of sprint.
• Average Actual Velocity: This should be the Actual Velocity averaged out over a number of sprints (provided the team i relatively stable – or this will have little or no meaning).
• Mandays in the sprint: This is simply the number of persons in the team multiplied by the number of workdays available in the sprint.
• Focus Factor: The Actual Velocity divided by the number of Mandays in the sprint.

Example: Let’s say I have a team consisting of 4 full-time individuals and one person working half-time with some other duties. Let’s say they work in 2 week sprints. Let’s further say that the team takes in a total of 35 story points at sprint start. Let’s say they manage to complete (Done-Done) a total of 28 story points. So;
Estimated Velocity = 35
Actual Velocity = 28
Average Actual Velocity = Avg(28) (So far only 1 sprint to average over)
Mandays in sprint = 4.5 persons * 10 days = 45.
Focus Factor = 28 / 45 = ~0.6.

So, what values do you use during Sprint Planning? I suggest the following:

• Average Actual Velocity, or at least last sprint’s Actual Velocity, to compare the new sprint’s Estimated Velocity to, as a sanity check.
• Probable Velocity = Mandays available in the new sprint X Focus Factor.

Example (continued from the example above): If the team and the sprint lengths remain unchanged then the team should probably not take in more than approximately 28-30 story points into the new sprint. If the new sprint is, for example, 15 days instead, you can use the previously measured Focus Factor to calculate the probable velocity; (4.5 persons * 15 days = 67.5 mandays) * 0.6 FocusFactor = ~41 story points.

The most important thing is that the team feels comfortable with the commitment on the sprint planning. The sum of the stories selected by the team should however be verified against previous velocity to make sure that the team hasn’t substantially under- or over-committed.

A team’s velocity is a measurement of the reality. It can’t be “too low” because the reality is what it is; making up excuses to why an actual velocity is lower than its estimated is not meaningful. If you see an actual velocity that is lower than your estimated then you will want to lower your estimated velocity next sprint. “But”, you say, “the velocity needs to be higher, it is too low, the project will take too long”. … Yeah, right. But it is not the velocity that needs to be higher, it is the environment in which the team operates which needs to be different. You will only fool yourself (and the stakeholders) if you make up excuses and count on an optimistic, higher velocity than your actual; all you’ll manage to do is to make your plan look good, but it will then – inevitably – be a lousy, incorrect, optimistic and unrealistic plan. The one thing (except resources) that affects velocity is impediments. Remove impediments and the team’s velocity will go up. This should be your approach if you feel the velocity “is too low”.