Done Environment

In addition to intelligent version control, I like to have a set of “test environments” that map closely to my Agile Version Control policy.

First, it is necessary to have some sort of “Development Environment” where the most recent code from the Development Branch in the version control system can be tested and worked on.

Similarly, I like to have a “Done Environment” (sometimes also known as “Next Release Environment”, or “NR Environment”) where the most recent code from the Done Branch can be seen. The power of this environment is that it allows (forces) you to integration test early. If your Done Environment is an accurate representation (e.g. replica) of your live environment then you will be able to integration- and system test as early as the first story being delivered there – and I always include in the Default Definition of Done that the story should be working and available for demo in the Done Environment. In fact, I like to setup an automatic build and distribution script here, that every night packages the most recent version of the Done Branch in the version control system and puts it on the Done Environment. This way you know that you will always be able to see the stories that are Done-Done on in this environment, without any risk of them being infected by code in progress. This also means that once the release day comes you have (hopefully) already done a lot of the steps required, and consequently making a release becomes less of a deal than it otherwise often is; avoiding or at least minimizing the Big Bang problem.

If you’re short on servers, server-space and/or money (actually, regardless) I think it is a great idea to use virtual servers for hosting the environments. It gives a great flexibility when it comes to setting up new environments and cloning existing ones. If you have a live environment consisting of 40 physical servers, and you want to replicate that environment as a Done Environment, it’s not always practical to buy 40 new physical servers. Especially not if you have three teams and each has its own Done Branch, and consequently needs its own Done Environment (3 x 40 servers ;-)). But maybe it is OK to buy just a few physical servers, and instead run the Done Environment servers virtually. Virtual servers is a great and powerful tool – try it. You can find a list of software here: http://en.wikipedia.org/wiki/Comparison_of_platform_virtual_machines

The velocity log - another Scrum tool

Do you measure velocity? If so, how do you record it? (And if not: start!)

You measure (or should be measuring) velocity in order to be able to adjust the intake of future sprints, allowing the team to optimize the size of its commitment, facilitating for keeping a constant pace. What you should be measuring, at the very least, is the number of story points completed according to the Definition of Done in each sprint. This is, as you probably know, called Actual Velocity. You should also measure how many story points the team originally thought they would complete when they started the sprint (on the sprint planning meeting). This is called the Estimated Velocity.

I like to do the counting as part of the Retrospection. The team review what they have delivered and we do the math and record it.

The actual (and estimated) velocity are dependent on the context in which the team work; plenty of disturbances, impediments, new people etc will yield in a lower actual velocity. The velocity is also dependent on the sprint duration and on the team size. You want to keep both as constant as possible, so that the team really has a continuous pace – a “beat”. However, there’s a reality, where people have vacations and there are parental leave and educations, and there are holidays and day-before-holidays and what not. Despite best efforts, the sprint duration and the team size will vary over time – a lot, sometimes. This makes it hard to compare the velocities of different sprints; one sprint resulted in 25 SP and another 40. Was that because the environment was such that it allowed the team to complete almost double the amount, or was it because the sprint had a few extra days and a couple of people less on vacation?

I like to remove sprint duration and team size from the equation so I get something comparable over time (across sprints); because variances in velocity should point out differences in the context, and not differences in the sprint duration or team size.

For this reason, I record also sprint duration (in number of working days) and team size (in number of people), together with every pair of velocity (actual + estimated). I try to keep the numbers to integers, but sometimes I have to count “half” people (because of e.g. parental part-time leave, educations, or similar). I try not to go into more details than halfs – if I’m just consistent in the way I measure and in the level of detail, it won’t matter in the end – so why waste time and effort trying to figure out if I have 5.18 persons or 5. Anyway, with this information (Sprint Duration and Team Size) I can calculate the number of available man-days in the sprint (number of days x number of people), and then I can use that to calculate the “Estimated Story Points per Man-day” and “Actual Story Points per Man-day”. It will say something like “1.5”. It means that for every man-day the team completes 1.5 Story Points.

I’ve made an Excel (I always do ;-)) for recording velocity like this. I call it “Velocity Log”. It also has some nifty charts to plot velocity and trends over time. It’s available for download here:
VelocityLog-v1.xls

As an example, here’s the velocity of my current team over a couple of sprints:

Let me know your thoughts.

Why estimate using Story points?

The other day I was talking about why bothering using Story Points instead of just estimating everything in hours. Here's a link to the slides.

Traditionally, the aim has been to try to estimate time by looking intensely at a problem (or a task, activity, requirement,…) – so traditionally there’s a lot of effort spent on breakdown, analysis & estimation. But at the same time, we all know that estimating time is very hard and often pretty inaccurate – because software development is research-oriented and there are things that surface only once we start digging in, which then change the picture. We all know this – yet those time estimates tend to become commitments down the line.

I argue that it’s not only the problem (task, activity, requirement,…) that is hard to foretell; the environment where we solve the problem also has a significant effect on the amount of time it will take to complete. And often the environment is much harder to foretell than the problem itself. So, in the end it doesn’t matter how long time you spend analyzing, you will still have a large unknown in the shape of the environment.

The point I want to make is that estimating in Story Points is about recognizing that Time is a result of the Size of a problem solved within a certain Context.
Time = Size x Context

Story Points is a unit for representing Size. This way Context becomes something we can grasp by comparing Size to Time – which is why we measure Velocity.

Agile Version Control - another magic bullet

I have previously posted articles about what the magic bullet of agile software development is; to achieve flexible scope. As you may recall, the article talks about the three most important aspects of your Scrum implementation which make up the fundament for being able to achieve this scope flexibility:

• Strict prioritization (having it and following it)
• Definition of Done (having one and following it)
• Story breakdown & formulation skills (understanding its importance, and constantly trying to improve)

In hindsight, I feel I have left out one important success factor when it comes to reaching a truly flexible scope; version control.

Most of us use one form of version control or another, but in my experience it is often (mistakenly so) only a matter of choosing the tool. The options are many, and range from “open source” to “incredibly expensive”. Once you decide on the tool, you use it for version tracking your codebase by checking out, committing and merging files. Perhaps you use branches in order to separate some work from other. But you may be missing an important tool; an Agile Version Control Policy.

If you do Scrum or some other form of Agile Development, you want to be able to actually finish your product increment every iteration, right? The purpose is to be able to achieve a flexible scope so that you can potentially ship on any day, at any given time. And what I am trying to get to here is that it doesn’t matter how well you succeed with your adherence to a Definition of Done if you don’t combine it with applying an Agile Version Control policy (completely regardless of which version control system you’re using). If you don’t have an Agile Version Control policy in place you risk mixing up your “Done-Done” code (code and assets related to User Stories that have been completely done according to the Definition of Done) with code which is in progress.

And guess what: if you mix Done code up with Code in progress, you’re no longer able to release at any given time!

Without Agile Version Control, shipping your software will mean that you first have to perform a series of tasks related to isolating code that is Done-Done from code that is in progress at that moment, and also often tasks related to doing last-minute polishing “while you’re at it”. The result is that shipping code is hard and a “big deal” and something that takes a lot of time. And what else does it cause? Yeah: crunch.

So how do you achieve this separation of Done-Done code from code in progress? Well, there are plenty of good articles out there to read: this one for example, written by Henrik Kniberg.

In short, what you need to do is dedicate a branch for containing that Done-Done code (let’s call this branch “Done Branch”), and define some rules for it. For example "all code on the Done Branch must adhere to the Definition of Done", "no code may be published to Done Branch without being reviewed", "code may only be published there by the branch owner", and so on.

Simplicity ftw! Here is a very simple Agile Version Control policy that I’ve been using;

Live Branch
Description: Create a branch from the trunk when you make a public/final release, that you will maintain for a while (until next release). Maintenance (patching) happens (for sake of simplicity) directly against this branch.

Branch policy:

• All code in Live Branch must be tested and working.
• All code in Live Branch must compile & build. In short: all code in Live Branch must follow Definition of Done.
• Code may only be published to Live Branch by branch owner (who may delegate it).
• Patches on Live Branch are immediately published to Trunk as the last step, and any conflicts are resolved on Trunk until code adheres to the branch policy. Note that you don’t resolve the conflicts on the Live Branch because it would imply updating Live with whatever is on Trunk (which can be a lot of changes).

Done Branch (which happens to be the Trunk, aka Main)
Description: Use the trunk (aka main branch) as “Done branch”. This will contain only code from stories that are Done-Done.

Branch policy:

• All code in Done Branch must be tested and working.
• All code in Done Branch must compile & build. In short: all code in Done Branch must follow Definition of Done.
• Code may only be published to Done Branch by branch owner (who may delegate it).

Development Branch
Description: Create a branch from the Trunk in which the team works during the sprints. The branch exists across several sprints.

Branch policy:

• All code in Development Branch must compile & build.
• The Development Branch should be updated from Trunk often (suggest daily).
• Conflicts with Trunk are resolved in Development Branch until code adheres to the branch policy.

Remember to update your code often from Development Branch (i.e. spark merges as often as possible!).

This simple policy relies only on three branch types; Live, Done and Development. Your Trunk (aka Main) is your Done Branch. There is a drawback with this policy though; if you have multiple teams who are developing (and finishing) stories which are supposed to be able to release independent of each other, then you will not be able to use the policy above (because Done-Done code from the two teams will be mixed up in the Done Branch). But, do you really have the requirement of teams to be able to release code independent of each other at different dates? Think carefully about this – do you really? If you do, fine: in that case you have to have a slightly more sophisticated setup (with a separate Done Branch + a separate Development Branch per team, and consequently you can’t use the Trunk as your Done Branch). You can read more about such a setup (Agile Version Control in a multi-team environment) in the paper by Kniberg that I referenced above.

As always, I’d be glad to hear about other peoples’ experience in this area. Please leave comments.