Follow-up on the Scrum Practitioners meeting Oct 15th

Thank you everyone who participated on the Scrum Practitioners meeting on the 15th of October at Axis Communications in Lund.

Some of the things discussed were;

• Distributed Scrum; does it work? A few of the participants had some experience in this, and the opinions were mixed. Bottom line is; yes it is certainly possible to get it to work. I.e. it doesnt automatically fail. It requires work, though. But that is true for any methodology; if you have team members on a distance you have to deal with the overhead cost in terms of extra communication, tools, activities, travel, whatever. Depending on how the distribution looks you can try different approaches; divide into teams per site, or distribute teams and work with conference solutions, digital whiteboards, etc for the daily meetings. Experiment. Distributed teams and distributed Scrum is not ideal, but if you're faced with a situation where you have a distributed team, don't automatically rule out using Scrum.
• How/if sort out design-issues before sprint start? The problem discussed was that of things taking a long time to finish if you do "design stories" in one sprint to discover what/how to implement, and actually implement in in the following sprint. With 4 week sprints it can take you 8 weeks to finish a story, even if it is not that big. The concept of "spikes" was discussed, i.e. prototyping and delivering a piece of working and valuable code in each sprint, rathen than doing "only design" separated from "only development". If you do "design stories" it may be wise to timebox rather than trying to estimate it, because the definition of done for "design" is hard to grasp. Another tip can be to consider shortening the sprints, to e.g. 2-3 weeks instead of 4.
• The problem of traditional, conservative customers, demaning fixed-price-fixed-scope contracts was discussed. The concept of "Money for Nothing and Change for Free" was discussed as an interesting theory. None of the participants had experience with that approach, though.
• Coping with less-than-fulltime resources in a Scrum Team: The problem of traditional customers and product owners who do not realize that Scrum requires a much higher degree of involvement. Not sure we came to any intelligent conclusions on how to address this problem, other than the importance of realizing that it is a problem if the customer/PO is not involved.
  
• Problems with large team size was discussed, and consequently the problem of daily scrums taking too long time.
• Is the traditional project manager automatically the best choice for Scrum Master when introducing Scrum? Not necessarily, was the conclusion. This seems to be the default choice in the industry, but it is not necessarily the best. It depends on your organization, the people, the problem domain, etc. The Scrum Master may be a good choice for the Project Manager, but Scrum Product Owner may be too.
  

Several other interesting things were discussed too, but I will not go into more details here.

I'm looking forward to the next meeting.

October meeting Scrum Practitioners South of Sweden

I'm planning the October meeting for the network Scrum Practitioners South of Sweden. I'll update this blog post as details are decided. For now, the meeting is planned for Thursday October 15. It will be between 17:00 and 19:00, hosted by myself, in Axis Communications' office at Emdalavägen 14 in Lund.

The topics of discussion are (under construction ;-)):

* Distributed Scrum; Does it work?
* How/if sort out design-issues before sprint start?
* How do I get the product owner more involved?
* Coping with less-than-fulltime resources in a Scrum Team
* Is the Project Manager the best choice for Scrum Master when moving from a Traditional to a Scrum organization? Who decides, and who is the best choice?
* Is the "Product Manager" the best choice for "Scrum Product Owner"?
* Story formulation, estimation, story breakdown into tasks and prioritization
* ...<insert topic here>

Follow-up on the Scrum Practitioners meeting Sept 7th

To follow up on our first meeting in the Scrum Practitioners South of Sweden network;

I don't intend to recap all the discussions and conclusions on the meeting, but rather I just wanted to briefly list the major topics discussed as I recall.

The meeting was held as planned at Labs2 in Lund with a total of 8 participants including Mathias Thinsz (Product Manager & Scrum Product Owner at Labs2) who was our host.

Mathias went through and explained the major parts of Labs2's Scrum implementation. It seems to me that they have a very solid and healthy view on how they development software. It's a clean and well-thought-through implementation of Scrum. A great example on how well Scrum works.

Related topics discussed were;
* How Labs2 use JIRA and a custom developed web application for managing the backlog
* How and when Labs2 estimate their work in the backlog, and how they use the storypoints unit
* How Labs2 has divided their development organization
* How and why Labs2 have divided their development time into fixed-length iterations
* How Mathias (as Scrum Product Owner) prepares for a new sprint and who is involved when
* How and why Labs2 use "Sprint Goals" as a major tool for directing development inside a sprint

There were tons of other interesting questions and discussions as well, but too much for me to go into here.

The general feeling, in my understanding, was that the meeting was interesting and inspiring and everyone had a lot of questions and topics which they wanted to continue discussing.
The number of participants was optimal and the length of the meeting as well. Hence; my conclusion is that there is a definate interest for meeting again and that the next meeting should be in the same format.

I want to take the opportunity to thank everyone who joined and not least Mathias Thinsz and Labs2 for the openess with how they work!

Mon Sep 09: Meeting Scrum Practitioners South of Sweden

On Monday the 7th of September 17:00-19:00 we're a bunch of people meeting at Labs2's office in Lund (Sweden :)) to discuss how Labs2 are working with Scrum. Anyone intersted in joining and sharing experiences is welcome.

Join the LinkedIn group "Scrum Practitioners South of Sweden" and post your interest if you want to join the meeting.

My idea with this is to get software professionals in the Öresund region to meet informally, network, and not least share experiences and help eachother improve our way of working with agile software development. Meetings will be hosted by a different group member each time, and the discussion theme will vary from meeting to meeting.

See you there!

Scrum but... test

This has been around for some time, but if you haven't taken it already then do it. It's a good way to compare yourself to others and help emphasize your points of improvement.

Go to the Scrumbut test!

I just took the test and my team currently scores 7.1 out of 9.

Estimating & prioritizing the product backlog

One of the things that I’ve been struggling with getting my head around since I started using Scrum is; how much effort do I put into estimations of the product backlog, and do I or don’t I estimate all at once? And do I re-estimate stories as I go along and learn more?

Originally I’ve been under the impression that I shouldn’t attempt to estimate it all up front. It's easy to argue that it is rather un-agile to do so: because I would be spending effort on low priority stuff, which is a big risk for waste. On the other hand, if I don’t estimate the backlog up front, I can’t tell its entire size and consequently I can’t (potentially) tell what I will have ready by when.

So far, I’ve been trying a couple of different approaches. One of my favorites has been to estimate in priority order up to a few months worth of stories (a "handful" of sprints). The remaining unestimated stories I apply an average size to based on the estimated ones. I know, it’s risky, but it’s at least something. And obviously, as time passes, I estimate more and more; trying to keep about 2-3 months worth of stories estimated at all times. For some time this has been my approach to release planning without estimating everything up front.

A problem I only recently realized I’ve had is that this approach requires me to prioritize before estimating; and consequently my prioritizations can’t take size into consideration. My prioritization has only been made based on “value” (whatever that is) without regard to cost. Is that optimal? I think the answer is No. Given I have a choice prioritization should take size into account. Or at least, if I don’t take it into account it needs to be a conscious decision not to. Which brings me back to my original question; how do I do it, do I estimate everything up front or not?

Recently, I’ve decided to try changing my approach (inspect and adapt, right?) and actually estimate the entire backlog up front, before prioritization. The good thing is that estimating in story points is very quick (the team usually doesn’t have to spend more than a few minutes per item) so the time spent on (potentially) low-priority items is minimal.

Once I have the story point estimates of all backlog items I use a prioritization technique called “Theme Scoring”. Let me try to describe it with a simple example;

The first step is to decide on a couple of aspects ("selection criteria") to judge stories on; for example “Improves user interface”, “Simplifies maintenance”, “Brings revenue in Q3” and so on. The combined criteria will represent what you call “business value” so whatever you put in that term should in one way or another be reflected by those criteria that you pick. Try, however, to keep it rather simple e.g. up to a maximum of 4-6 criteria or otherwise your scoring will become very tedious and the effort you spend on scoring (and consequently the risk of waste) I think will be too great compared to the benefit.

Once you’ve decided on your criteria you need to weigh them relative to each other. Maybe “Brings revenue in Q3” is the most important criteria to you right now, so let’s give it the weight of (for example) 5. “Simplifies maintenance” is not that important right now so you give it the weight 2 (it’s less than half as important as the revenue criteria). “Improves user interface” is almost as important as the revenue aspect, so you assign it the weight 4. Here are your criteria and their weights so far; “Brings revenue in Q3” = 5, “Improves user interface” = 4 and “Simplifies maintenance” = 2.

The next step is to go through all of your backlog items, one by one, and score them on a scale from 1-5 in terms of how they affect each of your selected criteria. For each criteria you can usually identify a “baseline story”; a story that gives you a fair amount of value in terms of that criteria, yielding in a 3 point score on that 1-5 scale. There should be stories that yield in a higher score as well as lower. Stories should be compared to that baseline story for each criteria to determine if it should get a higher or a lower score than the baseline.

This could be your result after going over all criterias for all stories;


The numbers marked red are the baseline stories for each criteria. The black numbers in the criteria columns are grades I assign each story on a scale from 1-5 relative to the baseline. 1 and 2 means much lower and lower than the baseline, and 4 and 5 means higher and much higher than the baseline. A grade of 0 means that the criteria doesn’t apply to the story. The Score is simply your grade multiplied by the weight, for each criteria.

Given your selected criteria and their weights, in the example here Story 3 is the one giving you the most value (37), followed by Story 2 (18) and then Story 1 (17), Story 5 (8) and Story 4 (6).

Now, it is time to compare value to cost (estimated size in Story Points);


So far this technique has been possible for me to do all along, even while I have been lacking estimates of parts of the backlog. But the following part – taking cost into the equation – obviously requires me to have the estimates. Putting the Benefit in perspective to Cost is a great idea; it gives me the opportunity to maximize benefit while minimizing cost.

So, divide Value (your total score for each story) with Cost (measured in story points) and sort your backlog by the result: the higher the value the more value I get back on the effort put in:


The above table is the resulting suggested prioritization based on the Theme Scoring technique and on calculating value/cost. Obviously it is worth noting that this is a suggestion only. It is an aid that I can use when prioritizing my backlog, it’s not something that replaces my thought-process – I still need to think and consider every item and e.g. be careful that I don’t miss any dependencies between stories (yes I know, we shouldn’t have any dependencies between stories, but sometimes we do anyway! :-)).

As always, I’m interested in input on how other people do things…!

Technical debt in Scrum projects

Code deteriorates with age. The older the system, the worse the maintenance. Why is this?

One answer may be that it is because of the maintenance itself: the things done over the years to keep the system afloat and the new features added, removed and changed. And people come and go. Some quit the team and new ones join. Sum it all up: deterioration.

Let’s talk about some of the causes and effects of deterioration.

Reinventing the wheel
An example of this is when a system contains several versions of the same function implemented at different times, often by different people, with only small differences. A similar situation is where a design (architecture) problem is solved in different ways in different places in the system. Either way, reinventing the wheel increases the complexity of the system and makes it harder to maintain. Developers won’t know which function is used where or even if they are still used at all (and consequently leave them there).

This is a negative spiral; the more complex the system gets the harder it is to understand, and consequently the harder it is to know if “my” problem has been solved already somewhere – which might lead me to unintentionally write my own version of a function that actually already exists. The lesson is to not reinvent the wheel. But it is easier said than done.

Maintainability is not something you can just add down the road. Building a maintainable codebase starts from the first line of code and it never ends.

Fear of changing what works – Legacy code
Once a system reaches a certain level of deterioration, people will be afraid to change certain parts. It’s often some critical and central component. Since it is central and critical it means it has been patched up numerous times, and changed and adjusted and changed again – over a long time. And hey, it works – now. And since it works, now, and since the code is so complicated and confusing, the developers don’t dare touch it. Consequently any features that would result in work in that critical component will be held back.

The way I see it, one of the root causes for this, is lack of regression testing abilities. With proper means for continuous and whole-covering regression testing, there is little to fear even when it comes to modifying old “legacy code”. Otherwise there are only two options when it comes to modifying legacy code; a complete rewrite, or just not touching it at all.

Lack of (efficient) regression testing abilities
The purpose of regression testing is to verify that every part of the system – even the less obvious parts – still works after a change or addition somewhere has been completed. The point with regression testing is to test a lot, over and over and over again. If you are in a situation where regression testing requires immense resources (which it does if you do it completely manually), then you will not be able or willing to do it as often or as extensive as is required. As a consequence your regression testing become inefficient, or even non-existent.

One of the steps in the right direction is to start automating your tests, and start running those automatic test cases continuously. The challenge, of course, is to;
(A) find a practical means - a tool - for automatic testing of your type of code, and
(B) figure out where to start.

I can’t help you with A (...psssst: JUnit, CPPUnit, CUnit, PHPUnit), but for B I suggest you just start somewhere. Don’t try to do it all at once – you won’t make it! Instead, just pick a simple starting-point, for example the most recent and newly added feature/module/component/part. Forget about the old stuff for now, just add automatic testing for the new things from now on. Something is better than nothing.

People joining and leaving
This is, unfortunately, unavoidable in most projects that are longer than just a few months. It happens. Either people get reassigned, or they choose to quit. And best-case, people join the team. Apart from a change in productivity caused by a team member leaving or joining, it obviously has an effect on the code itself too. People leaving will take knowledge with them, and people joining bring in new ideas (and misunderstand parts of what exists, too). One way to minimize the effects of this is to organize into “Feature Teams” that has a lot of close cooperation and joint commitments (like – tada! – Scrum suggests). This way you naturally spread knowledge among several people. It is also a pretty effective way of introducing new team members into the groove of things.

The classic method to minimize the problem of people leaving and joining is to write documents. I however argue that this is not the silver bullet for spreading & retaining knowledge. In fact, I think it's dangerous to rely on documents as the main tool for this; documentation is an extremely cost-inefficient and overrated way of transferring knowledge, and something that is often forgotten is the cost of keeping documentation up-to-date. As soon as documentation falls behind it becomes untrustworthy – and untrustworthy documentation will cause confusion and misunderstandings, and in the end no one will dare rely on documentation, and the time writing and updating it up to that point becomes waste.

In my opinion the guideline should be: don’t over-document – document just enough. Code Comments, I think, is a great benchmark of what level of documentation is “enough” for most situations. And remember; one excellent thing about code comments is that it is automatically (well…) kept up to date as the code changes – there’s little or no added cost for keeping it up to date.

Oh, and for the record, I’m not saying “Don’t write documents!”. If you really need to document then of course you should. I’m merely suggesting that you at the very least question the reason for doing that effort, and that you don’t forget to take into account the cost of keeping the document up-to-date as the system evolves.

Taking shortcuts - the Dirty that remains long after the Quick has been forgotten
“Well, we’ll do the quick-n-dirty fix now, just to get it done, and then we’ll go back later and clean it up...”. Have you ever said or heard something along those lines?

Short term gains such as reaching some immediate deadline makes it tempting sometimes to take shortcuts, and often it’s, sadly, a conscious decision. The problem with shortcuts is that they seldom or never get fixed afterwards, because there’s always that next deadline coming up with a new bunch of stuff to do with a new bunch of shortcuts that “has” to be made.

Doing things right from the beginning often requires a little more effort up front. And I think that different times call for different ways of acting. Sometimes it might be a correct decision to look only at short-term gains and cutting down on the immediate effort, and forgetting about those long-term consequences and drawbacks. But many teams and managers, I think, tend to be shortsighted by default – even when they don’t have to and there would in fact be room to do things properly. And in that case it is a matter of attitude (and competence). Do you do things fast and sloppy now and accept to pay for it later, or do you let it take a little longer now and reap the benefits of it later (for example in terms of costs saved on maintenance)?

It’s a challenge to figure out what is a shortcut and what is not. Remember Lean Software Development and the idea of “Extra effort” (and “Extra features”) being Waste. How do you know what is “Just enough effort”? There is no default answer to that. It depends. It’s up to you to figure that out for your system and for your business. But by figuring that out (or deciding on it) you will know what level to strive for; and anything below it is a shortcut and should not be accepted.

Don’t forget that you need to make sure that whatever your level is, it should be gut-felt by every team-member.

Bug fixes
Bug fixes have a tendency to deteriorate code. Enough patches in one place and the code will become more and more messy – at least if you have other problems too that cause code deterioration, such as people coming and leaving, inability to do regression testing, etc.

Bugs found in a production environment are often time critical to remedy. It can be tempting to take shortcuts to just fix the problem quickly and get a patch out there. But if you do that enough times but never take time to clean out the mess, you are destroying your system. See section above about taking shortcuts…

Summing things up – dealing with Technical Debt
A nice way to think of this deterioration of code is to think of it as “Technical Debt” - a term coined by Ward Cunningham.

Technical Debt is a long-term loan that we for some reason choose to, or have to, take from ourselves in order to achieve some short term gain. The Technical Debt increases for every individual loan and the debt never just goes away by itself. We have to pay “Interest” in the shape of things taking longer to complete because of this deteriorated code, and the only way we can decrease the Interest is by decreasing the loan – by paying “mortgages” e.g. by refactoring.

The most common (and worst) approach to dealing with Technical Debt is to ignore it. To pretend it doesn’t exist, and just push development forward without considering whether or not we’re taking loans.
The better approach to dealing with Technical Debt is to recognize it and have an active plan on how to deal with it in various situations.

I suggest you deal with technical debt into two steps: First stop increasing the debt further, and secondly start decreasing it. Only once you know that your debt is not increasing, you can start actively working with decreasing it.

To support you in the first step I suggest you insert a row in your Default Definition of Done that says “The Technical Debt has not increased”. It sounds trivial, but the intended effect is that it recognizes that it is OK that things take longer to complete if done in a way that doesn’t increase the debt - e.g. that it is OK to not take shortcuts. Remember to constantly remind people about this, and really do put your the money where your mouth is. Whenever faced with a choice, make sure you consider that the decision should be in-line with the attitude of letting things take longer in order to not increase the technical debt.

Once you’ve gotten used to that approach (it probably takes you a while and, if nothing else, will probably cause your velocity to drop significantly at first), the next step is to change the Definition of Done to instead say “The Technical Debt has decreased”. This is intended to recognize the fact that it is OK to also do some refactoring of things surrounding the current implementation “while you’re at it”. For example, urge all developers to modify the methods above and below the one currently worked in - even though it wouldn’t be necessary to complete the story itself! This way, for every new story completed the existing debt will decrease.
This type of refactoring will puts a lot of demands on your regression testing abilities. If you don't already have an automatic testing environment I suggest you start with introducing that first. Refactoring working code will (as explained in a section above) will otherwise be much too scary.

That's it from me for now. As always I'm interested in hearing other people's experiences and opinions in this matter.

Scrum Practitioners South of Sweden on LinkedIn

I just started a LinkedIn group: Scrum Practitioners South of Sweden.

The idea is to collect a bunch of people in the south of Sweden who want to meet face to face regularly, in an informal manner and share experience of Scrum, Agile and related topics with the intent of helping each other out and improving how we work.

Not sure if such a network already exists but I certainly see a need for it personally.

Follow this link if you're interested in joining.

Done Environment

In addition to intelligent version control, I like to have a set of “test environments” that map closely to my Agile Version Control policy.

First, it is necessary to have some sort of “Development Environment” where the most recent code from the Development Branch in the version control system can be tested and worked on.

Similarly, I like to have a “Done Environment” (sometimes also known as “Next Release Environment”, or “NR Environment”) where the most recent code from the Done Branch can be seen. The power of this environment is that it allows (forces) you to integration test early. If your Done Environment is an accurate representation (e.g. replica) of your live environment then you will be able to integration- and system test as early as the first story being delivered there – and I always include in the Default Definition of Done that the story should be working and available for demo in the Done Environment. In fact, I like to setup an automatic build and distribution script here, that every night packages the most recent version of the Done Branch in the version control system and puts it on the Done Environment. This way you know that you will always be able to see the stories that are Done-Done on in this environment, without any risk of them being infected by code in progress. This also means that once the release day comes you have (hopefully) already done a lot of the steps required, and consequently making a release becomes less of a deal than it otherwise often is; avoiding or at least minimizing the Big Bang problem.

If you’re short on servers, server-space and/or money (actually, regardless) I think it is a great idea to use virtual servers for hosting the environments. It gives a great flexibility when it comes to setting up new environments and cloning existing ones. If you have a live environment consisting of 40 physical servers, and you want to replicate that environment as a Done Environment, it’s not always practical to buy 40 new physical servers. Especially not if you have three teams and each has its own Done Branch, and consequently needs its own Done Environment (3 x 40 servers ;-)). But maybe it is OK to buy just a few physical servers, and instead run the Done Environment servers virtually. Virtual servers is a great and powerful tool – try it. You can find a list of software here: http://en.wikipedia.org/wiki/Comparison_of_platform_virtual_machines

The velocity log - another Scrum tool

Do you measure velocity? If so, how do you record it? (And if not: start!)

You measure (or should be measuring) velocity in order to be able to adjust the intake of future sprints, allowing the team to optimize the size of its commitment, facilitating for keeping a constant pace. What you should be measuring, at the very least, is the number of story points completed according to the Definition of Done in each sprint. This is, as you probably know, called Actual Velocity. You should also measure how many story points the team originally thought they would complete when they started the sprint (on the sprint planning meeting). This is called the Estimated Velocity.

I like to do the counting as part of the Retrospection. The team review what they have delivered and we do the math and record it.

The actual (and estimated) velocity are dependent on the context in which the team work; plenty of disturbances, impediments, new people etc will yield in a lower actual velocity. The velocity is also dependent on the sprint duration and on the team size. You want to keep both as constant as possible, so that the team really has a continuous pace – a “beat”. However, there’s a reality, where people have vacations and there are parental leave and educations, and there are holidays and day-before-holidays and what not. Despite best efforts, the sprint duration and the team size will vary over time – a lot, sometimes. This makes it hard to compare the velocities of different sprints; one sprint resulted in 25 SP and another 40. Was that because the environment was such that it allowed the team to complete almost double the amount, or was it because the sprint had a few extra days and a couple of people less on vacation?

I like to remove sprint duration and team size from the equation so I get something comparable over time (across sprints); because variances in velocity should point out differences in the context, and not differences in the sprint duration or team size.

For this reason, I record also sprint duration (in number of working days) and team size (in number of people), together with every pair of velocity (actual + estimated). I try to keep the numbers to integers, but sometimes I have to count “half” people (because of e.g. parental part-time leave, educations, or similar). I try not to go into more details than halfs – if I’m just consistent in the way I measure and in the level of detail, it won’t matter in the end – so why waste time and effort trying to figure out if I have 5.18 persons or 5. Anyway, with this information (Sprint Duration and Team Size) I can calculate the number of available man-days in the sprint (number of days x number of people), and then I can use that to calculate the “Estimated Story Points per Man-day” and “Actual Story Points per Man-day”. It will say something like “1.5”. It means that for every man-day the team completes 1.5 Story Points.

I’ve made an Excel (I always do ;-)) for recording velocity like this. I call it “Velocity Log”. It also has some nifty charts to plot velocity and trends over time. It’s available for download here:
VelocityLog-v1.xls

As an example, here’s the velocity of my current team over a couple of sprints:

Let me know your thoughts.

Why estimate using Story points?

The other day I was talking about why bothering using Story Points instead of just estimating everything in hours. Here's a link to the slides.

Traditionally, the aim has been to try to estimate time by looking intensely at a problem (or a task, activity, requirement,…) – so traditionally there’s a lot of effort spent on breakdown, analysis & estimation. But at the same time, we all know that estimating time is very hard and often pretty inaccurate – because software development is research-oriented and there are things that surface only once we start digging in, which then change the picture. We all know this – yet those time estimates tend to become commitments down the line.

I argue that it’s not only the problem (task, activity, requirement,…) that is hard to foretell; the environment where we solve the problem also has a significant effect on the amount of time it will take to complete. And often the environment is much harder to foretell than the problem itself. So, in the end it doesn’t matter how long time you spend analyzing, you will still have a large unknown in the shape of the environment.

The point I want to make is that estimating in Story Points is about recognizing that Time is a result of the Size of a problem solved within a certain Context.
Time = Size x Context

Story Points is a unit for representing Size. This way Context becomes something we can grasp by comparing Size to Time – which is why we measure Velocity.

Agile Version Control - another magic bullet

I have previously posted articles about what the magic bullet of agile software development is; to achieve flexible scope. As you may recall, the article talks about the three most important aspects of your Scrum implementation which make up the fundament for being able to achieve this scope flexibility:

• Strict prioritization (having it and following it)
• Definition of Done (having one and following it)
• Story breakdown & formulation skills (understanding its importance, and constantly trying to improve)

In hindsight, I feel I have left out one important success factor when it comes to reaching a truly flexible scope; version control.

Most of us use one form of version control or another, but in my experience it is often (mistakenly so) only a matter of choosing the tool. The options are many, and range from “open source” to “incredibly expensive”. Once you decide on the tool, you use it for version tracking your codebase by checking out, committing and merging files. Perhaps you use branches in order to separate some work from other. But you may be missing an important tool; an Agile Version Control Policy.

If you do Scrum or some other form of Agile Development, you want to be able to actually finish your product increment every iteration, right? The purpose is to be able to achieve a flexible scope so that you can potentially ship on any day, at any given time. And what I am trying to get to here is that it doesn’t matter how well you succeed with your adherence to a Definition of Done if you don’t combine it with applying an Agile Version Control policy (completely regardless of which version control system you’re using). If you don’t have an Agile Version Control policy in place you risk mixing up your “Done-Done” code (code and assets related to User Stories that have been completely done according to the Definition of Done) with code which is in progress.

And guess what: if you mix Done code up with Code in progress, you’re no longer able to release at any given time!

Without Agile Version Control, shipping your software will mean that you first have to perform a series of tasks related to isolating code that is Done-Done from code that is in progress at that moment, and also often tasks related to doing last-minute polishing “while you’re at it”. The result is that shipping code is hard and a “big deal” and something that takes a lot of time. And what else does it cause? Yeah: crunch.

So how do you achieve this separation of Done-Done code from code in progress? Well, there are plenty of good articles out there to read: this one for example, written by Henrik Kniberg.

In short, what you need to do is dedicate a branch for containing that Done-Done code (let’s call this branch “Done Branch”), and define some rules for it. For example "all code on the Done Branch must adhere to the Definition of Done", "no code may be published to Done Branch without being reviewed", "code may only be published there by the branch owner", and so on.

Simplicity ftw! Here is a very simple Agile Version Control policy that I’ve been using;

Live Branch
Description: Create a branch from the trunk when you make a public/final release, that you will maintain for a while (until next release). Maintenance (patching) happens (for sake of simplicity) directly against this branch.

Branch policy:

• All code in Live Branch must be tested and working.
• All code in Live Branch must compile & build. In short: all code in Live Branch must follow Definition of Done.
• Code may only be published to Live Branch by branch owner (who may delegate it).
• Patches on Live Branch are immediately published to Trunk as the last step, and any conflicts are resolved on Trunk until code adheres to the branch policy. Note that you don’t resolve the conflicts on the Live Branch because it would imply updating Live with whatever is on Trunk (which can be a lot of changes).

Done Branch (which happens to be the Trunk, aka Main)
Description: Use the trunk (aka main branch) as “Done branch”. This will contain only code from stories that are Done-Done.

Branch policy:

• All code in Done Branch must be tested and working.
• All code in Done Branch must compile & build. In short: all code in Done Branch must follow Definition of Done.
• Code may only be published to Done Branch by branch owner (who may delegate it).

Development Branch
Description: Create a branch from the Trunk in which the team works during the sprints. The branch exists across several sprints.

Branch policy:

• All code in Development Branch must compile & build.
• The Development Branch should be updated from Trunk often (suggest daily).
• Conflicts with Trunk are resolved in Development Branch until code adheres to the branch policy.

Remember to update your code often from Development Branch (i.e. spark merges as often as possible!).

This simple policy relies only on three branch types; Live, Done and Development. Your Trunk (aka Main) is your Done Branch. There is a drawback with this policy though; if you have multiple teams who are developing (and finishing) stories which are supposed to be able to release independent of each other, then you will not be able to use the policy above (because Done-Done code from the two teams will be mixed up in the Done Branch). But, do you really have the requirement of teams to be able to release code independent of each other at different dates? Think carefully about this – do you really? If you do, fine: in that case you have to have a slightly more sophisticated setup (with a separate Done Branch + a separate Development Branch per team, and consequently you can’t use the Trunk as your Done Branch). You can read more about such a setup (Agile Version Control in a multi-team environment) in the paper by Kniberg that I referenced above.

As always, I’d be glad to hear about other peoples’ experience in this area. Please leave comments.

When Scrum meets traditional quality assurance

The contents of this post has been growing on me for some time. It’s about the challenge of working with Scrum in an organization where there’s a lot of traditional (eg waterfall based) ways of doing things.

Let’s say you’re the manager of a team that develops and maintains some software system. Let’s say you’ve successfully introduced Scrum in that development team and they’re doing well and things are progressing. But, you don’t have any “Testers” on your team. Yet. Whatever testing is done is performed by the programmers themselves, in a more or less ad hoc way. Now, there’s a whole “QA Department” in your company, with resource(s) available for you. But it is an entirely different department that is not under your control. All you can do is request and allocate one or more resources from QA – but you’re not in a position to affect how they work. They are used to working according to a waterfall style project model. The processes that they follow rely on a “requirement specification” to exists ahead of time, before the implementation begins, together with a detailed and approved UI design ahead of time, a detailed and approved system design ahead of time, and so on. So the Testers from the QA Department will not be able to assist with any testing unless you have these things in place for them to start building test cases and test specifications on. What do you do?

One of the most central challenges for me right now with regard to Scrum & Agile Development is exactly that: How to continue working with Scrum in my development team and still manage to cooperate and integrate smoothly with another department that is not yet working according to Scrum or the Agile philosophy.

Is it possible to follow agile principles in only one part of the software development process?

So far, since I haven’t had any testers on my team, I have more or less been ignoring the problem and simply done things cowboy-style. I’ve run the project as a pure Scrum project, with user stories in a Product Backlog, with a Scrum Product Owner, with velocity measurement, etc, etc. And – in line with Agile – we’ve worked in a strictly prioritized order, so consequently we haven’t spent much time yet on those lower priority backlog items, and they would certainly not yet be possible to use for formulating any “Test Cases” in the QA Department’s meaning of the word. But the time is coming where my Scrum team and their way of working will meet the QA Department and its way of working. Will the two fit?

Don’t misunderstand me! I’m really looking forward to getting those QA resources on the team, and I’m 100% confident, regardless of philosophies and project models, that quality will benefit from the addition of QA resources – no question about it. But I’m worried that the benefit may be at the expense of “quality” in the development process itself (in the sense of me having to “sacrifice” certain Agile cornerstones).

In my view, “testing” and “quality assurance” is not something that is applied on top of programming as a separate stage. Quality assurance is something that is intertwined with the entire software lifecycle; from idea, through analysis, design and development, to integration and system testing, rollout, handover and maintenance. And since it is all intertwined inside the software lifecycle, it is not possible to limit Agile philosophies to programming only. To be precise, and to answer my own question above; No I don’t think you can be “agile” in an environment where you finalize a “Requirement Specification” at the beginning of the project, and then base everything on it; plans, designs, test cases, etc.

So, my plan is that together with the coming QA resource we will try to find a way for us to meet half-way; to find a way of working that requires minimal change in the QA processes and still minimal change in the Scrum implementation. I really think this can be done.

Here’s the approach so far;

What does the QA Department do? Well, they test software. What information do they need in order to be able to test? Well, they need a test plan, and they need a set of test cases. And what information is needed in order to create the test plan? A schedule, probably. Great! I can deliver a schedule, no problem. I just have a slightly different way of creating it than traditional projects.
And what information is needed in order to formulate test cases? “Requirements”! Could I deliver the backlog as “the requirements”? I suspect no. Darn. Why? Well, the User Stories alone are probably not detailed enough for QA to base test cases on; the level will be too high and there will be too much room for ambiguity, especially before we start working on the story. However, once we’re done with each story we’ll know more about the requirements. I don’t want to have to deliver the requirements ahead of time.

But wait. Didn’t I say that QA is an integral part of the software lifecycle? Yes I did :-). So what is Scrum? Iterative and Incremental. Can’t the “Requirement Specification” also be created Iteratively and Incrementally? And consequently; can’t the set of Test Cases (the Test Specification) also be created Iteratively and Incrementally?

So, the goal for us now is to make incremental deliveries of the “Requirement Specification” and of the “Test Specification”, in parallel to the incremental deliveries of software – sprint by sprint. We’re just getting started and are working out the practical bits in order to achieve this. For example, we’ll include in our Default Definition of Done that Requirements should be formulated. Secondly, we’ll keep track of which Stories are Done-Done (meet Definition of Done) at the end of each Sprint. The tester will use that information as input for his work. So at the end of each sprint there’s a new set of finished Stories accompanied with Requirements, that the tester can use as input to create a Test Specification increment. This way, the formulation of test cases is done one sprint behind the development. It might not be optimal, but considering that the tester is not full-time allocated to the development team, doesn’t sit with us, and that I’m not really attempting to reshape the entire QA department’s process all at once at this point :-), I think it’s a fair construction and a good trade-off.

So. That’s where I am right now. I’ll post an update as we progress!

I’d be glad to hear about other people’s experience about this type of situation.

How to adopt Continuous Refactoring?

This is a question that I have been asking myself, as well as received from others, a bunch of times. I am posting it here in the blog since I don’t have a ready-made answer. Please leave some comments on this if you have suggestions.

To clarify the question, let me begin by setting the scene for you;

Let’s say you are introducing Scrum. The team is, as always, a bunch of developers with varying experience, competence, and ages. You have read my blog ;-) as well as other even better sources of Scrum & Agile ideas and opinions, and you understand that the concept of “continuous refactoring” is central. You realize that anything is less likely to succeed if you try to solve everything at once in one single shoot; so it is better to work iteratively and deliver in increments, in order of priority. How do you get this message through to the developers, so that they really adopt it, wholeheartedly, in practice?

Because the problem here, in my experience, is that most people take a lot of pride in their work and “continuous refactoring” is easy to mistake for “rushing things out”. Some people just can’t get used to the idea of not attempting to do it “perfect” or “complete” from the beginning, and thereby find it extremely hard to deliver something that “just works” and is “good enough” for that particular product increment. Obviously, people are different (and so are programmers ;-)), so some will easily adopt this way of thinking, while it will be much harder for others.

How do you do it? It probably has much more to do with psychology and leadership than with Scrum per se, but I still think it is a relevant topic to bring up within the boundaries of this blog.

My suggestion on how to go about this is to educate. Talk about it. Have trainings. I have had lengthy discussions about this topic in the Scrum training sessions that I’ve held, and I can’t say that it has magically solved anything just like that, but I think the key to any change is for people to understand the need for it – and the first step for that is to talk about the problems surrounding BDUF (Big Design Up-Front, which is sort of the opposite of Continuous Refactoring).

Please leave some comments.

New employer - new Scrum & Agile challenges

Unfortunately I haven't had much time to write here lately. My excuse has to be that I just started a new job with an employer a little bit closer to home. I'm very excited about this opportunity, as the company I have now joined is more established in terms of its organization, practices and processes. First time really that I work for a company that isn't in the startup phase.

I'm just getting to know the people now, and studying the development and project models, so no new thoughts about Scrum or Agile just yet. Really looking forward to experiencing development practices in a mature organization (that supposedly have already adopted Scrum, in some way).

I'll be sure to post thoughts and experiences about this new Scrum & Agile journey, as I go.