Agile testing: Importance of a testing strategy

As you know from our history we didn't have much (if any) systematic testing going on, when we decided to introduce Scrum back in 2009.

A history of no testing equates to several challenges:

• The system is (was) not very well tested
• The codebase and architecture is (was) not very testable (it hasn't been written to be tested)
• The developers are (were) not used to having to care much about "testing"

It was important for our success that the strategy was clear from the beginning, from a management perspective. Testing had to be made important, it needed to be lifted up from the bottom to the top of the priority list. And, as they say, the proof is in the pudding. From a management perspective such as strategy has to be backed up by the attitude of allowing quality assurance to take time.

Our aim from the beginning was the following:

• We should introduce the Tester role in the team and clearly define its responsibilities (certainly harder than it sounds, more on this topic later).
• We should immediately stop taking new shortcuts (i.e. stop increasing our Technical debt)
• Test early and test often. All functionality implemented during a sprint should be tested and, if needed, corrected during the same sprint; while still having a reasonably short sprint length (i.e. two weeks maximum)
• Every user story that we implemented and delivered in every sprint had to be tested
• Create a minimal set of test cases that we could start using to verify the fundamental functionality of the system
• Be pragmatic and realize that we can't go the entire distance in the first sprint. We had to accept that despite our intentions we wouldn't have a truly releaseable product increment after every iteration. There would be critical bugs that we wouldn't have time to fix, there would be too few test cases to cover all functionality, etc. But striving towards the goal was a great start, and it meant we got better and better with every sprint.

Since we wanted to include testing in every sprint (and each sprint was just 1-2 weeks long) we had to figure out a way to expand test coverage without spending unreasonable amounts of effort. The solution (of course) was to automate testing. More on that in coming articles... :-)

Agile testing: Our background

I usually refer to the group of people I manage as "my team", but in fact we are several teams today.

When the development of the product first started back in 2004, as a prototyping/proof-of-concept effort, the "team" then consisted of only 1-2 developers and a very visionary leader (not me :-)). As the prototype became more and more potent and the market showed a positive response, the team was allowed to grow but it was still, in many ways, in a "prototyping/proof-of-concept" mode.

Up until 2008/2009 there was very little systematic testing going on. The development didn't follow any particular development model or strategy. But on the other hand the developers were very skilled, highly motivated and committed, passionate even, largely I think because they had a lot of influence on what went into the product back then. They all pretty much had their own specialist area and their own agendas in terms of what "cool stuff" they wanted to work on and put into the product.

And that was all ok, most of it was intentional, and it obviously worked well. In fact, it was a necessity, I think, to have that way of working in order to achieve what was achieved back then; to quickly get a proof-of-concept out there to check if the idea worked and if there was a market for it, and build it from there.

However. As the team size grew it became increasingly obvious that some structure was needed. As more and more customers installed the product the need for control from a product development perspective grew too; i.e. the need to be able to decide exactly what features the team should work on and when. Likewise, the need for maintaining project overview grew. And so did the need for testing. The result (i.e. the quality and contents of a release) needed to become controlled and predictable. Also, it was desired that the team should adopt development processes and practices used in other parts of the company - and hence no longer be allowed to work in such a chaotic unplanned/uncontrolled manner. It was time to align the product, the team and its way of working with the rest of the company.

It was at this point that I joined, early 2009, and we immediately started introducing Scrum and the Agile philosophy. There had been some efforts to introduce Scrum ("Scrumish") in other parts of the company but there were no success stories, and nobody had really tried to implement Scrum "all the way".

To give you an idea of what the situation was like before the Scrum introduction, here are a few characteristics (end of 2008):

• Relatively unorganized development efforts
• High entrepreneurial spirit in the group
• Plenty of shortcuts taken (Technical debt created)
• Unclear project control/steering
• Highly motivated, committed, self-organizing individuals
• High degree of freedom and innovation spirit
• Very little degree of cooperation among engineers
• Direct contact with customers
• No structured testing
• No "tester" role existed
• Definitely no automated testing
• No process improvement efforts
• Aging codebase, with a continuously increasing technical debt
• No requirement or test specifications
• The work didn't follow the established policies and processes within the rest of the company

And here is that wishlist, i.e. what we wanted to achieve:

• Maintain same or achieve better efficiency
• Maintain same or achieve better level of commitment and creativity
• Maintain same sense of freedom and influence
• Possibility for a "product manager" to be responsible for, and in control of, the direction & content of the product
• Reliable and repeatable delivery result
• Possibility for maintaining project overview
• Stop increasing the technical debt (no more shortcuts)
• Partial refactoring of the codebase/architecture to improve/remove some of the prototype code that remained
• Introduce systematic testing of some kind
• Use the resources available in the company's QA department
• Customer Support department to handle direct customer contacts
• Start following the established processes and policies within the rest of the company

Thankfully, largely due to good, flexible, patient and somewhat daring managers (such as Head of QA, Head of our department, the Head of R&D and also Product Management) my team was allowed to deviate somewhat from the established (largely waterfall-based) processes in favor for a "true" Scrum implementation.

Now, almost three years down the line, we've come far. The team is much bigger and we (think we) work in an agile way and have more or less fully adopted the Scrum methodology.

Here are a few characteristics of how we work today (2011) so you get an idea of what changes we've struggled with:

• We are now 4 cross-functional Scrum teams working in parallel on the same product
• All team members sit together
• 5-7 persons in every team, including 1-2 testers
• Still highly motivated and committed team members; both the "old" and the new
• Highly self organizing teams with a high degree of and natural focus on cooperation
• 1 Scrum Master (might not be optimal now that we've become as many as 4 teams)
• 1 Scrum Product Owner
• Each iteration is 9 work-days long (starts on a Thursday and ends on a Tuesday)
• Estimation is done in Story points
• We measure Velocity and we keep a Release plan up to date, and base time commitments on, measured velocity
• High degree of test automation; 1350 automated function (blackbox) tests, 100 automated "GUI tests"/"end-to-end tests", 2534 automated unit tests and around 110 manual test cases
• Small working (completed) product increments are delivered every iteration
• Our goal is to be continuously releaseable (truly releaseable after every sprint) but we are not there yet
• Natural and continuous focus on process improvement by every team and team member
• Continuous and reoccurring discussions about efficiency and quality
• We have a strategy (and resources) for handling "old" bugs found during sprints
• We have a strategy (and resources) for replacing some of the manual tests with automated tests
• We have a strategy (and resources) for writing automated unit tests for old (legacy) units and not just new ones

A lot of work remains and I'm sure we wouldn't pass any Scrum Compliance Test (even if there was one) with a 100% score, but we've come a long way compared to where we were only two-three years ago. And besides; we haven't adopted Scrum for the sake of adopting Scrum.

In following articles I'll describe and elaborate on some of the experience and lessons I think we've stumbled across during this journey, in particular lessons related to testing and quality assurance (as this article series focuses on Agile Testing :-)).

Agile testing: Start of article series

Wow. It's sure been quiet on my part for a while now. I guess that is what happens when you have kids; priorities in life change. With a 1.5 year old at home and a full-time job I've found myself having virtually no time to spare to write these posts about Agile development and Scrum.

However, I was recently asked by a colleague to come to the Q3 meeting of SAST Öresund (Swedish Association for Software Testing: http://www.sast.se) and hold a 45-minute presentation on the topic of Agile Testing (under the theme "Agile methods in practise"). This was the first time in a while that I've had the chance, and the challenge, to really focus my thoughts around an agile topic and had do an in-depth analysis of my own opinions and experiences.

The presentation was quite interesting and great fun to do, and from what I could tell the interest from the audience was quite high.

So. In the name of openness I figured that I'd share on this blog the topics I brought up in the presentation. I'll post this as an article series and do a bit more elaboration on each topic than what is possible in a brief concentrated 1-hour seminar.

So keep on popping by this blog from time to time, as I'll be posting articles in this series here as often as I have time.

And as always, feel free to drop me a comment with your opinions and own experiences, it's always interesting to hear what other people think.

How to fail with a Scrum introduction

The journey of introducing Scrum in an organization is a huge topic in itself, and nowadays there are loads of books and articles describing different aspects of this. Tens of thousands of companies have successfully adopted an Agile methodology. “Scrum” is no longer just a buzzword as it was a few years ago. The hype, the initial excitement and thereby, hopefully, some of the overconfidence has settled a bit. At least that is the feeling I get when speaking to managers and colleagues “out there”. One would think, then, that the process of introducing Scrum would be easier today. With all that collective experience in the industry, with all those books and articles.


But looking around at all those companies and organizations who claim they have adopted (or tried to adopt) Scrum, I wonder how many of them are/were really doing Scrum “by the book”? How many of them modified Scrum beyond what could still reasonably, with a clean conscience, be called Scrum – and struggled and had problems with what they thought was Scrum because of it?


It doesn’t really matter in the successful cases, where the modifications to Scrum work and the result is something that is at least as good as whatever was being done previously. I guess those organizations would still, in the end, claim that their Scrum implementation was successful – regardless of it really is Scrum or not. But what about all those cases where an organization tried Scrum, concluded it didn’t work, and maybe even decided to go back to dysfunctional..err..sorry, traditional, waterfall-based approach? How many of those ever stopped and pondered on the reasons – the root causes – of why Scrum didn’t work? Ever heard something like “Well, Scrum just didn’t suit our organization/business/team, I think Scrum is over-rated and as I thought it wasn’t the magic bullet answer to everything”?


Scrum and Agile is so simple and made up mostly of common sense behaviors and values, that if it really doesn’t work for you then you’re really special. And no offence but it is likely that you’re pretty much like everyone else – no matter how special you want to believe you are.


So why doesn’t Scrum work?


The person responsible for driving the Scrum introduction – the “Scrum driver” – is key to the outcome of the change. In my experience there are four (five) common traps that a Scrum adopting organization risk falling into. All relate to the Scrum driver:

1. He or she lacks sufficient knowledge about what the Scrum methodology and the Agile philosophy is all about.
2. He or she is also a Project Manager (or a group of Project Managers).
3. He or she lacks enough time to focus on things like coaching, monitoring, adjusting, educating and motivating the Scrum teams and the stakeholders.
4. There is no plan on how to introduce Scrum.

Maybe a fifth one should be listed also, for the record; the failure to realize that there is need for a “Scrum driver”.


Lack of conviction


I think a common reason for failing with Scrum is that the introduction is driven by someone who lacks conviction and (true) passion – i.e. someone who consequently lacks credibility in the eyes of all stakeholders. Introducing Scrum requires a firm and honest belief in its benefits. The need for conviction is not unique to a Scrum driver but apply to any change process. Introducing Scrum will mean drastic changes for everyone; team members, project managers, stakeholders and managers surrounding the development organization/team. The ways of thinking, the methods, attitudes and expectations will need to change. From day one. You can’t introduce Scrum in steps. It’s All in – or leave the table. And people are naturally conservative. It’s always easier to stick to what you know and have experience with. The unknown is scary. Especially at times of stress, when the going gets tough. That’s when conviction and passion will be required in order for the changes – the Scrum methodology – to persevere.


A Project Manager is put in charge


A Project Manager ought to be the one with the best understanding of how to run a software development project. Right? They are (mostly) educated, intelligent, communicative and multi-tasking individuals with loads of real-life experience of stressful situations, risk management and change management. They are used to following checklists, processes and project methodologies. If anyone should be able to introduce Scrum it would be a Project Manager. Right? Well. Would you let a taxi driver be responsible for introducing free subways? Silly analogy perhaps, but my point is that it would be fair to expect their level of commitment to be less than maximal.


Don’t get me wrong, some people are less conservative than others. Open-mindedness is a property that some people have more of – and some less. Introducing Scrum will without a doubt affect the every-day work of the Project Manager, their entire role and responsibility. It may even eliminate the need for a Project Manager altogether. Therefore, to me at least, it is simply not natural that the project manager by default becomes the person responsible for driving that change. They are an involved and affected party. Someone else should be in the driver’s seat, someone like a line manager, a manager of the project office, whoever is the project manager’s boss, or even someone external to the team.


Underestimating the challenge


Introducing Scrum isn’t free – and I’m not talking about a few thousand bucks to get someone a Scrum Master certification, or a couple of books. The adopting organization will need to put the money where its mouth is. In my mind it is not enough to only educate whoever becomes Scrum Master. The entire team needs to have a good (or great) understanding of Scrum; about what problems it solves, what are Agile attitudes (and what are not), what parts of Scrum are fundamental and about what parts can be tailored to the organization/business/team (and which can’t). Educating an entire team will cost time and money.


And education is only the beginning. Using Scrum will require lots of monitoring, coaching and adjustments along the way. Continuous improvement. Inspect and adapt. At first that will be managed by the person driving the Scrum implementation, and therefore will require a lot of time & effort from that person to stay close to the team, to keep up-to-date – by the hour – about what’s going on in the team and outside. But as the Scrum team grows used to the new way of working they will start helping each other out, correcting each other and helping each other with staying within the boundaries of Scrum.


Another aspect often underestimated may be that of getting everyone onboard. Getting the team members to accept Scrum is a pretty obvious challenge that has to be dealt with. The same should be true for the project manager(s), and maybe even the customer(s). But what about surrounding managers, departments and other roles and neighbors within the organization? Those are the people that will be indirectly affected by the Scrum introduction. It is important to have a good idea about who those persons are, so that you can actively work on their expectations. Take control. If surrounding expectations are not in-line with what Scrum is capable of you’re in trouble. That’s why it’s a good idea to form a plan on how to get them to accept Scrum.


Lack of strategy, lack of plan


It’s surprising (disturbing), I think, how little planning is generally made for introducing Scrum – considering that it happens in an environment where people are usually very used to making rigorous plans, managing risks, etc.
A sufficient “plan” in this case would mean taking the time to think about things like;

• Why are we introducing Scrum? What are the expected benefits? Are they realistic?
• What are the expected drawbacks of introducing Scrum? Will all stakeholders accept those?
• Who will be in charge of the Scrum introduction, i.e. who will be the “Scrum driver”? Is the person in charge also a Project Manager?
• Which persons will be directly affected by the change? Is the Scrum driver one of them?
• Which persons will be indirectly affected? Is the Scrum driver one of them?
• What is the level of credibility, passion and conviction of the Scrum driver? Is he/she able and ready to be a source of inspiration to others during the change process?
• Are all affected persons (directly and indirectly) onboard?
• Do we have any key people to get onboard as assisting informal Scrum advocates within the organization/business/team?
• What is our education plan (plan for getting people on the bus) for people directly as well as indirectly affected by the Scrum introduction?
• What is the budget for “education” (getting people on the bus), in people-time or in money, for the persons directly affected by the change?
• What is the budget for “education” (getting people on the bus), in people-time or in money, for the persons indirectly affected by the change?

Summary


Introducing Scrum is a challenge. But I’m pretty sure most would agree that it’s worth the effort, if you can get it to work for you. But please don’t underestimate the challenge. And please consider letting someone other than a Project Manager be in charge of the Scrum introduction.


Whoever is put in charge of the Scrum introduction it needs to be someone with:

1. mandate and authority to implement the change; and
2. someone with a wholehearted belief in the benefits of the change; and
3. someone with a budget for the extra effort caused by the Scrum introduction.

Let me be clear on this: I don’t think that Scrum really is the solution to everything. I’m sure there are many cases and circumstances where it simply doesn’t suit the organization/business/team. But if you tried it and failed, I think it’s worth to ponder on whether or not it was Scrum that really failed – or the implementation of it.


As always, I’m interested in getting some feedback via comments on this blog.


Cheers.

Overcommitment is better than undercommittment

This time I want to talk about the amount of user stories that a scrum team decide they feel comfortable committing to at the beginning of a sprint (at spring planning) - and what happens when they're approach the end of the sprint.

At sprint planning, have you ever had a discussion with (or in) your team about whether or not to include that last story or not in the sprint backlog? The one that they're unsure they'll be able to complete. What did you/they decide, and why? Did you hear anyone say something like "Well, let's stick with these fewer stories - we can always include that other one later during the sprint, if we should find ourselves with time to spare"?

If the team undercommits, i.e. includes a set of user stories that sums up to a lesser scope (a smaller sprint backlog), it means that they're reasonably likely to complete all stories committed to - and at the end of the sprint they have a good chance of getting a feeling of accomplishment and success. They delivered all the stories! Yay! And, as indicated by the quote above, they can always add that extra story at the end of the sprint if they complete the committed sprint backlog early. Right?

Constant overcommitment - on the other hand - might cause a feeling of failure, as the team never really manage to deliver what they commit to.

But, to be honest, how often has your team found themselves with time to spare at the end of a sprint? And how many times have they included that extra story from the top of the backlog at the end of the sprint? Somehow, the time available in a sprint always seem to be just enough for exactly (or less than) those stories committed to at the sprint planning - rarely more.

While I think we need to recognize and respect the potential negative consequences of constant failure by constant overcommittment, I think we also need to remember that people often need (and like) a healthy amount of stress/pressure in order for them not risk producing waste by extra effort (refer to Lean Software Development for more information about extra effort and waste).

The Agile approach is of course to try both alternatives a couple of times, and then inspect and adapt. Find out what suits your team(s). In my personal preference, however, I think reasonable overcommitment carries greater benefits than drawbacks.

Either way, the team owns the decision. It's their commitment. I for one want to remind my teams about and get them thinking about this for themselves. Keep it in mind at sprint planning.

There is a name of this phenomenon: Parkinson's Law; "Work expands so as to fill the time available for its completion" (http://en.wikipedia.org/wiki/Parkinson's_law).

Another approach used by some, is to not expect the team to include more stories during a sprint even if they finish early. As a general rule. The thinking behind that practice is that the team's commitment at the beginning of a sprint is what they agree on with the scrum product owner. And since that's what the scrum product owner expects, that's what is enough to deliver. Any spare time at the end of the sprint is used for other things; anything from free time (off work), lab activities, or whatever other fun things you could think of.

I think this approach is excellent in theory, and it sure sounds nice; if you finish early, you can go home. However as a general rule, in practice, when a deadline is approaching and you want to get as far as you can down the product backlog, i doubt that it would be possible to stick to this. Let me know if this works for you. I'm curious!

And as always, please give me some feedback. Let me know how you think about and handle this.

Back in business - with questions about Test and Development

Long time no see!

I've been silent for a while now, due to the extremely premature birth of my daughter this spring (she was born over 3 months early). Everything is OK now and she (Emelie, is her name by the way) is doing well. It was, not surprisingly, very chaotic times there for a while, and add to that getting used to being a parent which in itself is a big change.

Anyway, for these reasons I haven't really been focused on working with Scrum, but lately, as I'm getting back in to the groove of things here at work, I find myself thinking more and more about Why's and Why-Not's with regard to Scrum & Agile software development.

My team(s) here have expanded during the year. They're now a total of about 15 engineers (about 1/4 testers and 3/4 software developers). Which brings me to the topic of this post; how do other organizations cope with running Scrum in a traditional organization where there's one department for "Software development" and another for "Software Quality Assurance"?

I've managed to bring the Test Engineers into the team by means of physically locating them here where the Software Developers sit (instead of having the testers sit on another floor). Very cross-functional. Excellent! But, that was the easy part, now to the trickier bit; how do you get the team to think of themselves as one unit with a joint mission (and thereby sharing tasks without particular regard to their formal role) rather than one group of Software Developers and one group of Test Engineers? I often have to remind members of the team (more often Software Developers than Test Engineers, to be honest) that tasks such as "Write automatic test" or "Execute test case A" doesnt automatically have to be done by a Test Engineer just because the task description includes the word "test" - it may just as well be done by a Software Developer.

As you can probably guess I am of the firm opinion that every task on the Scrum Board should be up for grabs by anyone in the team (at least that should be the default), regardless of if says "Software Developer" or a "Test Engineer" on the individual's employment contract. They're all comitted to achieving the sprint goal and it's thereby in everyone's interest that every task is completed.

It all sounds good and is easy enough to grasp in theory, but in reality it's harder to achieve. Partly (I think) because this is a traditional organization where people are employed specifically as "Test Engineer" and "Software Developer". People tend to kling to that, and I sometimes have a hard time getting them to step outside the boundaries of their formal/traditional role. A complicating factor is that it is not naturally bidirectional; a Software Developer usually have no real problem of carrying out a task that says "Test" on it, but a Test Engineer is not as often able to carry out a task that says "Program" on it.
Does this support the notion not to try to share tasks across roles?

Well. I like to think of the role "Test Engineer" as a person with a (usually :-)) formal education and experience in the field of "software quality assurance" and that those guys and girls are those who are skilled in identify testing challenges, quality risks and generally ensure that what they (the Scrum team) build is tested "enough". But "Test Engineer" doesnt automatically mean "Those are the guys and girls who should carry out all the tasks with the word 'Test' written on them".

To sum this up; If it was completely up to me I would get rid of the formal roles and just hire "Scrum Team Members" with different backgrounds (some who are more skilled in programming and some who are more skilled in the field of software verification & validation).

As always, I like getting feedback and comments from you about this and other posts.

See you soon(er)!

It's all about priorities

I've been pretty silent on my Scrum blog lately (well, for the past few months even). The main reason is that we've been expecting a child and we've had a relatively problematic pregnancy; the climax of which led up to my beautiful daughter Emelie's very premature birth on the 17th of March 2010.

She was born in week 26 + 3 days only, so she is 3 months premature. She was 1230 grams, 38cm tall, when she came out. She entered the world at 09:57 in the morning, with three minutes to spare to my team's Daily Scrum!! :-)

Kidding aside, me and my girlfriend Helene are now living at the hospital in Lund, tending to little Emelie's needs and helping her grow and get stronger.

I'm currently on leave from work and I will continue to be somewhat out of the Scrum & Agile loop for some time ahead. We're taking it day-by-day here.

Such are priorities in life.

I keep a blog about my daughter's progress. You can find it here: Liten Emelie (sorry, only in Swedish - she doesn't know English just yet ;-)).

New version of Backlog Manager

Recently I've had some time to spare, so I decided to refactor my Backlog Manager into a new version.

Here it is: Backlog Manager v10.1

It is more robust than the previous versions, and has better support for Release Planning.

Moved/updated Excel tool links

Due to some technical issues the Excel tools that I've made has been unavailable for a week or so. The files have now been moved to a new temporary location.

Here are new links:

Backlog Manager
Velocity Log
Estimation Sheet

What unit to use in burndown?

What unit do you use for plotting your Burndown Chart? Do you use “Remaining Hours” of the tasks? Or perhaps “Story Points” of the stories in the sprint backlog? Or something else?

Up until recently I’ve taught Scrum teams to just sum up the number of remaining hours for all tasks on their Scrum Board, and plot that sum on the burndown chart. That has been my default approach and I haven’t really experimented with it.

Why even bother breaking down a Story into Tasks?
The immediate answer to this question is of course that we do it to be able to sum up estimated Remaining Time. Duh :-). But I think there’s more to it, because a Story is more than just the sum of its Tasks. The real reason, I believe, is to get the team thinking; to get the team to reflect over what challenges the Story hides so that they maintain an as clear picture as possible of where the implementing is heading and how the sprint is progressing. This implies that breakdown of a story shouldn’t be done just once (e.g. at sprint planning). The view of what tasks a story contains changes as the implementation progresses and things are discovered. It’s natural that the contents of the story vary over time. Nothing of what I have written here is controversial. It’s all part of Scrum.

What happens in practice?
Plotting a burndown based on the sum of Remaining Time obviously requires that the team first breaks down the Stories into Tasks. This, in my experience, has always been somewhat of a challenge; for the team to actually end up with useful tasks. A good breakdown of a scope into User Stories is hard enough, and then to also successfully break a Story down into good Tasks … Well, it is hard and it requires lots of practice. I’ve always struggled getting teams to continuously reflect over their breakdown and about how to improve it. But in the end, tasks often seem to end up saying more or less meaningless things like “Implementation”, “Testing” and “Review”. Fail.

Failure in Task breakdown like this effectively prohibits the team from having any use of it. A breakdown like this doesn’t say much about the Story and consequently there is little or no use in continuously reviewing the breakdown to try to learn more about the Story as implementation progresses. The only remaining purpose of the breakdown, then, is to force a daily re-estimation, so that numbers can be summed up and plotted on the chart. The burndown and the estimation of Remaining Time becomes an end in itself.

We started counting "Number of remaining tasks"
My team, being in the situation described above, felt that the daily task re-estimation was (more or less) meaningless, and proposed that they should try and completely skip the estimation all together. If nothing else, it would save them that estimation-time that was borderline waste anyway. But they also realized that the burndown has an important purpose (yesterday’s weather, indicating progress, early indication of problems, etc) so they felt that they still needed burndown. But if they don’t estimate Remaining Time, then what do they plot on the burndown chart? Well, they agreed to try to plot the number of remaining (not done) Tasks. Simple! And not completely new; it’s a fairly common practice, if I’m not mistaken.

Interesting consequences of this change
This change in the process had a very interesting side-effect: all of a sudden the Task breakdown produced a large number of highly meaningful tasks. Instead of just saying “Implementation” the tasks that the team now produce actually describe what needs to be done, in pretty deep detail. The number of tasks produced both at the sprint planning and on the Standups is significantly higher now than before, and they say a lot more about what needs to be done.
It just seem to come more naturally now. The only difference is that we no longer estimate the tasks afterwards. That’s it. I think this is very interesting. It’s like if there was a mental block before, that has now been removed once the team doesnt have to bother estimating the tasks. I don’t know, but whatever the reason, I think it's great. The team is satisfied with the change. I feel that the quality of the task breakdown of the Sprint backlog is much higher now, and consequently the understanding of the team’s progress is much better (both for themselves and for me as a stakeholder). The project is more transparent.

The drawback of this change might be that the burndown probably is a little less accurate, because the size of tasks vary a lot. On the other hand, how accurate was the burndowns before, when we were estimating remaining time almost as an end in itself? Hopefully the accuracy now is not much worse. And if it is, I think it is outweighed by the gain in insight and increased feeling of progress.

Let me know how you work with tasks, and why… As always I’m curious to get some feedback.

Agile projects are never late

In the post I want to talk alittle about "late projects". It is a topic that is somewhat current for me and I have spent some time pondering about this phenomenon :-).

The question I ask is: What does it mean for a project to “be late”?

Well, in my view, in order to “be late” there has to be a target-date or a target-state (e.g. deadline or a certain scope or both) that the project’s current state is compared against, and the result of the comparison is that the state is such that either expected date or expected scope isn’t achievable.

If we can agree that this would be some reasonably valid definition of “being late” then it implies that the following factors are involved in “becoming late”:
• Date
• Scope

When a project “is late” it means that (for whatever reason) what was expected to be ready at a given time, isn’t. Date doesn't match scope - or vice versa.

Now, let’s assume that I fix the date, i.e. I commit to delivering my software on a certain date, but I don’t commit to an exact scope by that date. Instead, I make sure that the scope is allowed to flex so that I can deliver only whatever is ready by the time I reach the date that I have committed to. As you all know by now this is what Agile is all about. It is OK to commit to either date or scope, but never both.

Is it now possible for my Agile project to “be late”?

And I believe the answer is "No". An Agile project can never “be late”, because it is impossible. “Late” is a concept that does not apply to Agile projects. By allowing the scope to flex you effectively remove one of the two required factors for becoming late in the first place. Consequently; if the project is late, then it is not an Agile project.

One could of course argue that “being late” can also be when a stakeholder desires the scope to be in a certain state sooner than it actually is. However, in my book this is called “wishful thinking” on the behalf of the stakeholder, and it has nothing to do with the project “being late”. That needs to be handled by proper management of expectations – which Agile and Scrum projects in particular solve by opening up the project with full transparency to stakeholders (velocity measurements, release plan based on velocity, highly visible burndown charts, deep involvement of customers and other stakeholders, short feedback loops with regular deliveries and demos, etc).

What the stakeholder above has to realize is that the scope will be delivered as fast as reality allows. The state the project will be in on any given date is determined largely by the “context” in which the project is being realized. When I say “context” I mean all those factors that are affecting how fast the realization can happen: the competence of team, the cooperation of team members, the availability and effectiveness of tools used, the ability for the team to focus on the task (i.e. disturbances), the mood & motivation of individual team members, the availability of information, the clarity/understanding of what is requested, and so on.

So, let’s agree to never again claim that an Agile project is “late”.

OK? :-)

Scrum. You're doing it wrong when...

…your sprint is so long that you feel you have to make a time-plan for it.
…your Scrum Master tells people in the team what to do and when.
…the team report their progress to the Scrum Master.
…estimates are not done collectively by the team-members who will be doing the work.
...you hear one team-member say to another; “Sorry, I’m really focusing on my own tasks here in order to finish by the end of the sprint, so you will have to solve your own problem yourself.”.
…all stories in the sprint finish on the last day of the sprint.
…lower-priority stories in the sprint gets done before higher-priority ones.
…the team has as many stories in progress as it has team-members.
…the Scrum Product Owner can’t explain every single story in both the Product Backlog and the Sprint Backlog.
…you still think it’s a good idea to write down all requirements before you start coding.
…you consider customer feedback a risk.
…the team doesn’t know who their Scrum Master is.
…your system testing happens all at once, after a couple of sprints.
…you still think that a detailed Gantt chart is a useful planning tool for the software development tasks.
…your sprints are 6 weeks long.
…the team’s burn-down has been flat-lined for three days and they’re not discussing why.
…you think the customer knows what he wants.
…you think the team knows how to build what the customer says he wants.
…the team doesn’t know how much work is remaining of the sprint.
…you think it’s OK if the team has 45-minute Daily Scrum meetings.
…you come running to the team after a sprint and yell “Hey, we have to release now!” and someone in the team responds “…Oh, but wait, we still haven’t done any system testing… And there’s some more code review to be done too.”.
…the team doesn’t know when and where the sprint demo will be held.
…the team constantly over-commits.
…you think it’s OK that the team has their Daily Scrum meetings every other day instead; “It’s their choice, and meetings take so long, so….”.
…the team is persuaded to add more to the sprint than they really want.
…the team accepts to include stories that have not been estimated.
…you don’t collect feedback from the customer after every sprint.
…the team doesn’t know who their Scrum Product Owner is.
…your team is larger than 8 team members.
…the team doesn’t know what the sprint goal(s) are.
…your Scrum Product Owner misses a sprint demo or a sprint planning every now and then.
…the team decides (or doesn’t follow or care about) the order of the stories in the sprint backlog.
…you think the Scrum Master is sort of like a Project Manager.
… you think Scrum doesn’t require or need planning.
… you think you can’t have a deadline or commit to a scope "because you’re using Scrum".
… you think you don’t need to document "because you're using Scrum".
… you commit both to a certain scope and to a certain date.
...you need to add stuff to the sprint half-way through.
…you think the Scrum Product Owner is sort of like a Project Manager.
…the team doesn’t know their velocity, or the meaning of it.
…the team haven’t met the Scrum Product Owner for a week.
…the contents of a sprint demo comes as a surprise to the Scrum Product Owner.
…the team doesn’t know or understand the overall vision of what they’re working towards.
…you can’t release the software after every sprint.
...Scrum doesn't work.

Follow-up on the Scrum Practitioners meeting Oct 15th

Thank you everyone who participated on the Scrum Practitioners meeting on the 15th of October at Axis Communications in Lund.

Some of the things discussed were;

• Distributed Scrum; does it work? A few of the participants had some experience in this, and the opinions were mixed. Bottom line is; yes it is certainly possible to get it to work. I.e. it doesnt automatically fail. It requires work, though. But that is true for any methodology; if you have team members on a distance you have to deal with the overhead cost in terms of extra communication, tools, activities, travel, whatever. Depending on how the distribution looks you can try different approaches; divide into teams per site, or distribute teams and work with conference solutions, digital whiteboards, etc for the daily meetings. Experiment. Distributed teams and distributed Scrum is not ideal, but if you're faced with a situation where you have a distributed team, don't automatically rule out using Scrum.
• How/if sort out design-issues before sprint start? The problem discussed was that of things taking a long time to finish if you do "design stories" in one sprint to discover what/how to implement, and actually implement in in the following sprint. With 4 week sprints it can take you 8 weeks to finish a story, even if it is not that big. The concept of "spikes" was discussed, i.e. prototyping and delivering a piece of working and valuable code in each sprint, rathen than doing "only design" separated from "only development". If you do "design stories" it may be wise to timebox rather than trying to estimate it, because the definition of done for "design" is hard to grasp. Another tip can be to consider shortening the sprints, to e.g. 2-3 weeks instead of 4.
• The problem of traditional, conservative customers, demaning fixed-price-fixed-scope contracts was discussed. The concept of "Money for Nothing and Change for Free" was discussed as an interesting theory. None of the participants had experience with that approach, though.
• Coping with less-than-fulltime resources in a Scrum Team: The problem of traditional customers and product owners who do not realize that Scrum requires a much higher degree of involvement. Not sure we came to any intelligent conclusions on how to address this problem, other than the importance of realizing that it is a problem if the customer/PO is not involved.
  
• Problems with large team size was discussed, and consequently the problem of daily scrums taking too long time.
• Is the traditional project manager automatically the best choice for Scrum Master when introducing Scrum? Not necessarily, was the conclusion. This seems to be the default choice in the industry, but it is not necessarily the best. It depends on your organization, the people, the problem domain, etc. The Scrum Master may be a good choice for the Project Manager, but Scrum Product Owner may be too.
  

Several other interesting things were discussed too, but I will not go into more details here.

I'm looking forward to the next meeting.

October meeting Scrum Practitioners South of Sweden

I'm planning the October meeting for the network Scrum Practitioners South of Sweden. I'll update this blog post as details are decided. For now, the meeting is planned for Thursday October 15. It will be between 17:00 and 19:00, hosted by myself, in Axis Communications' office at Emdalavägen 14 in Lund.

The topics of discussion are (under construction ;-)):

* Distributed Scrum; Does it work?
* How/if sort out design-issues before sprint start?
* How do I get the product owner more involved?
* Coping with less-than-fulltime resources in a Scrum Team
* Is the Project Manager the best choice for Scrum Master when moving from a Traditional to a Scrum organization? Who decides, and who is the best choice?
* Is the "Product Manager" the best choice for "Scrum Product Owner"?
* Story formulation, estimation, story breakdown into tasks and prioritization
* ...<insert topic here>

Follow-up on the Scrum Practitioners meeting Sept 7th

To follow up on our first meeting in the Scrum Practitioners South of Sweden network;

I don't intend to recap all the discussions and conclusions on the meeting, but rather I just wanted to briefly list the major topics discussed as I recall.

The meeting was held as planned at Labs2 in Lund with a total of 8 participants including Mathias Thinsz (Product Manager & Scrum Product Owner at Labs2) who was our host.

Mathias went through and explained the major parts of Labs2's Scrum implementation. It seems to me that they have a very solid and healthy view on how they development software. It's a clean and well-thought-through implementation of Scrum. A great example on how well Scrum works.

Related topics discussed were;
* How Labs2 use JIRA and a custom developed web application for managing the backlog
* How and when Labs2 estimate their work in the backlog, and how they use the storypoints unit
* How Labs2 has divided their development organization
* How and why Labs2 have divided their development time into fixed-length iterations
* How Mathias (as Scrum Product Owner) prepares for a new sprint and who is involved when
* How and why Labs2 use "Sprint Goals" as a major tool for directing development inside a sprint

There were tons of other interesting questions and discussions as well, but too much for me to go into here.

The general feeling, in my understanding, was that the meeting was interesting and inspiring and everyone had a lot of questions and topics which they wanted to continue discussing.
The number of participants was optimal and the length of the meeting as well. Hence; my conclusion is that there is a definate interest for meeting again and that the next meeting should be in the same format.

I want to take the opportunity to thank everyone who joined and not least Mathias Thinsz and Labs2 for the openess with how they work!