Monday, February 16, 2009

Forget Change Management - it undermines your chances of success. All changes should be free!

When you commit yourself to delivering a certain set of requirements ("requirement specification", "statement of work" or whatever you call it) by a certain deadline you automatically create a need of ways to manage change – because change brings risk and disorder, and it risks ruining your time-plans that you have spent so much time figuring out. It risks ruining the architecture that your designers & architects spent so much time thinking through, and it risks turning those already completed programming & testing efforts into waste because a feature is made obsolete.

What's even worse, if you're the project manager then you're likely to be the lucky person stuck with the bill and you have to work out who's going to pay: your company or the customer?
It can be a nerve-wrecking job to be the one that have to make that call to the customer and explain that "this recent request will end up costing 5.000 EUR more and it will delay the project with 3 weeks". And if your requirement specification is the least bit vague or ambiguous at this point, you might end up in an irritated argument or negotiation with the customer about him considering the feature part of the “original scope”, whereas you consider it a Change Request. Not really an optimal situation. But under such circumstances of course you will require a clear way of managing these ugly changes.

So what can you do to minimize these risky aspects of software project management? The traditional way to go is to become even more thorough the next time during Requirements Analysis. I.e. you put even more focus on planning and on trying to figure out even more of the details up front. You have longer pre-studies. You do bigger and more detailed design up front. This way you'll be able to give yourself a better picture of the project before you sign off the requirements and set any deadline. In essence, you build your walls higher and thicker around you. You strengthen the foundation for your Change Management Process and thereby make your project less likely to be exposed to Change and the risks they bring.

With the above reasoning change is something ugly that you want to avoid. But wait. Is change really a bad thing?

I argue that change is great. Change is fantastic.

Screw Change Management processes and the administration they bring!
Yes, I said it. In fact, I think we need to give the customer all changes for free, always!

Change means the customer has gotten a different or stronger opinion than he had before, and if I don't adapt to it then the customer will be less happy than otherwise. And what is our number 1 objective? …No, silly, not to make money. Our number 1 objective is to make the customer happy - because happy customers are returning customers. And as a consequence we will make money.

The problem here is much larger than the Change Management process alone. The problem is in the entire traditional approach to software project management and software development.

With a traditional methodology you will definately need Change Management - no doubt about it. There are many great project managers and companies out there, I'm sure, who are working with traditional methods and who are excellent at Change Management.

But I think remarkably many of those never thought twice about the software development methodology they adopted. They just chose one that everyone else was using (for that reason: because everyone else was using it) instead of adopting a software development methodology that maximizes their chances of success. At the end of the day, if you're not NASA and you're not working against the FDA with tons of beurocracy and laws to adhere to, then you're much better off using an Agile development methodology.

You have to accept that you can’t foretell the future. It would be very nice if you could, but you can’t. Sorry. And consequently; change is inevitable – you can’t wish it away. You also have to realize that the customer doesn’t know what he wants – he discovers that as he goes along. Consequently; change is inevitable. Furthermore, you have to realize that developers don’t know how to build the software up front – they discover that as they go along Yes, consequently; change is inevitable.

You can sit there and sob and dislike it all you want, but it doesn’t change the facts.

This is important, and a huge difference to traditional assumptions, so I’ll recap:
  1. Unforeseeable sh*t happens that will affect your plans.
  2. The customer doesn’t know what he wants – he discovers that as he goes along.
  3. The developers don’t know how to build it – they discover that as they go along.
Most people will agree that the above is true. So, why do we rely on dusty old project management methodologies that assume the exact opposite??

The first thing you need to do is to stop committing to both scope and deadline up front. Commit to date rather than scope. This is sometimes called “date-driven development”. You achieve it by putting the requested features in one single list that is prioritized by the customer. Allow changes to priorities, as much and as often as the customer likes. But make sure the priorities are always kept unique for each item in the list and that the priorities are strictly followed, i.e. work is carried out in strict order of priority. As a consequence of adopting this approach there will be no such thing as an “original scope” – because that would imply that there was a “before” and an “after” some event (e.g. a “change”). “Feature creeping” is no longer a bad thing. As long as the customer sticks to the one single prioritized list of features he can have as many features “creep” in as he wants; he can replace all the items with others from one day to another. As long as the customer is happier in the end. With this approach there is no such thing as a “scope change” anymore: the scope instead is something that evolves over time by continuous readjustment .

Secondly, you need to stop making detailed plans. Plan for God’s sake, don’t misunderstand me, but just don’t overdo it. Forget about figuring out what exactly will be done, by who, when, before or after what, two-three months or more away from now. Try to ask yourself; are you creating the time-plan the way you WISH things will happen or as you KNOW they will happen? I’m guessing most of the time the answer will be that it’s the former. The effort spent on creating silly wishful-thinking time-plans is much better spent on something more constructive in terms of bringing the team closer to the end goal (a happy customer) – like getting coffee for the developers. Note that I am not telling you NOT to plan. I am just telling you that all plans are wrong and rarely survive more than one minute in contact with reality. So just minimize the amount of time you spend making them.

Thirdly, start involving the customer. Do it early on. And make sure you have as short feedback loops as you can. With a traditional approach, presenting the product to the customer is often scary because you fear there will be changes, and those changes can be ugly. But you have to remember that those changes are a necessity for the customer to become maximally happy. The problem is not the feedback, the problem is when you have a long lead-time between points of customer involvement (and consequently risk having put in a lot of effort into something that is wrong or will change). So, remember to keep feedback loops (very) short, from the very start of the project to the very end. I suggest 1-3 week loops. Notice that this puts a different demand on the amount of customer involvement than with a traditional approach: the customer needs to be involved (much) more, and earlier. If your customer is not comfortable with that then you’re better off with a traditional approach – but you will less likely to succeed, and the project will be much more expensive because you will have to put risk margins in your pricing.

Fourthly, derive a release plan from reality rather than dream one up ahead of time. Don’t lie to yourself or (worse) the customer by claiming you know exactly what you will be able to deliver by when. You don’t know this! And anyone who claims to is a liar. Instead, commit only to time. Then continuously observe your actual progress rate and use those observations to calculate what parts of the scope it is likely that you’ll be able to finish before the deadline date. You work in priority order and have short feedback loops involving the customer that spark “change” often and early. When you reach the deadline date you have the most important things done.

Fifthly, test early and continuously, and always finish things (completely). Don’t leave some parts of testing for later. Don’t postpone code reviewing. Don’t wait with code merges. The sooner you do things the easier and cheaper they are. Started, half-finished work has no value. For every new feature you start without finishing completely, you add to a pile of more or less known remaining tasks. Over time this pile grows to a mountain of unknown stuff that will get you in the end. Avoid this by really completing things early and do all types of testing early. Continuous integration is key here, and automated testing is a helpful tool.

Since change and unpredictability is inevitable, we have to find a way of working where change is not made ugly by processes. Where there is as little waste of effort as possible when change happens. This is what Agile Development is about – a software development philosophy which covers process frameworks such as Scrum, XP, FDD, DSDM and others. But remember: tThis isn’t the silver bullet that automatically solves all your problems, but it brings maximally good odds of succeeding. Software development and project management is still hard, but at least you’re not making it harder than necessary.

If you’re interested in reading more about Agile and Scrum here are some links:
My blog: http://scrumftw.blogspot.com/
Agile Manifesto: http://www.agilemanifesto.org/
Wikipedia about Agile: http://en.wikipedia.org/wiki/Agile_development

Thursday, February 5, 2009

Working smarter is better than working harder - How to increase velocity

"Why is progress slow? Why does everything have to take so freakin' long? You have to push harder! Work more."

This is something that I've heard a lot, especially from certain types of managers. Let’s look at this more closely, and what it means, and what it is an expression of.

I think it is reasonable to assume that it’s not a matter of the manager actually wanting people to work harder, faster or more just for the sake of doing so. No, instead I think (I might be naive ;-)) that it indicates the manager’s wish to have stuff done sooner. "Stuff done sooner", that is key here. Can this really be done without having people work harder/more? Yes! If you work smarter!

The time that something takes to complete (from its very start, to the time that it is ready) is called “Cycle Time”. You can learn a lot more about this if you study Lean Software Development, I won’t go much further into details here.

Let's investigate what factors affect Cycle Time. How “hard” people work is one factor, yes absolutely. At least up to the point where negative stress kicks in and its consequences appear; defects and demoralization, lacks in communication, loss of cooperation and team synergy effects, and many other ugly things.

At this point I could dive into Value Stream analysis, talk about Value Stream Maps and discuss how to apply the Lean Software Development tools supplied by Mary and Tom Poppendieck et al. But I won’t do that now. In this blog post I will limit myself to only the “implementation part” of the development process. You should know that there is a lot more room for shortening “Cycle Time” than what I will talk about below. With Lean and its tools you should go far beyond only the sprints and observe the steps before and after implementation and focus on the whole chain – the entire “value stream” that flows through your business.

So, back to the subject. In Scrum we continuously measure at least a part of the Cycle Time: We measure velocity. The velocity is the speed with which the team finish things, according to your Definition of Done. Velocity is measured in “Story Points”. You can observe it over a sprint (SP/sprint) or over man-days (SP/manday), or variations of this sort. The point is that you measure the velocity in a consistent way, and you record your findings (in something that I call “the Velocity Log”).

If you say you want to shorten your Cycle Time, i.e. shorten the time it takes to finish something, then what you really want to do is increase your velocity. The bitch here is that the velocity is not something that you can tune by itself. It’s not a knob that you crank up and voila stuff goes faster. The velocity is a result of the environment in which you work. By changing the work environment you will change the velocity. It doesn’t work the other way around. Sorry.

I think we generally need to be much more focused on “helping” the velocity up. It is easy to stare oneself blind on a deadline and all the stuff that has to be done by then, dig in and spend all energy on completing those stories. during such times I think we often forget, or at least seriously underestimate, the power of improving the work environment and the effect it would have on the velocity. This is what "Continuous Improvement" is all about.

Now, what does “work environment” mean?

Unplanned items. They happen all the time, during your sprint and are typically part of your work environment. These are the things which you for some reason just have to do, but that you really shouldn’t (because the team’s job is really to work on the sprint backlog). These are those “negative additions/changes” to your sprint. They affect your team’s ability to focus on the sprint commitment. What you want to do is record your unplanned stuff by putting a note in your “Unplanned”-section on the Scrum Board, for everyone to see. Then you need to work on removing the source of those unplanned items. In my opinion, unplanned items is the number 1 opportunity to increase velocity. Find the root-cause of every unplanned item, and consider if you can’t insert a task into your Impediment Backlog about removing that root-cause. Either way, it is the Scrum Master’s and the Scrum Product Owner’s number one priority to shield the team from these things so that the team is allowed to focus 100% on the sprint backlog.

Concentration breakers: Email. What can I say. As a communication tool it is great, but the way email has been integrated into our daily life is not. If you get a “You’ve got mail!” dialog every time you receive one then your concentration will be broken as many times a day as the number of emails you get. Some days I get hundreds of emails. Jeez! If I was a developer today I would consider not checking my email more than once or twice during the day, and have the client shut down in between. In a Scrum team we rely on face-to-face communication more than written stuff anyway, so believe me, the really important stuff will reach you anyway without you automatically checking your email every minute. And get used to the fact that if you want someone something, then you need to walk over to them and speak to them face to face instead of writing an email.

Concentration breakers: Skype. It is indeed a great tool. In our company we rely a lot on short-lived Skype group chats that we create for various purposes, and we use it a lot for internal communication. It’s a great way of communicating and cooperating. But it can also be a huge concentration breaker. Skype announces new messages by popping up a new window (in the background), with a system tray popup and/or by coloring the chat window taskbar button orange. So every time you receive a message (personal or in a group chat) your concentration will be broken. I guess you learn how to ignore this after a while, but I’m pretty sure that ability is individual. My advice: be careful if you’re using Skype. I suggest killing it altogether and rely on face-to-face communication instead. If you decide to go that route though, make sure it’s a team decision and not enforced as a policy or company rule.

Concentration breakers: Background noise. We work in an open office landscape with very few closed offices. Most of the people sit next to each other in that landscape in order to promote cooperation and communication. The clear drawback with an open office landscape is the noise level from people talking, writing, from computers buzzing and so on. If you’re a manager reading this: don’t be a cheap moron. Invest in sound absorbing cubicle dividers, even if they are more expensive than regular ones (or compared to not buying any dividers at all). If there’s one thing that directly affects the concentration of every single person in the open office landscape – all day, every day! – it is the background noise level. Yes, we can always ask everyone to talk quietly and step into a meeting room for those longer discussions. But also consider: how much are you crippling freedom of communication (and cooperation) by not buying those sound absorbing dividers? It is a low cost for ensuring optimal work environment! While on the subject: seriously consider installing roof- and wall-mounted sound absorbents too.

Unnecessary manual work. Stop and reflect: are there any tasks/actions that you perform often which could be partially or completely automated? Even if the effort of automating it is somewhat large (maybe several days or even weeks of work to build automation or refine tools), it might be worth it in the end. Is it only you that have to do this manual task every time? Maybe it is every developer. 5 minutes a day, for every developer in a 25 person organization, equals 45 man-hours every month. Consider automating many of those manual tasks, and optimize the efficiency of those that you can’t completely automate. If you think about it I’m sure you can come up with a bunch of things. Have a workshop. List stuff and prioritize. I’m sure your Scrum Product Owner will listen if you run the numbers. If you have tips on tasks that can be automated, please feel free and add them as comments to this blog post!

Technical debt. The higher technical debt you have the longer things will take. The system is more complex, harder to test, things are harder to implement, and so on. Addressing technical debt is a subject on its own, which I will come back to in later blog posts. For now I'll just advise you to keep in mind that it directly affects your velocity. You should do what you can to (at the very least) immediately stop increasing your technical debt even further, by letting it take longer to do things "right". It is a short term investment for a long term gain. The bad approach is to do it the quick and dirty way. Remember: the dirty remains long after the quick has been forgotten.


So, if you think progress is too slow, consider working smarter instead of harder. It has a much bigger impact on your cycle time than crunch and overtime has. I really hate crunch. And in so does everyone. Trying to continuously improve the work environment and thereby increasing velocity should be way up high on your agenda! That is how you get things finished sooner, without killing yourself and without sacrificing quality.

I would appriciate comments on your tips and tricks and experiences regarding how to increase velocity!