Ok so I admit it, I am a staunch advocate of waterfall methodologies. I am very much a ‘plan your work and work your plan’ kinda guy. But recently I started to think that waterfall may be limited and that there are other options that could help – or dare I say it – even improve deliveries…
Recently a couple of projects did not go as well as planned. At first I though this was because of one of the usual culprits that usually cause waterfall projects to fall over:
- not enough up-front planning
- failure to spot, manage and act on change
- bad communication
But after initial investigation I am not so sure this is the only reason and it is making me uncomfortable. I am not uncomfortable with the issues that have come up or the missed dates – these thing happen. I am uncomfortable because when analysing the problems with hindsight the biggest issue appears to be the fact that waterfall does not have a method of managing the unknown.
Now I am sure there are many who would disagree and please feel free to comment and tell me where I am going wrong. But the fact remains that there is no way to ‘know’ the unknown.
Sure, there are some techniques that help us to manage unknowns:
- we can add contingency – personally I hate contingency as it is just guess-work and confirms that the plan is incomplete
- we can go through a ‘discovery’ stage – fine but how long do you need to discover what you need to discover?
- One alternative I was playing with was to try flipping between a pre-planned delivery then when the “challenges” hit the fan to flip the team into discovery mode – but again how long will this last?
So then I started to think maybe waterfall is great for delivering known and more predictable projects, for managing change and delivering in a controlled fashion. But we must accept its limitations:
- there are large overheads when managing change (impact analysis, going through the change process, getting acceptance, updating the plan, product breakdown etc etc.)
- changes are ‘managed’ rather than embraced (could be great to change that button from green to red – let’s try it!)
- if we hit problems that we don’t understand late in the game – how do we plan to discover what the problems are and keep to the plan in a controlled fashion???
And then I thought – hang on there are methods that allow delivery of unknowns so I started to read more on agile, specifically Test Driven Development and continuous Integration.
I have always said “agile is great for delivering small, less defined changes such as enhancement and bug fixes – but it doesn’t allow me to plan the delivery properly, or communicate to my customer what we are going to do, for that I must plan a waterfall approach and manage the changes as they appear”. I have even had agile phases in a waterfall project and it worked well. So maybe agile is not limited to smaller deliveries, maybe I am wrong and maybe what needs to change is my concept of a delivery and the expectations of my customer. Maybe there are advantages to delivering to an unknown plan:
- flexibility of scope
- a lighter management touch
- allowing people to be people and for them to contribute to the overall delivery – not just their bit. Who says a DBA can’t come up with a great usability feature?
- A better more, more appropriate product???
So where to go from here? Well, as I said I have been doing a bit of research and here are a couple of gems:
Delivery of a large project including the integration of many separate systems http://www.thoughtworks.com/bt-westpac
Video of Martin fowler (one of the daddys of agile) discussing WHY agile works (rather than how) at a conference in Paris a couple of weeks ago http://universite-du-si.com/fr/conferences/6/sessions/909
At this point the jury is still out – but I cannot ignore the limitations of waterfall and the shouts from the agile community that it is delivering more and more successful projects. I feel like a new road of possibility is opening up and I am excited!