For innovative work: Agile is about delighting customers, Waterfall about CYA (Cover Your A**)
For innovative work: Agile is about delighting customers, Waterfall is about CYA (Cover Your A**).
If you work isn’t innovative, there is nothing wrong with a waterfall approach. If you only have a standard set of options to choose from, like choosing the standard features of a car, there is nothing wrong with a waterfall approach. You know exactly what the options are, you know exactly how long each takes to add, you know exactly what the customer expects.
My hope is that if your job is to customize software like that you not waste your time (and more importantly your creative heart) on this type of work. Instead spend time creating an administrative system that allows non-developers to configure it on behalf of the client (or even better, let the client set those options).
However, if like most jobs software developers are hired (and desire) to do, there is likely going to be a lot of unknowns -- the client will not know exactly what they need, the developers have to figure out the best way to deliver what the clients want, and there may even be unknowns about a new technology being used for the client solution.
Agile methodology recognizes that these unknowns are inherent in software development, and allows the client and development team discover what is unknown on an iterative basis. Constantly delivering production ready software increments and constantly re-adjusting where the product is going based on what the team and client learns. The ultimate goal being deliver the highest value features to the client within a limited time or budget.
So why do clients sometimes fight this approach? Because they want the feeling of certainty of knowing exactly what they are getting for their money. It’s a perfectly reasonable reaction. Who wants to pay more than they have to or not get what they paid for? However, it can lead to behaviors that do exactly the opposite.
If a client insists on that certainty, companies pull out the waterfall approach. The company makes sure to talk and document to the nth degree upfront hoping that they can capture exactly what the client needs and any risks associated before engaging development team. The company makes the client sign off that the final specification document is exactly what the client is paying for the company to build and that any variation from that document requires an agreement of change to scope, time, and/or budget.
What does the client get out of this. They get an illusion of certainty. They get to see a plan with exactly what the agreed laid out across a timeline with milestones of what pieces will be delivered when. They get to think that if the development team just follows that plan they will get something they want in the end.
After hitting the first milestone, the illusion begins to deteriorate. The client gets a demo and realizes that yes they may have asked for this but they really needed it is this or they may find they have forgotten a requirement (perhaps regulatory) that has to be met. No problem for the Project Manager pulls out the handy Change Order and asks whether the client wants to pay more, take more time, and/or cut scope. Clearly the client agreed to the original specification so now any variance from that specification is their fault and they must pay. It about the company CYA, not helping the client discover what is of highest value to them.
Any good project manager will know that a development team’s estimates aren’t reflection of certainty… it’s a best approximation given prior knowledge (i.e. educated guess). The project manager will have added their own padding in an attempt to make up for the lack of certainty. However, they must be cautious not to put too much or else they might make the price too high to win the bid. If the estimates prove to be wrong, the “good” news is that client can pull the CYA card. It doesn’t matter that higher value features might have been delivered or that the feature may have been discovered to be not needed, it just matters that it was documented in the specification.
Instead of working together with the limited time and budget to find the highest value for the client, both sides waste time making sure they get what they agreed to and that any variance not in the favor is “paid for” by the other side.
Whether following either Agile or Waterfall, the client will not end up with the product they first envisioned because it's impossible to fully envision what client really needs and what it’s really going to take complete it without actually doing the work. Agile is honest about that fact and promises to the time and budget and uses it to the client’s highest value. Waterfall sells an illusion and then punishes both sides when the illusion isn’t a reality.