Waterfall - an accident

Origins - the real Waterfall

An article released in 1970 by Winston Royce - "Managing the development of large software systems" -  it is the most known explicitly description of a sequential approach of the development, where the sequence of phases is in fact the sequence of disciplines (one type of activity per phase) : Requirements , Analysis, Design, Coding ,Testing (simplified).  Then, this life-cycle model was adopted by military standards for software acquisition and later was considered as a reference model by the industry.

Anyway, this was an accident! 

The mentioned sequence is just an early picture from that article, that have the following comments from the author:  "I believe in this concept, but the implementation described above is risky and invites failure. […]. The testing phase which occurs at the end of the development cycle is the first event for which timing, storage, input/output transfers, etc., are experienced as distinguished from analyzed."

The real proposed models are describe after that an contains some improvements. Here are some of them:

DO IT TWICE - The development must perform first a pilot model the product  - "If the effort runs 30 months then this early development of a pilot model might be scheduled for 10 months." - to reduce the risks: "Without this simulation the project manager is at the mercy of human judgment."

Somehow Evolutionary design - preliminary design (high risks are addressed first) must be performed before analysis and design. Also, the design document must be updated after implementation - "AS BUILT". That mean "implementation phase" could change also the design.

History of failures

The single-step Waterfall has a history of failures, with a important rate of project failures on significant  software acquisitions.

1999 Review:   “Of a total $37 billion for the sample set, 75% of the projects failed or were never used, and only 2% were used without extensive modification.” (S. Jarzombek - Proc. Joint Aerospace Weapons Systems Support, Sensors and Simulation Symp., Gov’t Printing Office Press, 1999)   

The software industry was always a "top" domain with the highest rate of project failures , but in the Waterfall dominance era, it was the biggest problem.

Reactions to Waterfall

The DoD and others similar acquirers have reacted to these bad results with changes in the acquisitions standards (evolutionary instead of single step) and with support for new development and improvement approaches.  Unified Process was inspired also from  a big iterative project of DoD. CMM and then CMMI (Capability Maturity Model Integration) become a required process improvement approach.

The community of professionals & software engineers has reacted with the creation of more iterative approaches: Unified Process (Ivar Jacobson, Grady Booch and James Rumbaugh), XP - Extreme programming (Kent Beck and  Ron Jeffries), Scrum (Jeff Sutherland and Ken Schwaber) and others.

Then, the main creators of iterative approaches from the (... later called) Agile family and the main creators of Agile related knowledge published in 2001 the Manifesto for Agile Software Development. From what I know, the idea of the meeting that generate this manifesto belong to Robert C. Martin.

Results "after" Waterfall

The results of the software industry were improved after larger usage of iterative approaches. The most known reports and polls (as chaos reports of the Standish Group) indicates that Iterative and Agile approaches have better results than Waterfall (and Iterative-UP and Agile has similar numbers).

 ?

Waterfall it is still used for software development (with mentioned results), but has no support and no recommendation from known authors from software development area (at least I do not know any). In many cases, the motivation is the need of a "planned" approach instead of adaptive-Agile methods. This motivation is not valid because there are iterative alternatives for "planned" case: Agile works for some projects , Unified Process for others and Spiral for high risks cases.

Remember Wiston Royce words about single-step Waterfall: ("Without this simulation") "the project manager is at the mercy of human judgment."

Agile fundamentals: Working software

This concept it is highly used when we talk about Agile. It is mentioned in Agile Manifesto, first as a main value and then as part of the agile principles. In the same time it is highly misunderstood, even by some claimed agile practitioners.

Waterfall versus Working Software

The main dilemma concerning the process approaches in software development is not "Agile versus Waterfall" - that is false and confusing - but rather Waterfall versus Working Software, that mean Waterfall versus Iterative Development. Why? Very simple answer: the process approach that has a main attribute the Working Software is Iterative Development. Ok... but what about Agile? Agile is one of the main variants of Iterative Development, and probably most used.

There are 4 main Agile values and associated principles. There are also few others values and principles related to software development. Anyway - Working Software/Iterative development is the first one, if not as value, but at least as attribute that define an entire category of non-Waterfall approaches .

"Traditional" PM main problem

A "traditional" PM for software domain is not the one that use PMBOK based PM style, but the one that one that use Waterfall as project life-cycle model.  (... PMBOK does not propose any life-cycle model because that is out of its scope). The first thing that should know a such project manager is: if Agile is not needed, that could be ok, but Iterative Development is not optional for a good project management in the software development case.

Working Software it is a must!

Agile related confusions

Good understanding of working software it is fundamental for a real Agile process. 

The Bad and the Ugly

An often misinterpretation of the Agile Manifesto is the following: we do not need documentation, we just craft some software. That is not agile, and it is very harmful.

The Good

The right interpretation of W.S. concept from the A.M. is when we are thinking to the working software resulted at the end of an iteration: analyzed, designed , implemented and tested & fixed software. Nothing is missing? The complete "formula" should be: continuous integrated (working) software.

Recap:  we should use iterations (smaller than one month, but better of two weeks of less) and working software should be the main result and measure of the progress at the end of the iteration.

The Great

Softwarecraftsmanship Manifesto states also that we need not only working software, but also "well-crafted" software. Something similar could be found also in the Agile Manifesto principles, where the used terms are "good design" and "technical excellence". Finally, that mean that for Agile we need "well-crafted" working software. Why? That will enable, enhance and offer steadiness to the agility. Also, that mean we will produce less Technical Debt, that is a condition for Agile (and agile methods as Scrum) and also for any good process.