Document Essentials - a missing (agile) practice

Agile documenting: something is missing

Agile principles and practices such as KISS, YAGNI, TAGRI, Single Source of Information, Agile Documentation, are responses to the waste, ineffectiveness or even lack of feasibility of approaches such as BRUF – Big Up Front Requirements, BRUD – Big Up Front Design, and of so called “comprehensive documentation”.

See also previous post: Practices and principles for an economic documentation. 

There is still a missing part: most of the good advices are about reducing the amount of documentation, but no one tell us what is the minimum that could not be ignored.

What is missing: Document, Protect and Use the Essentials

With both “big”/”comprehensive” and “simple”/ “agile” approaches we could have a very unpleasant surprise: the most important, essential information is not available for the ones that must use it.

When “comprehensive” destroy the essentials

Too many time the “comprehensive” documentation is practically done in these ways:

• Unstructured, hard to read and navigate text – the essentials are hard to find, understand and validate
• Only the volume of the documents it is evaluated, but not the coverage of the essentials parts

Finally, we have a lot of text, that cannot be validated and we cannot guarantee even that the main decision areas are covered. The focus is on quantity and not on quality and value of the information and a lot of effort is consumed for poor results.

When “agility” destroy the essentials

One of the main reasons that allow a much lighter documentation in case of Agile is usage of Executable Specification (aka TDD). If you do not have such things it is better to start to think what documentation it is missing.

The main start questions in case of “agility” are:

• I need to “travel light”. What is the minimum needed documentation ?
• I have the right support to distribute the “System Metaphor” to all team members?

Essentials: System Metaphor

Any team member has the right to have access to the System Metaphor – simple explanation of the system design. Beyond self-explained code, you could need a lightweight document(s) that capture this information.  Examples: architectural views, description of main design mechanisms.

Essentials: Business and System Functionality  

Any good developer (and, in particular, an Agile developer) should be able to understand the target business and the main functions of the system. It is very unlikely (!) that this kind of information could be acquired accurately, and in time, only by working from time to time to some details of the system. A lightweight documentation is necessary (beyond practices such as pair programming and moving people around).

Essentials: Data   

A specific part of the of the business and functionality is the managed data & knowledge: meaning, relationships, usage. Examples of some useful lightweight overview documentation: Glossary of Terms, Domain Model.  

Essentials: Main functional flow   

Examples of some lightweight overview documentation: Epics, High Level Use Cases.

Essentials: YOU ARE GONNA NEED IT!   

Here are some real examples of possible consequences for the cases when team members have no easy access to essential knowledge about the product:

• No other alternative for team members to get in-time this knowledge
• Team members work for years on products without knowing essentials
• Team members implement change request and fixes on components where they do not know the functionality and the design
• Testing is not effective with severe consequences on quality and economic
• The undesired complexity and technical debt increase too much with all consequences (fragility and similar)
• The system fragility increase too much (also rigidity, viscosity, immobility) and customer trust will be lost
•  New team members are hard to integrate
• Product ownership handover is a high risk operation

Principles for “Document Essentials”   

Document essentials! – you have no other alternatives to timely, effectively and efficiently distribute this information to the team members.

Present the essentials! – avoid TAGRI. Face to face conversation is the best way to present the documented essentials. 

Protect the essentials! – this kind of information shall not be sunken in an unstructured high volume information. Keep it clear, clean and updated!

Use the essentials! – already having this information and not using it – that it is the greatest guilt and the greatest waste considering the lost benefits.

 Remember: It is very unlikely to make essentials known without document it & present it.

Practices and principles for an economic documentation

Intro - Agile principles and practices for documentations are responses to the waste, ineffectiveness or even lack of feasibility of traditional approaches such as BRUF – Big Up Front Requirements, BRUD – Big Up Front Design, and to so called “comprehensive documentation”.  We will try  to enumerate and comment here these principles and practices.

Observations – This article is not intended to address also cases where requirements are subject to specific compliance and regulatory environments. Also are out of current post scope some more specific practices like Product Backlog, User Stories or Effective Use Cases.

KISS principle - “Keep It Simple Stupid” - was introduced by Kelly Johnson, system and aeronautical engineer. Unnecessary complexity is “stupid” in a context where complexity it is already an inherent problem. The software domain is one where the unnecessary complexity is a fundamental problem. Here is an example of a software related interpretation: (by Filip Hanik)

https://people.apache.org/~fhanik/kiss.html

Requirements will be changed (!) because of a multitude of reasons: clarifications, business changes, customer understanding changes and others. In such conditions, any comprehensive documentation will require a significant effort of change and in the same time a high risk of not being updated. The first solution is to keep the requirements representation lightweight. That was very clearly said by Robert C. Martin in its book “UML for Java Programmers” (Chapter 5 - Use Cases): “The real trick to use cases is to _keep them simple_. Remember, tomorrow they are going to change” (why requirements as use cases? and why use cases in a programming book? Please discover yourself in the book, or wait for a later post).

 YAGNI principle – “You aren't gonna need it” principle was introduced by XP founders and it is described by Ron Jeffries as: "Always implement things when you actually need them, never when you just foresee that you need them.". That was translated in XP practice “no functionality is added early”.

A good insight about Yagni (and its origins) could be found here: http://martinfowler.com/bliki/Yagni.html 

Martin Fowler states that “only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.” (For deeper considerations about the last part see Clean Coders series of Robert C. Martin.

Speaking of documentation, you should not “capture” presumptive features, but rather wait until you have good enough information, that mean right balancing between Envisioning and Just In-Time.

TAGRI principle - They Ain't Gonna Read It principle was explained by Scott Ambler here:

 http://www.agilemodeling.com/essays/tagri.htm .

The start idea was that “documentation was written and then ignored by the intended audience”. To avoid this problem, Scott propose more rules. Here are some of them: “Create documents with a clear audience”, “Work with the true audience to identify their actual needs”, “Write agile documentation”.

Documentation it is witted with the intention of being used and used effectively and efficiently. Something is very wrong if the TAGRI is manifesting.

Agile Documentation – a more robust work about effective and efficient documentation for software development is included in Agile Modeling and Disciplined Agile methods. Here are some recommended readings:

Agile Documentation - http://agilemodeling.com/essays/agileDocumentation.htm . Some interesting topics:

• What is the relationship between documentation and project success?
• What Are The Issues Associated With Documentation?
• What Does it Mean to Travel Light? 
• Recommended practices to increase agility of the documentation

When we should document

• http://www.disciplinedagiledelivery.com/agile-documention-strategy/

Agile Modeling Practices for documentation

• Executable Specification – by using TDD, your tests become in fact detailed specification, in a language that is known by the programmer. One of the problems is to know the differences between the flows (what you build) and the scenarios (what you run or test) from coverage point of view. 
• Single Source of Information – it is the equivalent of DRY (Do Not Repeat Yourself) for documentation. One typical example is to duplicate the requirements specs in the test specification. Duplication are definitely not Agile and not Lean. 
• Document Continuously – in Agile we start usually with an Envisioning and continuously to refine it during each iteration. 
• Document Late – a good Agile strategy is to wait until the information is stabilized. Detailed information is rather captured on JIT -Just In Time bases. Also we need to fill the gaps in overview documentation that it is necessary to for the team in the future work on the product.

Important: We can have lightweight requirements documentation in an Agile process only because, in fact. we have also the Executable Specification.

Still … something is missing – To be continued.