Something it is very wrong with state of agile

"Who responded" (for below mentioned report):

• 24% Project/Program Managers
• 18% Developments Staff
• 14% Developments Leadership
• 16% Other (Scrum Master or Internal Coach)
• 10% Consultant/Trainer
• 7% Product Owner 
• 5% Business Analyst
• 3% IT staff
• 3% C-level  

Data extracted from <VersionOne 10th Annual State of Agile Report> :

•   https://versionone.com/pdf/VersionOne-10th-Annual-State-of-Agile-Report.pdf

Something it is very wrong with both <State of Agile> and with state of agile if 24% or respondents are PMs and only 18% is Developments Staff.  PMs and Scrum Masters rule over an "Agile" world of Scrum-only. Recommended further reading <What Killed Waterfall Could Kill Agile> by Robert C. Martin.

Practices and principles for an economic documentation

Intro - Agile principles and practices for documentations are responses to the waste, ineffectiveness or even lack of feasibility of traditional approaches such as BRUF – Big Up Front Requirements, BRUD – Big Up Front Design, and to so called “comprehensive documentation”.  We will try  to enumerate and comment here these principles and practices.

Observations – This article is not intended to address also cases where requirements are subject to specific compliance and regulatory environments. Also are out of current post scope some more specific practices like Product Backlog, User Stories or Effective Use Cases.

KISS principle - “Keep It Simple Stupid” - was introduced by Kelly Johnson, system and aeronautical engineer. Unnecessary complexity is “stupid” in a context where complexity it is already an inherent problem. The software domain is one where the unnecessary complexity is a fundamental problem. Here is an example of a software related interpretation: (by Filip Hanik)

https://people.apache.org/~fhanik/kiss.html

Requirements will be changed (!) because of a multitude of reasons: clarifications, business changes, customer understanding changes and others. In such conditions, any comprehensive documentation will require a significant effort of change and in the same time a high risk of not being updated. The first solution is to keep the requirements representation lightweight. That was very clearly said by Robert C. Martin in its book “UML for Java Programmers” (Chapter 5 - Use Cases): “The real trick to use cases is to _keep them simple_. Remember, tomorrow they are going to change” (why requirements as use cases? and why use cases in a programming book? Please discover yourself in the book, or wait for a later post).

 YAGNI principle – “You aren't gonna need it” principle was introduced by XP founders and it is described by Ron Jeffries as: "Always implement things when you actually need them, never when you just foresee that you need them.". That was translated in XP practice “no functionality is added early”.

A good insight about Yagni (and its origins) could be found here: http://martinfowler.com/bliki/Yagni.html 

Martin Fowler states that “only applies to capabilities built into the software to support a presumptive feature, it does not apply to effort to make the software easier to modify.” (For deeper considerations about the last part see Clean Coders series of Robert C. Martin.

Speaking of documentation, you should not “capture” presumptive features, but rather wait until you have good enough information, that mean right balancing between Envisioning and Just In-Time.

TAGRI principle - They Ain't Gonna Read It principle was explained by Scott Ambler here:

 http://www.agilemodeling.com/essays/tagri.htm .

The start idea was that “documentation was written and then ignored by the intended audience”. To avoid this problem, Scott propose more rules. Here are some of them: “Create documents with a clear audience”, “Work with the true audience to identify their actual needs”, “Write agile documentation”.

Documentation it is witted with the intention of being used and used effectively and efficiently. Something is very wrong if the TAGRI is manifesting.

Agile Documentation – a more robust work about effective and efficient documentation for software development is included in Agile Modeling and Disciplined Agile methods. Here are some recommended readings:

Agile Documentation - http://agilemodeling.com/essays/agileDocumentation.htm . Some interesting topics:

• What is the relationship between documentation and project success?
• What Are The Issues Associated With Documentation?
• What Does it Mean to Travel Light? 
• Recommended practices to increase agility of the documentation

When we should document

• http://www.disciplinedagiledelivery.com/agile-documention-strategy/

Agile Modeling Practices for documentation

• Executable Specification – by using TDD, your tests become in fact detailed specification, in a language that is known by the programmer. One of the problems is to know the differences between the flows (what you build) and the scenarios (what you run or test) from coverage point of view. 
• Single Source of Information – it is the equivalent of DRY (Do Not Repeat Yourself) for documentation. One typical example is to duplicate the requirements specs in the test specification. Duplication are definitely not Agile and not Lean. 
• Document Continuously – in Agile we start usually with an Envisioning and continuously to refine it during each iteration. 
• Document Late – a good Agile strategy is to wait until the information is stabilized. Detailed information is rather captured on JIT -Just In Time bases. Also we need to fill the gaps in overview documentation that it is necessary to for the team in the future work on the product.

Important: We can have lightweight requirements documentation in an Agile process only because, in fact. we have also the Executable Specification.

Still … something is missing – To be continued.

The Obligation of the Programmer (Robert C. Martin ) versus reality

After "Clean code", the book and the rules, are inspired from Hippocratic Oath, Robert C. Martin has elaborate now "The Obligation of the programmer" , a code of ethic inspired from the "Order of the engineers" (http://en.wikipedia.org/wiki/Order_of_the_Engineer) code.

http://blog.cleancoder.com/uncle-bob/2014/11/15/WeRuleTheWorld.html 

Some significant ideas from this code:

• Human progress depend on "those who manipulate information"
• The result of any programmer is depended on the "accumulated knowledge and experience " - that mean this experience must be known and used
• It is important to participate only to honest enterprises 
• The programmer skills and experience it is a public good

Starting from Uncle Bob sentences, here some ideas about why this code it is important. How the reality looks like:

• Human progress it is stalled because of software development problems (high costs and low productivity, where defects management it is included)
• Too many programmers does not learn enough from "accumulated knowledge and experience" -  the re-invent the (...squared) wheel syndrome is not rare - and too many of them blocked in stone-age of the design (... no design).
•  "Honest enterprises..." - the short term goals (combining with poor skills) are destroying the products and productivity

The programmers and the others involved in this domain are smart people, but the domain is very difficult and we need to be also disciplined and deliberate dedication.

Oath of a Clean Coder (cloned from Hippocratic Oath)

As you see in the previous post, Uncle Bob has a list of expectations from the software professionals. He use also the term Clean Coder to describe a such professional. He also says that our life will depend more and more on software, and for this reasons "We need to be!" professionals.

Our life.

It is the same thing as in the healthcare domain. Even the healthcare is depending more and more on software. The physicians take the Hippocratic Oath , beyond any other rules and official laws. Why? Because it is important. What about the software used by these physicians? This Oath has no effect for the software...

We will need a similar Oath - a kind of Clean Coder Oath:
(Note - In fact, Martin book Clean Coder start (some years before this post) from the idea of Hippocratic Oath and present a code of conduct for the professionals developers. )

I swear by Apollo, the REFACTORER and Hephaestus, and I take to witness all the gods, all the goddesses, to keep according to my ability and my judgment, the following Oath and agreement:

To consider dear to me, as my parents, him who taught me this art; to live in common with him and, if necessary, to share my goods with him; To look upon his children as my own brothers, to teach them this art; and that by my teaching, I will impart a knowledge of this art to my own sons, and to my teacher's sons, and to disciples bound by an indenture and oath according to the ENGINEERING  laws, and no others.

I will prescribe DESIGN for the good of my SOFTWARE according to my ability and my judgment and never do harm to ANY CODE.

I will give no SPAGHETTI CODE to any one if asked, nor suggest any such counsel [...]

But I will preserve the purity of my life and my arts.

I will not cut for BUG, even for SOFTWARE in whom the ERROR is manifest; I will leave this operation to be performed by practitioners, specialists in this art.

In every SOFTWARE house where I come I will enter only for the good of my CODE, keeping myself far from all intentional ill-doing [...].

The main question is: how many times we have break such rules and principles? How many times the programmers start to "cut"  or build parts that are beyond their skills? How many times the programmers have delivered a (too) bad code?  How many time the programmers have harm the code?
Too often ...

Are we professionals? (1)

If...
... Robert C. Martin will be your new CTO, then his first question will be:

Are we professionals?

His answer is:

We need to be.

Here is a list of what the professionalism demand in his view:

I Expect...

...We will not ship shit.
...You will always be Ready.
...Stable Productivity.
...Inexpensive Adaptability.
...Continuously Improvement.
...Fearless competence.
.. Extreme quality.
...QA will find nothing.
...We cover for each other.
...Honest Estimates.
...You to say "No".
...Automation!
...Continuous Aggressive Learning.
...Mentoring.

I will post the source later. Please try this exercise:

• try to find the motivations behind each of these sentences
• try to find yourself the source... maybe you will find more interesting subjects from "Uncle Bob".

The Story of Agile Manifesto

You can find the full article here:

http://www.informit.com/articles/article.aspx?p=1743276

>>>
InformIT: You were a co-author of the Agile Manifesto in 2001. What drew you to the gathering?

Bob: In 2000 I asked Martin Fowler to lunch to talk about the idea of a "Lightweight Process Summit." We were both part of the Extreme Programming movement, but we had seen that there were several other similar processes, most notably SCRUM, FDD, DSDM, and Crystal. We thought it would be a good idea to get the advocates of all those processes together with other industry leaders to see if there wasn't some way to find and define common ground. I suggested to Martin at the time that we might be able to create some kind of manifesto. He and I put an invitation list together. I composed the invitation and sent it out. After that the process took on a life of its own.

>>> ("Ten Years Of Agile: An Interview with Robert C. "Uncle Bob" Martin")

The idea (and the initiative) of a "Lightweight Process Summit" and of the manifesto that could define the common ground of that kind of movement belong to Robert C. Martin. "Uncle Bob" and Martin Fowler have created also the invitation list... The resulted manifesto is quite outstanding, at least for the ones that have good experience with software development and could serve as guidance for every practitioner.

Yes, we can say that all those methods are "implementing" these principles, but the great thing is that it was first the demonstration: methods and practices resulted from experience and proven to be working. This "theory" (the values and associated principles) are just the abstraction of real experience.

Thank you, Uncle Bob! :)

Note: "The Manifesto of Softwarecraftsmanship" has the same creator.

Hyper-productivity - just a short run ?

The Scrum marketing claims Hyper-Productivity. Sometime is true, sometime it is not. That depends on who make that claim...Which is the best way to validate that? We can measure, for example, the team velocity for a period of few months. It is that a correct approach?

Maybe the most know and severe problem in software development is the accumulation of the undesired complexity - Technical Debt. The main consequence of a such debt is the deterioration - in time - of the overall productivity and quality. 

Short "hyper-productive" runs with a lot of accumulated debt it is not Agile and it is not Scrum. Remember the principle from Agile Manifesto that require "sustainable development": "The sponsors, developers, and users should be able to maintain a constant pace indefinitely.". Scrum suppose that you will assess your work progress using the criteria of "Done": no other work is necessary on that product part, and that include very few Technical Debt.

Anyway, this kind of  "shortness of breath" it is very common and we must be able to identify and address the root causes. One cause it is the lack of skills of the involved team of developers, but another is related to the management.

"So how can you incent a Scrum team to not make a mess?"

This is a very good question, proposed by Robert C. Martin, in the article "The Land that Scrum Forgot"  that could be found here (with a forward by Mike Cohn):

http://www.scrumalliance.org/community/articles/2010/december/the-land-that-scrum-forgot/

Spoiler (from the article): "If the team goes fast and stays clean, then there is a reward!"

Usually only the project viewpoint it is the of interest for the management, because is focused on short term results. Anyway, the overall business should remain steady, and we should be interested on the product internal state, the amount of the accumulated undesired complexity.

The Scrum proposed by Jeff Sutherland it is Hyper-Productive (!), because in the same package we will found the requirement for technical excellence and outstanding enginnering practices such as TDD. For a such performance you need to read more than Scrum Guide and you need more others Agile skills and knowledge.