Saturday, April 18, 2020

Essential Lean - Waste Red Flags

We need to find out the sources of waste, but that requires a very good and close observation of the overall process. Some wastes, tough are important, are difficult to identify. For example:
  • It is difficult to find that you have a lot of extra features or junk stories in your product. 
  • It is difficult to find that you have a lot of hidden work (for example product parts that are hard to read or are hard to change because of the technical debt) 
Some other sources of waste are easier to detect, and are induced by most of the others (see "Avalanche effect"):
  • Waiting
  • Defects

This means that, although we could successfully use Waiting and Defects wastes as red flags, finally we need to identify and eliminate the root causes. We should also remember that we cannot reduce the Lean goal of avoiding waste only to these two sources. Waiting is the main subject for the methods that use a visual control of the work. However, if something is harder to draw, it does not means that it is less important. There is no generic ranking between the main sources of waste and their impact is different from one context situation to another.

Essential Lean Series

Read first: Principles of lean thinking (by Mary Poppendieck)     

Essentializing Lean - essay
Essential Lean - Red Flags

Essential Lean - Waste avalanche effect

Avalanche effect 

Many sources of waste will induce others, and it is hard to tell where this chain reaction ends, as it causes an "avalanche effect".  The following are some examples.

Overproduction - Extra features and junk stories increase the product and the release scope amounting to a bigger inventory.  Big inventory - A big work in progress increases work complexity. A higher complexity for both problem and solution results in more defects. More defects introduce more unexpected interruptions of the work in progress and that produces more waiting. Waiting ...  

Vicious circle


An even worse situation is a vicious circle effect between wastes. If wastes are not avoided, the difficulties not only persist but will increase in a never-ending spiral.    

Task switching affects people's focus and that increases the number of defects. More defects amount to more unexpected interruptions, which results in more waiting. With more waiting, the delivery is delayed, and the target business changes more than we previously expected. With this extra time, we receive more change requests that will increase the release and the product scope (inventory). With a bigger inventory, task switching increases: we cannot good enough slice the work inside a release, we can't say that something is done and the work for different things will interfere.         

A situation where the vicious circle could be even faster occurs when two wastes are directly feeding each other.  

Task switching reduced focus and that in turn increases the number of defects. More defects amount to more unexpected interruptions, which in turn result in more task switching.

A lesson to be learned 

We cannot disregard any important source of waste since each one creates and feeds other wastes.

Essential Lean Series

Read first: Principles of lean thinking (by Mary Poppendieck)     

Essentializing Lean - essay

Essential Lean - Goal Driven Lean

Goal-Driven Lean

Lean should be goal-driven and not prescriptive. Work optimization - the main purpose of Lean – could be done only by discovering the problems and finding the solutions that are specific for a team's and organization's context. We already have a good goal-driven approach in Disciplined Agile [1] and that could be an inspiration for organizing and essentializing Lean.
The followings are my expectations for such an approach.
  • Provide a Lean goals map 
  • For each goal, provide guidance including the followings:
    • Discover concrete problems & root causes description 
    • Recommended “default” practices and possible options
    • Discussion about options & trade-offs in each specific context
    • Reference to more detail guidance (as DA toolkit [2]) 

Principles, practices, and goals - some existing Lean principles are not really principles, but rather practices. For example, the “deliver in small batches” principle is the XP “small releases” practice. DA Manifesto [3] value “Responding to feedback over following a plan” is a proper root principle in this case. Avoiding one significant source of waste (defects, waiting, and others) is the middle ground between principles and practices, and could be a good goal to achieving a lean way of working.

Lean Development Goals

I propose the following list of lean development goals:

Avoiding waste – This means avoiding the main sources of waste (see Essential Lean - Analyzing existent solutions). There exists already a translation of Toyota Production System proposed wastes, initiated by Mary and Tom Pooppendick. There is a “coincidence” here: the described lean solutions are some of the main practices of Extreme Programming (XP). Agile and Lean are convergent [4],  and we cannot disregard this aspect. Of course, we can and we should extend [5] this initial set of lean solutions.

Early delivery of value – This is needed for several reasons. While an early delivery could have a higher market value, a delayed delivery would increase the incertitude about its real value. At the same time, early delivery implies managing complexity through more frequent feedback and reducing the work scope at the release level
Note: this goal has similarities with avoiding Overproduction waste.        
Maximizing delivered value - We need to discover, focus, and improve the activities that have an effective contribution to creating and delivering value. Ineffective and unnecessary work is an undesired complexity. Some examples include:
  • Core agile practices [6] – Helps to avoid unnecessary work, and are proved at the industry level
  • Adapt to context [7] - Work optimization is context-dependent
Note: this goal is similar to a good understanding/generalization of avoiding Overprocessing waste.   

A less known impediment is related to the management of knowledge & skills. It is not only about teamwork versus individual work. We need to collaborate and to share knowledge also at the organization and community level.  

When we evaluate the wastes and the workflow of the value we should optimize the whole and not only some local parts. Some examples include:
  • Assess the whole value stream [8]
  • Do not rush only for short term results if you want to stay in business

Comment: the first three goals above are covering the current "lean methods" or solutions set. However, we need to add any other aspect related to complexity and incertitude. 

Managing complexity – Software development has an inherent complexity that affects work results. Adding undesired and unnecessary complexity will complicate all development aspects, possibly in an exponential manner. We should consider any problem and any solution related to complexity management, not only those related to avoiding main sources of waste, early delivery of value, and maximizing value.

For example, the undesired complexity of a software product is a critical aspect of software development, creates many severe problems and it is an impediment to overall process optimization. Often feedback on all levels is necessary to keep complexity on manageable values. Some examples include: 
  • Reduce the delivery cycle with smaller releases [9], often delivery 
  • Use iterations, working software to slice the work 
  • Use consumable solution [10] increments instead of just working software, to have the full feedback circle with usable and desired software (not just functional)
  • TDD - automated test could be used more often  
  • Collaborative work - get in time feedback from your collaborators 
Managing incertitude – This is a similar discussion with that related to the management of complexity. When incertitude is inherent, we should act accordingly  - for example by using more feedback -  and stop pretending that we have “predictions”. In many cases, the incertitude is a result of the high complexity of the surrounding context for both the business and the development.    
Pull based approach (from Lean and lean inspired Kanban, Scrum) is an also feedback oriented way of working. The command is given in small increments starting with the real needs and not with a big upfront estimation of these needs. Pull based systems are addressing sources of waste such as Waiting, Overproduction, Inventory by reducing the incertitude.

Context! Waste, late delivery, complexity, incertitude are not our problems. These are just some categories of problems that a lean approach could solve. Concrete problems are always dependent on our situation/context. The "theory" (read: existing experience, in a structured form ~ categories of solutions) could help us not to fail and to succeed earlier in assessing of the current situation or in our intention to build a more effective/efficient way of working.

Essential Lean Series

Read first: Principles of lean thinking (by Mary Poppendieck)     

Essentializing Lean - essay


[1] Goal-Driven Approach (Disciplined Agile) 

[2] DA Toolkit

[3] DA Manifesto

[4] What IS Agile, that is the question (essay)

[5] Work Optimization: Avoiding Waste with XP, DA, and Scrum

[6] Core Agile Practices 

[7] Process Alphabet - Adapt to context

[8] Principles of Lean Thinking, Optimize the whole, by Mary Poppendieck

[9] Extreme Programming Explained: Embrace Change, Second Edition - Chapter 1. What is XP? , By Kent Beck, Cynthia Andres, Publisher: Addison Wesley Professional, Pub Date: November 16, 2004, ISBN: 0-321-27865-8 

[10] Consumable Solutions (Disciplined Agile)

Essential Lean - Analyzing existent solutions

Existing problems/solutions inventory

The software domain's generic problems and traits are related to complexity, incertitude, and knowledge work. The current Lean generic solutions study the way of working and try to improve it by:
  •     Avoiding waste
  •     Maximizing produced value
  •     Producing the value earlier
What we need first is traceability between the domain's problems/specifics and Lean solutions. I will start by analyzing the currently proposed solutions and describing the avoidance of best-known sources of waste, inspired by the Toyota Production System. The presented description of the wastes goes beyond the variants outlined in the Poppendick &
Poppendick writes.
Hint: search complexity and incertitude in the below text 
Avoid Defects Waste – Some significant root causes for defects are the lack of built-in quality and the late defects catching. No build-in quality [1] is equivalent to the undesired complexity of the software solution (poor design). Late defects catching [1][2] means late validation/feedback, which is another wrong approach for managing complexity

Avoid Motion Waste – Motion: finding information [2], too many handoffs [3]. Lost time and energy in finding information are results of a poor organization and of a poor abstraction of complex information. Too many handoffs cause information loss and distortions, another case of undesired complexity.     

Avoid Transportation Waste – Task switching [3] causes unnecessary complexity when the mental context for performing a single job is abandoned, then re-created, and mixed with the aspects of others. As a result, the “focus” on the task at hand will be deteriorated.   

Avoid Big Inventory Waste - A bigger scope for our work in progress [1] is equivalent to a bigger complexity. At the same time, the work will be more affected by the incertitude of the real value of the requirements, which results in sky-rocketing the complexity. 

Avoid Overprocessing Waste – Overprocessing: Any extra process [2], execution of not necessary/not effective work (and not only [1] extra paperwork) is just another form of undesired complexity.  

Avoid Overproduction – Overproduction: Too much guesswork when dealing with incertitude will generate unneeded work results - such as extra features [2], junk stories [1] - that unnecessarily increase the inventory and complexity.   

Avoid Waiting Waste – This type of waste - waiting for dependencies [1] - is easily generated if we mismanage the complexity and the incertitude caused by the dependencies related to various factors from our work. Other sources of waste such as big inventory, defects, and task switching will also induce waiting.
Early value delivery – While an early delivery could have a higher market value, a delayed delivery will increase the incertitude about the real value (similar to Overproduction waste). At the same time, early delivery means more frequent feedback, which is one of the best ways to deal with complexity.
Maximizing delivered value - We need to discover, focus, and improve the activities that have an effective contribution to creating and delivering value (similar to Overprocessing waste logic). Ineffective and/or unnecessary work is an undesired complexity.  

Analyzing Lean Solutions

Failing to avoid wastes will produce an avalanche effect, whereby each source of waste produces others and so on. We cannot deliberately ignore any of the significant sources of waste without serious underestimation of the overall waste.  

All lean categories of solutions  - avoiding waste, value maximization, early delivery - are related to the domain's
main problems/traits: complexity and incertitude. Instead of just enumerating older or newer Lean solutions we should consider exploring and experimenting with any approach that addresses these generic domain problems.  

Essential Lean Series

Read first: Principles of lean thinking (by Mary Poppendieck)   

Essentializing Lean - essay
Essential Lean - Red Flags


[1] Work Optimization: Avoiding Waste with XP, DA, and Scrum

[2] Principles of Lean Thinking  - Eliminate Waste, by Mary Poppendieck 

[3] Lean Software Development: An Agile Toolkit By Mary Poppendieck, Tom Poppendieck   Publisher: Addison Wesley Pub Date: May 08, 2003, ISBN: 0-321-15078-3

Essential Lean - Searching for a Definition

Starting with the need - Taiichi Ohno, Toyota Production System 

Searching for a definition

Lean is a systematic approach that offers a specific capability and/or offers solutions to problems from specific domains (for example software development).

Why do we need a clear, generic definition? Existing ones are focused on enumerating the “how”-s, and consequently exclude some of the great contributions to Lean.
Initial promised capability: efficiency.
For both manufacturing and knowledge work, there is a catch! You will not get efficiency by blindly following this goal. Effectiveness and efficiency are working together in any context defined by complexity and incertitude.

 Concrete promised capability: work optimization (efficiency & effectiveness)

Lean Definitions Pitfalls

Jumping to solutions is prescriptive – The currently existing definitions are "enumerations", describing some of the main categories of solutions and do not fully address the promised capability or the problems to solve. This approach – jumping into solutions - is somehow prescriptive and cannot offer an optimum (leaner) response in context.
Example: The recommendation to reduce the delivery level feedback cycle with Kanban and continuous flow is a good general guideline, but for a specific context, the leaner option could be an iteration-based life cycle with Kanban used internally. In other contexts, the reverse could be true. Given these prescriptions, we will start to "believe" in labels. For example, one big confusion is the one between Kanban and Lean, where Kanban is just one of the Lean practices.

Disregarding significant problems - The most common problems for software development efficiency should be also covered by Lean. Some examples include:
  • undesired complexity
  • lack of collaborative work
Both categories of problems are generating a lot of well-known wastes such as waiting, excessive handovers, and defects.  
Most known Lean approaches have inherited from manufacturing the subject of the waste produced by defects but failed or forgot to discuss the main root cause of the defects - undesired complexity and poor design - that is specific to software development. Unfortunately, non-software thinking mode still predominates in lean.

No problem-solution traceability – we should start from identifying the main problems and then find the best solution in context instead of starting from a list of enumerated good practices. Unfortunately, Lean is currently still delivered in a "methods" based package: here is our solutions list, just do it.   

A point of start

We should extract and use knowledge from different sources as Al Shalloway points out here: ".. if we take from Scrum what we’ve learned, not about Scrum, but about the methods it pulls from, about the ideas, about the concepts it pulls from, about why getting quick feedback is good, why having teams is good. Now, what if we go to Kanban and say what have we learned about these axiomatic truths from using Kanban, why is managing queue size good, why is managing flow good, why are explicit policies so good. And then kind of throw away both Scrum and Kanban so to speak, but keep all that knowledge of this axiomatic knowledge and then say, well, what if we created a new method, what if the people who know Scrum and Kanban are really good at it and now understand these axiomatic methods were to come up with, what would we build now if we knew back then what we know now?

It is interesting that the same thinking could be found in the foundations of world philosophy: “The fish trap exists because of the fish. Once you've gotten the fish you can forget the trap. The rabbit snare exists because of the rabbit. Once you've gotten the rabbit, you can forget the snare. Words exist because of meaning. Once you've gotten the meaning, you can forget the words. Where can I find a man who has forgotten words so I can talk with him?” - Zhuangzi, Chuang Tsu, Chapter XXVI.

Returning to software, we might have some similar goals in different software efforts, but the context is different, so we can't rely on prescribed process solution. So we should pay attention to what is a principle, an axiomatic knowledge, and what is closer to concrete practices. An excellent approach could be one expressed by Scott Ambler and Mark Lines:: "Our philosophy is to look for great ideas regardless of their source and to recognize that there are no best practices (nor worst practices). When we learn a new technique, we strive to understand what its strengths and weakness are and in what situation to (not ) apply it." [2]. We should creatively apply our domain principles and select the practices/process solutions that work better in our specific context.     

Essential Lean Series

Read first: Principles of lean thinking (by Mary Poppendieck)   

Essentializing Lean - essay
Essential Lean - Red Flags

[1] - Al Shalloway Discusses the Lean-Agile Method
 [2] - Choose your WoW: A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working, 2020 Edition, by Scott Ambler, Mark Lines 

Essentializing Lean - Essay

Starting with the need - Taiichi Ohno, Toyota Production System

We suggest reading first: Principles of lean thinking article (by Mary Poppendieck)   

SEARCHING FOR A DEFINITION... – We need a clear definition, with a clear promised capability, and that will starts from the needs, and that will not exclude any great existent contribution.   

LEAN DEFINITIONS PITFALLS... – lean should be goal-driven. Just enumerating some of the lean solutions is merely a prescriptive approach. Software development's biggest problems -  such undesired complexity and lack of collaborative work - are too often disregarded in these solutions. We need traceability between industry problems/specifics and lean solutions.
PROBLEMS/SOLUTIONS INVENTORY... – Main software domain's known generic problems and traits include: 
  • complexity, incertitude, and knowledge work
Existing lean solutions have these generic recommendations:
  • studying the way of working & improving it by avoiding waste, 
  • maximizing the produced value
  • producing the value early.  

ANALYZING LEAN SOLUTIONS... – We cannot disregard any of the major sources of waste. There is an avalanche effect: one source of waste will produce more others and so on. All lean categories of solutions (avoiding waste, maximizing value, delivering early) are related to the main domain problems/traits: complexity and incertitude. Instead of just enumerating lean solutions we should consider exploring and experimenting with any approach that will address these main problems.
GOAL DRIVEN LEAN... - Lean should be goal-driven. Work optimization - the main purpose of lean – could be performed only by discovering the problems and finding the solutions that are specific for a team and organization context. Since we already have a good goal-driven approach in Disciplined Agile that could be an inspiration for organizing and essentializing Lean.
LEAN DEVELOPMENT GOALS... – instead of jumping to directly to practices, we could propose a list of Lean goals:
  • avoiding waste
  • delivering value early
  • maximizing delivered value
  • managing complexity
  • managing incertitude
For avoiding waste, the current works that start from the sources mentioned by the Toyota Production System are a good starting point.    

GUIDANCE EXAMPLE: AVOID DEFECTS GOAL... – First we need to evaluate and assess the impact of the defects. Product complexity and its criticality should influence quality politics. Then we should identify the root causes and the recommended practices in context. Some examples of causes are an unnecessary big scope of work, no built-in quality, and lack of skills. 
RED FLAGS... – if we want a quick, but not necessarily accurate, assessment of work there are some red flags, such as two sources of waste that are easier to measure: Waiting and Defects. Anyway, one should remember the “waste avalanche effect” and search for all significant sources of waste and their root causes.   

Essential Lean Series

Essentializing Lean - essay