Thursday, December 29, 2016

Agile: the hypotheses


The basic promise of a good software development process approach is to provide good results in the context of the target domain – software development – with capabilities to address this domain specific problems & traits -  knowledge work, incertitude and changes, complexity. We need some start hypotheses for what works in managing these specific problems.

Domain: Software development
Domain Problems and Traits: knowledge work, incertitude and changes, complexity.
Promise: Address target domain specific problems and traits by providing effective, efficient and steady capabilities
Solutions hypotheses: Incertitude and complexity could be steady managed ("solved") by being adaptive and lean in the problem context.
Adaptive - Reason 1: Complex problems, with high incertitude, can be solved only with adaptive solution approaches. We need feedback & adapt loops, but these loops must be effective/efficient in each problem context. Reason 2: we need the capability to <follow the (customer) business>, were this target is in continuously change and evolution. More: technologies will change, context will change.
Lean - Incertitude and complexity will increase with any ineffective practice: the ones that not address the problems. In fact, in this generic context - complexity and incertitude - any ineffective practice could instead exponentially increase the initial problems: complexity and incertitude. More: in these conditions any waste create more waste! We need to avoid this spiral of waste.
Context Tailoring we need to solve complex problems, but each problem is different. Each process solution must address both generic traits and specific traits. We need to find what it is generic and what is specific and provide categories of solutions for the categories of problems.
  

A process approach must provide capabilities to address target domain specific problems and traits, where capability suppose both understanding and (deliberate) practice. Also a process approach must provide capabilities to adapt the process to context specific problems and traits.
 

Why Adaptive & Lean? Here also an intuitive answer: we need to be able to continuously change the software products in order to follow the needs of the target (customer) business and to do that without waste. That will serve the economics of both development and customer side.     

I just want to remember some thoughts of Robert C. Martin:
  • The main responsibility of a programmer is to be able to change the software  [1]
  • "I expect ...Inexpensive Adaptability...You will always be Ready...Stable Productivity" [2] 
 [1] - See "Clean Coders" videos ; [2] - see video "Demanding Professionalism in Software Development "

Quality? Lean suppose "No defects!" approach and Adaptive suppose built-in quality, few and easy to fix defects. 

Collaborative Work? Both Lean capability and Adaptive capability are based on a lot of collaborative work practices. 


Sunday, December 4, 2016

Work Optimization: Avoiding Waste with XP, DA and Scrum


Avoid Waste (!Waste)


Here are some of the most known work optimizing principles related to Avoiding Waste, where many of them have as source TPS philosophy to avoid sources of Muda/Waste (it is better to return to original ideas, beyond various further Lean evolutions; see biblio-TPS) The other two are the ones that bother me the most, and could be specific for knowledge work: Not Used Talents and Improvement Stalemate.  In our case, these sources of waste will be interpreted from software domain point of view.



Principles & practices. XP, Scrum and DA  


As in the case of Core Agile Capability we need have a traceability between principles, values and the practices that bring them to live. A such trace could be found at the end only from what is useful in the real work.

In the following parts, we will try to present how this realization is done by practices from original main Agile methods: XP-Extreme Programming and Scrum and how we can get a more robust approach using DA - Disciplined Agile. We mention only practices that are explicitly mentioned by the methods & their guidance ... and what we have found that works in real practice. Here it is summary, I will come back with details for each part.   

"What we need" - description of what we have find that it is working to manage that aspect of the waste.

Important: if Core Agile Capability – Responding to Changes - is not met, the software development is not optimized from the start: this core capability address all the optimizing work principles at their fundamentals.
.

No Inventory: reducing work in progress


What we need
·   First, we need to “make the work easy” (See biblio-Kent_Beck): a big work in progress (inventory) will exponentially increase the complexity & incertitude, will reduce reaction time (core required capability) and quality.
·      Reduce the work in progress (WIP) on all possible dimensions, from making the release smaller to avoid multi-tasking   

Avoid Defects


What we need
·    To prevent defects & catch them as early as possible ~ provide build-in quality; all the practice that “shift left” the quality risks will be useful  
·    First, to make the work easy: see Avoid Overproduction and Avoid Big Inventory    


Avoid Waiting


What we need
·  Work should be “Ready” – actionable (!) -  before start using main resources: we do not want to wait for dependencies related to information, artifacts and others; what is missing usually is the overall perspective, that a Rolling Wave approach could bring: product, release, iteration, inside iteration   
·    We want to avoid bottlenecks in work, such skills (specialization) bottlenecks   




Avoid Overproduction


What we need
·   To avoid “Big work” that bring many risks: it is hard to include & manage required changes, it is a high probability to deliver results that are no more needed       
·   To develop and be ready to deliver big value first; will be much easier drop not started “junk stories” on request
·  To not guess about production needs: mixing envisioning and JIT on requirements and solution, implement what and if is needed


Avoid Over-Processing


What we need
·   Avoid extra work, not required by customer
·   Avoid process ceremonial that does not bring value
·  See also dedicated sections for:  Avoiding Defects (Built-In Quality), Avoid “Transport & Motion”, Avoid Re-Work  



Avoid “Transport & Motion”


What we need
  • If, in case of manufacturing, products could be damaged by transport & motions, we can found an equivalent for this waste, also for knowledge work/software development: unnecessary long chain of information handover & information duplicate in a long chain of artifacts.
  • Successive handovers could be caused by lack team ownership over information and lack of cross-functional skills   


Avoid Not-used talents


What we need
·        Practices support to quick and effective learning & skills improvement
·        To develop cross-functional skills. Example: a programmer with requirements & testing skills will perform TDD and design better   


Continuous Improvement & no persistent impediments


What we need
·        To avoid blockage on process improvements 
·        A process that make the impediments visible & it is effective on removing them




Avoid rework


What we need
·        To avoid “Waterfall rework”: big work with too few feedbacks could increase rework to infinite  
·        To avoid “Scrum-Agile rework”: lack of look ahead on requirements & solution could induce need of rework 
·        To avoid late defects detection – see Avoid Defects
·        Finally, we need to balance envisioning and defer of the commitment (JIT) and also get as much feedback it is possible   


Final thoughts


Work optimization it is related to some main ideas:
·        Teams should choose the optimum practices and ways of work in context
·        A good range of skills and known practices maximize the also the range of available options  

Disciplined Agile principles are about choices and growing team’s members, offer a solid fundament and guidance for work optimizations, while including also XP and Scrum related practices.

Notes about practices


Specific for Agile is that with one practice we can get multiple benefits. If the same practice could address more sources of waste, we could consider that practice as more valuable for our process.
See specified bibliography for each method for more details about mentioned practices
  

Bibliography


Biblio-TSP
·        Toyota Production System: Beyond Large-Scale Production 1st Edition, by Taiichi Ohno, Productivity Press; 1 edition (March 1, 1988)
·        https://en.wikipedia.org/wiki/Toyota_Production_System
Biblio-Kent_Beck
Biblio-DA Disciplined Agile
·        Agile Modeling Practices - http://www.agilemodeling.com/
·        “Disciplined Agile Delivery:  A Practioner’s Guide to Agile Software Delivery in the Enterprise”, by Scott Ambler and Mark Lines
Biblio-XP Extreme Programming
·        Extreme programming explained: embrace change, Kent Beck, Addison-Wesley Professional; US ed edition (October 5, 1999)  
Biblio-Scrum

·        Software in 30 Days: How Agile Managers Beat the Odds, Delight Their Customers, And Leave Competitors In the Dust 1st Edition, by Ken Schwaber , Jeff Sutherland

Sunday, October 23, 2016

Core Agile capability - responding to changes - with XP and DA

Core Agile Capability – responding to changes – it was stated by Agile Manifesto values and principles and has a primary backup a set of Core Agile Practices that already realize these principles at Manifesto time. Most of them were part of the XP – Extreme Programming (See [Biblio – XP1]):    
  • Small releases
  • System Metaphor
  • Simple design
  • Testing First Development
  • Refactoring
  • Pair programming
  • Collective ownership
  • Continuous integration
  • On Site Customer
  • 40-hour week
  • Coding standards
  • Exploration, solution spikes

The beauty of this initial set of core practices is that each practice will add multiple values and also will support the use of the others.  More: with this small set, we can backup the full Agile promise: a good and adaptive process with the capability to respond to changes: quick, often and even late in development.



What will we need more?

We can extend these practices to fill the possible gaps and/or make their usage more robust on achieving the Core Agile Capability – responding to changes. Please find above such extensions using the good help of Agile Modeling and Disciplined Agile practices.


Exploration, Solution spikes – extension
  • Proven-architecture milestone - solution main decisions are proven with an end-to-end exploration skeleton, based on spikes and other working code, early in the release construction
  • Multiple-models – develop and use skills related various techniques for explorations     
  • Requirements Envisioning, Architecture Envisioning – envisioning over requirements and solution it is what need to cover during the explorations and are necessary before Release Planning 


On-Site Customer – extension
  • Active Stakeholder Participation – we need to involve the customer beyond on-site participation: not all customer side stakeholders could be on site and there are also other involved stakeholders beyond customers. Also, this it is a more robust approach, that does not depend on a single product owner. Note: One of the first XP projects (C3 Project) had big problems because of “customer representative” bottleneck (see [Biblio – XP2]).  


Collective ownership – extension
  • Collective ownership it is not possible without spreading the knowledge and the skills around the team
  • In order to spread the knowledge around the team, some of the enumerated XP practices are very useful: Test-first encourage work and refactoring on others code by offering knowledge (tests as requirements specification) and a safety net for quality; Pair-Programming (and move people around) help a lot on information distribution; System-Metaphor – high-level information it is available to the whole team
  • Anyway, using only above enumerated practices we will not cover all the needs of spreading knowledge and skills inside the team. It is very important to involve the team members in key moments of the developments with other non-solo work practices: Envisioning, Look Ahead Modeling. More, Model Storming, opportunistically used, could cover the remaining gaps in both knowledge and skills
  • The DA Architect Owner, acting as a coach, can help making this collective ownership possible    

Refactoring – extension
  • For significant amount of Technical Debt, where Test-First is not available, we will need more help, in order to make refactoring possible
  • High Technical Debt usually comes together with missing knowledge about functionality, knowledge needed for tests (normal approach of "reading" the code is very hard). More: recovering this info from refactored code is difficult because debt hide buggy functionality (... and the recovered info it also distorted). We will need to involve others in order to validate this recovered info and DA practices could help: Model Storming – go beyond pairing and discuss with more persons from your team; Active Stakeholders Participations - discuss with customers, domain experts, and others.     


From Working software to Consumable Solutions
  • Working Software and Potentially Shippable as concepts and practices will help have the work Done in an incremental/iterative manner that is a fundamental condition for being adaptive and responding to the changes
  • Consumable Solution it is an enhanced/improved concept versus the above mentioned: product it is really Done if is consumable: functional, usable, and desirable to its end users.


TDD as Executable Specification
  • In fact, XP are using TDD as Executable Specifications but in is not so explicit in its intentions
  • In Scott W. Ambler, Agile Modeling and in Robert C. Martin work this intention was made clear and explicit
  • Executable specification and TDD can support the strongest Agile promises related to responding to changes       


Changes in product life-cycles
  • In XP books, Kent Beck has noticed that Exploration it is more important in the early period of the product life. Recently he makes that more clear in Explore, Expand, Extract concepts
  • Responding to change it is different during the product life (because of incertitude and complexity differences) and we should act accordingly. DA offer clear agile delivery life-cycles variants, that could be associated with these product life phases. A start example: when a product is born (and it is in its Exploration phase) we can use Exploratory (Lean Start-up Life-Cycle)  


Add other practices
  • There are other outstanding Agile practices like (Robert C. Martin) Clean Code and Clean Architecture that are fundamental to Core Agile Capability – responding to changes. With DA it is easy to include them in the team & product process
  • Example: Clean Architecture goal could be discussed in the Envisioning and also using various form non-solo work (Look Ahead Modeling, Model Storming) and then will become strong support for TDD (strategic testable design) 


Summary of Disciplined Agile/Agile Modeling mentioned practices:


Bibliography and references