BOOK: Trust - The social virtues and the creation of prosperity

A couple of posts ago I mentioned the book Trust: The social virtues and the creation of prosperity by Francis Fukuyama. I hadn't read the book yet, but Austin Hastings, an esteemed SCM colleague, has and I promised that if he was willing to post his summary and notes on the book that I would publish them on my blog here (and not just as a comment). I also recommend reading one of Austin's recent blog-entries entitled "Being a trust specialist", which reinforces why, even for CM professionals, it is often all too true that "The first thing to 'build' is TRUST." The rest of this blog-entry is Austin's writing.

---

Francis Fukuyama is noted for writing "The End of History" and "The Great Disruption" as well as for "Trust: The Social Virtues and The Creation of Prosperity." As such, it is pretty easy to draw inferences about his political viewpoint(s). Many reviewers have made the assertion that his books are a product of his politics. I (personally) don't know enough to comment.

That exact argument -- questionable causality -- is one they make against some of the correlations that Fukuyama cites in Trust, but they seem to miss it when it applies to their own positions. So beware: the book may be written in deliberate support of his politics, or his politics may have evolved from the studies he has done. It's your call.

Fukuyama claims that trust is a form of "Social Capital." That is, trust is something that you can invest in, something that you can create or obtain more of, something that you can use to achieve an economic end, and something that has value.

He further claims that the presence or absence of trust in a society has a significant, measurable impact on the economic indicators for that society, and that it probably has other, less clear effects as well. All of this is generally in agreement with other writings on trust that Brad has cited here.

His argument, then, is that there should be a way to objectively test for the amount of trust in a society. His test criterion is the number of employees that work for medium-sized businesses, where business size is a function of the number of employees. (That sounds circular, but it isn't.)

For example, my neighborhood pizza shop is run by George, his wife Rosanne, and their son Steve. There are two other cooks, and two delivery drivers. So the business has 7 employees, which makes it a small business.

By contrast, GM just announced that they intend to close down the Pontiac car brand, and lay off 21,000 workers. Having 21,000 workers to lay off makes GM a really, really HUGE business.

Somewhere between the two extremes lies a medium sized business. I don't recall exactly what numbers Fukuyama used, but let's say it's 100 <= n <= 1000. So we have small is less than 100 employees, medium is up to 1000, and large is anything over that.

With those boundaries in mind, the question is simply how many employees in a particular country work for a business that can be classified as "small," or "medium," or "large." Well, if there are 200 small businesses, and their average size is 30 employees, then 6,000 employees work for small businesses. (That's how we figured the average size, I guess.)

What Fukuyama found was that some regions or countries exhibited a noticeable "saddle shape" in their graph of business size versus number of employees. There were lots of people working for small businesses, and there were lots of people working for large businesses, but few people working for medium sized businesses.

He argues that large businesses are a distraction, because governments can create large businesses by fiat. France, Mexico, and most of the OPEC countries have huge businesses that were created by the government (as opposed to being grown up from small businesses).

Small businesses, naturally, are the starting point for almost everything. Somebody has an idea, they start a business with their close friend from college or their brother or parents, and if they make money they start hiring.

The problem comes when all the family members are hired. If the entire family is hired -- all the brothers, uncles, cousins, etc. -- and they're all doing some kind of management thing, with "outsiders" brought in to do the simple labor, the business has reached a critical point.

At this point, the business may or may not be *capable* of bringing in a qualified stranger, and handing that stranger an appropriate amount of power. That transition, from family shop to "real company," is the dividing line that Fukuyama is really looking for with his arbitrary criterion of 100 employees.

And he argues that if you see a disproportionately low number of workers that have jobs at medium-sized companies, it's because there are a low number of medium-sized companies. And *that* is because there is not enough social trust. When grandpa and dad and uncle Cletus can't let go of the reins, the company can't get any bigger. And in fact, the company will likely fail shortly thereafter, resulting in a much smaller business after the smoke clears.

Using his metric, there are some genuine surprises. Germany is a high-trust country, but France is low-trust. Southern Italy is virtually a no-trust area. Japan and Korea appear low-trust, until you refactor your statistics to deal with the Zaibatsu. China and most of the Asian mainland are low-trust.

This is interesting, but not necessarily controversial. What *is* controversial is the correlation with "The Protestant Ethic and the Spirit of Capitalism" (Max Weber, 1905 !!). That offends a lot of people, for a lot of reasons. Weber's point, made back when people were giving serious credence to "racial studies" and other stuff, was that Protestant countries did better economically than Catholic ones. Fukuyama makes a similar point, but claims that the effect is corollary, not causal.

Fukuyama's point is that there are a lot of flavors of protestantism, and countries that are majority protestant don't always have social mechanisms for creating and maintaining trust networks. If you can't generate trust, it doesn't matter how Protestant you are.

Things like the Rotary Club, and the Moose Lodge, and the Veterans of Foreign Wars in the United States do that job for us. These little mini-networks enable people of similar creed to reach each other, so that there are many networks of high trust -- "I trust him because we go to the same meetings" -- working parallel to each other. This enables Catholics to network with Catholics, Baptists with Baptists, Scrum fans with other Scrum fans, etc.

The rest of the "trust equation" is pretty straightforward. Nearly all of the trust literature agrees on these things: high trust leads to efficiency. Fukuyama's point is illustrated in any American business transaction. If you make noises of intent to engage in such a transaction, the expectation is that both sides intend to do so fairly.

That simple fact -- that you can go into a sandwich shop, for example, and place an order, and they will start making your order before you pay for it -- is one that is hard to see if you aren't looking for it.

The reverse situation pertains low trust areas. If you try to do business in these places, nothing is done until the cash changes hands, or at least until it is displayed for all to see.

A similar thing occurs in my own line of work, where implementing change tracking lifecycles is surrounded by requests to create explicit status codes for each possible situation. ("You should have a 'completed by development, but QA will not start testing due to other commitments' status code!")

It's important to keep in mind that the business-size metric is an indicator, nothing more. And that Fukuyama uses that metric as a way to set expectations for research, not as a causative for other social ills. A small number of medium-sized businesses doesn't cause poor social trust. It is an indicator that social trust is likely poor. (Or, as in the cases of Japan and Korea, that the metrics need to be refined.)

One of the reasons for many of the negative reviews is Fukuyama's assertion that trust is not correlated with equality, or fairness. American society has historically been more unfair and inequal than otherwise. Every single minority has been discriminated against at some point, which has led most of them to creating their own separate "civil associations" -- networks of trust.

The obvious inference is that eroding the separate networks of trust will result in an overall low-trust society. This isn't particularly politically correct, and so you can probably imagine how it was received in academic and/or liberal circles

---

Thanks again to Austin Hastings for taking the time to comment so comprehensively on this important work. And don't forget to read his recent blog-entry "Being a trust specialist."

More Articles on Trust

Since I blogged about a couple books on this subject I wanted to give a few other resources as well. First off, the reason I came across these resources is because back in January I participated in a discussion with Jim Coplien, Diana Larsen, and Doug Shimp about sharing, trust-loops, and team interaction.

Aside from each of us sharing our own thoughts we also shared some resources. I already mentioned the books. There were also some online articles/materials that were discussed and I wanted to share those here.

• One recent article is Creating Trust by Rick Brenner -- Rick has a lot of good articles at his site, and I'm a regular subscriber to his free PointLookout newsletters


• I also read some interesting things in "Trust and Trust Building" by Roy J. Lewicki and Edward C. Tomlinson, particularly the definitions of "calculus-based trust" and "identification-based trust" (which I admit were new terms to me)


• Fear, trust and Truth in IT Project Teams


• Building Trust in Emergency Response Teams
  


• The Meanings of Trust, by some folks at the University of Minnesota (not sure if this is a research project/report or someone's in-progress thesis)


• Diana Larsen has a presentation entitled "The First Thing to Build": Leveraging Trust on Agile Teams, which builds upon something I said several years back ("The first thing to build is TRUST!" -- which Alistair Cockburn liked a lot and started using it in the signature of his email & forum posts)
  

Diana Larsen mentioned the following:

Another useful book is Dennis & Michelle Reina's Trust and Betrayal in the Workplace. They've developed an interesting model they call Transactional Trust which is a further explanation of the behaviors involved in "you've got to give it to get it?". They include three kinds;

• Contractual trust - managing explicit and implicit expectations, establish boundaries, delegate appropriately, encourage mutually serving intentions, be consistent, keep agreements (do what you say you'll do)
• Communication trust share information, tell the truth, admit mistakes, give & receive effective feedback, maintain confidentiality, speak to good purpose (avoid gossip, a.k.a "be impeccable with your word")
• Competence trust - acknowledge people's skills and abilities, allow people to make decisions, involve others & seek their input, help people learn new skills

Robert Hurley wrote an article for Harvard Business Review, Winning Your Employee's Trust. It's more about the relationship between leaders and staff. The interesting part, to me, is a model he developed out of his research. It links back to that idea that people's capacity for trust comes both from within themselves and from situational context, in a sophisticated (and unconscious) calculation of numerous elements. Fascinating, I thought.

Another article you might find interesting is called, Promises, Lies and Apologies: Is it Possible to Restore Trust? and is about what circumstances contribute to whether trust can be rebuilt. Particularly apt here in Portland as we deal with the lies our newly elected Mayor got caught in.

If you have some links to some other good resources on trust, please add your comment and share them with me!

BOOK: Building Trust

Last time I blogged about Stephen Covey's The Speed of Trust: The One Thing That Changes Everything (see www.speedoftrust.com). I have a few other books about trust on my bookshelf, including:

• Trust and Betrayal in the Workplace, by Dennis & Michelle Reina
  
• The Trusted Leader, by Robert Galford and Anne Seibold Drapeau, and ...
  
• Building Trust: in Business, Politics, Relationships, and Life by Robert C. Solomon and Fernando Flores

This time I'd I like to share some of the thoughts from Solomon and Flores' book Building Trust: in Business, Politics, Relationships, and Life. I bought it on the recommendation of David Anderson in his blog-entry You are what you read!. That blog-entry also mentions The Trusted Leader, as well as Trust: The social virtues and the creation of prosperity by Francis Fukuyama (which David felt was the most important book he'd seen on the topic so far).

Anyway, since we started exploring what trust is and what it means, here are some excerpt's from the introduction of "Building Trust: in Business, Politics, Relationships, and Life, by Robert C. Solomon and Fernando Flores (italicized comments appearing in square brackets outside of quotation marks are mine):

"Building trust begins with an honest understanding of trust, but it also requires everyday routines and practices. Without the practices, that understanding comes to nothing." [what happens if I replace "trust" with "agility" in this sentence]

"Trust is the essential precondition upon which all real success depends. The key to trust is action, and, in particular, commitment: commitments made and commitments honored."

"The problem of trust has clearly emerged as the problem in human relationships and organizations. What makes most companies falter-leaving aside market forces, bad products, and incompetent management-is the lack of trust."

"Trusting is something we make, we create, we build, we maintain, we sustain with our promise, our commitments, our emotions, and our sense of our own integrity. "

"Trust is not merely reliability, predictability, or what is sometimes understood as trustworthiness. It is always the relationship within which trust is based and which trust itself helps create."

"The freedom provided by trust is the freedom to to engage in projects that one could not or would not undertake on one's own. The freedom provided by trust is the freedom to approach and engage with strangers whom one may in fact never lay eyes on. The freedom provided by trust is the freedom to think for oneself and speak up with one's ideas. It includes as its consequence (not its cost) the freedom to be questioned and criticized -- and the right to be recognized and (if deserving) rewarded."

"Trust is a matter of making and keeping commitments, and the problem is the failure to cultivate commitment making.

"Trust involves sincerity, authenticity, integrity, virtue, and honor. It is a matter of conscientious integrity."

"Authentic trust is going into the unknown together."

"The worst enemies of trust are cynicism, selfishness, and a naïve conception of life in which one expects more than one is willing to give. Resentment, distrust, and inauthenticity are the result."

"Self-trust is the most basic and most often neglected from of trust. Distrust is often a projection of missing self-trust."

"Trust goes hand in hand with truth. Lying is always a breach of trust. What is wrong with lying, in turn, is that it breaches trust. ...telling the truth establishes trust and lying destroys it."

"Authentic trust can never be taken for granted, but must be continuously cultivated through commitments and truthfulness. True leadership, whatever else it may be, can be based on nothing less."

"cordial hypocrisy: the strong tendency of people in organizations, because of loyalty or fear, to pretend there is trust when there is none, being polite in the name of harmony when cynicism and distrust are active poisons, eating away at the very existence of organizations [or relationships]."

"How we think about trust ... makes trust possible, difficult, or even impossible. Trust (like love and freedom) involves any number of self-promoting and self-defeating prophecies."

"Trust(ing), not trustworthiness, is the issue. The existential question is how to trust, not just who can be trusted. (Trust is not only earned; it must be given.)

Trust is a matter of reciprocal relationships, not of predictions, risk and reliance.

Trust is transformative. It is not a matter of trusting or being trusted so much as it is a matter of changing each other and the relationship through trust."

"The German sociologist Niklas Luhmann stresses that trust is a way of dealing with complexity in an increasingly complex society. There is a deep truth to this. The paradigm of trust is not found in the simplicity of a familiar relationship. Rather, it exists in the new complexity of the world and the global economy. Trust not only lets us increase complexity in our lives (and thus simplify them at the same time); it also changes our lives in dramatic ways, allowing us to explore in new directions, to experiment and express ourselves in our relationships in ways that would otherwise be unthinkable. And it allows us to grow and change and mellow and deepen in all the ways that merely provincial trust and distrust distort and prohibit."

"Trust is not always a good thing. Trust can be foolish, naive, gullible, and blind. And trust ought never to be taken for granted. That is why we insist the issue is building trust -- that is, creating trust, maintaining trust, restoring trust once it has been lost or betrayed. We want to suggest that this requires a radical revision of our conception of trust. Our thesis, to put it simply, is that trusting is something that we individually do; it is something we make, we create, we build, we maintain, we sustain with our promises, our commitments, our emotions, and our sense of our own integrity.

Trust is not, contrary to what some authors have written, a medium, an atmosphere, a 'lubricant,' social 'glue,' a lucky break for one society or another, or some mysterious social 'stuff.' Trust is an option, a choice. It is an active part of our lives, not something that is there from the beginning, or that can be taken for granted. It involves skills and commitment, not just good luck or mutual understanding.

The focus of trust -- or what we will call authentic trust -- is not just the hoped for outcome of this or that event or transaction. Trust is not merely reliability, predictability, or what is sometimes understood as trustworthiness. It is always the relationship within which trust is based and which trust itself helps create. Authentic trust does not necessitate the exclusion of distrust. To the contrary, it embraces the possibilities of distrust and betrayal as an essential part of trust. To be somewhat grim in our initial characterization of trust, it entails the possibility of betrayal.

The loss of trust is not mere disappointment. That is why trust is often evident only in the event of a breakdown. Like love, trust often becomes most palpable in the breach. (“You don't miss your water till the well runs dry.”) Building trust means coming to terms with the possibility of breach and betrayal."

"Trust, like love, may seem to fail us, but truly, we fail at trust or love. But then we get more sophisticated. We learn that trust, like love, is an emotional skill. It requires judgment. It requires vigilant attention. It requires conscientious action. It involves all of the intricate reciprocities of a human relation­ship (even in cases in which it remains “unrequited”)."

"Trust. like love, is an emotional skill, an ongoing dynamic aspect of relationships. We don't just fall in love, we decide to love. So, too, we do not simply find ourselves trusting, after months or perhaps years of comfortable familiarity. We make decisions to trust. We make promises and tacit communication. We see them through. We come to have expectations of others, and we respond to the fulfillment or frustration of those expectations. Trust isn't something we 'have,' or a medium or an atmosphere withing which we operate. Trust is something we do, something we make. Our mutual choices of trust determine nothing less than the kinds of beings we are and the kinds of lives we will live together."

Note some of the similarities and differences between the above, and what Stephen R. Covey writes in The Speed of Trust. Next time I'll give a few more resources on trust!

BOOK: The Speed of Trust

I read Stephen Covey's The Speed of Trust: The One Thing That Changes Everything (see www.speedoftrust.com). I listened to it on audiobook during my commute a few months ago and parts of it definitely struck a chord with me.

I like how he described the relationship between "trust" and speed+cost, and how low trust makes things slower and more costly. He also defined "5 levels of trust" (more like concentric circles) as follows:

1. Self-Trust ("giving trust" - do you trust yourself? are you willing & able to trust of others?)
2. Relationship Trust (establishing trust within interpersonal relationships)
3. Organizational Trust (establishing trust within & across an organization)
4. Market Trust (establishing trust within & across your market - stockholders, patrons, consumers. This is like "Brand trust")
5. Global Trust (I forget the examples of this one)

There is a good summary of the book here, and another one here. There is also an early draft of chapters 1-2 available online.

Covey actually doesnt try to define trust very precisely. He simply quotes Jack Welch saying "I know it when I feel it." He says trust implies confidence in something/someone, and that lack of trust implies suspicion.

Trusting someone is a function of our perception of their character, and their competency.

• "Character" includes a person's integrity, motives/agenda, intent and behavior with people, where
• "Integrity" is mostly congruence (with an appropriate dash of humility and courage thrown in). Lack of congruence results in a lack of credibility.
• "Competency" includes a person's capabilities, skills, results and "track record"
• "Capability" is defined in terms of TASKS: talent, attitudes, skills, knowledge & style.

In addition to observing the five "waves" (or "levels of evolutionary scale") of trust above, he says about the first level/wave that it is very much about credibility, and describes the "four cores" of credibility are a person's integrity, intentions, capability and results. Demonstrating those things builds credibility in your words and actions. Credibility is a necessary (but not sufficient) ingredient for trust. People are less able to trust you if they don't find you credible.

The rest of the book is about the so called "13 behaviors" that, when demonstrated, will help you "build trust". Those are:

1. talk straight
2. demonstrate respect
3. create transparency
4. right wrongs
5. show loyalty
6. deliver results
7. get better (improve)
8. confront reality
9. clarify expectations
10. practice accountability
11. listen first
12. keep commitments
13. extend trust

Regarding "higher" (more evolved) levels of trust, he refers to the following principles:

• the principle of alignment
• the principle of reputation
• the principle of contribution

He also describes some myths about trust:

MYTH	REALITY
Trust is soft.	Trust is hard, real, and quantifiable. It measurably affects both speed and cost.
Trust is slow.	Nothing is as fast as the speed of trust.
Trust is built solely on integrity.	Trust is a function of both character (which includes integrity) and competence.
You either have trust or you don’t.	Trust can be both created and destroyed.
Once lost, trust cannot be restored.	Though difficult, in most cases lost trust can be restored
You can’t teach trust.	Trust can be effectively taught and learned, and it can become a leverageable, strategic advantage.
Trusting people is too risky.	Not trusting people is a greater risk.
You establish trust one person at a time.	Establishing trust with the one establishes trust with the many.

The Agility Cycle - Part 2

We saw in the previous blog-entry several definitions of the Business Agility Cycle. We also mentioned that in order to derive the Software Agility Cycle from this, we needed to explicitly include more close collaboration.

The Software Agility Cycle is:

• Sense the Problem/Opportunity
• See the Problem in the Context of the "Whole"
• Socialize the Goals and Constraints
• Swarm the Solution
• Show the Results
• Share the Knowledge Learned

Here's how I derived the above... Once again, I'll refer to Jim Highsmith to represent the "people and collaboration" component of software agility.

In his book Adaptive Software Development, Highsmith compares software development to a complex adaptive system (CAS) and uses CAS with elements of chaos theory and complexity science to derive the critical importance of people and collaboration in software development. He does this using the concepts of intelligent agents, self-organization and emergence within a turbulent (ever-changing, complex and seemingly chaotic) environment:

• When we treat people as "intelligent agents" who both cooperate and compete to get work done in a turbulent environment, the final outcome is not the result of the work from any particular individual or process.


• The outcome instead emerges from the interaction between the collaborating individuals to produce a result that cannot be predicted from their individual behavior (it is non-linear, defying simple cause-and-effect reasoning).


• The details of the collaboration (and the final solution) cannot be pre-ordained and then commanded-and-controlled


• Instead, successful collaboration and creativity must be nurtured, and must trust and empower the individuals to direct and organize themselves together "in real-time", learning and adjusting as they go.

This is the phenomenon of emergent behavior from self-organization to achieve successful results and adapt to unpredictable circumstances.

The other problem with the business agility cycle is that it seems to presume that decision-making about what solution to attempt is done by a smaller, separate group of people than those who will implement and deliver the solution, and we merely need to communicate to them and have them act to execute the solution.

It's not clear whether this assumes knowledge or design of the solution "up front" with a "handover" to an implementation team, or whether it can mean that determining the solution needs to be just as collaborative as its implementation and must involve many of the same people, all working together at the same time.

The collaborative aspect of software agility demands that the solution emerges from those who must create and deliver it, and that they are empowered to make the decisions about what that solution is and how best to do it. Rather than having the decision made for them and simply "executed" by them, once the problem became known, the request or opportunity would be presented to them as a problem to be solved, together.

The goals and objectives would be socialized, along with the needs and constraints, and then those who must collaborate across the value-stream to devise and deliver a solution would get together to make it a reality. They would learn what they needed to know, show results to customers and stakeholders and then get feedback to try and learn and adapt.

This yields a slightly different cycle for software agility than the one we had for business agility:

• Sense the Problem/Opportunity using whatever local feedback mechanisms have been put in place to become aware of it. This is the same as it was in the business agility cycle.
  


• See the Problem in the Context of the "Whole" system. Rather than "strategizing" this is more like the "orient" step from Boyd, or the "interpret & evaluate" step from Oosterhout. This lets us properly frame the problem within the organization and its "delivery system."
  


• Socialize the Goals and Constraints among all those who will work together to devise and deliver the result. This is similar to the communicate step from Gartner, but we didn't really decide what the solution should be. We decided what to do (the vision) at a high-level but not the details of "how." We haven't made a detailed plan or spec.
  


• Swarm the Solution, collaborating intensely and self-organizing as needed across functions, organizational and geographical boundaries, etc. This combines the "act" and "decide" steps into a single, self-organizing group responsible for the result. We delegated the decision-making authority to them about what exactly to build and how to go about building it.
  


• Show the Results, demonstrating the knowledge gained about the solution in "executable" form so we can get feedback as to its validity. Then finally ...
  


• Share the Knowledge Learned with the rest of the organization so they can inspect, adapt, and move forward. And most importantly so they can remember how they learned what they now know.
  

The thing to remember is that this cycle happens at every level of scale in our delivery system. If we do this cycle once for a large problem, we have ourselves a waterfall. The above cycle applies to ALL the feedback loops that need to be put into place. There is the delivery-cycle, the release-cycle, the planning-cycle (the timebox), the development-cycle (backlog-item), the integration-cycle (e.g., TDD and CI), and the peer-review cycle (e.g., pair programming). They all go thru these same six steps in some form or another:

• Sense & See (sense locally, see globally)
  
• Socialize & Swarm (socialize the problem, swarm the solution)
• Show & Share (show the results, share the knowledge)
  

These are the steps of the software agility cycle that we use to Evaluate change, Collaborate together, Validate the results, and then Iterate over the cycle again, applying our newly found knowledge to learn and adapt was we go.

The Agility Cycle - Part 1

Continuing the "What is Agility?" series of posts ... we have looked at business agility and how it combines with the "people factor" from the agile manifesto to yield software development agility. So now that we know the meaning of agility, the next two questions  I want to answer are "how do you do it?" and "what does it look like?"

In this posting I will attempt to generally answer the "how do you do it?" question. This is basically a question of process. What is the overall process for "swiftly sensing and rapidly responding to change & uncertainty in close collaboration with stakeholders to create simple, sustainable structures with sufficient flexibility to dynamically adapt & evolve business processes, products & plans."

This process is essentially an overall cycle. Something that you do, and then "rinse, lather & repeat." It is called the Agility Cycle. Let's look at a few different descriptions of this cycle (mostly from the realm of business agility) and then formulate our own for software development agility.

Gartner describes the business agility cycle as:

• Sense the need for change in the environment (includes the proactive initiation of change)
• Strategize the available options and develop alternatives
• Decide which path to take and commit to the approach
• Communicate internally and externally to everyone who needs to know
• Act to produce results and follow-through efficiently

Gartner has a number of reports on agility (some of which are even freely available :). You can read about them all at this summary.

John Boyd describes it as an OODA loop:

• Observe your environment (yourself; your threats and opportunities; the physical, mental, and ethical situation; and potential allies and opponents)
• Orient yourself to grasp what it all means and identify patterns, approaches and likely outcomes
• Decide upon a course of action and tactical plan
• Act on the plan (execute while minding the terrain)

Another source describes it as:

• Scan emerging trends and issues through gathering information and analysis
• Sense opportunities to translate information into actionable solutions
• Respond to opportunities and risks by being sufficiently flexible at tactical and strategic levels
• Shape future environments through driving change.

In Sense, Respond and Learn, Marcel v Oosterhout describes the business-agility cycle as:

• Monitor & Sense: monitor the business environment and sense critical market signals, events, and changing conditions
• Interpret & Evaluate: interpret the information and evaluate if it is something to (re)act to or ignore.
• Decide & Plan: decide which response is best and plan to implement it
• Reconfigure & Adapt: reconfigure or adapt business, operations, or IT capabilities
• Respond & Execute: respond by executing with new or adapted capabilities
• Learn: Record the cycle, learn from previous cycles, share to improve future cycles
• Manage: manage the sense-respond-learn cycle to support completing the process as rapidly and effectively as required

Notice that, as with the definitions of business agility we saw before, there is no explicit mention of the kind of close collaboration and self-organization that is deemed an essential component of software agility. Any attempt we make to derive the software agility cycle from the business agility cycle needs to take into consideration this close collaboration of the participants and emergence of the solution as a result (rather than knowing it all up front).

We'll discuss this next time when we try to describe the software agility cycle.

What is Agility? Part 2 - Software Agility

In Part 1 of "What is Agility?" we looked at numerous definitions and descriptions of business agility.

The overwhelming majority of them had the following elements in common among their descriptions:

• Agility means swiftly sensing and rapidly responding to change and uncertainty.
• Demand for change may be internal or external to the organization/environment.
• Responding to the change utilizes resources & staff that are both internal and external.
  
• Changes can be unpredictable and unanticipated, as well as foreseeable and anticipated.
• Unanticipated changes are typically handled with dynamic adaptation through short cycles of feedback and learning.
• Anticipated changes are typically handled with simple yet flexible structures that are resilient when modified.
• And of course all this is achieved with solutions that are effective, efficient, and economical to implement and deliver!
  

So what is missing from the above that is an essential characteristic of agile development as defined by the values and principles of the Agile Manifesto?

I think it is the people factor, and what Jim Highsmith calls "a focus on talent and skills of individuals and teams." Close collaboration across functions and other internal & external boundaries is the "secret sauce" that best leverages the talents & skills of individuals & teams. This is what enables the rapid learning which, when combined with those talents and skills, delivers the most innovative solutions and best results in the shortest time.

Put ALL of the above together, and we have a definition of software development agility that looks something like the following:

Software development agility is the ability to ...

• swiftly sense and rapidly respond to change & uncertainty
• in close collaboration with internal and external stakeholders
• to create simple, sustainable structures with sufficient flexibility
• to dynamically adapt & evolve business processes, products & plans
• that realize results with efficiency in motion, economy of effort, energy in execution, and efficacy of impact!

And there you have it! Note that the last bullet above is really just a fancy phrasing of mandatory business jargon that ultimately means "Lean", but we'll explore that connection further in a later blog-entry.

BOOK: Clean Code - A Handbook of Software Craftsmanship

My review of Robert Martin's (and the rest of the folks at ObjectMentor) book Clean Code: A Handbook of Software Craftsmanship was just published in the April 2009 issue of the Agile Journal.

Robert ("Uncle Bob") Martin and the folks at ObjectMentor have written a new book that should be required reading for all programmers! When it comes to writing clear and maintainable code, cleanliness is indeed next to godliness, and we should all follow the Boy Scouts' Rule whenever we write or modify any piece of code: leave the place cleaner than when you found it!

Robert C. Martin is one of the giants of object-oriented design, and a founding father of the agile manifesto and of agile software development. More recently, he helped kickstart the software craftsmanship movement, which gained much attention from his keynote at Agile2008 and subsequent talks on craftsmanship and ethics.

...

If you are at all serious about programming, then as soon as you finish reading this review you should get your hands on Clean Code as fast as you can! Get it. Read it. Learn it. Then live it!

Read the full review

Additional resources related to the book are:

• Uncle Bob's article "What is Clean Code?" which is an online version of the first chapter of the book
• PDF of an excerpt from the book, including the TOC, Foreword, Introduction, and Chapter 1
• Clean Code Tip of the Week #1 and other Clean Code tips available online from the "Extras" tab of the InformIT site for the book

CRISP Builds

Jeff Langr and Tim Ottinger have a pretty sweet and useful "gig" going on at The Agile In a Flash project, which will "result in a book and replenishable deck of at least 52 flash cards."

I made a suggestion to them late last week about Mike Clark's "CRISP" acronym for builds, and they turned it into a CRISP flashcard. Here is the comment I made that led to that result ...

Mike Clark gave a 2004 presentation on Pragmatic Project Automation that included a description of what he called the "CRISP" criteria for build:

• Complete (recipe lists all ingredients)
• Repeatable (version control time machine)
• Informative (radiate valuable information)
• Schedulable (complete and repeatable)
• Portable (machine-independent)

There is a similar description in the 2007 presentation “All Builds are Good”, and a more detailed description in this 2007 CT-SPIN presentation on project automation:

Complete:
• Build from scratch and independently without human intervention.

Repeatable:
• Must be able to create exactly the same build at a later time.
• Store build scripts in source control.

Informative:
• "Detector of unexpected changes".
• Provide information on why a build failed.

Scheduled:
• Let the builds run automatically.

Portable:
• Build should be runnable from any system (same platform), not just that of the developer.
• For cross-platform software, it should build on all platforms.

I'm curious as to whether or not others have come across this acronym and if they like it and find it useful? What corrections, clarifications or additions do you think should be made to the acronym or its description?

What is Agility? Part 1 - Business Agility

Kicking off my series of posts on the subject of "What is Agility?" is this first posting on Business Agility, and how agile methods got its name from those roots...

Some of you may already know that before Agile methods used the word "Agile", the term "Lightweight methods" was being generally used to describe them. It wasn't until the February 2001 gathering at Snowbird, Utah that they began using the term "Agile", and the "Agile manifesto" was born.

At that time, the term "Agile" was already in use to refer to "business agility" by those in the field of organizational change and learning. I know of at least a few people (Mike Beedle for example) who were there at Snowbird that knew this and even liked the term and used it from time-to-time before that now famous gathering.

So the then lightweight methods community intentionally adopted the terms "agile" and "agility" in this context. Let's explore our roots and see exactly what "Business Agility" means. I surfed the net for some time and gathered a number of definitions.

Business Agility is ...

“the ability to both create and respond to [anticipated and unanticipated] change in order to profit in a turbulent business environment.” —[James Highsmith and Highsmith2]

“the ability to adapt rapidly and cost efficiently in response to changes in the business environment.” —[Wikipedia]

“the ability to sense environmental change and to respond efficiently and effectively to it. Sensing the need for change also includes the proactive initiation of change.” —[Gartner1 and Gartner2]

“the capability to be flexible, responsive, adaptive, and show initiative in times of change and uncertainty”—[BNET]

“the ability of enterprises to cope with unpredictable changes, to survive unprecedented threats from the business environment, and to take advantage of changes as opportunities” —[Agile Business: The competitive weapon. Market research study in four business segments (2004)]

“the ability to swiftly change businesses and business processes beyond the normal level of flexibility in order to effectively deal with unpredictable external and internal changes.” —[Business Agility in the Netherlands: Research Study Public Sector (2005)]

“the capacity to gain competitive advantage by intelligently, rapidly and proactively seizing opportunities and responding to threats.” —S. Meredith and D. Francis (2000)

“the ability to cope with unexpected changes, survive unprecedented threats from the business environment, and take advantage of these changes as opportunities.” —Z. Zhang and H. Sharifi (2000)

“The ability to thrive in a continuously changing, unpredictable business environment.” —R. Dove (1999)

Those are some nutshell definitions. They are focused on the dynamic capability to swiftly sense and rapidly respond to change using a combination of adaptation (for unforeseen changes) and flexibility (for foreseeable changes).

Let's look at some more details from a few more sources ...

From Process Transformation for Reaching Agility: CIO Role

• Business agility calls for quick decisions and action. Agility is the ability of a business system to sense environmental change and respond efficiently and effectively to that change. Any framework for agility must address issues that go well beyond selection of the latest technologies.

• Business system's willingness to be agile, its understanding of its own business system building blocks and its enablers of agility, and its adherence to an "agility cycle" are just as important as the judicious use of agility-influencing technologies.

• The following four fundamental capabilities enable a business system to increase agile performance across the agility cycle. They are essential to being able to allocate corporate activities to measurable categories:

  • Awareness (the right information through data and event monitoring mean knowing what is going on)
  • Flexibility (the right options by rule modeling and simulation render possible confronting expected change)
  • Adaptability (the right reactions by rapid rule modification enable confronting unexpected change)
  • Productivity (the right policies, procedures and operations through automation for executing well day to day).

From What Lessons can the Agile Community Learn from a Maverick Fighter Pilot? and Thriving on Chaos:

• The key to thriving on chaos is a skill Boyd calls agility, which is “the ability to rapidly change one’s orientation—[or] worldview—in response to what is happening in the external world.”

• In the context of battle, agility involves four distinctive activities: observe your environment (yourself; your opponent; the physical, mental, and moral situation; and potential allies and opponents), orient yourself to decide what it all means, reach some type of decision, and attempt to carry out the decision. Observe, orient, decide, and act.

• It turns out that the most critical step in the agility cycle is the first one: observe. This is much more than a process of looking around. Rather, it is a relentless search for the truth about your situation. The idea is to “go out and get all the information you can by whatever means possible.” Why? Because you can never be sure beforehand which stray idea will prove essential.

• In Boyd’s conception, the first step—observe—is the only input from outside yourself. For this reason, how well your orientation matches the real world is largely a function of how well you observe. You are looking for mismatches between your current worldview and the world as it actually is. “A general rule is that bad news is the only kind that will do you any good….

• You must seek out and find data that doesn’t fit with your worldview and you must do this while there is still time.” Otherwise the world will change and you will find yourself disoriented. You will have lost the initiative, which is dangerous in any conflict. However, if you are able to get the facts right and are willing to orient yourself to them, then astute decisions and effective actions will follow.

From Towards Agile Government:

• In practice, agility features the following four characteristics: short term frontline responsiveness, strategic adaptation, outcomes focus. preventing or reducing problems before they arise.

• Agile organisations are ‘hyper strategic’, tackling challenges wrought by turbulent external environments, while also preparing for future changes that are not yet apparent. They move through an agility cycle, seeking out and interpreting information to inform short, medium and long term decision making and action. The agility cycle is a four-step process through which organisations:

  • scan emerging trends and issues
  • sense opportunities to translate information into actionable solutions
  • respond to opportunities and risks
  • shape future environments.

From Agility thru SOA (Master’s Thesis)

• Business Agility is the ability to sense internal and external changes, as well as being able to swiftly adapt, in reaction to sensed changes, businesses and business processes beyond the normal (operational) levels of flexibility, effectively using internal and external resources, to effectively manage unpredictable external and internal changes.

• Different types of agility:

  • Product Agility – the ability to easily change between various products to address changing needs
  • Process Agility – the ability to easily adapt the internal processes to address changing circumstances
  • Market Agility – the ability to easily enter and change markets
  • Network Agility – the ability to easily coordinate and cooperate with various partners, ranging from suppliers to customers

So what did we learn? Share your thoughts! (I'll share mine in my next blog-entry.)

What is Agility?

I'm involved in an Agile adoption effort that has done of lot of communication as of late. And one of the things we always cover in any intro training is the question What is Agile Software Development?"

Our standard answer and presentation materials usually involves citing the text of the Agile Manifestoand either this definition by Scott Ambler and/or this comment by James Highsmith that "Ultimately, Agility is about embracing change rather than attempting to resist it, [and a] focus on talent and skills of individuals and teams."


I've noticed our responses attempt to describe what Agile Development is by describing what it looks like (Ambler's definition), or the Agile values (from the manifesto), or the emphasis on change and collaboration (Highsmith). What they don't do is describe what Agility is! The Highsmith comment comes closest, but none of them really describe what it means for something to have the ability called agility.

I think it is key to understand what Agility is, and why/how it works, in order to understand Agile software development. I think without that understanding, it can be much harder to understand and apply the principles of lean/agile development, and to "inspect and adapt" to improve the right things in the right direction.

Toward that end, I did a bit of research on a lot of different definitions of agility (beyond some of my previous blog entries like Nutshell definitions of Agile development, Agile development distilled, and Business Agility defined) and of various attributes of agility. And I plan to share them in some blog-entries for each of the following:

• Business Agility
• The Agility Cycle
• Software Agility
• Self-Organization
• Collective Intelligence
• Social Creativity
  
• Swarm Behavior
• Emergence
  
• Collective Ownership
• Change & Uncertainty
• Simplicity, Sufficiency and Sustainability

So "stay tuned" to this blog over the next several days because I intend to post on this more than weekly.

I'm ba-a-ack!

I must apologies to my readers of this blog (or at least whomever is left). I haven't blogged in 9 months. I became heavily immersed in an agile adoption effort that had just managed to succeed in getting the senior-most management fully bought-in to Agile.

I've had a LOT of ideas I've been wanting to blog, many of them queued up and waiting, but I kept having too many other higher-priority things on my plate. I finally have a bit more breathing room now and plan to "catch up". So expect a burst of activity over the next few months.

Summer of Books

I'm going on some long needed (and hard earned) vacation. I won't be blogging again for about one month (so this will likely be my only entry for July).

I've got a lot of REALLY GREAT and interesting books to try and catch up on. I hope to blog about them when I return. Here is what's on my summer reading list:

Theory U: Leading from the Future as it Emerges, by C. Otto Scharmer

I saw the executive summary and other chapters at www.TheoryU.com (also see www.dialogonleadership.org). This looks to be THE penultimate book on leadership for the Agile Organization. It doesn't even use the word "Agile" anywhere, but the values and principles from the book are so well aligned with Agile values and principles, it is positively uncanny.

Advanced Rails Recipes (by Mike Clark), and Deploying Rails Applications (by Ezra Zygmuntowicz, Bruce Tate, and Clinton Begin)

The latest Ruby & Rails books from the Pragmatic Bookshelf.

Implementing SOA: Total Architecture in Practice, by Paul C. Brown

I was extremely impressed with Paul Brown's earlier book on Succeeding with SOA: Realizing Business Value through Total Architecture, and am looking forward to this follow-up work.

Eating the IT Elephant: Moving from Greenfield Development to Brownfield, by Richard Hopkins and Kevin Jenkins

I received a review copy of this. Don't exactly know what to make of it just yet - but it does seem intriguing

Emergent Design: The Evolutionary Nature of Professional Software Development, by Scott L. Bain

This books looks to be an excellent "one stop shop" for learning the theory and practice of Test-Driven Development, Refactoring, Simple Design, and Design Patterns, all as part of a single integrated and coherent method

Agile Adoption Patterns: A Roadmap to Organizational Success, by Amr Elssamadisy

Amr has been writing for the Agile Journal and InfoQ.com and I have been looking forward to seeing this book in hardcopy.

The Software Project Manager's Bridge to Agility, by Michelle Sliger and Stacia Broderick

I think this is going to be THE book for all traditional PMPs trying to make the transition to agile project management.

Changing Software Development: Learning to Become Agile, by Alan Kelly

This books seems to have an interesting take on the connection between Agile, Lean, and Learning Organizations.

I have a few other books too, but they are much shorter :-)
Have a great July everybody!!!

Assigning Code Ownership-Policy Ownership

Jurgen Appelo has an interesting article on StickyMinds entitled "Code Ownership Re-Visited"

Jurgen prefers the term "artifact assignment" rather than "code ownership" and explains there are 4 methods of artifact assignment:

1. Local Artifact Assignment (LAA) delegates policy to subsystems and subsubsystems (etc.)
   
2. Authoritarian Artifact Assignment (AAA) assigns change/access-control access-control of of ALL related artifacts to a single individual "benevolent dictator" who approves/authorizes all changes and who may also assign individual change-tasks to developers
   
3. Collective Artifact Assignment (CAA) assigns the whole team (rather than any one person) as collectively accountable for all its artifacts
4. Individual Artifact Assignment (IAA)

He also provides a nice set of criteria to help decide which policy to use.

This seems very different from the 4 kinds of ownership I described in "Situational Code Ownership: Dynamically Balancing Individual -vs- Collective Ownership" where I define what amounts to Non-Ownership (which can sometimes be the result of Dictatorship), Individual Ownership, Stewardship, and Collective Ownership and show how each maps to a corresponding leadership-style of the Situational Leadership Model.

So what gives? What explains this difference?

• I think Jurgen would probably consider Stewardship as a weak form of Individual ownership (many others would too, though I staunchly disagree for reasons elaborated in the aforementioned article).
• Authoritarian Assignment would be akin to the form of Non-Ownership that results from Dictatorship (or "director-ship" to be more precise) where assignments are made per modification/tasks by a director (or "benevolent dictator")
  
• I would argue that the first two methods Jurgen describes above aren't really artifact-assignment policies, but instead are assignment-ownership policies: They're not so much about making a decision among ownership-policies as they are about making a policy for ownership decisions. Rather than deciding "who should own which artifacts", they decide "who should own the decision" to make such artifact assignments.

In other words, the first two policies Jurgen defines are about decision-rights to assign modification-rights to owners, and not about the modification-rights (or ownership assignments) themselves. As such, it raises an important point taken for granted in my article and in so many other discussions on this topic. Most of the prior discussion probably has assumed that the decision about which ownership-policy to adopt was made either "by the team" or by the team's "leadership" (that might be a manager, a technical-lead, a project-lead, or any combination thereof).

Another common assumption is that such ownership is defined along "architectural" boundaries such as individual artifacts/files, classes/modules, or packages, components and subsystems. Other possibilities are:

• Functional boundaries (e.g., by feature/feature-set, story/theme/epoch, or use-cases)
• Project boundaries (e.g., work-breakdown-structures, tasks, activities)
• Role/discipline specialization boundaries (e.g., requirements, tests, models, database, UI, programming-language, user/help documentation, etc.), and even ...
• Configuration/variation boundaries (e.g., version, variant, branch, platform). In fact some of these stretch across multiple dimensions of a project/product and might even be used in combination.

With Agile development, the emphasis is to break-down communication boundaries and any corresponding separation related to role, or phase, or physical boundaries and to instead prefer customer-determined boundaries of "scope" and "deliverable value" (e.g., stories, features or MMFs, use-cases, etc.).

So you will see a definite (but time-constrained) assigning of things like tasks to owners and stories (though the "owners" sign-up rather than "being assigned"). Those kinds of boundaries encourage closer and more frequent communication rather than separate & isolated (and less frequent) communication.

In the end, with Agile methods, it's all about maximizing learning and knowledge sharing & transfer rather than compartmentalizing knowledge into pigeon-holed roles and responsibilities. One opts for "separation of concerns" without "separation of those concerned" (work is isolated and separated, but not people).

Four Rules for Simple Codelines

Some of you may be aware of Kent Beck's Four Rules of Simple Code that state simple code:

1. Correctly runs (and passes) all the tests
2. Contains no duplication (OnceAndOnlyOnce and The DRY Principle)
   
3. Clearly expresses all the ideas/intentions we needed to express (reveals all intent and intends all it reveals)
4. Minimizes the number of classes and methods (has no superfluous parts)

(I've seen some boil this down into some of the same rules for writing clear prose: correct, consistent, clear, and concise.)

Lately I've been noticing some parallels to the above and rules for what I would call "simple codelines" and I think there may be a similar way of expressing them...

Simple codelines:

1. Correctly build, run (and pass) all the tests
2. Contain no duplicate work/work-products
3. Transparently contain all the changes we needed to make (and none of the ones we didn't)
4. Minimize the number and length of sub-branches and unsynchronized work/changes

To elaborate further...

Correctly build, run (and pass) all the tests

This is of course the most obvious and basic of necessities for any codeline. If the codeline (or the "build") is broken, then integration is basically blocked, and starting new work/changes for the codeline is hindered.

Contains no duplicate work/products

The same work and work-products should be done OnceAndOnlyOnce! Sometimes effort is spent more than once to introduce the same change/functionality. This is sometimes because of miscoordination, or simply lack of realization that what two different developers were working on required each of them doing some of the same things (and perhaps should have been accomplished in smaller chunks).

Other times, rather than modify or refactor a common file, some will simply copy-and-paste the contents of one or more files (or directories/folders) because they don't want to have to worry about reconciling what would otherwise be merges of concurrent changes to the common files.

This is akin to neglecting to refactor at the "physical" level (of files and folders) as opposed to the "logical" level of classes and methods. It adds more complexity and (over time) inconsistency to the set of artifacts and versions that make up the codeline, and also eventually adds to the time it takes to merge, build, and test any integrated changes.

If content is being added to the codeline, we want that content to have to be added only once, without any duplicate or redundant human effort.

Transparently contains all the changes we needed to make (and none of the ones we didn't)

The above is sometimes the cause of much undesirable additional effort that is imposed for the sake of attaining traceability and ensuring process compliance/enforcement. Here, I mean to focus on the ends rather than the means, and I say transparency rather than traceability for that very reason.

If people are working in a task-based and test-driven manner, it should be simple to report what changes have been made since a previous commit and that only intended tasks were worked-on and integrated.

If a codeline is truly simple, then it should be very simple and easy to reveal all the changes that went into it without adding a lot of overhead and constraints to development. It should be easy to tell which changes/tasks have been integrated and what functionality and tests they correspond to. One very simple and basic means of tying checkins (or "commits") to backlog-tasks and their tests can be found here; others are mentioned in this article.

Minimizes the number and length of sub-branches and unsynchronized work/changes

Branching can be a boon when used properly and miserly. It can also add a heck of a lot of complexity and redundancy for maintaining two or more evolving variants of the project. The additional effort to track and merge and build many of the same fixes and enhancements in multiple configurations can be staggering.

Sometimes such branches are useful or even necessary (and can help with what Lean calls nested synchronization and harmonic cadences). But they should be as few and as short-lived as possible, preferably living no longer than the time it takes to complete a fine-grained task or to integrate several fine-grained tasks.

Even when there are no sub-codelines of a branch, there can still be un-integrated (unsynchronized) work-in-progress in the form of long-lived or large-grained tasks with changes that have not yet been checked-in or synced-up with the codeline. Keeping tasks short-lived and fine-grained (e.g., on the order of minutes & hours instead of hours & days) helps ensure the codeline is continuously integrated and synchronized with all the work that is taking place.

Another (possibly less obvious form) of unsynchronized work is when there is a discrepancy between the latest version of code checked-in to the codeline, and the latest version of code that constitutes the "last good build." Developer's lives are "simpler" when the latest version of the codeline (the "tip") is the version they need to use to base new work off of, and to update their existing workspace (a.k.a. "sandbox").

When the latest "good" version of the codeline is not the same (less recent) than the latest version, it can be less obvious to developers which version to use and become less likely that they use/select it correctly. Some use "floating tags" or "floating labels" for this purpose where they "move" the LAST_GOOD_BUILD tag from its previous set of versions to the current set of versions for a newly passed/promoted build. Sometimes the developers always use this "tag" and never use the "tip" (except when they have to merge their changes to the codeline of course).

Even with floating tags however, it is still simpler and more desirable when the last good version IS the latest version. Even if the latest version is known to be "broken", the lag between "latest" and "last good" version of a codeline can be a source of waste and complexity in the effort required to build, verify and promote a version to be "good" (and can introduce more complexity when having to merge to "latest" if your work has only been synchronized with "last good").

Plus, this lag-time often leads many a development shop to separate merging (and integration & test) responsibilities between development and so called integrators/build-meisters, where the best developers can attempt is to sync-up their work with the "last good build" and then "submit" that work to a manually initiated build rather than being directly responsible for ensuring the task is "done done" by being fully integrated and passing all its tests.

Such separation often leads to territorial disputes between roles and build/merge responsibilities. This in turn often leads to adversarial (rather than cooperative and collaborative) relationships and isolated, compartmentalized (rather than shared) knowledge for the execution and success of those responsibilities.

So there we have it! Four rules of simple codelines.

Simple Codelines should:

1. Correctly build, run (and pass) all the tests
2. Contain no duplicate work/work-products
3. Transparently contain all the changes we needed to make (and none of the ones we didn't)
4. Minimize the number and length of sub-branches and unsynchronized work/changes

Sometimes there are legitimate reasons why some of the rules need to be bent, and there are important SCM patterns to know about in order to do it successfully. But any time you do that, it makes your codeline less simple. So you want those scenarios to be few and far between, and to keep striving for the goal of simplicity. (Other SCM patterns, such as Mainline, can help you refactor your codelines/branches to be more simple.)

Traceability Matrix in an Agile Project

InfoQ.com summarized an email-list discussion thread on the subject of using a Traceability Matrix in an Agile Project.

I contributed quite a lot to the thread, and InfoQ apparently included many of the key things I said along with the related URLs to articles I've written. (Thanks guys!)

Iterative and Incremental redefined redux

The agile community has written much about this in the past year or so:

• Iterative vs Incremental - from the first (and original) Wiki Web
• The Neglected Practice of Iteratation - by Jeff Patton
• Difference between Iterative and Incremental Development - nice visual depiction
• No Waterfall Trap -- an even better visual depiction of the difference between iterative and incremental
• Alistair Cockburn (pronounced "Coh-burn") has written several papers on the subject:
  • Using Both Incremental and Iterative Development -- Crosstalk, May 2008
  • Iterative and Incremental Development: How & Why you should be doing BOTH -- Better Software, April 2008
  • Incremental means adding, Iterative means reworking -- Jan 2008 (I dont care for this definition, but the article has lots of good references/resources)
• Iterative and Incremental Development: A Brief History -- by Craig Larman and Victor Basili, IEEE Computer, June 2003

Apologies in advance for being a "stick in the mud" on this one - I'm not particularly happy with the definitions so far. I searched around some more on the WWW and came across one I like a lot that I think better meets our needs.

It is from the paper What is Iterative Development? (part 1), by Ian Spence and Kurt Bittner,

Iterative and Incremental Development:
A style of development that involves the iterative application of a set of activities to evaluate a set of assertions, resolve a set of risks, accomplish a set of development objectives, and incrementally produce and refine an effective solution:

• It is iterative in that it involves the successive refinement of the understanding of the problem, the solution's definition, and the solution's implementation by the repetitive application of the core development activities.²


• It is incremental in that each pass through the iterative cycle grows the understanding of the problem and the capability offered by the solution.


• Several or more applications of the iterative cycle are sequentially arranged to compose a project.

Sadly, development can be iterative without being incremental. For example, the activities can be applied over and over again in an iterative fashion without growing the understanding of the problem or the extent of the solution, in effect leaving the project where it was before the iteration started.

It can also be incremental without being truly iterative. For example, the development of a large solution can be broken up into a number of increments without the repetitive application of the core development activities.

To be truly effective the development must be both iterative and incremental. The need for iterative and incremental development arises out of the need to predictably deliver results in an uncertain world. Since we cannot wish the uncertainty away, we need a technique to master it. Iterative and incremental development provides us with a technique that enables us to master this uncertainty, or at least to systematically bring it sufficiently under control to achieve our desired results.

I like that this definition separated iterative from incremental and then defines them together. I would summarize it as follows (but I like the above better, even if it is longer):

Iterative development is the cyclical process of repeating a set of development activities to progressively elaborate and refine a complete solution. The “unit” of iterative development is an “iteration”, which represents one complete cycle through the set of activities.

Incremental development is the process of developing and integrating the parts of a system in multiple stages, where each stage implements a working, executable subset of the final system. The “unit” of incremental development is an “increment”, which represents the executable subset of the system resulting from a particular stage

Iterative and Incremental development is therefore ...

the application of an iterative development lifecycle to successively develop and refine working, executable subsets (increments) of a solution that evolves incrementally (from iteration to iteration) into the final product.

• Each iteration successively elaborates and refines the understanding of the problem, and of the solution's definition & implementation by learning and adapting to feedback from the previous iterations of the core development lifecycle (analysis, design, implementation & test).
• Each increment successively elaborates and refines the capability offered by the solution in the form of tangible working results that can be demonstrated to stakeholders for evaluation.

An Agile Iteration is a planned, time-boxed interval (typically measured in weeks) whose output is a working result that can be demonstrated to stakeholders:

• Agile Iterations focus the whole team on collaborating and communicating effectively for the purpose of rapidly delivering incremental value to stakeholders in a predictable fashion.
• After each iteration, the resulting feedback and data can be examined, and project scope & priorities can be re-evaluated to adapt the project's overall performance and optimize its return-on-investment

So in addition to the non-agile-specific definitions above, we see that Agile iterations are adaptive, in that they use the previous results and feedback to learn, adjust and recalibrate for the next iteration. And Agile increments are tangible, in that they can be executed and made accessible to stakeholders for demonstration and evaluation.

That's my story and I'm sticking to it!

The Laws of Codeline (Thermo)Dynamics

Some of the discussion with my co-authors on our May 2008 CM Journal article on Agile Release Management spurred some additional thoughts by me that I hope to refine and work into a subsequent article later this year.

Release Management is about so much more than just the code/codeline (and it being "shippable") it's not even funny. Some other articles to reference and mention some key points from are:

• Breaking the Major Release Habit
• Release Management - Making it Lean and Agile

Kevin Lee has written some GREAT stuff on Release Management that relates to Agile. The best is from the first and last chapters of his book on "The Java™ Developer's Guide to Accelerating and Automating the Build Process" but bits of pieces of it can also be found at:

• Agile SCM in the Enterprise
• Software Release Management Best Practices
• Agile Software Delivery
• Architecting the Build Process

ANY discussion about Release Management also needs to acknowledge that there is no single "the codeline", not just because I may have different codelines (Development-Line plus Release-Line) working toward the same product-release, but ESPECIALLY because no matter how Agile you are, the reality is that you will typically need to support MULTIPLE releases at the same time (at the very least the latest released version and the current under development version, but often even Agile projects need to support more than one release in the field)

So, when dealing with multiple release-line, and any "active development lines" for each of those, and the overall mainline, we really should say something overall about how to manage this "big picture" of all these codelines across multiple releases and iterations:

• What is the relationship between development line, release-line and release-prep codeline?
• How do the above three "lines" relate to "mainline"
• What is the relationship between the different release-lines for the different supported releases
• What is the overall relationship between the mainline and the release-lines (and if the mainline is also a release-line, which release is it?)

The above questions and the ability to give some big picture "advice" on relating it all together (the stuff of pattern languages) is precisely where Laura Wingerd's writing on "channeling the flow of change" and her chapter on "How Software Evolves" fits in! It tells us

• The overall Mainline model
• The different types of codelines ("line" patterns), and what kinds of builds take place on each of them
• The relationships of those to the mainline
• When+Why to branch (and from which "types" of codelines)
• When+Why to merge across codelines (as a general rule)

These are where Laura's rules for "the flow of change" apply. And her concept of "change flow" is very much applicable to the Lean/Agile concept of "flow of value". The Tofu scale and "change flow" rules/protocol have to do with order+flow of codeline policies across the entire branching structure when it comes to making decisions about stability -vs- speed. One codeline's policy might make a certain tradeoff, but it is the presence of multiple codelines and how they work together, and how their policies define the overall flow of change across codelines, that forms the "putting it all together" advice that is key to release management across multiple releases+codelines.

In some way's you could make an overall analogy to the Laws or Thermodynamics and the realities of codeline management. Software and codelines tend, over time, to grow more complex and, if unchecked,
"Entropy" (instability) quickly becomes the most dominating force to contend with in their maintenance. See

• "The 8 'Laws' of Software Evolution", and ...
• "The Laws of Software Complexity"

The "entropy" (instability) doesnt just happen within a codeline. It can actually get far more hideous when it happens across codelines via indiscriminate branching from, or merging to, other codelines. This is what happens when you don't respect the principles and rules of "change flow" (from Wingerd) which ultimately stem from the rules of smooth and steady (value-stream) flow from Lean.

The Laws of Thermodynamics are about energy, entropy, and enthalpy. In the case of release management and codelines ...

• energy relates to effort & productivity
• entropy relates to stability/quality versus complexity
• enthalpy relates to "order" (i.e., in the sense of structure and architecture as Christopher Alexander uses the term "order"). It is the "inverse" of entropy.

We could call them them "Laws of Codeline Dynamics" :-)

Energy misspent degrades flow, creates waste, and hurts productivity/velocity. In traditional development, we often see "fixed scope" with resources and schedule having to vary in order to meet the "scope" constraint. IN Agile development we deliberately "flip" that triangle upside down (see the picture in the article at here under the title "The Biggest Change: Scope versus Schedule - Schedule Wins"). So we are fixing "resources" and "schedule" and allowing scope to vary.

This might be one way of viewing the law of conservation of energy. If we fix resources and time (and insist on "sustainable pace" or "40hr work week") then we're basically putting in the same amount of effort over that time-box, but the key difference is how much of that effort results in "giving off energy" in the form of waste ("heat" or "friction") versus how much of that energy directly adds value. Both "Value" and "Enthalpy" degrade or depreciate over time, and adding more energy (effort) doesnt necessarily mean value is increased.

To make sure that energy goes toward adding value (and minimizing waste) we need to focus on the flow of value, and hence the flow of change/efforts to create value (the latter is one reasonable definition of a "codeline" or a "workstream"). to ensure a smooth, steady, and regular/frequent flow, there are certain rules we need to impose and regulate stability within and across codelines to better manage all those releases.

Zeroth Law of Thermodynamics (from Wikipedia)

- If two thermodynamic systems are each in thermal equilibrium with a third, then they are in thermal equilibrium with each other.

Translation to codelines ... this law of "thermal equilibrium" is a law of "codeline equilibrium" of sorts. (Does this mean If two codelines are are "in equlibrium" with a third codeline, then they are "in sync"? and with each other? here "in sync" doesnt mean they have the same frequency, it means their is some synchronization pattern regarding their relative stability and velocity. In Lean Terms, this would refer to "nested synchronization" and "harmonic cadence"). This might imply the "mainline" rule/pattern or one of Wingerd's rules of change-flow.

First Law of Thermodynamics

- In any process, the total energy of the universe remains the same.

This is the statement of conservation of energy for a thermodynamic system. It refers to the two ways that a closed system transfers energy to and from its surroundings - by the process of heating (or cooling) and the process of mechanical work.

This relates to effort & changes expended resulting in the creation of value and/or the creation of waste. We have activities that add value (which we hope is development), activities that preserve value (which is what much of SCM attempts do, given that it doesnt directly create the changes, but tries to ensure that changes happen and are built/integrated with minimal loss of energy/productivity/quality), and then we have activities (or portions of activities) that create waste (and increase entropy rather than preserving or increasing enthalpy/order)

Second Law of Thermodynamics

- In any isolated system that is not in equilibrium, entropy will increase over time

So this is the law of increasing instability/complexity/disorder. The "key" to preventing this from happening is achieving and then maintaining/preserving "equilibrium". How do we achieve such equlibrium? we do it with the "release enabler" patterns for codeline management (which help ensure "nested synchronization" and "harmonic cadence" in addition to achieving a balance or equilibrium between stability and velocity (to smooth out flow).

Third Law of Thermodynamics

- As temperature approaches absolute zero, the entropy of a system
approaches a constant minimum.

In our case, "Temperature" could be regarded as a measure of "energy" or "activity". As the energy/activity of a codeline approaches zero (such as a release in the field that youve been supporting and would LOVE to be able to retire that codeline sometime real soon), it's instability approaches a constant minimum.

This is perhaps another more polite way of saying something we already said in our article on "The Unchangeable Rules of Software Change", namely that "absolute stability" means dead (as in, "no activity"), and should serve as a reminder that our goals is not the prevention of change in order to achieve some ideal "absolute stability", for such an absolute would mean the project not just "done" but "dead".

On the other hand, it also speaks to us as a guideline for when it is safe to retire old codelines, and when to change their policy in accordance with their "energy level"