skip to main |
skip to sidebar
Extending the analogy of software architecture views and quality attributes for software process architecture, I'd like to spend some time discussing how software product lines relate to software process architecture and "common processes" across an enterprise.
Many organizations strive for standard common processes, often as part of a CMM/CMMI-based process improvement. All too often I have seen the mantra of "common process" misused and abused to make the practitioners serve the process instead of the other way around.
Processes don't create great software. People and Teams do!
And while the process needs to meet the needs of the business and the needs of the customer, it has to first and foremost serve the needs of the practitioners so that they in turn may better serve the needs of the business to deliver operational business value.
Many in management seem to have the mis-impression that "common process" means "no tailoring" and everyone does everything the same way across products and projects throughout the organization. Process variation across products and projects is regarded as something to eschewed and stamped out, beating the offenders into compliance with top-down dictates and mandates and sanctions. If everyone does everything the same way then the people are more or less "plug-and-play replaceable" and can quickly and easily be reallocated to another project or product with zero learning-curve and associated start-up costs.
This is a dangerous myth that causes irreparable harm to process improvement and common/standard process efforts. Anything that focuses on individuals and interactions as subservient to common processes and standard tools is doomed to fail, and those organizations often end-up with the processes they deserve (along with many disgruntled, frustrated workers).
The purpose of such common processes and tools is not to be a rigid restrictive straightjacket for replaceable people. The intended purpose is to recognize that such people are irreplaceable and to provide a flexible knowledge framework to guide and enable them as the help each other collaborate to learn, grow, and lead in the discovery, practical application and effective execution of practices and improvements that are the best fit for a particular product, product, community and business-environment.
The intended purpose common software processes is quite simply that of process and knowledge reuse! And as such it shares many of the same fundamental problems and solutions as that of software reuse. Indeed it could even be argued that software process reuse is but a special case of software reuse. And current prevailing industry wisdom on the subject suggests that software product-lines show the greatest promise of leveraging software reuse for greatest business value.
In software reuse, we seem to recognize that "one size does not fit all." We acknowledge that even though different products, components, and platforms may share common features, that each one may have different project parameters and environments with different quality attributes and engineering-tradeoffs that need to be "preferred" and optimized that particular application: dynamic versus static, performance versus memory, storage versus latency, throughput versus bandwidth, single versus multi processing, optimistic versus pessimistic concurrency, security versus availability, and on and on.
Software process reuse is no different. Different products and projects have their own uniquely differentiating value proposition (if not there would be no business-need to attempt them in the first place). And those differentiating aspects warrant many kinds of process variation across projects, products, technologies and teams.
Those coming from a SixSigma background may point out how SixSigma strives for reducing process variation. But it's all too easy to forget the context for that is when repeatably reproducing the same output from the same inputs for the same desired set of quality attributes and tradeoffs (not to mention the "kinds" of variation SigSigma is appropriate for trying to eliminate).
So I would advocate the translation and application of software product-line practices to software processes (software "Process-Lines" or "Process Families" if you will) and the treatment of such common processes as first class architectures that need to accommodate the views and perspectives of ALL their critical stakeholders, and which should identify their essential quality attributes and tradeoffs, approaches to managing commonality and variability, and apply appropriate patterns and tactics (such as modifiability tactics for software processes and projects) to meet those objectives.
In light of the above, Lean & Agile software development define an architectural style for such process families and their architecture. Lean and Agile principles identify some of the critical process-quality attributes for such efforts. And the corresponding enterprise and its product offerings and their market segments may identify additional quality attributes that needs to met (such as security, regulatory auditability/compliance, large-scale and/or distributed projects and teams, etc.)
Design for Trustworthy Software is a rather impressive tome that comprises a compendium of the latest and greatest methods, tools and techniques for system and software design applied to software. It covers Design for Six-Sigma (DFSS) techniques, TRIZ, Taguchi Methods, Quality Metrics, Poka Yoke, 5S, QFD, FMEA, and more.
I expect this book to become a standard reference and graduate-level software engineering text. It is so comprehensive and and yet modern/up-to-date in its coverage. It's a very dry read (and I'm not finished with it yet) and looks to be a comprehensive synthesis of what has been working best in industry for the last 25 years that has only recently (in the past 10 years) been getting some visibility in complex systems software of any significant scale. And it shows how to apply them to software.
My surgery went well. I returned from the hospital one week ago and am now able to get around on my own (and do some email :-). My remaining kidney is still learning to do the work of two and will be growing larger in the next couple months (in order to compensate). I'm still dealing with the usual to-be-expected-stuff and hopefully within another couple of weeks I will be mostly back to normal (except that I still need to wait another 2months before trying to lift anything >10 lbs, like my 3yr old & 4 yr old :).
I received a few more books in the mail that look like they will be very helpful to someone like myself trying to introduce/increase Agility in a large corporation that has already committed to the likes of SEI CMMI, ISO-9000 (TL9000 for Telecoms), and Six Sigma.
These books will help someone like me to "speak the lingo" when presenting Lean/Agile principles and practices to those well versed in the above. The Lean Sigma book will (I hope) especially show me how to use the methods and tools of Six Sigma itself to support Lean concepts and techniques.
The July 24 issue of Fortune Magazine has an article entitled The New Rules as the cover story, with the cover saying "Sorry Jack! Welch's Rules for Winning Don't Work Anymore (But We've Got 7 New Ones That Do)"
I think the new rules it discusses are very much about "Agile is better than Bigger!" and "Bigger isn't necessarily better!" The list of new rules is:
Old Rule: Big Dog Owns the Street.
New Rule: Being Agile is Best; Being Big can Bite You!
Old Rule: Be #1 or #2 in Your Market.
New Rule: Find a Niche, Create Something New.
Old Rule: Shareholders Rule.
New Rule: The Customer is King.
Old Rule: Be Lean and Mean.
New Rule: Look Out, Not In.
Old Rule: Rank your Players; Go with the A's.
New Rule: Hire Passionate People.
Old Rule: Hire a Charismatic CEO.
New Rule: Hire a Courageous CEO.
Old Rule: Admire my Might.
New Rule: Admire my Soul.
All in all I thought it was pretty fair-minded. There was even a sidebar to the article that gave Welch a chance to respond to the criticisms. You'll probably need to read the article for further insight into what exactly is meant by each of the "new rules" above. There was plenty of commentary across the industry on the article! (Just Google on "The New Rules" +Fortune +"Sorry Jack" and look through the results)
I had some positive feedback on my previous entry about Six Sigma and Good vs. Bad Variation.
The Six Sigma methodology is largely about eliminating or reducing [destructive] process variation. In the case of software design (and what should be the case in software process design, but all too often is not) we recognize that change and uncertainty is inevitable and we use techniques to minimize and localize the impacts of changes. Three of the most common techniques come straight out of the classic Design patterns book from the "Gang of Four":- Identify what changes and Encapsulate the thing that varies
- Program to an interface, not to an implementation
- Prefer composition over inheritance
These principles could just as easily apply to process design and the design of process-families (a Product-Family or Product-Line for a family of processes). I attempted this in an earlier blog-entry entitled CM to an interface, not to an implementation.
So how do we find this variation, and how to we know what to do with it? Both Lean and TOC give us some tools to do this (as does Six Sigma). Six Sigma's process-maps are similar to (and quite possibly borrowed from) Lean's Value-stream maps. These are one way to form the "current reality tree" of TOC's Thinking Process and then look for things like waste, non-conformance, or conflict (e.g. a "process" impedance mismatch).
When we find something, what do we do? ...
- If it is waste, we attempt to eliminate it or reduce it using Lean.
- If it is variation, we should ask if it is destructive variation (causing poor quality) Is the variation the cause of the problem? Or is it the inherent uncertainty and/or our inability to adapt in the face of change?
- If the variation is destructive, seek to eliminate it. Sometimes automation helps to eliminate variation as long as there is enough certainty in the procedural details and outputs of what to automate.
- If the variation is not destructive, or if the problem is uncertainty and/or our ability to be resilient and adapt to change, then try to isolate and localize the variation by encapsulating it. Use the GoF principles and patterns to separate policy (process interface) from mechanism (procedural implementation).
- If the problem is conflict, then look to see if it is destructive or constructive interference, or impedance mismatch.
- Impedance mismatch needs to be corrected, and destructive interference (friction) needs to be eliminated or minimized.
- Constructive interference, and possibly other forms of conflict may need to be retained, in which case we would again look to encapsulate the conflict by trying to separate policy from mechanism, or interface from implementation, and discovering the higher-level rules/forces that they still have in common.
In all the above cases, TOC's Five Focusing Steps can help with the identification of the appropriate patterns/practices to apply.
Comments? Did that make sense? Did it seem like a reasonable application of using OOD patterns and principles in conjunction with TOC, Lean and Six Sigma?
I briefly touched on Six Sigma (the methdology, not the number/measure) in my previous blog-entry on Cost, Cruft and Constraints, and how Six Sigma methods are about reducing what I called destructive variation:Six Sigma is about eliminating process variation, but not just any kind of process variation. It's about eliminating "destructive" variation. Destructive variation is value-subtracting variation (rather than value-adding).
Some go too far with Six Sigma and interpret it to mean that ALL process variation is "bad." I don't subscribe to that belief. I think there is intentional variation that adds value, as well as "essential" variation that is either unavoidable, or else emerges naturally (and beneficially) out of the creative & learning processes.
Many have tried for repeatability & reproducibility where it isnt always appropriate. The knowledge creating aspects of software development aren't always appropriate places for procedural repeatability and removing variation. The feedback/validating aspects often are. I think Software CM and testing are appropriate places for this: anything that is appropriate and desirable to automate (like building and testing) would be an appropriate for removing variation and SixSigma techniques.
So just like not all conflict is bad, not all variation is bad! And Six Sigma efforts should (I think) limit their focus to destructive variation for the portions of their processes where it make sense to have both repeatable procedures and reproducible results.
I think I'm actually starting to come up with a combination of Agile, Lean, TOC and Six Sigma that is both cohesive and coherent. I credit David Anderson with starting me down that path. (Look out! CMMI might be next :-)
My earlier blog-entry on "Feedback, Flow, and Friction" met with mixed reviews.
Many commented that agile development is ultimately about lowering the cost of change. Others noted feedback is important, but unless you respond to it, and take appropriate action, it's not the be-all and end-all it may seem. A few felt the that "friction" wasnt quite close enough to "contraints."
It seems when it comes right down to it, Agile, Lean and TOC are all about trying to eliminate and/or remove the following:- Cost-of-change: Agile is about reducing the cost of change by making change easy and catching the need to change much earlier thru continous feedback and close, frequent stakeholder interaction.
- Cruft: It (cruft) is waste! And Lean is all about eliminating waste in order to maximize flow.
- Constraints: TOC is all about eliminating constraints, and using the five focusing steps to do it.
While we're at it - Six Sigma is also about eliminating something: variation (but not just any kind of variation). It's about eliminating "destructive" variation. Destructive variation is value-subtracting variation (rather than value-adding).
The phrase "Cost, Cruft and Constraints" doesnt sound as attractive as "Feedback, Flow and Friction." A large part of that may be due to its nonconstructive focus on what to eliminate rather than on what to create.
For each thing I'm allegedly eliminating, I'm gaining something else:
- Reducing the cost-of-change makes it easier to accommodate change and be adaptive/responsive
- Eliminating waste helps maximize flow of the production pipeline.
- Eliminating constraints help maximize throughput
- Eliminating destructive variation helps maximize quality in terms of correctness, reliability, availability, operability, maintainability, etc.
I've been reading bits and pieces about "Lean Six Sigma" for the past couple years. It seems a reasonable mix of Lean Production and the Principles of Lean Thinking with Six Sigma methods and the SEI's description of Six Sigma. Lately it seems to be getting abbreviated to "Lean Sigma"
More recently, I've been hearing about "Design For Six Sigma (DFSS)" and "convergences" between "Lean" and Goldratt's "Theory of Constraints" (TOC), and techniques like the "The 5 Focusing Steps", "Throughput Accounting" and "Drum-Buffer-Rope." (There was a nice ASQ article comparing Lean, Six Sigma, and TOC awhile back.)
So I wanted to be the first to try and coin the phrase "Agile Six Sigma" - except I'm not real fond of the resulting acronym, plus someone else might have come up with it already (if only in passing). So I wanted to embellish it a bit to create an even better acronym before I commence the marketing madness for my new "cash cow" idea. Thus I have decided upon:"Agile Six Sigma - Holistic, Organic, Lean, Emergent."
Seriously tho! I actually think there is a lot of GREAT stuff in and synergies between Agile, Lean, TOC, and Systems Thinking. I think DFSS has some useful tools in its toolbox. I'm less sure of the overall methodology for SixSigma being compatible with Agile methods -- tho I admit David J. Anderson has some GREAT articles that seem to show a connection, particularly the one on Variation in Software Engineering.
I am getting weary of lots of hype that simply throws these buzzwords together (hence my marketing slogan and acronym above :-) but I think they have a lot to offer, and I would be interested in applying them to CM.
I'm particularly curious about using the Lean tools of value-stream mapping along with TOC in analyzing anti-patterns and bottlenecks that often occur in building, baselining and branching & merging (since there seems to be a fairly direct correlation to "code streams" of "change flows" and a "value stream" or "value chain"). Has anyone already done this for CM? (I wonder if something like this could better substantiate the "goodness" of the Mainline pattern.)
