Scaling Agility: Agile Program Management

Over the past months I've come across a bunch of good links & papers on the topic of "Going Agile" at the program-level:

Michele Sliger (of Rally Software Development) has several good articles and presentations on Relating PMBOK Practices to Agile Practices

• Relating PMBOK Practices to Agile Practices - Part 1 of 4
• Relating PMBOK Practices to Agile Practices - Part 2 of 4
• Relating PMBOK Practices to Agile Practices - Part 3 of 4
• Relating PMBOK Practices to Agile Practices - Part 4 of 4
• A Project Manager's Survival Guide to Going Agile
• The Agile/Waterfall Cooperative

On using Agile methods in organizations with a stage/gate approach to program management, see some of Per Runeson's work in this area:

• Combining Agile Methods with Stage-Gate Project Management
• Integrating Agile Development into Stage-Gate Managed Product Development

Murray Cantor has some good papers on Governance and Variance as it applies to Agility:

• Estimation, Variance and Governance
• Governing the Business Process of Software and Systems Development
• Systems Engineering & Architecting Challenges: Application to Agile Development
  

Some other papers & resources:

• Challenges of Migrating to Agile Methodologies
• Management Challenges to Implementing Agile Processes in Traditional Development Organizations
• Wikipedia page on "Adaptation of Agile methods"
• Agile Project Management Methods for IT Projects (by Glen Alleman)
• Agile Project Management Methods for ERP (by Glen Alleman)
• Agile Program Management: A Critical-Chain Multi-Project Solution (by David Anderson)

Those interested in some advanced agile planning concepts should look at Jeff Sutherland's paper on Scrum II - The Future of Scrum: Parallel Pipelining of Sprints in Complex Projects (and the presentation slides that go with it)

There are several REALLY GOOD whitepapers on Adopting & Scaling Agile at Rally's Agile Knowledge Portal, including the following in particular:

• Five Levels of Planning ... To Agility & Beyond
• Tactical Management of Agile Development: Achieving Competitive Advantage
• A CIO's Playbook for Adopting the SCRUM method of Software Agility

There's gotta be some other good stuff out there and Agile Portfolio, Program and Multi-Project Management! If you know of any - please add a comment and hyperlink or URL!

Scaling Agility: Agile Systems Engineering

Over the past months I've come across a bunch of good links & papers on the topic of "Going Agile" at the program-level for large systems and systems of systems. Some of these relate to Agile program Management and others are more about Agile Systems Engineering (and some relate to both). I'll mention the ones on Agile Systems Engineering in this blog-entry and leave the ones on agile program management for a subsequent entry:

• Barry Boehm's XP2006 Keynote: Product and Process Architectures for Integrating Agile and Plan-Driven Methods
• The Future of Systems Engineering and Software Processes
• A View of 20th & 21st Century Software Processes
• 21st Century Processes for Acquiring 21st Century Software-Intensive Systems of Systems
• Some Future Trends and Implications for Systems and Software Engineering Processes


• Guidelines for Lean Model-Based (System) Architecting and Software Engineering
  
• Agile COCOMO II Cost Estimation
  
• Systems Engineering and Architecting Challenges: Application to Agile Development, by Murray Cantor
• Integrating Agile Methods, and Teaching the Elephant to Dance: Agility Meets Systems of Systems Engineering and Acquisition from the CSE 2005 Annual Research Review


• Agile SYSTEMS ENGINEERING versus AGILE SYSTEMS engineering
  
• Fundamental Principles for Agile Systems Engineering


• Agile PM and Systems Engineering, by Glen Alleman
  
• Large Project Management as a Systems Engineering Discipline, by Glenn Alleman
  
• Lessons Learned Using Agile Methods on Large Defense Contracts
  
• Agile Systems Engineering for Creative Anti-Terror Solutions
  
• Engineering Complex Systems @ MITRE

That's the best I came up with. If you know of other good links on this topic, please send me a comment!

Scaling Agility: Adapting Agile to the Organization

Here is a list of resources I've found that I feel are applicable in figuring out how to scale Agility for a large organization and project. (On the subject of metrics and values, I personally find Sam Guckenheimers work to be of greatest interest):

• A Value-Up Paradigm, by Sam Guckenheimer
• Appropriate Agile Measurement: Using Metrics and Diagnostics to Deliver Business Value
• Scaling Agile Methods (IEEE Software Paper) [also see a related paper, and its presentation slides]

• Scaling Agile Processes (slides), Jutta Eckstein [also see the Intro from Jutta's book Agile Software Development in the Large]
• Supersize me by Scott Ambler
• The Agile System Development Lifecycle, by Scott Ambler
• Scaling Agile Software Development for Larger Projects, by Steven Thomas
• Scaling Down Large Projects to meet the Agile sweet spot, by Phillipe Kruchten (also see corresponding presentation slides)

• Using an Agile Approach in a Large, Traditional Organization, Dot Tudor & George A. Walter, Agile2006
• Agile Can Scale: Inventing and reinventing SCRUM in Five Companies, by Jeff Sutherland
• Agile Software Development Across the Entire Project, by Granville Miller
• System Analysis and Design in a Large-Scale Software Project: The Case of Transition to Agile Development
• Extending Agile Methods
• How BMC is Scaling Agile Development

• Introducing Lean Principles with Agile Practices at a Fortune 500 Company, from Agile2006
• Three-Way Cultural Change Introducing Agile within Two Non-Agile Companies and a Non-Agile Methodology
• The Agile Marathon, by Bob Schatz and Ibrahim Abdelshafi, Agile2006
• Agile at Scale: 7+7 Practices for Enterprise Agility
• 5 C's of Agile Management

• Calculating Earned Business Value for an Agile Project
• Driving Agile Adoption through Audits?
• Seven Agile Team Practices that Scale
• ShopZilla Scales Software Agility

• Lessons Learned using Agile Methods on Large Defense Contracts
• Bridging Agile and Traditional Development Methods: A Project Management Perspective
• Army Simulation Program Balances Agile and Traditional Methods With Success
• A Governance Model for Incremental, Concurrent, or Agile Projects

Additions and corrections are welcome!

Scaling Agility: Seamless Agility across the Enterprise

David Anderson writes about the recent Agile2006 conference in his blog-entry Thoughts for Agile2006:

Scaling Agile. The BIG issue for this year is scaling agile across a whole organization. I see this as having three parts - program or multi-project management and the rollup of schedules and resource plans to a Director or VP level; architecture and enterprise level modeling of a domain and data center; and finally configuration management including build, integration, branch and merge strategies, and work-in-progress batching and related communication.

Ive been dealing with this topic a LOT lately in my own organization as part of efforts to spread amd adapt Agile methods across a large distributed enterprise working with large systems and teams. Ive been researching and collecting lots of resources, including some earlier blog-entries on Agile CMMI and Dancing Elephants and Agile Adoption across the industry.

My perceptions of where the "seams" of the enterprise are that are hardest to introduce Agility into are the close collaboration and alignment required across organizational (lifecycle discipline) boundaries and geographic boundaries (and I find the former to be more difficult to surmount than the latter.)

If I try to categorize them as different areas or aspects that each require the ability to be agile, I come up with something like:

• Process - Adapting Agile to the Organization (making processes responsive to change)


• Product - Agile Systems Engineering/Architecture (making the requirements & architecture be responsive to change)


• Project - Agile Program Management & Governance (making the project be responsive to change)


• People - Distributed Agile Development (collaborating across multiple sites, teams, and timezones)


• Organization - Agile Metrics/Reporting, Governance, and Organizational Design
  


• Environment - Agile CM, deployment, operation/support, etc.

I'll be blogging separately with lists of resources of found for several of the above.

TEA-Time - a metric to elicit TDD behavior?

I've been thinking about a metric that might elicit Test-Driven behaviors in my organization. As a first step to TDD, we definitely want folks to create automated tests as much as feasible and execute them frequently. Once they get that, I've been thinking about what sort of metric might encourage them to actually work in short, test-driven cycles, where requirements are elaborated test-by-test (given a use-case or story, write the first test, write the code to pass the test, refactor, rinse-lather-repeat).

Some of these folks are very much ingrained in a systems engineering V-model lifecycle that does a lot of big-requirements up-front. So ensuring they work to automate tests and execute them frequently isn't enough to enourage them to use an interative approach of fine-grained TDD-style elaboration. An idea I had for what to measure is something I chose to call Test-Execution Availability Time, or TEA-Time (I figure if nothing else, my friends in the U.K will like the name :-).

As proposed, Test-Execution Availability Time (or TEA-Time) would be defined as the mean time between when a system requirement description is first baselined, and the time at which a "sufficient" number of automated tests for the requirement were implemented and ready (available) to be executed.

I was thinking that if a group was measuring this and wanted to behave in a way that minimized TEA-Time, it might encourage them to elaborate requirements in an iterative and incremental fashion, in smaller and smaller "functional slices". One thing I'm not sure of is what "a sufficient number of automated tests" should be in the above.

Any thoughts or comments?

REVIEW: Practices of an Agile Developer

Are you a developer who wants to improve your personal development habits in a way that helps not just yourself, but also incrementally improves your project and your team? If so, then run, don't walk, and get your hands on Practices of an Agile Developer: Working in the Real World by Venkat Subramaniam & Andy Hunt, from the Pragmatic Programmer's Bookshelf.

Want to know more about why? Then read the complete review in the September issue of The Agile Journal (the theme for September is Collaboration and Reuse).

Agile CM on YahooGroups

As mentioned in an earlier blog-entry, I have created an Agile CM Yahoo Group. The description is:

The agile-cm group is for the discussion of ideas relating to Configuration Management for Agile development and of applying Agile concepts, methods and tools to the practice of Configuration Management itself (and CM "Patterns"). This includes aspects of CM that relate to agile development, refactoring & design patterns, agile project-management, Lean, Theory of Constraints (TOC), even SixSigma and CMMI to the extent that they can help CM and related practices to be more "agile".

As I wrote earlier, I hope this new agile-cm group will have a healthy balance of both SCM folks and Agile development folks so we can have some constructive multi-faceted discussions.

How I Blogged my Summer Vacation

Some of you may have wondered what the heck happened to my blog during August and the first half of September. Well - I lost some write-access to my website for a lot of the summer (while my host was recovering from a denial of service attack) and I couldn't login and update it.

I had several blog-entries "queued up" simply needing some final cleaning-up before I made them live, and that got interrupted, then I couldnt access them, then I had some vacations/travel, then I needed to "catch up" when I got back to work, and so on ...

So over the next week or two I'll be "pushing out" the entries I wrote-up in August, using the date they were originally written. I'll also be making available the talk I gave in July at Architecture & Design World 2006 on the subject of "SCM Patterns for Agile Architectures."

In October/November I may not be blogging much at all as I'll be trying to recover from major surgery (I'm giving one of my kidney's to a family member).

Relating SCM Patterns to SCM Principles

Our August Agile SCM column in the CM Journal is about Relating SCM Patterns to SCM Principles. The article is pretty flimsy, barely a skeleton, and that's pretty much entirely my my fault. I meant to write more "meat" about how certain principles are the underlying forces behind several patterns. I wanted to show how the principles are strongly related, much the same way patterns in a pattern language are related. And I wanted to show how that structure shaped the relationships between the patterns as well. Rob did a really good job trying to put together what I had with what he could come up with, but I didn't really give him enough and couldn't easily convey it in a way that made it easy for him to "run with it!"

It's not that I don't see those relationships, I do. And I'm not lacking for words to describe them either. I'm having trouble describing them clearly and concisely. A big part of that is because I still don't like the names of the SCM Principles as I've described them so far. Their names currently relate to the OOD Principles they were derived from. I think that might speak to programmers, but not to folks trying to do version control (even if they also wear "developer" hats). I really want to go back and rework the names of the principles to be more simple and direct.

This is something Ive been working in my mind on and off for alkmost a decade now. To me, it is of profound important - perhaps even the most significant contribution I'll have made to the field of SCM to date. THe interest level doesn't seem to be so high on the scm-patterns list and the cmcrossroads.com forums (but that hasn't deterred me - yet :). I really do think it's not a coincidence that principles of object-oriented design also manifest themselves as fundamental principles of SCM solution design (because I think both are all about architecture - and minimizing and managing dependencies).

I'd really like feedback. On this stuff so if you've had a chance to read thru the June, July and August Agile CM columns, I'm going to create a new YahooGroup about Agile-CM for discussing these and other Agile CM issues. Hopefully this new agile-cm group will have a healthy balance of both SCM folks and Agile development folks so we can have some constructive multi-faceted discussions.

More SCM Books in 2006

Some more SCM books have come on the market in the past 6 months or so. Several of them are worth a look:

• Practical Perforce, by Laura Wingerd
• 2nd edition of Pragmatic Version Control using Subversion, by Mike Mason
• Global Outsourcing with MS Visual Studio 2005 Team System
• IBM Rational Clearcase, Ant, and Cruise Control, by Kevin Lee
• Implementing IBM Rational ClearQuest, hot off the presses by Christian Buckley and Darren Pulsipher

I realize Practical Perforce isn't new - I just can't stop gushing about how great it is! Laura Wingerd really has a gift for elegantly and simply stating important (and sometimes profound) insights about SCM, particularly when it comes to component/container-based SCM and codeline branching and management.

Global Outsourcing with VSTS looks interesting - not really sure if I like it enough to recommend it just yet. It certainly has an attention grabbing title.

The 2nd edition of Pragmatic Version Control using Subversion is just as outstanding as the first edition. It is my first choice for an introducitory book on Subversion and Version Control.

Kevin Lee's book on using ClearCase, ANT and CruiseControl is nothing short of outstanding. If you're a Java-shop using ClearCase and want lots of practical tips and tactics for doing automated builds and accelerating the build process, this book is a MUST HAVE! Kevin's website is also a great resource. Now that IBM/Rational has acquired BuildForge I imagine their will be updates at his website (and eventually a 2nd ed?) about using Clearcase with BuildForge (and maybe some stuff regarding Maven2?). He apparently has some new (online PDF-only) book out called The BuildMeister's Guide from a new series/publisher called Buildmeister books.

Pulsipher and Buckley did a great job in their earlier book The Art of ClearCase Deployment. Now they have a similar book about ClearQuest entititled Implementing IBM Rational ClearQuest: An End-to-End Deployment Guide. So far it looks very good. It's also the only ClearQuest book I know of - so If youre using ClearQuest for anything but a small or informal project you probably should get your hands on this book in a hurry!

Everyone wants to be LUVEd!

I had a posting on Scott Berkun's project management (pmclinic) list that was rather well received. I thought folks here might be interested in it ...

Someone else had written:

> I've found that powerful people respect people who do not back down from them. Make your point forcefully.

I responded with:

That could backfire really badly! That might work with some, for others it will only make things worse. Either way, what they are looking for is respect for their perspective. They want to be LUVed:

• Listened to,
• Understood, and ...
• Valued.

Making any assumptions about what it means to them to be LUVed and what behaviors they feel demonstrate LUV could easily be jumping to conclusions down a very wrong and unconstructive path.

Chances are the "annoyance" is because, somewhere, somehow, the two people involved have different notions of what it means to feel respected and to be treated respectfully and be LUVed. What you regard as disrespectful and disruptive may seem just the opposite to him. He may feel that by giving what you say substantial thought and vocalization on multiple sides to be an act of mindfulness rather than dissension.

I would suggest getting together one-on-one for an open, honest, and candid yet compassionate discussion about what each of you need and what you should/can expect from one another. Dont attribute feelings/motives/intent to him, instead describe his (or her) actual, factual, observed behavior and how it makes you feel, what that makes you think, and its consequences/impact on you and others.

Something is certainly out of alignment between the two people in question, but it may not be objectives or intent, but rather differences in values and belief as to what constitutes respect, progress (versus disruption) and what things are most/more important for overall success.

So in terms of advice, I would add an 'E' to the end of the LUV acronym to make it LUVE: Listen, Understand, Value/Validate, Empathize:

• Listen attentively to what the other person has to say
• Understand it to the best of your ability (you don't have to agree with it, just "seek first to understand")
• Value/Validate what was said. That doesn't mean agreeing with it; It does mean finding the value in it and validating that value to the speaker.
• Empathize with the speaker. Try to identify the feelings they are experiencing and demonstrate your understanding of it, and that it is okay for them to have those feelings.

I wish it was as easy to do habitually as it was to write about. If I could develop the above habit, I would be 10X more effective in both my personal and professional life!

Agile Adoption across the Industry

I've been needing to do some research into the apparent state of Agile adoption & usage across the industry. For those who are interested, here are the URLs I have gathered (some of them are pointers to excerpts or abstracts that you have to register and/or pay to read the entire article - but I found other URLs that summarized the information in them).

Agile Survey Results

• Early Mainstream: Agile Develops in the Enterprise
• Agile Survey Results: Solid Experience And Real Results
• The 2006 "State of Agile Development" Survey Results
• Scott Ambler's Dr Dobbs' Agile Adoption Rate Survey
  (see also a related article on infoq.com)
• Agile Alliance 2006 White Paper on Current Trends
• The Digital Focus Agile 2006 Market Survey Unveiled at Industry Conference
• Agile Software Development Gaining Momentum, Survey Finds
• Are Agile Methods Gaining Ground?
• Facts from a 2005 MethodsandTools.com Survey on Agile Adoption

Other Articles and Reports on Agile Adoption:

• Forrester report: Offshore Outsourcing and Agile Development
• Forrester report: Adopting Agile Development Processes
• Forrester report: Corporate IT Leads the Second Wave of Agile Adoption (document excerpt)
• Gartner Report: Agile Requirements Definition and Management Will Benefit Application Development
• blog-entry about the above report
• Analyst Finds Agile Customers are Happy
• Forrester report: Metrics for Application Development
• Forrester report: Applying Open-Source Process in Corporate Development Organizations
• Other Forrester reports on Agile development
• Agile Delivery at British Telecom, a story of a large Telecom company's experience transitioning to Agile
• Wiki page with info on Agile Adoption

[updated 14 Sept with articles from AgileJournal.com and VersionOne.net]

Relishing Ruby and Rails

I've been trying to keep-up with what's going on in the scripting world. Perl6 still isn't out yet. And Ruby and Groovy seem to be the hottest and most popular & promising.

Groovy is still trying to win support and evolve to a JSR implementation, and achieve a big following. Not clear yet if it's going to succeed or if it has stalled out. The attempted revival of the "Practically Groovy" series at IBM developerWorks suggests maybe it's still alive and kicking.

Ruby is positively on Fire! And Rails is a big part of the reason for that (though there is a similar "Grails" project for Groovy). I've been wading my way through books on Ruby. The best ones were from the PragmaticBookshelf, but now O'Reilly has a some good Ruby books too. So now my Ruby reading list is as follows (I'm about half-way through it):

• Programming Ruby (2nd ed.), by Dave Thomas and Andy Hunt (read this one first)
• Enterprise Integration with Ruby
• Ruby Cookbook
• Agile Web Development with Rails
• Rails Recipes
• Ruby on Rails Up and Running
• Ruby in a Nutshell
• From Java to Ruby: Things Every Manager Should Know

So far, it seems like Ruby and Rails is an amazingly hyperproductive language and fraemwork that takes what many feel are the best of Smalltalk, Python, and Perl, makes it fun and elegant, and makes web development and enterprise integration for simple to glue together production-quality stuff in less than half the usual time.

So right now I would say that Ruby (and Rails) seems to be winning out over Groovy, and some (including Bruce Tate) even think it's beating out Java. I guess we'll see. Right now Ruby (with Rails) appears to lay claim to the title of scripting language with the fastest growing popularity and user-base.

Some Good SQL Books

I've been needing to "brush up" on my SQL and RDBMS skills (let's just say it's been a while :-). I found the following books from O'Reilly and Addison-Wesley to be extremely useful, in order of increasing usefulness:

• SQL Queries for Mere Mortals
  
• Database Design for Mere Mortals
  
• SQL Cookbook
  
• The Art of SQL

The first two above are from Addison-Wesley. I found them to be good for a "quick review" for someone who used to do SQL and RDBMS extensively 10+ years ago but whose knowledge is a bit rusty and outdated.

The second two books above are from O'Reilly. I really like the two O'Reilly SQL books! I refer to them the most right now. The SQL Cookbook updated a lot of my existing knowledge and added to it, and the Art of SQL is fantastic! I just love the way it uses Sun Tzu's "Art of War" to apply to winning the "war" on DBMS and DBM complexity!

SCM as the Architecture of the Evolution of Architecture

One thing I particularly liked about my presentation of "SCM Patterns for Agile Architectures" at the Dr Dobbs Architecture & Design World 2006 conference in downtown Chicago...

I offered up several definitions or perspectives of SCM, including a traditional description, an Agile one, and an architectural one. I liked my architectural definition (I hadn't used that one before - I had used the other two in previous definitions). I said that, from an architectural perspective:

"Configuration Management is the Architecture of the Evolution of your Architecture!"

I rather liked that! I sort of used that to justify the title of the presentation, and presented an IEEE definition of architecture, along with Kruchten's 4+1 views model of software architecture. I then used that to justify my 4+2 model views of SCM/ALM solution architecture. I updated my drawing of the 4+2 model a bit. I think the addition of the Who+When+When+Where+Why+How shows how it may be a possible "bridge" between the Zachman Framework and the 4+1 views model.

Once again, the presentation is here. Feedback is welcome!

SCM Patterns for Agile Architectures

Back in Mid-July I gave a presentation on "SCM Patterns for Agile Architectures" at the Dr Dobbs Architecture & Design World 2006 conference in downtown Chicago.

I gave a 90 minute presentation, and I had 100 powerpoint slides. I had no intention of covering all of the slides. I had the presentation split up into an intro, plus 4 subject areas. In "Agile" fashion, I planned to do "dynamic prioritization for fixed-scope time-boxing" by polling the audience to see which sections were most interesting to them (and which patterns within which sections were of greatest interest). I figured we'd get through the introduction (which has an updated picture of my 4+2 views model of SCM/ALM architecture), and then 2 of the 4 remaining sections (3 at the very most).

I ended up making it through all of the sections and finishing on time - but I couldn't have done it without skimming quickly over the individual patterns/slides that were of least interest for my audience. Overall, I think it went pretty well. I probably wasn't animated or charismatic enough in my presentation style and should vary my tone of voice a bit more. The pictures I used were very cool and very apropos, and very well received (which is good cuz I spent a TON of time finding ones I liked that had copyrights I could use).

For those who are interested, I've made the slides available here. I welcome your feedback in improving them!

The New Rules: Agile beats Big

The July 24 issue of Fortune Magazine has an article entitled The New Rules as the cover story, with the cover saying "Sorry Jack! Welch's Rules for Winning Don't Work Anymore (But We've Got 7 New Ones That Do)"

I think the new rules it discusses are very much about "Agile is better than Bigger!" and "Bigger isn't necessarily better!" The list of new rules is:

Old Rule: Big Dog Owns the Street.
New Rule: Being Agile is Best; Being Big can Bite You!

Old Rule: Be #1 or #2 in Your Market.
New Rule: Find a Niche, Create Something New.

Old Rule: Shareholders Rule.
New Rule: The Customer is King.

Old Rule: Be Lean and Mean.
New Rule: Look Out, Not In.

Old Rule: Rank your Players; Go with the A's.
New Rule: Hire Passionate People.

Old Rule: Hire a Charismatic CEO.
New Rule: Hire a Courageous CEO.

Old Rule: Admire my Might.
New Rule: Admire my Soul.

All in all I thought it was pretty fair-minded. There was even a sidebar to the article that gave Welch a chance to respond to the criticisms. You'll probably need to read the article for further insight into what exactly is meant by each of the "new rules" above. There was plenty of commentary across the industry on the article! (Just Google on "The New Rules" +Fortune +"Sorry Jack" and look through the results)

Agile SCM Principles - From OOD to TBD+CBV+POB

I finally finished a set of articles I'd been working on for almost 10 years on and off on the subject of "translating" principles of OOD into principles of SCM. See the following:

• Principles of Agile Version Control - Part I
• Principles of Agile Version Control - Part II

The principles of OOD translated into principles of Task-Based Development (TBD), Container-based Versioning (CBV), and Project-Oriented Branching (POB).

Here are the principles that I translated. Most of them are from Robert Martin's book Agile Software Development: Principles, Patterns, and Practices, but a couple of them are from The Pragmatic Programmers:

Principles of Class Design
SRP	The Single Responsibility Principle	A class should have one, and only one, reason to change.
OCP	The Open-Closed Principle	A class should be open for extension (of its behavior) but closed against modification (of its contents).
LSP	The Liskov Substitution Principle	Derived classes must be substitutable for (require no more, and ensure no less than) their base classes.
DIP	The Dependency Inversion Principle	Depend on abstract interfaces, not on concrete details.
ISP	The Interface Segregation Principle	Make fine grained interfaces that are client specific.
LOD	The Law Of Demeter (Principle of Least Assumed Knowledge)	Objects and their methods should assume as little as possible about the structure or properties of other objects (including its own subcomponents).
DRY	The "Don't Repeat Yourself" Principle	Every piece of knowledge should have one authoritative, unambiguous representation within the system.
Principles of Package Design
REP	The Release Reuse Equivalency Principle	The granule of reuse is the granule of release.
CCP	The Common Closure Principle	Classes that change together are packaged together.
CRP	The Common Reuse Principle	Classes that are used together are packaged together.
Principles of Package Coupling
ADP	The Acyclic Dependencies Principle	The dependency graph of packages shall contain no cycles.
SDP	The Stable Dependencies Principle	Depend in the direction of stability.
SAP	The Stable Abstractions Principle	Abstractness increases with stability.

Here is what I ended-up translating them into. Note that some of the principles translated into more than one principle for version control because they applied to more than one of changes/workspaces, baselines, and codelines. I'm not real thrilled about the names & acronyms for several of them and am open to alternative names & acronyms:

General Principles of Container-Based Versioning
The Content Encapsulation Principle (CEP)	All version-control knowledge should have a single authoritative, unambiguous representation within the system that is its "container. In all other contexts, the container should be referenced instead of duplicating or referencing its content.
The Container-Based Dependency Principle (CBDP)	Depend upon named containers, not upon their specific contents or context. More specifically, the contents of changes and workspaces should depend upon named configurations/codelines.
The Identification Insulation Principle (IDIP)	A unique name should not identify any parts of its context nor or of its related containers (parent, child or sibling) that are subject to evolutionary change.
The Acyclic Dependencies Principle (ADP)	The dependency graph of changes, configurations, and codelines should have no cycles.
Principles of Task-Based Development
The Single-Threaded Workspace Principle (STWP)	A private workspace should be used for one and only one development change at a time.
The Change Identification Principle (CHIP)	A change should clearly correspond to one, and only one, development task.
The Change Auditability Principle (CHAP)	A change should be made auditably visible within its resulting configuration.
The Change/Task Transaction Principle (CHTP)	The granule of work is the transaction of change.
Principles of Baseline Management
The Baseline Integrity Principle (BLIP)	A baseline's historical integrity must be preserved - it must always accurately correspond to what its content was at the time it was baselined.
The Promotion Leveling Principle (PLP)	Define fine-grained promotion-levels that are consumer/role-specific.
The Integration/Promotion Principle (IPP)	The scope of promotion is the unit of integration & baselining
Principles of Codeline Management
The Serial Commit Principle (SCP)	A codeline, or workspace, should receive changes (commits/updates) to a component from only one source at a time.
The Codeline Flow Principle (CLFP)	A codeline's flow of value must be maintained - it should be open for evolution, but closed against disruption of the progress/collaboration of its users.
The Codeline Integrity Principle (CLIP)	Newly committed versions of a codeline should consistently be no less correct or complete than the previous version of the codeline.
The Collaboration/Flow Integration Principle (CFLIP)	The throughput of collaboration is the cumulative flow of integrated changes.
The Incremental Integration Principle (IIP)	Define frequent integration milestones that are client-valued.
Principles of Branching & Merging
The Codeline Nesting Principle (CLNP)	Child codelines should merge and converge back to (and be shorter-lived than) their base/parent codeline.
The Progressive-Synchronization Principle (PSP)	Synchronizing change should flow in the direction of historical progress (from past to present, or from present to future): more conservative codelines should not sync-up with more progressive codelines; more progressive codelines should sync-up with more conservative codelines.
The Codeline Branching Principle (CLBP)	Create child branches for value-streams that cannot "go with the flow" of the parent.
The Stable Promotion Principle (SPP)	Changes and configurations should be promoted in the direction of increasing stability.
The Stable History Principle (SHIP)	A codeline should be as stable as it is "historical": The less evolved it is (and hence more mature/conservative), the more stable it must be.

You can read the 2nd article to see which version-control principles were derived from which OOD principles. Like I mentioned before, I'm not real thrilled about the names & acronyms for several of them and am open to alternative names & acronyms. So please share your feedback on that (or on any of the principles, and how they were "derived").

Agile SCM Principles - Design Smells

I finally finished a set of articles I'd been working on for almost 10 years on and off on the subject of "translating" principles of OOD into principles of SCM. See the following:

• Principles of Agile Version Control - Part I
• Principles of Agile Version Control - Part II

In an August 2005 blog-entry on SCM Design smells, I tried to translate the design smells that Robert Martin. wrote up in his book on Agile principles, Patterns and Practices. In the first of the two articles above, I think I was much more successful at translating them into the version-control domain. Here is what I came up with ...

Symptoms of Poor Version Control

Rigidity/Inertia:

The software is difficult to integrate and deploy/upgrade because every update impacts, or is impacted by, dependencies upon other parts of the development, integration, or deployment environment

Fragility/Inconsistency:

Builds are easily “broken” because integrating new changes impacts other fixes/features/enhancements that are seemingly unrelated to the change that was just made, or changes keep disappearing/reappearing or are difficult to identify/reproduce

Immobility/Inactivity:

New fixes/features/enhancements take too long to develop because configurations and their updates take a long time to integrate/propagate or build & test.

Viscosity/Friction:

The “friction” of software process against the development flow of client-valued changes is too high because the process has an inappropriate degree of ceremony or control

Needless Complexity/Tedium:

Procedures and tasks are overly onerous and/or error-prone because of too many procedural roles/steps/artifacts, too fine-grained “micro” tracking/status-accounting, overly strict enforcement or rigid and inflexible workflow

Needless Repetition/Redundancy:

The version-control process exhibits excessive branching/merging, workspaces or baselining in order to copy the contents of files, changes and versions to maintain multiple views of the codebase.

Opacity/Incomprehensibility:

It is difficult to understand and disentangle the branching/merging and baselining schemes into a simple and transparent flow of changes for multiple tasks and features developed from multiple collaborating teams working on multiple projects from multiple locations for multiple customers

In my next entry I'll start describing the actual principles and their translations.

Codeline Flow, Availability and Throughput

There has been an interesting discussion on codeline build+commit contention on the XP yahoogroup initiated by Jay Flowers' post about a proposed build contention equation ...

The basic problem is that there have been some commit contention issues where someone is ready to commit their changes, but someone else is already in the process of committing changes and is still building/testing the result to guarantee that they didnt break the codeline. So the issue isnt that they are trying to merge changes to the codeline at the same time, the issue is that there is overlap in the time-window it takes to merge+build+test (the overall integration process for "accepting" a change to the codeline).

Jay is being very agile to the extent that he wants to promote and sustain "flow" on the codeline (see my previous blog-entry on the 5 Cs of Agile SCM Codelines). He is looking at the [average] number of change-packages committed in a day, and taking into account build+test time, as well as some preparation and buffer time. Here the "buffer time" is to help reduce contention. It makes me think of the "buffer" in the Drum-Buffer-Rope strategy of critical-chain project management (CCPM) and theory-of-constraints (TOC).

Several interesting concepts were mentioned that seem to be closely related (and useful):

• The concept of Takt time, from lean production


• Also from Lean production, the notions of process-batches and transfer-batches (especially batch processing-time versus batch transfer-time), along with the concept of single-piece flow


• The concepts and measures of system availability, reliability and survivability for codelines

If we regard a codeline as a production system, its availability to the team is a critical resource. If the codeline is unavailable, it represents a "network outage" and critical block/bottleneck of the flow of value through the system. This relates to the above as follows:

• Throughput of the codeline is the [average] number of change "transactions" per unit of time. In this case we'll use hours or days. So the number of change-tasks committed per day or per hour is the throughput (note that the "value" associated with each change is not part of the equation, just the rate at which changes flow thru the system).


• Process Batch-size is all the changes made for a single change-task to "commit" and ...
• Transfer Batch-size would be the number of change-tasks we allow to be queued-up (submitted) prior to merging+building+testing the result. In this case, Jay is targeting a one change-task per commit (which is basically attempting single-piece flow).


• Processing-time is average duration of a development-task form the time it begins up until it is ready-to-commit. And ...
• Transfer-time is the time it takes to transfer (merge) and then verify (build+test) the result.


• Takt time in this case would regard the development as the "customers" and would be (if I understand it correctly) the [average] number of changes the team can complete during a given day/hour if they didnt have to wait-around for someone else's changes to be committed.


• System outage would occur if the codeline/build is broken. It could also be unavailable for other reasons, like if corresponding network or hardware of version-control tool was "down", but for now let's just assume that outages are due to failure of the codeline to build and/or pass its tests (we can call these "breakages" rather than "outages" :-)


• MTTR (Mean-time to repair) is the average time to fix codeline "breakage," and ...
• MTBF (Mean-time before failure) is the average time between "breakages" of the codeline

Note that if full builds (rather than incremental builds) are used for verifying commits, then build-time is independent of the number of changes. Also note that it might be useful to capture the [average] number of people blocked by a "breakage," as well as the number of people recruited (and total effort expended) to fix it. That will helps us determine the severity (cost) of the breakage, and whether or not we're better off trying to have the whole team try to fix it, or just one person (ostensibly the person who broke it), or somewhere in between (maybe just the set of folks who are blocked).

Anyway, it's an interesting service-model of codeline availability and reliability for optimizing the throughput of codeline changes and maximizing collaborative "flow."

Has anyone ever captured these kinds of measures and calculations before? How did you decide the desired commit-frequency and how did you minimize build+test times? Did you resort to using incremental builds or testing?

I think that giving developers a repeatable way of doing a private development build in their workspace, even if its only incremental building+testing, gives developers a safe way to fail early+fast prior to commiting their changes, while sustaining flow.

I don't particularly care for the practice "build-breaker fixes build-breakage." At the very least I think everybody who is blocked should probably try to help (unless the number of people blocked is more than recommended size of a single team), and I'm sure the person who broke the build probably feels bad enough for causing the blockage (maybe even more so if multiple people help fix it). I think the build-breaker should certainly be a primary contributor to fixing the build and may be most familiar with the offending code, but they may need some help too, as they might not be as familiar with why/how the breakage happened in the first place since it slipped past them (unless of course it was "the stupid stuff" - which I suppose happens quite a bit :-)

So is anyone out their measuring serviceability, availability, reliability of their codelines? Are any of you using these other concepts to help balance the load on the codeline and maximize its throughput? I think that same of the more recent build automation tools (BuildForge, Maven, MS Build + TeamSystem, ParaBuild, etc.) on the market these days could help capture this kind of data fairly inobtrusively (except for maybe MTTR, and the number of people blocked and people+effort needed to effect the repair).