WANTED: Seeking Single Agile Knowledge Development Tool-set

I'll be presenting at Agile2009 in Chicago on the Tools for Agility stage on Tuesday 25 August, 4:45pm-5:30pm.


Here is my session description from the Agile2009 conference website (download PDF here)

WANTED: Seeking Single Agile Knowledge Development Tool-set

Aren’t code, backlog-items, tests, designs & documents all just different forms of system knowledge at different levels of detail? Why can’t the same tools help refactor, browse, search, and provide build/test automation for non-code forms of knowledge without requiring a separate tool/repository for each format? This talk is intended as a challenge to tool vendors/developers to see how this simple treatment of all non-code items as part of a single, unified project knowledge-base can be at once both immensely powerful, and imminently practical, without requiring too much added complexity.

Process/Mechanics

Approximately ~30 minutes of slides/presentation to “make the case” for why this approach is useful and desirable, followed by discussion of challenges (to and from tool developers/vendors, as well as the rest of the audience) as to its usefulness and benefits, and why their current tools can’t easily do so and why the should or should not be easy to enhancement them to do it.

Outline

Software development as knowledge development

• Source-Code as knowledge
• Requirements (Stories) and Tests as knowledge
• Other usable forms of project knowledge (e.g., build scripts & configuration, build/test result reports version control history/comments, online help & other supporting docs)

How would I do this?

• Refactoring Knowledge (thinking about the rest of the “knowledge” the way we think about the code, and its habitability, navigability, and structure/refactoring)
• Applying other agile-practices (like TDD, Continuous Integration, etc.) to non-code knowledge development
• Wiki-based skins, DSLs, and use-cases/design as domain-driven Wiki-word definitions
  • Patterns (giving things names), Retrospectives results and lessons learned
• Viewing, searching EVERYTHING (even the code) via wiki?
• The “Wu-Wei” of Traceability (Tracing without Tracing)
• Versioning and operating on wiki-like entities, just like with code (e.g., making “working copies”, branches and tags)

DISCUSSION & CHALLENGE: Why can’t (or how can) YOUR tools do this!

Begin audience discussion/dialogue: Why can’t a tool like Eclipse or a Wiki-based CMS (such as Confluence) be used as a front-end to browsing/refactoring and navigating ALL the knowledge of the system? (not just code, but stories, tests, build/config data, CI reports, backlog, retrospective lessons).
What makes it easy/hard for these and other tools like any of the following to do this, or to be simple “skins” or plug-ins on top of only a handful of tools instead of a whole kitchen sink full of separate tools and repositories.

• Eclipse, Confluence, Twiki?
• FIT, Fitnesse, Contour
• Doxygen / Javadoc
• Trac / Agile-Trac?
• Jira / Remedy

See also a blog-entry of mine entitled Can I have just one repository please?

Learning outcomes

• Thinking about Agile development as knowledge management & creation
• Current barriers to using agile techniques and supporting tools for non-code artifacts
• What do refactoring, TDD, continuous integration, etc. mean for non-code artifacts (and why you should care)
• Use of wikis for organizing, structuring and refactoring ALL system knowledge
• Why manual traceability is unnecessary (and even comes for free) when using such an approach

Primary target persona

• Tomas Tool-smith/Tool-vendor

Other target personas

• Billy, Business Analyst
• David, Agile Developer
• Patricia, Project Manager
• Tara, Tester

(download PDF here)

Traceability Matrix in an Agile Project

InfoQ.com summarized an email-list discussion thread on the subject of using a Traceability Matrix in an Agile Project.

I contributed quite a lot to the thread, and InfoQ apparently included many of the key things I said along with the related URLs to articles I've written. (Thanks guys!)

Trustworthy Transparency over Tiresome Traceability

If there was an Agile CM Manifesto, then this statement should be part of it!

Trustworthy Transparency over Tiresome Traceability

Note that my position on traceability is more "lean" than "agile" I suspect. I base this on the XP & Scrum centric views that were expressed in the March 2004 YahooGroup discussion thread Why Traceability? Can it be Agile? I think the "tests over traceability" is probably a valid summary of the XP/Scrum perspective from that thread.

I think myself and David Anderson would probably say something more along the lines of "transparency over traceability", where we acknowledge the important goals that traceability is trying to fulfill (I'm not sure the XP community embraces all of those that I identified as the "8 reasons" and "6 facets" that I identified in my paper on traceability dissected). David in particular has written in the past about "trustworthy transparency" and "naked projects" (projects that are so transparent and visible in their status/accounting that they seem "naked").

I also differ strongly with the many vocal opinions expressed in XP community when it comes to the use of tools for tracking requests/changes: I'm strongly in favor of using a "good" tracking tool. I think index cards are a great and valuable "tool" for eliciting dialogue and interaction with the "customer" (and I use them for this purpose, along with post-it notes). But I believe index cards simply do not "cut it" as a serious means of storing, tracking, sorting, searching, slicing & dicing development/change requests).

I do believe an extent of traceability is necessary, and that it's not necessarily "agile", but that it can be, and should be, "lean" and streamlined, and should serve the purpose of transparency, visibility and status-accounting rather than being a goal in itself. And I think there are several strategies and tactics that can be employed to achieve "lean" traceability in service to "trustworthy transparency and friction-free metrics."

I think that a "lean" approach of traceability would focus on the following:

1. Flow: If one uses "single piece flow" and does changes at the granularity that TDD mandates, then software-level requirements, design, coding, and testing are all part of the same task, and tracking them to a single record-id in the change-tracking system and version-control tool would actually go a long ways toward traceability (its much more work & intermediate artifacts when those activities are all separated over time (different lifecycle phases), space (different artifacts) and people (different roles/organizations). When traceability efforts noticeably interfere with "flow" is when agilists will start screaming.


2. Minimizing intermediate artifacts and other perceived forms of "waste" (overspecifying requirements or too much requirements "up front") because fewer artifacts means fewer things to trace.
   


3. Collocating both people & artifacts (the former for communication, the latter for "locality of reference") for those artifacts that are deemed necessary.


4. Coarse-Granularity and Modularity/Factoring of what is traced: tracing at the highest practical level of granularity (e.g., is it practical to trace to the individual requirement or the use-case? To the line of code, or to the method/subroutine, or to the class/module) - this would be about "simple design" and "(re)factoring)" as it applies to the structure of the traced entities and their relationships.


5. Transparent, frictionless automation of the terribly taxing and tiresome tedium of traceability: focus on taking the tedium out of manual traceability and have it streamlined and automated as much as possible, ideally happening seamlessly behind the seems (like with Jane Huang's event-based traceability (EBT), or thru the use of a common environment "event" catcher within Eclipse or MS Team System server), probably using a task-based, test-driven (TDD), or feature-driven (FDD) approach.

Many of these concepts and more are embodied in Sam Guckenheimer's recent book on Software Engineering with Microsoft Visual Studio Team System. I found this book to be surprisingly good (outstanding even), and not at all what I was expecting given the apparent tool/vendor-specific nature suggested by the title. The value-up paradigm and most of the other concepts and values in the book are very well aligned with agility while still meeting the needs of more rigorous ceremony in their software and systems engineering efforts.

I'll close with a description of a recent presentation by David Anderson on Changing the Software Engineering Culture with Trustworthy Transparency:

"Modern software tooling innovation allows the tracking of work performed by engineers and transparent reporting of that work in various formats suitable for everything from day-to-day management and team organization to monthly and quarterly senior executive reporting. Modern work item tracking is coupled to version control systems and aware of analysis, design, coding and testing transitions. This makes it not only transparent but trustworthy. Not only can a tool tell you the health of a project based on the state of completion of every work item, but this information is reliable and trustworthy because it is tightly coupled to the system of software engineering and the artifacts produced by it.

The age of trustworthy transparency in software engineering is upon us. Trustworthy transparency changes the culture in an organization and enables change that unleashes significant gains in productivity and initial quality. However, transparency and managing based on objective study of reality strains existing software engineering culture as all the old rules, obfuscation, economies of truth, wishful thinking and subjective decision making must be cast aside. What can you expect, how will you cope and how can you harness the power of trustworthy transparency in your organization?"

As someone with a strong Unix and Open-Source heritage, I've long regarded Microsoft as "the evil empire" and loathed their operating system and browser and ALM tools. But in the last 3 years or so they've acquired a number of people in the Agile and ALM community that I highly respect (Brian White, Sam Guckenheimer, David Anderson, Ward Cunningham, James Newkirk) and the products these folks have worked on look incredibly impressive to me (even tho not all of them are still with Microsoft), plus I'm quite impressed with the whole of their Software Factories vision and approach ...

I actually may have to start liking them (or at least part of them :-). Don't get me wrong! I'm still a big fan Unix (and Mac OS/X), Open-Source, and more recently Eclipse, ALF and Corona; But the competing stuff from the folks in Redmond is looking exceedingly more and more impressive to me. Working on those kinds of things with those people would be an incredible experience I think (now if only I could do that without having to relocate from Chicago or spend 25% or more of my time traveling ;-).

Extreme Traceability

I gave the following response in a March 24 posting to the extreme programming list on the subject of XP-style traceability:

dsrkkk wrote:

Whatever process we use, we need to track requirements, user stories to design and test cases. The traceability provides obviously many benefits. Why don't we automate traceability? How can we humanly track these complex relationships? If we do manually, we may miss something.

I responded . . .

The less you work in a waterfallish-way, and the more you work in an agile fashion with very short feedback loops, the less necessary it becomes to make additional efforts to track/trace requirements.

If I first do lots of requirements, then lots of design, then lots of code, etc., I have to "put the pieces together." The task I performed to write requirements was different from the one that did the design which was different from the one that did code, tests, etc.

If I work in very short cycles (particularly using test-driven development), then an individual development task looks something like:

• write a single test
• write the code to pass the test and no more
• refactor the code
• commit my changes (Note I might commit my changes before refactoring too).

If, when I commit my changes to the repository, I associate my "commit" with the name or id of the story the test was for, then I have a single, fine-grained change-task that went thru the complete cycle of specification, implementation/design, verification all in a matter of minutes for a single "commit" and user-story.

Working in such fine-grained full-lifecycled tasks automatically associates all the elements that were created & updated as part of the task, which in turn is associated with a user-story. Ta da!!!! Traceability comes along for the ride almost automatically.

From Extreme Locality, in the Feb 2004 "Agile Times", pp.37-40

In Agile methods, we often hear a lot about the importance of simplicity: simple design; do the simplest thing that could possibly work; simple tools; minimal documentation ... Much of the documentation and artifacts created in larger software development methods is for the sake of capturing historical knowledge: the rhyme and reasons behind why something is there, or is designed a certain way.

The desire for such information is often used to justify the need for formal traceability and additional documentation for the sake of maintainability and comprehension. Some very powerful and sophisticated tools exist to do this sort of thing. And yet, there are basic fundamental principles and simple tactics to apply that can eliminate much of this burden.

WHENCE FORMAL TRACEABILITY?
The mandate for formal traceability originated from the days of Department of Defense (DoD) development with very large systems that included both hardware and software, and encompassed many geographically dispersed teams collaborating together on different pieces of the whole system. The systems were often mission critical in that a typical “bug” might very likely result in catastrophic loss of some kind (loss of life, limb, livelihood, national security, or obscenely large sums of money/funding).

At a high level, the purpose of formal traceability was three-fold:

1. Aid project management by improving change Impact Analysis (to help estimate effort/cost, and assess risk)
   
2. Help ensure Product Conformance to requirements specs (i.e. ensure the design covers every requirement, the implementation realizes every design element and every requirement)
   
3. Help ensure Process Compliance (only the authorized individuals worked on the things [requirements, tasks, etc.] they were supposed to do)

On a typical Agile project, there is a single team of typically less than two-dozen. And that team is likely to be working with less than 10 million lines of code (probably less than 1 million). In such situations, many of the aforementioned needs for formal traceability can be satisfactorily ensured without the additional rigor and overhead of full-fledged formal requirements tracing.

Rigorous traceability isn’t always necessary for the typical Agile project, except for the conformance auditing, which some Agile methods accomplish via test-driven design (TDD). A “coach” would be responsible for process conformance via good practices and good “teaming,” but likely would not need to have any kind of formal audit (unless obligated to do so by contract or by market demand or industry standards).

Agile methodologies turn the knob up to 10 on product conformance by being close to the customer, by working on micro-sized changes/increments to ensure that minimal artifacts are produced (and hence with minimal reconciliation) and that communication feedback loops are small and tight. Fewer artifacts, better communication, pebble-sized change-tasks with frequent iterations tame the tiger of complexity!

For more then you ever wanted to know about the why&wherefore behind traceability, see The Trouble with Tracing: Traceability Dissected

TDD/BDD + TBD + IDE = EBT 4 Free?

I've been thinking a bit more about inter-relationships between Test-Driven Development (TDD), Task-Based Development (TBD), a spiffy interactive development environment (IDE) such as Eclipse, and the trouble with traceability ...

One thing that occurs to me that might actually make traceability be easier for agile methods is that some agile methods work in extremely fine-grained functional increments. I'm talking about more than just iterations or features. I mean individually testable behaviors/requirements:

If one is following TDD, or its recent offshoot Behavior-Driven Development (BDD), then one starts developing a feature by taking the smallest possible requirement/behavior that can be tested, writing a test for it, then making the code pass the test, then refactoring, then going on to develop the next testable behavior etc., until the feature is done.

This means, with TDD/BDD, a single engineering task takes a single requirement through the entire lifecycle: specification (writing the test for the behavior), implementation (coding the behavior), verification (passing the test for the behavior), and design.

That doesnt happen with waterfall or V-model development lifecycles. With the waterfall and V models, I do much of the requirements up front. By the time I do design for a particular requirement it might be months later and many tasks and engineers later. Ditto for when the code for the requirement actually gets written.

So traceability for a single requirement thru to specs, design, code, and test seems much harder to establish and maintain if those things are all splintered and fragmented across many disjointed tasks and engineers over many weeks or months.

But if the same engineering task focused on taking just that one single requirement thru its full lifecycle, and if I am doing task-based development in my version control tool, then ...

The change-set that I commit to the repository at the end of my change-task represents all of that work across the entire lifecycle of the realization of just that one requirement, then the ID of that one task or requirement can be associated with the change-set as a result of the commit operation/event taking place.

And voila! Ive automatically taken care of much of the traceability burden for that requirement!

If I had a spiffy IDE that gave me a more seamless development environment integration and event/message passing with my change/task tracking tool, and my version-control tool, and the interface I use to edit code, models, requirements, etc., then it would seem to me that:

• The IDE could easily know what kind of artifact Im working on (requirement, design, code, test


• Operations in the IDE and the version-control tool would be able broadcast "events" that know my current context (my task, my artifact type, my operation) and could automatically create a "traceability link" in the appropriate place.

I realize things like CASE tools and protocols like Sun's ToolTalk and HP's SoftBench tried to do this over a decade ago, but we didnt have agile methods quite so formalized then and werent necessarily working in a TDD/TBD fashion. I think this is what Event-Based Traceability (EBT) is trying to help achieve.

If I had (and/or created) the appropriate Eclipse plug-ins, and were able to develop all my artifacts using just one repository, then if I used TDD/BDD with TBD in this IDE, I might just be able to get EBT for free! (Or at least come pretty darn close)

Wouldn't I?

The Single Configuration Principle

I'm wondering if I tried to bite off too much at once with my Baseline Immutability Principle. Maybe there needed to be another step before that on the way from the Baseline Identification Principle ...

The baseline identification principle said that I need to be able to identify what I have to be able to reproduce. The baseline immutability principle said that the definition of a baselined configuration needs to be timesafe: once baselined, the identified set of elements and versions associated with that baseline must always be the same set of elements and versions, no matter how that baseline evolves in the form of subsequent changes and their resulting configurations.

Maybe somewhere in between the baseline identification principle and the baseline immutability principle should be the single configuration principle:

The Single Configuration Principle would say that a baseline should correspond to one, and only one, configuration.

Of course the baseline itself might be an assembly of other baselined configurations, but then it still corresponds to the one configuration that represents that assembly of configurations. So the same baseline "identification" shouldnt be trying to represent multiple configurations; just one configuration.

What does that mean? It means don't try to make a tag or label serve "double-duty" for more than one configuration. This could have several ramifications:

• maybe it implies that "floating" or "dynamic" configurations, that are merely "references", should have a separate identifier, even when the reference the same configuration as what was just labeled. So maybe the identifiers like "LATEST or "LAST_GOOD_BUILD" should be different from the one that identifies the current latest build-label (e.g., "PROD-BUILD-x.y.z-a.b")


• maybe it might also imply that when we use a single label to capture a combination of component versions, that we really want true "
  composite" labeling support. This would literally let me define "PROD_V1.2" as "Component-One_V1.1" and "Component-Two_V1.0" without requiring the label to explicitly tag all the same elements already tagged by the component labels


• maybe it implies something similar for the notion of a "composite current configuration" or even a "composite codeline" where a product-wide "virtual" codeline could be defined in terms of multiple component codelines

What do you think? Is the single configuration principle a "keeper" or not?

The Baseline Identification Principle

Yesterday (actually just a few hours ago) was my 40th birthday. I had a really nice celebration with my wife and kids at a picnic in the park. I really dont feel like I'm 40. My body thinks I am 50 - at least that how it seems to be acting. My mind still isnt used the the fact that I'm now more than just a little bit older than all those leading men and leading ladies on TV and movies. (Guess I can no longer identify them as part of my historical "baseline" :-)

Back again to describing The Principles of SCM! Last time I described The Baseline Reproducibility Principle. Now we'll take the next logical step and talk about the need to identify baselines.

If the ability to reproduce a baseline is fundamental to SCM, then it stands to reason that the ability to identify a baseline that I must be able to reproduce should also be pretty fundamental. If I have to be able to "show it", then I must first be able to "know it." If I can't uniquely identify a baseline, then it's pretty hard to reproduce it if I'm not sure what I'm trying to reproduce.

So the baseline reproducibility principle gives rise to The Baseline Identification Principle: a baseline must be identified by a unique name that can be used to derive all the constituent elements of the baseline. In other words, we have to have a name, and a way of associating that name with all the object (e.g. files) and their revisions that participate in the baseline.

How do we identify a baseline? By defining a name (or a naming system) to use, and using that name to reference the set of elements that were used to build/create the baselined version of the product.

A "label" or "tag" is one common way that a version control tool allows us to identify the sources of a baseline. This lets us associate a name with a specific set of repository elements and their corresponding revisions. Or it lets us associate a name with an existing configuration or event from which the set of elements and versions may be derived.

Sometimes tagging all the "essential" files and revisions in the repository is sufficient. Sometimes I need more information. I can always take any files or information that werent previously in the version control repository, and put them in the repository:

• I can put additional information in a text file and checkin the file
• I can export a database or binary object into some appropriate format (e.g., XML, or other formatted text)
• some tools let me directly checkin a binary object (e.g, compilers, libraries, images, models) to the repository

If you currently have to label or tag more than just source-code and manually created text-files, then tell me about the other kinds of things you checkin and tag, and what special things you do to ensure they are identified as part of a baseline.

Customer-Oriented Requirements Architecture - The Next Big Thing?

Robert Martin writes about what is "The 'Next Big Thing." From structured programming, to modular programming, object-oriented, extreme, aspect-oriented and service-oriented .... All have been heavily hyped in their "hey day."

Bob notes almost all of those started off as just "programming" and then each "trifurcated", having "design" and later "analysis" come afterward - completing the waterfall "triplet." Robert (Uncle Bob) Martin writes:

“I hope the next big thing is the big thing that we’ve needed for the last thirty years and haven’t had the guts to actually say. I hope the next big thing is professionalism. Perhaps a better word would be Craftsmanship.”

I notice that, within the past year, Aspect-Oriented Programming (AOP) and Aspect-Orientation has now "trifurcated", completing the waterfall triplet: the last few years have seen books and articles on Aspect-Oriented "Design", and within the last 6-8 months we have a book on "Aspect-Oriented Analysis and Design" and another on Aspect-Oriented Use-Cases.

Aspects and analysis/requirements (use-cases) interests me because I wonder if the trend we're seeing isnt so much aspects, but the same trend indicated by Charles Simonyi's emphasis on Intentional Programming (he now calls it "Intentional Software"), and by the emphasis on Domain-Specific Languages (DSLs) in Microsoft's "Software Factories", and to some extent even eXtreme Programming's representation of tests as precise requirements specification, and the way ObjectMentor's own FitNesse framework is almost a DSL-like front-end for that.

I think programming languages and software specifications have evolved beyond the domain of the coder and designer into the domain of the analyst and the customer.

1. Assembly language was just some ever so slight mnemonic sugar (it wasnt even syntactic sugar) on "raw" machine code. We used some short, cryptic but mnemonic names, but it was still making the programmer think in terms of the way the computer's "processing" was organized. We had to think in terms of opcodes and registers and addresses in storage. We had to try and think like the machine.


2. Then we got structured languages. You know, the kind where all that pseudo-code we used to write with words like "if then else", "while" and "for" could now be written directly in the language. We made the programming language try and represent the way the sequential logic seemed to go down in our heads. But it was still coding.


3. Then with abstractions and objects we made programming languages cross the threshhold from mere coding & programming, to design, where we could now express not merely logical processing directly in the language, but design abstractions and encapsulation and various associations and interfaces.


4. Then we built on that with patterns and libraries, and have now adorned it not just with inheritance and templates, but now Aspects and Java's "Annotations"
   

But one of our biggest technical problems in software development is still accurate communication of the customers and users needs and wants to the folks that have to translate that into working software. All those "shall" and "should" statements and "formal methods" like VDM and Z werent as much help as we hoped.

Enter test-driven development, where we let the customers talk to us in their native language, but we try to write the technical requirements not as vague prose, but rather as executable code in the form of tests, so we can get immediate feedback and use short-cycles. Fit and FitNesse attempt some of the goals of intentional-programming by making a little DSL and GUI to let us write something that looks closer to prose but still generates source-code to make our executable tests.

What this also does, along with use-cases, is bring the world of encapsulation and abstraction into the requirements/analysis domain. We can more formally and precisely attempt to package or encapsulate requirements into logical entities more formally manage the dependencies between them.

Use-cases were a start at this, though it was rare to see the various use-case-relationships (extends, specializes, generalizes, uses, etc.) utilized all that much. Rick Lutowski’s little known FREEDOM development methodology also gets very formal about applying encapsulation to requirements – it might become a little more well known now that his book is out as of May 2005 Software Requirements: Encapsulation, Quality and Reuse

With DSLs and Intentional software combined with the ability to encapsulate requirements, we can then start talking about managing their dependencies much the same way we do for code/design today, which means we’ll be able to talk about things like Refactoring of requirements, and “Design Patterns” of Requirements Design/Engineering (and ultimately “requirements reuse”).

And if we ever get even close to that, much of what we call “traceability” today will become a thing of the past. The dependencies in the requirement will be precisely specified and apparent from how we express them in encapsulations and their formal relationships (just like C++/C#/Java code today specifies classes, interfaces, packages, and their interitance, composition, and uses relationships)

So, if it is true that:

• Computers and software are useful, primarily to the extent that they allow us to visualize & virtually manipulate the concepts we can previously only imagine within our minds.
• The evolution of programming languages has gone from trying to make it easier for the programmer to understand the machine’s representational language, to making it easier for the language to represent the thoughts of programmer’s and designer’s
• And with software becoming ever more pervasive and ubiquitous along with the increasing demand for regulatory traceability

Then perhaps Aspects and Sarbanes-Oxley combined with DSLs (and/or Intentional Software) and Test-Driven development will get us from Object-Oriented past Aspect-Oriented or Service-Oriented to arrive at Customer-Oriented Requirements Architecture of encapsulated requirements and the corresponding automated "acceptance" tests.

Then maybe the next big thing for software development may well be Customer-Oriented Requirements Architecture (CORA) and the evolution of expressive environments (I wont call them languages because I don’t think they’ll be strictly textual) that allow the business analyst and customer to express their needs and requirements in terms closer to their own thoughts and vocabulary, and to be able to directly transform that into encapsulated entities from which tests can be automatically generated and executed.

It would basically be creating a simulation environment with rapid feedback for exploring thoughts about what the software might do and analyzing the consequences. And just maybe it would enable the kind of Craftsmanship that Bob Martin's been doing for programming and design, only with the actual requirements!

But will we ever get there? Or will we be too busy chasing the next “Next Big Thing”? Or maybe I just need more sleep! What do you think? Is it too far fetched?

The Trouble with Traceability

I've taken many of my thoughts from my previous blog-entries on traceability and expounded & expanded upon them in my May 2005 "Agile SCM" column of CMCrossroads Journal. The article is entitled The Trouble with Tracing: Traceability Dissected. It describes:

• Ten Commandments of Traceability
• Nine Complaints against Traceability
• Eight Reasons for Traceability
• The Seven Functions of SCM:
  
• The Six Facets of Traceability
  
• The Five Orders of Traceability
• The Four Rings of Stakeholder Visibility
  
• Three Driving Forces for Traceability
  
• Two Overarching Objectives: Transparency & Identification
• One Ultimate Mission: Fostering a Community of Trust
  

On a separate note ... I recently realized that the order/date on which I first author my blog-entries has been very different than the published date of the entry. I was saving them, but not publishing them.

Normally I may take a day or two to "clean-up" an entry (add the URLs, fix the formatting and fight with the WYSIWYG blog-composer) and publish them within a couple days of creating them. But we had a grave illness (and then death) in my family that consumed most of May for us, and I didnt realize until recently that my blog-entries for the last half of April and most of May hadnt been published yet.

So for that I must apologize. And I'll try to do better about publishing more regularly (ideally at least weekly).

Traceable + Transparent = Trustable?

After some more reflection ... I think Ive changed my mind regarding someting I wrote about Traceability and TRUST-ability:

Traceabilityis a means of providing transparency while facilitating impact-analysis, conformance, compliance, accountability, reproducibility and learning. I stopped short of including transparency as one of the goals because I guess Ive seen it fall far short of that too many times. I think that if done well, transparency is an effect of traceability. And at the same time I think Ive seen enough examples of people managing to achieve those other goals without being terribly transparent, that as much as I wanted to include transparency and as an overarching goal of traceability, I just couldn't do it!

I think Joe Farah's response in the "Why Traceability" discussion thread on CMCrossroads was right on target. Transparency is supposed to be a by-product of traceability, if its done "right." It's not a "goal" the same way that facilitating impact-analysis, conformance, compliance, accountability, reproducibility and learning are goals. Achieving transparency in each of those areas is what facilitates those goals:

• Architectural (structural) transparency facilitates impacts analysis
• Functional (behavioral) transparency facilitates product conformance
• Process (procedural) transparency facilitates process compliance
• Project (managerial) transparency facilitates project accountability
• Build/Baseline (physical) transparency facilitates reproducibility
• Decision-making (logical) transparency facilitates organizational learning and root-cause analysis

If my attempt at traceability (including the kind mandated by Sarbanes-Oxley) didnt achieve transparency, then I'll go so far as to say it wasnt done "right" or it wasnt done effectively. Traceability may not imply trustability, but traceability done "right" should achieve transparency, and it is transparency that engenders trust by visibly giving an open, honest, accurate and forthcoming accounting of decision-making and work-efforts.

So how can I do that without creating so much additional manual maintenance and administration as to become insurmountable and/or unwieldy? How do others feel they have achieved this?

The Five Orders of Traceability

Traceability is supposed to help us track and link related pieces of knowledge as we progress thru the software development lifecycle. The software development lifecycle is devoted to creating knowledge that transforms fuzzy functional concepts into tangible working results. It is the process of transforming theoretical concepts into executable reality.

Phil Armour, in his magnificent book The Laws of Software Process, maintains that software is not a "product" in the usual production-oriented sense of the word, but that software is really a medium for capturing executable knowledge. He then uses this to derive that software is therefore not a product-producing activity but rather a knowledge creating and knowledge acquiring activity.

Armour goes on to describe "The Five Orders of Ignorance" and how, if software development is a process of knowledge-acquisition and knowledge-creation, then it is also ultimately a process of "ignorance reduction" whereby we progressively reduce our ignorance of what the system is, what it needs to do, how it needs to do it, and how we need to do it and manage it.

Perhaps it should follow then, as a corollary to all of this, that the mission of traceability is to connect all the dots between these various pieces and "orders" of knowledge to help those of us auditing or reviewing them to learn the lessons of how and why that knowledge came to be created in its current form. To that end, I hereby propose The Five Orders of Traceability:

• 0^th Order Traceability – Existence: Tracking Knowledge Content
• 1^st Order Traceability – Structure: Tracking Knowledge Composition
• 2^nd Order Traceability – History: Tracking Knowledge Context
• 3^rd Order Traceability – Transformation: Tracking Knowledge Creation (causal connection in the knowledge creation/derivation process)
• 4^th Order Traceability – Meta-Traceability: Tracking Knowledge of the Five Orders of Traceability.

It's not entirely clear to me whether I have the 2nd and 3rd items mixed-up in the above (perhaps structure should come after context). When deriving the ordering and identification of the above, I basically took my cue from Armour's 5 orders of ignorance, and made the assumption that it is the intent of a particular "order" of traceability to eliminate or address the corresponding "order" of ignorance! Hence 3rd order traceability should help resolve 3rd order ignorance, 2nd order traceability should help resolved 2nd order ignorance, and so on.

With that in mind, I'll elaborate further upon each of the five orders of traceability Ive "coined" above ...

0th Order Traceability is merely the existence of knowledge content. There are no additional linkages or associations to navigate through that content. There is no explicit traceability - the content is simply there.

1st Order Traceability is an attempt to structurally organize the knowledge content and provide links that navigate the decomposition from one level of organization/detail to another. This would be like an outline structure and cross-referencing/indexing capabilities. A number of tools give us a means of organizing a particular portion of system knowledge:

• Basic requirements document management tools provide way of organizing and viewing the requirements, and even design documentation for a project
• Modeling tools provide a way of organizing and viewing problem-domain and solution-domain abstractions
• Many interactive development environments give a logical (e.g., object-based) view of the code and/or a physical (e.g., file-based) view of the file+folder structure of the codebase.
• Many of these tools even provide a way to link from a section of a document to a model entity (e.g., a UML object or package) and/or a code-construct (e.g., a class, method, or module)

2nd Order Traceability goes beyond mere content and structure to provide contextual awareness. Not only are there links, but there is also contextual information (typically in the form of metadata) giving a clue as to the events that transpired to create it: who authored the content, when they did it, where they did it (physically or virtually).

This type of historical/log information assists in auditing and recovery, and is typically most-suited for automatic capture/recording by the application used to create/modify and store the information (perhaps using a mechanism like dynamic event-based traceability). The information might also include application integration data (for example, recording the identifier of a requirement or a change-request and associating it with the corresponding portion of the design, code, or tests when it is created or updated).

3rd Order Traceability is the "nirvana" of traceability for many. Not only do we have structure and context, but we have additional associations and attributes (e.g., metalinks between [meta]data) that capture the causal connection between related pieces of knowledge in the value chain. Some call this "rich traceability" and there are some high-end requirements management tools capable of doing this.

Still, tracing all the way through to design models, and code remains very effort-intensive unless all knowledge artifacts are captured in the same repository (requirements, models, code, tests, project-activities, change-requests) where these advanced "rich tracing" capabilities exist.[MKS claims to have taken a step toward achieving this with its new RM tool that tracks requirements in the same repository as the source-code version control system].

With 3rd order traceability, we are effectively capturing important decisions, criteria, constraints, and rationale at the various points in the knowledge creation lifecycle where one form of knowledge (e.g., prose, model, code) is being either transformed into another form of knowledge, or is being elaborated to another level of decomposition within the same form of knowledge.

4th Order Traceability is meta-traceability, or tracking of knowledge about the five orders of traceability within or across systems. (Sorry - I couldn't come up with anything better that is analgous to Armour's 5th order of ignorance - a.k.a. meta-ignorance. If you have a different idea of what 4th order traceability should be, feel free to comment.)

What about Ontology or Epistemology? I don't honestly know. I would imagine there must be some way of tying together the above with terms and concepts from "knowledge management," such us transforming tacit knowledge to explicit knowledge, and maybe even relating XML schemas and ontologies back to all of this. I leave that as an undertaking for someone much more versed than I in those domains.

Traceability and TRUST-ability

Traceability is one of those words that evokes a strong "gag" reflex among many hardcore agilists. They are all in favor of tracing tests to features, which is extremely straightforward when one is doing test-driven development (TDD). When it comes to tracing requirements thru to design and code, images of manually maintained traceability matrices that are hopelessly effort-intensive and never quite up-to-date seem to spring to mind.

So what are the main goals that traceability supposedly serves? Based on sources like CMMI and SWEBOK, and several others, I think the goals of traceability are to assist or enable the following:

1. change impact-analysis: assess the impact and risk of a proposed change to facilitate communication, coordination and estimation [show the "know-how" to "know-what" to change, "know-where" to change it, and "know-who" will change it]


2. product conformance: assure that necessary and sufficient requirements were implemented, and ensure the implementation of each requirement was verified/validated [prove we "did the right things" and "did the things right"]


3. process compliance: assure that the necessary procedural activities (e.g., reviews and tests) were executed for each feature/requirement/code-change and ensure they were executed satisfactorily [prove we "walk the walk" and not just "talk the talk"]


4. project accountability: assure that each change of each artifact was authorized and ensure that they correspond to requested functionality/business-value [safeguard against "gold plating"]


5. baseline reproducibility: assure that the elements necessary to reproduce each baseline have been captured and ensure that the baselines can be reproduced [so "all the king's horses and all the king's men" can put your fallen "humpty-dumpty" build back together again]


6. organizational learning: assure that the elements necessary to rediscover the knowledge of the system have been captured and ensure that the rationale behind critical decisions can be reproduced -- e.g., for root-cause analysis, or to transfer system knowledge to a deployment/support team. ["know-why" you did what you did when you did it]

Many would argue that these all boil down to questions of trust, and communication:

• Do I trust the analysts/architects to do what I said and say what they did?


• Do I trust the product or its producers to correctly realize the correct requirements?


• Do I trust the process or the engineers following it?


• Do I trust the project or the people managing it?


• Do I trust the environment in which it is built and tested?


• Do I trust the organization to be able to remember what they learned and learn from their mistakes?

Whether or not traceability efforts successfully achieve any of these goals is another matter entirely. Often, traceability efforts achieve at most one of these (conformance), and sometimes not even that. Many question whether the amount of effort required actually adds more value than what it subtracts in terms of effort (particularly when traceability efforts themselves may create additional artifacts and activities that require tracing).

Traceability is often desired because it's presumed to somehow alleviate fears and give us "TRUSTability." I suspect that's really an illusion. Traceability is useful to the extent that it facilitates more effective communication & interaction between individuals, teams, and stakeholders. And traceability can be useful to the extent that its helps us successfully convey, codify and validate system knowledge.

So to the extent that traceability helps us more quickly identify the right people to interact with and the right information to initiate that interaction, it can be extremely useful. It is the people and the quality of their interactions that provide the desired "trustability." Beyond that, unless unintrusively automated, it can quickly turn into "busy work" with very high "friction" on development of new functionality realized as "working software".

To that end, I have a few slides in a presentation I gave on Agile Configuration Management Environments (see slides 24-25) that talk about "Lean Traceability", and a big part of that is using encapsulation, modularity, and dependency management (a.k.a refactoring) to track at the coarse-grained level rather than a fine-grained one. This alone can reduce the traceability burden by 1-2 orders of magnitude. A more detailed discussion is near the end of the paper The Agile Difference for SCM.

The LoRD Principle is also mentioned (LoRD := Locality of Reference Documentation). You can read more about LoRD on the Wiki-Web, in Volume 3 of the Agile Times (pp. 37-40), and in Scott Ambler's essay Single Source Information. For automating traceability, there is some interesting work from the DePaul University Applied Requirements Engineering Lab and the SABRE Project (Software Architecture-Based Requirements Engineering) on the subject of dynamic event-based traceability.

Building Trust with Transparency

I was catching up on some of my trade journals last weekend and I came across a coincidental confluence of several seemingly divergent streams of thought, all revolving around this recurring theme of building trust ...

First, The January 2005 issue of Software Development, with the featured them "RFID Everywhere: a primer for the new-age of self-tracking products" plastered on the cover. Then the February 2005 CACM had an article entitled "Trust in E-Commerce". Lastly, the February 2005 issue of Software Development had two fantastic articles, one by Kirk Knoernschild entitled "Benefits of the Build" and an interview profiling David Anderson about his new position "Managing at Microsoft."

The CACM article on e-commerce trust was about building trust between the vendor and the consumer. But it got me thinking about building trust between the agile development "vendor" (e.g., an agile team in a large organization) and the quite probably non-agile "consumer" with whom they need to develop a trustworthy working relationship.

Then Kirk's article on the benefits of the build talked about one such benefit being the Regular Frequency Indicating Development status/health (which of course made me think of RFID). A frequent build usually happens at regular intervals that many refer to as the rhythm, pulse, heartbeat, or cadence of the team's development progress and health. When the results of the build are visibly reported in a public location/webpage, it's like a "blinking beacon" that the team and its stakeholders can monitor.

Such build status reports are a part of the part of CM more formally known as configuration status accounting. Basically, status accounting lets us account for the status of any change-request, development-task, build, iteration, or release, at any time (sounds even more like RFID doesnt it).

Then I read the interview with David Anderson. I know David. I had occasion to meet him in July of last year when he gave a presentation about FDD and its relationship/history from Peter Coad and "Color Modeling" Pattern to the Chicago Agile Developers group (ChAD), and then a few days later he presented a Business-Case for Agile Management to the "Agile track" at the 2004 Motorola Engineering Symposium. [I was the "track chair" that arranged to have David speak both to ChAD and to Motorola, and I also had the chance to chat with him at length. I'm unendingly impressed by David and am in awe of his mindfulness, vast knowledge, keen insight, and systems-thinking abilities when it comes to the union of agility, lean, theory of constraints, software development and management theory.]

Anyways, near the end of his interview with SDMagazine, David says the following little gem about how Transparency Enhances Trust when he was asked how a manager can introduce and encourage transparency in a culture that is opaque and compartmentalized:

Economists talk about measuring the level of trust in a society. It's been proven that societies where there is greater trust experience faster economic growth—"My word is my bond." The same is true in software. To have trust, you must not be afraid that someone is hiding something from you. Transparency enhances trust.

However, to have transparency, you must drive out fear. Many people believe that data hiding is required because figures can be misinterpreted. Hence, they hide information from people whom they believe will draw the wrong conclusions ... I prefer to educate people to draw the right conclusions ... Information sharing (or transparency) is an enabler to team power. Team software development is about collaboration ....

And there you have it: transparency engenders trust! When we do regular builds and other regular development activities and publish build-status reports and burndown-charts and cumulative flow diagrams in a visible fashion, we are giving agile stakeholders a form of RFID for each task, feature, build, iteration and release.

This type of frequent and regular feedback gives transparency of an agile development team's activities to groups like CM, V&V/QA, Systems Engineering, and Program Management. That transparency helps establish trust, which in turn enhances cooperation, and ultimately enables collaboration.