Rise of the Development Environment Architect

Peter Eeles and I must be subconsciously on the same page. Because at the same time I was blogging about Software Architecture Views and Perspectives and Software Architecture Quality Attributes and their direct applicability to SCM/ALM solution architecture (and software process in general), Peter was working on an article for IBM developerworks entitled The Rise of the Development Environment Architect:

[Development] environments present challenges; and, interestingly, these challenges are similar to those of the systems they support. For example, development environments have to deliver against the required functionality and properties (such as performance and usability), often have to coexist with legacy systems (such as, in the case of a development environment, existing methods and tools), and have to acknowledge other constraints (such as the distributed nature of development teams, and existing skills and infrastructure).

All in all, creating a well-oiled development environment that accelerates, rather than hinders, project performance is a science unto itself. This is why IBM® Rational® has spent many years specifically developing a services capability that understands the challenges faced by organizations that 1) want to improve developer productivity, and 2) regard their development organization as a strategic differentiator, rather than simply a cost center.

Our experience has led the Rational team to define a role within the software development lifecycle called the "development environment architect." In October 2007, one hundred of Rational's most experienced development environment architects from across the globe gathered together in the first conference dedicated to this role to share their experiences. This article is a result of that conference and the discussions that took place.

As you read the concepts presented here, you may well question whether the development environment architect should be a role itself, or whether the individual or team who normally functions in the software or systems architect role should simply add consideration of the development environment to their list of architectural concerns. I believe that both propositions are valid. Furthermore, whenever the role of the "architect" is discussed, it is always qualified with the domain under consideration; thus we speak of a "building architect," "software architect," "systems architect," "enterprise architect," etc. The development environment is simply one of these domains, and one that is not traditionally a concern for the "software architect" role. I therefore believe that the "development environment architect" role is one that hasn't been emphasized before -- hence this article.

This article has several audiences and objectives. It is relevant to organizations undertaking an improvement to their development environment and who need to understand the value of a development environment architect to help guide their initiative. It is also relevant to those who are responsible for the technical content of the development environment -- i.e., development environment architects -- because this article introduces this responsibility as a role not previously defined. Finally, this article may supplement material contained within a development environment, in helping communicate its content, the role of its architect, and the benefits of having such a role in place.

Read the full article here. I may blog later about the similarities and differences between the sort of architecture that Eeles describes versus my 4+2 Views Model of SCM/ALM Solution Architecture

BOOK: Outside-in Software Development

My review of Outside-In Software Development is in this month's edition of The Agile Journal.

Kessler and Sweitzer's Outside-in Software Development should resonate deeply with all those who genuinely value the principle of customer collaboration in the Agile Manifesto, and with anyone who has played the role of Product Manager for a software project. This 2008 Jolt award Finalist is not a book about eliciting or prioritizing requirements (or "user stories") for an Agile project. This book goes beyond mere user-stories and their ranking or velocity to focus on uncovering the underlying needs and goals of your stakeholders and understanding what truly adds value for the customer and the business.

... I think Outside-in Software Development is a profoundly important book for anyone in the Agile or Lean "camps" because it addresses and embraces the often neglected pieces of the customer-relationship puzzle that emerge from the stakeholders' perspective, often after the software is released. It shows us how many of those same Lean and Agile values of collaboration, responsiveness, waste-elimination, and respect for people can be successfully applied to the users' experience with our software, and to the stakeholders' experience with ourselves in the service of realizing the very business value we strive to deliver.

Read the full review.

BOOK: Programming Groovy and Groovy Recipes

I just received an advance copy of Programming Groovy from the Pragmatic Programmer's Bookshelf. This complements their work that came out last month on Groovy Recipes.

From the Programming Groovy book webpage:

Groovy brings you the best of both worlds: a flexible, highly productive, agile, dynamic language that runs on the rich framework of the Java Platform. Groovy preserves the Java semantics and extends the JDK to give you true dynamic language capabilities⎯programming in Groovy feels like you’re using an augmented Java. Programming Groovy will help you learn and take advantage of the latest version of this rich dynamic language, so you can be a more productive Java Platform developer.

From the Groovy Recipes book webpage:

If you’re a busy Java professional who needs quick solutions to everyday problems, then Groovy Recipes is for you. The Groovy language and Grails web framework give you seamless integration with your legacy Java code while adding the flexibility and dynamism of a scripting language and giving you modern, agile, time-saving techniques. Groovy allows you to write code the way you always thought you should—you’ll never look at Java the same way again.

For those who like Ruby and Rails and the ability to access other Java frameworks and APIs, but also really want their Java-like syntax (and hence more than just JRuby), these are the books to read. Groovy even has its own answer to Rails called "Grails"

See also:

• An Introduction to Groovy and Grails
• Groovy with Grails – Java’s fight back to Ruby on Rails
• Groovy Zone | Everything for the Groovy & Grails developer
• AboutGroovy.com
• Mastering Grails : Read more in this series to gain a further understanding of Grails and all you can do with it.
• Groovy homepage
  
• Grails homepage
  
• Grails Framework Reference Documentation: The Grails bible.
  
• Practically Groovy : This developerWorks series is dedicated to exploring the practical uses of Groovy and teaching you when and how to apply them successfully.
  
• Grails Plugins: Documentation for all Grails plug-ins.

Software Process-Line Architecture and Common Processes

Extending the analogy of software architecture views and quality attributes for software process architecture, I'd like to spend some time discussing how software product lines relate to software process architecture and "common processes" across an enterprise.

Many organizations strive for standard common processes, often as part of a CMM/CMMI-based process improvement. All too often I have seen the mantra of "common process" misused and abused to make the practitioners serve the process instead of the other way around.

Processes don't create great software. People and Teams do!

And while the process needs to meet the needs of the business and the needs of the customer, it has to first and foremost serve the needs of the practitioners so that they in turn may better serve the needs of the business to deliver operational business value.

Many in management seem to have the mis-impression that "common process" means "no tailoring" and everyone does everything the same way across products and projects throughout the organization. Process variation across products and projects is regarded as something to eschewed and stamped out, beating the offenders into compliance with top-down dictates and mandates and sanctions. If everyone does everything the same way then the people are more or less "plug-and-play replaceable" and can quickly and easily be reallocated to another project or product with zero learning-curve and associated start-up costs.

This is a dangerous myth that causes irreparable harm to process improvement and common/standard process efforts. Anything that focuses on individuals and interactions as subservient to common processes and standard tools is doomed to fail, and those organizations often end-up with the processes they deserve (along with many disgruntled, frustrated workers).

The purpose of such common processes and tools is not to be a rigid restrictive straightjacket for replaceable people. The intended purpose is to recognize that such people are irreplaceable and to provide a flexible knowledge framework to guide and enable them as the help each other collaborate to learn, grow, and lead in the discovery, practical application and effective execution of practices and improvements that are the best fit for a particular product, product, community and business-environment.

The intended purpose common software processes is quite simply that of process and knowledge reuse! And as such it shares many of the same fundamental problems and solutions as that of software reuse. Indeed it could even be argued that software process reuse is but a special case of software reuse. And current prevailing industry wisdom on the subject suggests that software product-lines show the greatest promise of leveraging software reuse for greatest business value.

In software reuse, we seem to recognize that "one size does not fit all." We acknowledge that even though different products, components, and platforms may share common features, that each one may have different project parameters and environments with different quality attributes and engineering-tradeoffs that need to be "preferred" and optimized that particular application: dynamic versus static, performance versus memory, storage versus latency, throughput versus bandwidth, single versus multi processing, optimistic versus pessimistic concurrency, security versus availability, and on and on.

Software process reuse is no different. Different products and projects have their own uniquely differentiating value proposition (if not there would be no business-need to attempt them in the first place). And those differentiating aspects warrant many kinds of process variation across projects, products, technologies and teams.

Those coming from a SixSigma background may point out how SixSigma strives for reducing process variation. But it's all too easy to forget the context for that is when repeatably reproducing the same output from the same inputs for the same desired set of quality attributes and tradeoffs (not to mention the "kinds" of variation SigSigma is appropriate for trying to eliminate).

So I would advocate the translation and application of software product-line practices to software processes (software "Process-Lines" or "Process Families" if you will) and the treatment of such common processes as first class architectures that need to accommodate the views and perspectives of ALL their critical stakeholders, and which should identify their essential quality attributes and tradeoffs, approaches to managing commonality and variability, and apply appropriate patterns and tactics (such as modifiability tactics for software processes and projects) to meet those objectives.

In light of the above, Lean & Agile software development define an architectural style for such process families and their architecture. Lean and Agile principles identify some of the critical process-quality attributes for such efforts. And the corresponding enterprise and its product offerings and their market segments may identify additional quality attributes that needs to met (such as security, regulatory auditability/compliance, large-scale and/or distributed projects and teams, etc.)

Commonality and Variability Management

Continuing the previous discussion on software product-lines ...

Central to the notion of product-lines and product-families are tracking and managing three different kinds of software assets:

• common/core assets that are shared by all the products in the product-line
• shared assets that are common to some products but not others, and ...
  
• product-specific assets (or custom-components) that are specific to a single product in the product-line.

Architecture for such product-lines is all about managing commonality and variability, and easing their evolution to achieve a diverse family of products to achieve economies of scale from reusing common assets. Change/Configuration Management for SPLs is a very challenging problem. And variability management techniques often come down to a matter of binding-times. There are also more advanced strategies (some involving mathematical models).

A few resources on the subject of Commonality and Variability are as follows:

• A good (albeit old) article on Variability Management in Software Product Lines has a nice overview of some basic variability enabling techniques and the use of binding time
• A dissertations on Variation Mechanisms and Multi-view Architecting in Platform-based Product Family Development
• Expressing Product Diversification -- Categorizing And Classifying Variability In Software Product Family Engineering
• Coplien, Weiss and Hoffman wrote a nice overview article on Commonality and Variability in Software Engineering
  
• Alan Shalloway has an excellent introductory article on Commonality and Variability Analysis in conjunction with Design Patterns.
  

In July 2006 I presented at the Dr Dobbs' Architecture & Design World conference about SCM Patterns for Agile Architectures, which included a section on managing variations. I summarized that portion of the presentation as follows:

Use Late-Binding instead of Branching:
• Build/Package Options
• Feature Configuration/Selection
• Business Rules

Think about which of the following needs to "vary" and what needs to stay the same:
• Interface vs. Implementation vs. Integration
• Container vs. Content vs. Context

Commonality & Variability analysis helps identify the core dimensions of variation for your project

Use a combination of strategies based on the different types of needed variation and the "dimension" in which each one operates

Software Product-Line Architecture and Product-Families

Extending the analogy of software architecture views and quality attributes for software process architecture, I'd like to spend some time discussing software product lines. According to the SEI website on software product-lines, A Software Product-Line is defines as follows:

A software product line (SPL) is a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way.

At SoftwareProductsLines.com, Charles Krueger defines them as follows:

Software product lines refers to engineering techniques for creating a portfolio of similar software systems from a shared set of software assets using a common means of production.

The key objectives of software product lines are: to capitalize on commonality and manage variation in order to reduce the time, effort, cost and complexity of creating and maintaining a product line of similar software systems.

• Capitalize on commonality through consolidation and sharing within the software asset inputs, thereby avoiding duplication and divergence.
• Manage variation by clearly defining the variation points and decision model, thereby making the location, rationale, and dependencies for variation explicit.

Closely related to software product lines is the notion of software product families and Product Family Engineering. In many cases the terms product-line and product-family are used interchangeably. Sometimes a product-family is slightly more general in that a product-family may comprise one or more product-lines. The SEI has established a Framework for Software Product-Line Practices that encompasses topics such as architecture, organization, patterns, business-case, and even a section on configuration management for software product-lines.

Software Process Architecture Views and Quality Attributes

After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.

Well, about a year ago I wrote that Software CM is NOT a Process! ...

Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.

The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.

So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:

• What are the views and perspectives of an SCM solution architecture?


• How do software architecture quality attributes relate to software CM (or even process) quality attributes?


• What are the quality attributes that, if attained, will make the resulting software CM and/or process environment Agile?

I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove most challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.

The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):

• Process Availability might correspond to the availability of the underlying tools and technology that support the process. But it might also need to include both the physical and cognitive "availability" of the process itself. It probably also needs to include the availability of key information (i.e., metrics and reports) to create information radiators, and big visible charts & reports.
  


• Process Modifiability is the ease with which the process itself can be adapted, extended/contracted, perhaps even "refactored", and ultimately improved. Rigid processes can't be changed very rapidly in response to a change in business need or direction.
  


• Process Performance is probably what most closely translates to flow or throughput of software development. (Although for tools and the supporting computing environment, it clearly has the usual meaning there.)
  


• Process Security is ... hmmn, that's a tough one! Would it mean "safety" as in keeping the practitioner safe/secure? Would it mean process quality? Or might it mean the security of the process itself in terms of making sure that only authorized/authenticated persons have access to the knowledge of the system (its requirements, designs, etc.) which may be proprietary/confidential and a significant competitive advantage, and that only those authorized individuals are allowed to execute the roles & workflows that create & modify that system knowledge? Perhaps "security" in this context is all about trust and trustworthiness: How well does the process ensure the trust and integrity of the system and of itself? How well does it foster trust among its practitioners and consumers?
  


• Process Testability might correspond to the ease with which the process and its results can be tracked/reported (transparency) and audited (auditability). Perhaps it is also related to the ease with which parts of the process can be automated.
  


• Process Usability probably has to do with the amount of "friction" the process imposes on the flow/throughput of development. Is it too big? too complex? a poor fit? easy to understand and execute? easy to tell if you did it correctly?

What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?

How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?