Saturday, September 17, 2005

Can I have just one repository please?

One of the things I spend a lot of time dealing with is integration between application lifecycle management tools and their corresponding process areas: requirement management, configuration management, test management, document management, content management, change management, defect management, etc.

So I deal with process+tool architecture integration for a user community of several thousand, and the requirements, version control, change-tracking, and test management tools almost always each have their own separate repositories. Occasionally the change-tracking and version-control are integrated, but the other two are still separate.

And then if there is a design modeling tool, it too often tries to be a "world unto itself" by being not merely a modeling environment but attempting to store each model or set of models as a "version archive" with its own checkin/checkout, which makes it that much more of a pain in the you-know-what to get it versioned and labeled/baselined together with the code, particularly if code-generation is involved and needs to be part of the build process.

And what really gets to me is that, other than the version control tool, the other tools for requirements and test management, and typically change management usually have little or no capability to deal with branching (much less merging). So heaven forbid one has to support multiple concurrent versions of more than just the code if you use one of the other tools.

The amount of additional effort for tool customization and configuration and synchronization and administration to make these other tools be able to deal with what is such a basic fundamental version-control capability is enormous (not to mention issues of architectural platforms and application server farms for a large user base). So much so that it makes me wonder sometimes if the benefit gained by using all these separate tools is worth the extra integration effort. What if I simply managed them all as text files in the version control system?

At least then I get my easy branching and merging back. Plus I can give them structure with XML (and then some), and could easily use something like Eclipse to create a nice convenient GUI for manipulating their contents in a palatable fashion.

And all the data and metadata would be in the same database (or at least one single "virtual" database). No more having to sync with logically related but physically disparate data in foreign repositories and dealing with platform integration issues, just one big (possibly virtual) repository for all my requirements, designs, code, tests, even change-requests, without all the performance overhead and data redundancy and synchronization issues.

It could all be plain structured text with XML and Eclipse letting each artifact-type retain its own "personality" without having to be a separate tool in order to do it.

Why can't someone make that tool? What is so blasted difficult about it!!!

I think the reason we dont have it is because we are use to disconnected development as "the rule" rather than as the exception. Companies that shell out the big bucks for all of those different tools usually have separate departments of people for each of requirements (systems/requirements engineers), design (software architects), source-code ("programmers"), test (testers), and change-management.

It's a waterfall-based way of organizing large projects and it seems to be the norm. So we make separate tools for each "discipline" to help each stay separate and disconnected, and those of us doing EA/EAI or full lifecycle management of software products have to deal with all the mess of wires and plumbing of integration and platforms and workflow.

Oh how I wish I could take a combination of tools:
  • a good, stream-based version control tool like Accu-Rev
  • a fully Java/XML extensible issue-tracker like Jira (or combination of the two, like SpectrumSCM)
  • a framework like Eclipse
  • and a collaborative knowledge/content management system like Confluence
and roll them together into a single integrated system with a single integrated repository.

Notice I didn't mention any specific tools for requirements-management or test-management. Not that I dont like any of the ones available, I do, but I think it's time for a change in how we do those things with such tools:
    they basically allow storing structured data, often in a hierarchical fashion with traceability linkages, and a way of viewing and manipulating the objects as a structured collection, while being able to attach all sorts of metadata, event-triggers, and queries/reports
I think a great wiki + CMS like Confluence and Jira can do all that if integrated together; Just add another "skin" or two to give a view of requirements and tests both individually and as collections (both annotated and plain).

The same database/repository could give me both an individual and hierarchical collection-based views of my requirements, designs, code, tests and all their various "linkages." Plus linking things in the same database is a whole lot easier to automate, especially thru the same basic IDE framework like Eclipse.
  • the requirements "skin" gives me a structured view of the requirements, and collaborative editing of individual requirements and structured collections of them;
  • ditto for the test "skin";
  • and almost "ditto" for the "change-management" skin (but with admittedly more workflow involved)
  • the design tool gives me a logical (e.g., UML-based) view of the architecture
  • the IDE gives me a file/code/build-based view of my architecture
  • And once MS-Office comes out with the standard XML-based versions, then maybe it will be pretty trivial to do for documents too (and to integrate XML-based Word/Office/PPT "documents" with structured requirements and tests in a database)
Oh why oh why can't I have a tool like that! Pretty please can I have it?

3 comments:

Frank said...

If tool vendors would spend their effort and money on making it easy to integrate with other tools (of other vendors or of other divisions within the same company), then they can spend less money on features that customers need. As a result, the other tool vendor (that spent less on integration and more on functionality) will have the commercial benefit. A weak business case, I would say.

On the other hand, if a tool vendor is able to integrate the complete software development lifecycle transparantly into their tools, they could use it internally to build their tools more efficiently and thus beat the competition. However, this requires enormous investments and causes enormous business risks, which translates into a high price tag for the tool, which in turn chases away customers who rather select a cheap solution with similar or better functionality and performance.

So they have to sell their omnipotent tool at a very low price. But having high initial costs and a low price is again a weak business case.

I think that the central issue is not that it is difficult to develop such a tool, but that it is difficult to define a solid business case.

Frank.

Anonymous said...

OK, we produce a tool that doesn't have the UML engine (though it will integrate with a UML tool - e.g. Rose).

It does Requirements Tracking, Document Management, Change Management, Version Control, Problem/Issue Tracking, Project/Activity/Task Management, Build and Release Management, Test Case Management, Test Run Management, and a few other things, all in one tool. One interface, one database. It didn't cost a bundle to build. In fact we saved enormously by not having to manage integration of a whole set of tools.

It rolls out easily with very low risk and low cost ($1K/user for all apps), how a process workflow capability, can be interactively configured without down time, requires near-zero administration and support very low admin multiple site management for all applications in one go.

Is this what you want Brad?
Joe

Brad Appleton said...

Hi Joe!
What you describe certainly makes me start salivating :-) Of course I'd also want either API-level and/or command-line access to all the functionality (so I could automate tasks), and I'd need to be able to customize any workflow, as well as the data-attributes (schema) that are tracked for the various work-product types (requirements, models, code, tests, etc.).

And for the piece de resistance, I'd want the ability to view/manipulate/modify a hierarchy of objects both as individual elements, and as an overall collection.

So for requirements and test-cases, that means the ability to treat the collection as a single document (or set of related docs) as well as individual requirements/tests. And for code that means an Eclipse-like environment with refactoring-type operations.

Oh yeah, and Eclipse-compatibility would be good too :-)

But even if it didnt have all those things, I'd still want to take a close look at it just from what's been described so far.