Friday, February 10, 2006

Agile vs MDE: XP, AMDD, FDD and Color Modeling

The February 2006 issue of IEEE Computer is devoted to Model-Driven Engineering (MDE). MDE is actually a bit broader than MDA/MDD, because MDE (allegedly) covers more of the lifecycle, and corresponding process and analysis. Doug Schmidt's Guest Editor's Introduction to MDE is a pretty good overview of the current theory and practice and the obstacles to overcome.

A co-worker of mine is very interested in combining Agile methods with Model-Driven Engineering. He feels that the benefits of agility and of model-driven code-generation show tremendous promise as a breakhrough combination in productivity and quality and he is stymied that there aren't a lot more folks out there trying to do it.

He attended UML World in June 2005 and had some discussions with Scott Ambler (AgileModeling), Steve Mellor (co-creator of the Schlaer-Mellor OOAD method, and co-author of "Agile MDA", Executable UML and MDA Distilled), and Jon Kern, Agile MDA Evangelist (who helped Peter Coad launch TogetherSoft). He found most of what they had to say supported the possible synergy between Agility and MDA, but was very surprised to see AMDD folks and XP/Scrum folks throwing away their models once they had the code for it.

Upon hearing the above, I noted that Peter Coad is quite possibly the missing link between MDE and Agility:
The potential mismatch between MDA, with AMDD and XP-like Agile methods, is that:
  • Full/pure MDA strives for 100% generation of all code and executables directly from the models.

  • Ambler's AMDD, and "domain modeling" espoused by the likes of Robert Martin, Martin Fowler, and others in the XP community strives for "minimal, meaningful models", where they model only as needed, as a means of gaining more understanding, and then embed the knowledge gained into the code and/or tests.
I beleive FDD has the potential to bridge the gap. It strives for a comprehensive domain model, but from that point the code is written by hand (using coding practices that are traditionally non-Agile in nature, including strict-code-ownership, and formal code reviews/inspections). FDD doesn't say anything about using MDA/MDD techniques to auto-generate code, but the method is extremely amenable to doing exactly that.

Furthermore, doing so would remove a lot of the manual parts and practices of FDD that many consider to be the least "Agile". And much of the FDD "Color Modeling" patterns and techniques are very much the equivalent of refactoring and design-patterns that are currently used for code. See the end of this message for some more resources on Color Modeling.

In my own humble opinion, I think the "sweet spot" is somewhere in between 100% code generation and "hand-crafted" code. I realize that 100% is the ideal, but I'm thinking about the 80/20 rule here, and whether trying to eliminate that last 20% is perhaps not always practical.

I think the biggest barrier to doing that today is tools:
  • The modeling tools are good at handling the structure, but not as much as the behavior.

  • High-level programming languages like Java and C# and their IDEs are more convenient for specifying behavior (which is still largely textual in UML 2 and Action-Syntax Languages).

  • It is extremely difficult to maintain the non-interface code for a "class" or "package" unless it is either 100% manually coded or else 100% auto-generated. If it is 50-50, or even 80-20, then the "nirvana" of seamless and reversible round-trip design to code and back just isn't quite there yet.
What would get us there and help close that gap? I think the "melding" of the IDE with the modeling tool is what is needed - and it would have to allow specifying code such as Java or C# as opposed to only allowing ASL "code" (most of which looks pretty darn close to Java and C# anyway :) as well as a means indicating if/how a chunk of code was auto-generated or if it was to be hand-crafted, but "navigable" and editable via the Model.

The Eclipse framework shows a lot of promise in helping us get to that point, and has a lot of the groundwork and building blocks already in place, but still has a lot more work to be done.

I hear some of you saying, "Okay Brad, I see what this has to do with Agility. But what does this have to do with CM?" Well, in my January 2005 Agile SCM column, among other "crystal-ball gazing" predictions, I talked a little about "Model-Driven CM" and how it would resurrect the once popular SCM research-area of Software/System Configuration Modeling:
  • MDE/MDA would readily lend itself to allowing the developer to focus on the logical structure of the code, letting the physical structure (files and directories) be dictated by the code-generation tool with some configuration scripting+profiles.

  • This in turn would allow models and modeling to be easily used to analyze and design the physical structure of the code, including build & configuration dependencies.
Of course, we have a ways to go until we get there, but I do believe the trend is on the rise and it's only a matter of time.

Some other resources related to Agility and MDE:


Scott W, Ambler said...

I think that a serious problem we face in this industry is the confusion surrounding modeling. Many people relate modeling to CASE tools, and refuse to accept that whiteboards, index cards, post-it notes, ... and other simple tools can be used to model very complex software. Yes, agilists throw away a lot of our models -- we get the benefit from them and then choose to forgo the cost of pretending to maintain them (sadly, traditionalists often keep models but don't invest the time to update them, which later slows them down).

Agilists model, we're just very smart about it. Sometimes we do use complex tools, as I show at , if the tool adds value to our effort.

- Scott

Brad Appleton said...

Hi Scott! I think the issue and surrounding confusion isn't so much about whether or not tools are used to create the model. I think it's more about what (if anything) is used to capture the knowledge that the model represents.

Some Agile methods (perhaps most of them), want the source-code to be the primary means of knowledge capture. So you can model on whatever you want (post-it notes and napkins), but you capture the knowledge in the code.

Model-driven methods (MDE, MDA, MDD) are concerned with capturing the knowledge in the model. FDD is an agile method that happens to share this value. The problem arises when we try to be lean/agile about minimizing redundant and intermediate artifacts, and try to Single Source Information (an excelllent essay by Scott). It's really hard to abide by that practice if some knowledge is in the code, and other knowledge is in the model. So we end up with waste/redundancy, or worse, the two get out-of-synch and we try to maintain them anyway (ouch! Dr. it hurts when I do that!)

So how do we resolve that? Most agile methods treat the code as the final and authoritative form of knowledge capture, and treat the model as the "intermediate artifact." MDE tries to keep the model "primary" and the code "intermediate." I think the underlying problem is that we have to make this choice at all ...

I think the "sweet spot" is where model and code are just different, compatible and co-existing views of the same knowledge, and that if we had tools that allowed us to treat them that way (instead of thinking of them as separate), we could more readily overcome the major problems. [The model is the source; and the source is the model]

I think if we thought of all the artifacts of software development (requirements/specs, designs/models, code, tests, etc.) as "knowledge" in its differing views and formats, and it was all part of a single "knowledgebase" that knew the difference between which entities were of which type/format, but still allowed us to impose the same structures and manage the same kinds of dependencies, then the "disconnect" goes away.

Jason Rutberg said...

I absolutely agree 100% with your perspective but I would go further.

I don't really believe that most corporate cultures can utilize MDA to get to level 4 modeling awareness (see my blog for reference -

The reason for this is that without SCRUM or some other kind of feedback driven development process, the potential of people is not unlocked. Acheiving level 4 modeling awareness requires a fundamental paradigm shift that most organizations can't even conceive.

You have to unlock the potential of people before you can create roles like Modeling Development Engineer which spans multiple "traditional" roles. This is not easy but MDA puts a focus on modeling that many traditional developers find uncomfortable (i.e. modeling is documentation).

I agree also in many ways with Scott Ambler but I differ in that you do need a place to capture the results of that firehose of creativity once you unleash it. That is when you seriously consider embracing MDA.