Thursday, August 20, 2009

SOA, Mashups, Mashed Knees and Surgery

Today is my birthday - I had arthroscopic knee surgery last night and am feeling pretty good so far (happy birthday to me). I know I still have a lot of meds/painkillers in my system and that its going to feel more uncomfortable the next few days. I'm still hobbling around on crutches but I'm feeling much more confident that I can attend (and present at) Agile2009.

I received some very good books on Mashups and SOA a few months back and I'm finally getting a chance to look at them in some more detail. They really are quite good! I think Mashups are the "latest frontier" for realizing the promise of SOA, and a natural evolution from Wiki-webs. Here are the books:

If you follow the links above to the InformIT.com homepage for each of the above books, you'll find some good excerpts and related articles that will give a preview of what to expect so you can judge for yourself.

Thursday, August 13, 2009

BOOK: Running an Agile Project

First, on a personal note, I had the misfortune to tear cartilage in my right knee a couple days ago and will require surgery to repair/remove it. I'm hobbling around on crutches for the time being. I hope I can still attend (and present at) Agile2009.

Anyway, my review of Mike Holcombe's book Running an Agile Software Development Project appears in this month's Agile Journal.

Running an Agile Software Development Project is an interesting book. On the surface it looks like it would be very academic, because the author, Mike Holcombe, was a University Professor at the time, and running an “agile software development factory” of students (albeit for a real commercial development shop). And yet what is described in the contents is very much the practical, real-world results of running agile projects with those same people for real IT software development work.

[...]

Overall, I found Running an Agile Software Development Project to be interesting and enjoyable. It still seems just a bit academic for my taste, and probably wouldn’t be the first book I would recommend on the subject unless it was for a classroom audience (in which case this book would be an excellent one to use).

You can read the full review here!

Friday, August 07, 2009

WANTED: Seeking Single Agile Knowledge Development Tool-set


I'll be presenting at Agile2009 in Chicago on the Tools for Agility stage on Tuesday 25 August, 4:45pm-5:30pm.


Here is my session description from http://agile2009.org/node/2762


WANTED: Seeking Single Agile Knowledge Development Tool-set

Aren’t code, backlog-items, tests, designs & documents all just different forms of system knowledge at different levels of detail? Why can’t the same tools help refactor, browse, search, and provide build/test automation for non-code forms of knowledge without requiring a separate tool/repository for each format? This talk is intended as a challenge to tool vendors/developers to see how this simple treatment of all non-code items as part of a single, unified project knowledge-base can be at once both immensely powerful, and imminently practical, without requiring too much added complexity.

Process/Mechanics

Approximately ~30 minutes of slides/presentation to “make the case” for why this approach is useful and desirable, followed by discussion of challenges (to and from tool developers/vendors, as well as the rest of the audience) as to its usefulness and benefits, and why their current tools can’t easily do so and why the should or should not be easy to enhancement them to do it.

Outline

Software development as knowledge development

  • Source-Code as knowledge
  • Requirements (Stories) and Tests as knowledge
  • Other usable forms of project knowledge (e.g., build scripts & configuration, build/test result reports version control history/comments, online help & other supporting docs)

How would I do this?

  • Refactoring Knowledge (thinking about the rest of the “knowledge” the way we think about the code, and its habitability, navigability, and structure/refactoring)
  • Applying other agile-practices (like TDD, Continuous Integration, etc.) to non-code knowledge development
  • Wiki-based skins, DSLs, and use-cases/design as domain-driven Wiki-word definitions
    • Patterns (giving things names), Retrospectives results and lessons learned
  • Viewing, searching EVERYTHING (even the code) via wiki?
  • The “Wu-Wei” of Traceability (Tracing without Tracing)
  • Versioning and operating on wiki-like entities, just like with code (e.g., making “working copies”, branches and tags)

DISCUSSION & CHALLENGE: Why can’t (or how can) YOUR tools do this!

Begin audience discussion/dialogue: Why can’t a tool like Eclipse or a Wiki-based CMS (such as Confluence) be used as a front-end to browsing/refactoring and navigating ALL the knowledge of the system? (not just code, but stories, tests, build/config data, CI reports, backlog, retrospective lessons).

What makes it easy/hard for these and other tools like any of the following to do this, or to be simple “skins” or plug-ins on top of only a handful of tools instead of a whole kitchen sink full of separate tools and repositories.

  • Eclipse, Confluence, Twiki?
  • FIT, Fitnesse, Contour
  • Doxygen / Javadoc
  • Trac / Agile-Trac?
  • Jira / Remedy

See also a blog-entry of mine entitled Can I have just one repository please?

Learning outcomes
  • Thinking about Agile development as knowledge management & creation
  • Current barriers to using agile techniques and supporting tools for non-code artifacts
  • What do refactoring, TDD, continuous integration, etc. mean for non-code artifacts (and why you should care)
  • Use of wikis for organizing, structuring and refactoring ALL system knowledge
  • Why manual traceability is unnecessary (and even comes for free) when using such an approach

Primary target persona
  • Tomas Tool-smith/Tool-vendor

(download PDF here)

Saturday, August 01, 2009

Studies on Effectiveness of TDD?

This question came-up in a discussion earlier this week: Do we know of published studies on this subject? A quick Google-search turned up the following for me ...


Any others? Any comments on the above?

Sunday, July 26, 2009

Resources on Retrospectives

I found a really good resource-list from George Dinwiddie on Introspection and Retrospectives that includes the following list of resources (mostly patterns & techniques) about conducting retrospectives. It contains many (but not all) of the links below:



Friday, July 17, 2009

Refactoring @ Scale

In my previous post, Refactoring for Agility, I posted an outline and some thoughts for Part I of an Overview on Refactoring. Now I'm ready to post on Part II which is about refactoring @ scale. By "at scale" I mean in the larger context of other agile practices, as well as for large projects.

PART II - REFACTORING @ SCALE

1. Scaling-Up
  • To scale refactoring for larger projects, some additional techniques & issues must be added to the mix.
  • Note that this is “in addition to” (not “instead of”)
    Refactoring In-the-SmallRefactoring @ Scale
    Small, Fast & Frequent RefactoringsLarger, Periodic & Planned Restructurings
    Emergent DesignIncremental Design & Evolutionary Architecture
    Deferred RefactoringRestructuring & Technical Debt
    Code SmellsArchitecture Smells
    Design Principles & PatternsSoftware Modifiability Tactics
    Simple/Clean CodeSupple/Domain-Driven Design

2. Emergent Design
  • Emergent Design is a fancy name for the resulting design that “emerges” from the synergy of combining Refactoring together with TDD, Continuous Integration and Automated Testing.
  • Emergent Design means that ...
3. Technical Debt [a.k.a. Design Debt]
4. Restructuring Technical Debt
  • If we accrue a non-trivial amount of technical debt, we can’t simply “refactor” it away.
  • Paying it off typically requires restructuring efforts (or even reengineering) that must be planned.
  • Iteration plans must accommodate specific tasks for these restructuring efforts (or even be dedicated to restructuring).
  • Ignoring it, or deferring it for very long is not a viable option!
5. Overview of Restructuring
  • Identifies higher-level issues (“architecture smells”) that typically represent violations of known principles of good software architecture & design.
  • Periodically applies larger-scale “refactorings” and/or many small refactorings that were previously deferred.
  • The goal is to “pay down technical debt” in order to limit the increasing costs of accumulated complexity.
  • Typically requires a concerted effort that must be separately planned.
  • Uses not only design patterns/principles, but also architectural patterns/principles, as well as software modifiability tactics.
6. Refactoring vs. Restructuring
7. Restructure Periodically
  • Restructuring is often associated with absent or neglectful refactoring and/or design.
  • But … Any large software project spanning multiple teams eventually needs restructuring.
    • Even in the presence expert-level architecture, design & continuous refactoring
    • This is just a reality of software evolution/entropy
  • Therefore … Large software projects should assume that periodic restructuring will be necessary, and should plan accordingly to:
    • Clean-up accumulated code-smells and apply numerous refactorings that were deferred but are now sorely needed,
    • Address architecture smells by applying restructurings, patterns and modifiability tactics that have broader impact.
8. Architecture Smells
See Stefan Roock's and Martin Lippert's book Refactoring in Large Software Projects (There was an earlier version of their work entitled Large Refactorings").
  • Smells in Dependency Graphs
  • Smells in Inheritance Hierarchies
  • Smells in Packages
  • Smells in Subsystems
  • Smells in Layers
9. Software Modifiability Tactics
  • Localize Changes (increase cohesion)
  • Prevent Ripple Effects (reduce coupling)
  • Defer Binding-time (defer decision-making)
10. When to Consider/Plan Restructuring?
11. Evolutionary Architecture
  • Software Architecture concerns infrastructure elements that must exist before you can begin execution.
    • Architecture is about things that are hard to change later, it is difficult to allow an architecture to emerge.
    • For large projects, this includes high-level organization of the system into functionality/elements that will be allocated to separate teams.
  • Key techniques of Evolutionary Architecture include:
    • Deferring Irreversible Decisions to the “Last Responsible Moment” (LRM Principle)
    • Architectural “Spike”, Architecture Iteration and/or Spike Iteration
12. Incremental Design
  • Design evolves incrementally, iteration by iteration, based on current business priorities and discovered technical limitations.
  • Incremental Design …
    • Does not prohibit thinking about higher-level design.
    • Does encourage planning in detail only what will be constructed soon.
  • Focus is on “Just Enough, Just-In-Time” :
    • Specifying too much detail too soon causes more rework later.
    • But doing less now and saving the rest for later should not require significantly more work later than it would today.
  • We must do Just Enough Design Initially to attain the right balance of anticipation and adaptation.
13. Just Enough Design Initially (JEDI)
  • Initial design (before coding) is still necessary.
    • This use of “JEDI” was coined by Stephen Palmer as part of Feature-Driven Development (FDD)
  • Basic rule of thumb to tell when “JEDI” is achieved:
    • At iteration-scope, when, after one pass through the iteration backlog, modeling in small groups does not produce any new classes or associations of real significance.
    • At task/TDD scope, when we have defined enough structure & interface(s) to know specifically what code to write/test, precisely where to write it, and exactly how to invoke it.
  • Techniques of “the JEDI way” include:
    • Collaborative Domain Modeling and Color Modeling [from FDD]
    • Supple Design techniques & patterns
    • Domain-Driven Design (a.k.a. DDD -- see domaindrivendesign.org)
    • Design Blitz & other Agile Modeling techniques (see agilemodeling.com)
14. Supple Design & DDD
  • Domain-Driven Design (DDD) approaches modeling the core logic of the software by focusing on the domain.
    • The basic idea is the design should directly reflect the core business domain and domain-logic of the problem to solve
    • This helps understanding the problem as well as the implementation and increases maintainability of the software.
  • DDD uses common principles and patterns as "building blocks" to model & create a "supple design”
    • Supple: pliant, malleable, limber, yielding or changing readily.
    • The design is firm yet flexible, with structure and intent both clearly conveyed and deeply realized by the code.
  • Patterns of Supple Design include: Intention-Revealing Interfaces, Ubiquitous Language, Side-Effect-Free Functions, Assertions, Conceptual Contours, Standalone Classes, Closure of Operations, Declarative Style

15. Resources:

Resources on Emergent Design and Evolutionary Architecture Resources on Restructuring Resources on Technical Debt Resources on Modifiability Tactics Resources on Supple Design & DDD

Thursday, July 16, 2009

Refactoring for Agility

Some of you might have guessed from my recent posts on Emergent Design, Technical Debt, JEDI Programming, and 5S Qualities of Well Designed, Well-Factored Code, that I've been looking into trying to teach the fundamentals of refactoring and how it scales to larger projects. I've gathered some references and quotes and some ideas for slides that I wanted to bounce around on my blog.

Here is an outline and some thoughts for part I of some slides ...

PART I - REFACTORING FOR AGILITY

1. Overview of Refactoring
  • Identifies design-maintenance issues (“code smells”) that typically represent violations of known principles of good design.
  • Incrementally and iteratively applies a set of design improvement techniques (“refactorings”).
  • The goal is to minimize complexity & duplication in order to maximize simplicity & ease-of-change.
  • Encourages the “right” design details to emerge “just-in-time” with minimal guesswork/rework.
  • Scaling-up includes the use of periodic restructuring, initial & incremental design (“just enough”), and evolutionary architecture.

2. Refactoring Defined [cite definition(s)]

3. Refactoring Myths -- Refactoring is NOT …
  • “Rework” – redesigning things that could, and should, have been designed correctly in the first place.
  • Design “gold-plating” – work that adds no business value, and merely serves to stroke the egos of perfectionists who are out of touch with business reality.
  • Miscellaneous code “tidying” – the kind that is “nice to have,” but should only happen when the team has some slack-time, and is a luxury we can do without, without any serious consequences.
  • A license to “hack” – avoiding any and all initial design & analysis and instead jumping straight to coding with no “real” design.
  • Reengineering – large-scale restructuring that requires a concerted effort over the course of several weeks/months to re-write or re-architect significant parts of the system.
4. Refactoring IS …
  • A systematic approach to source-code “hygiene” that minimizes the chances of introducing bugs
  • Improving the design of the code after it has been written
  • A behavior-preserving transformation of source-code structure
  • The process of simplifying & consolidating a work-product by making several, small, successive revisions focused on: preserving correctness, removing redundancy, revealing thoughts & intentions, and improving clarity & conciseness.
  • A disciplined way of making changes while exposing the project to significantly less risk.
  • An effective means to address the economic reality of software growth/complexity by reducing & amortizing its cost throughout the daily business of code development & maintenance activities.
5. Why Refactor?
6. How to Refactor
7. Rules of Clean Code
8. Rules for Simple Code
9. The Steps of Refactoring
10. Code Smells
11. Categories of Refactorings
  • Small Refactorings
  • Larger Refactorings/Restructurings
  • Each category contains as many as a dozen or more refactorings, most of which are catalogued at http://refactoring.com/catalog/

12. Refactorings (Some refactorings from real projects)
  • See http://refactoring.com/catalog/ for an up-to-date list (and the “Refactoring to Patterns” catalog too)
13. What to do if …?
  • I spot a “smell” that is not already known or catalogued?
  • There is no specific known/catalogued “refactoring” for what I think I need?
14. When to Refactor?
  • While adding functionality
  • While fixing a bug
  • While reviewing code
  • After coding the same/similar thing for the third time (to “factor out” the duplication)
  • A.k.a.: The Rule of Three: 3 strikes and you refactor.
  • After the third time you deferred refactoring a change, for any reason [The Rule of Three, again]
  • Before the end of the iteration if you haven’t been following The Rule of Three
15. Refactor Continually
16. When NOT to Refactor?
  • When the build is broken or tests don’t pass
  • When it would compromise meeting an impending deadline or commitment
  • When the code in question really just needs to be re-written “from scratch”
  • When it would modify code/interfaces that could significantly impact/break other work (e.g.: Published/public interfaces and protocols, Database schemas/tables/operations)
  • Sometimes we must defer refactoring for later and/or plan for subsequent restructuring
17. Refactoring to Patterns & Principles
Software Design Principles and Design Patterns are the underlying foundation for Refactoring:
  • Code smells (a.k.a “code pathologies”): Signal a possible violation of design principles, Suggest which refactoring may be needed
  • Refactoring: Correct a design principle violation (at least partially), Converge toward common design patterns
  • Design Patterns: Reconcile forces among conflicting design concerns,Restore balance between competing design principles
  • Design Principles: Lead us to attain desired design qualities/attributes
18. Design Attributes/Code Qualities
Qualities of Highly Maintainable Software:
  • Loose Coupling & High Cohesion
  • Hierarchy (Structural Decomposition)
  • Abstraction, Encapsulation & Modularity
  • Sufficiency, Parsimony and Primitiveness
  • Readability
  • Testability
  • Modifiability
  • Serviceability
19. Design Principles: SOLID, SoC, DRY, Shy
  • The SOLID Principles of Object-Oriented Design (from Uncle Bob)
  • The SoC Principle: Separation of Concerns — separate interface from implementation, policy from mechanism, behavior from construction, commands from queries, ...
  • The DRY Principle: Don’t Repeat Yourself (Eliminate Duplication), Single Point of Truth
  • The “Structure-Shy” Principle: (“Tell, Don’t Ask!”), The Law of Demeter, Principle of Least Assumed Knowledge
20. Other Acronyms of Simple/Agile Design
  • OAOO – Say Things Once And Only Once (restatement of the DRY principle)
  • DTSTTCPW – Do The Simplest Thing That Could Possibly Work! (restatement of the KISS principle)
  • YAGNI – You Aren’t Gonna Need It!
  • The LRM Principle: Defer Commitment of Irreversible Decisions to the Last Responsible Moment!
  • BDUF – Big Design Up-Front! (vs. JEDI)
  • JEDI – Just Enough Design Initially/In-front!
  • DDD – Domain-Driven Design
21. Design Patterns
22. Summary: Refactoring for Agility
  • Successively applies small behavior-preserving transformations to eliminate code smells
  • Based on proven design principles and patterns for achieving maintainability & modifiability
  • Good automated testing is a prerequisite
  • Refactoring is not rewriting, rework or restructuring
  • With refactoring, we continuously invest nominal effort to reduce the risk & cycle-time of changes
  • The goal is to minimize complexity & duplication in order to maximize simplicity & ease-of-change.
  • Practiced in a highly disciplined manner, it promotes:
    • Sufficient functionality
    • Simple & clean code
    • Supple design
    • Serviceable software
    • Sustainable team velocity

23. Resources:
Code Smells:
Design Principles:
Design Patterns:
- Online Resources:
- Books:
Other Agile Design Slogans: