Thursday, 13 October 2011

Agile is a Cop-Out?

In a blog entry entitled "Agile Software Is A Cop-Out, Here’s What’s Next", Forrester's Mike Gualtieri makes some bold statements about what he sees as hype and a lack of empirical evidence of success from the Agile community.

Now, Mike's post is bound to raise the hackles of many people in the Agile community, but I do agree he has some good points.

He doesn't specifically use the term "hype", but certainly the promise of hyperproductivity coming from Agile thought leaders like Jeff Sutherland doesn't help.  I have witnessed trainers routinely telling their students to expect incredible gains in productivity once they drink the Scrum kool-aid.  It's a great sales pitch - others are doing this and if you aren't achieving those sorts of increases then you must need more training or coaching!  The amount of Bad Agile or Bad Scrum that has resulted is a testament to both the hype that has pulled people into trying Agile and the difficulty in actually doing it well.

I will defend the authors of the Agile Manifesto, though, in that they were making a statement about the world of software development as it was in the late 90's and into early 2001.  Mike cites Steve Jobs using "insanely great" to describe products, but in February 2001 the world still hadn't seen it's first iPod.  OSX was a month away from going GA.  The aluminum-cased MacBook Pro and MacBook Air were 5 and 7 years away respectively.  The iPhone was a pipe-dream.  Tablets had come and gone several times at that point, long before the iPad was developed.  So, most of that insanely great design was still well ahead of us when the Manifesto was written.

Also, as an industry we sucked at building software, and the dot-com boom hadn't helped much to fix that.  You had either code and fix or some hard-ass waterfall process to follow, and precious little in between.  It's no wonder that the Manifesto's authors focused on that problem.  "Working Software" wasn't narcissistic, as Mike states, it was something that was sorely lacking in early 2001 (and arguably still is today).

Mike mentions that he first wrote about his approach, Parallel Immersive Software Studio (he acknowledges the acronym that creates!), in 2008 and has refined the ideas since then.  Well, the software development world is a little different today than it was in 2001.  The success of the products from Apple are mainly due to the total user experience, which is something that has grown and been refined over the ensuing decade.

I see a lot of good ideas in Mike's approach.  I certainly agree fully with the ISS parts of his approach, but I do have issues with Parallel.  Any time that groups go off in isolation to work in parallel, you incur multiple risks:
  • One or more groups go in the wrong direction;
  • You have to integrate the work at some point, and the longer you wait the more difficult the integration will be;
  • Based on Mike's description of parallel work, you are isolating experts in different areas thereby decreasing your Truck Number
Another point Mike makes is, "Parallel work streams can reduce uncertainty and the number of costly iterations."  Could I please see some empirical data to support that?  If he's going to criticize the Agile community for not providing empirical data, then I believe it behooves him to do the same.

Mike concludes the post with 5 key points about his process:
  1. Software is not code. It creates experience.
  2. Development teams are not coders. They are experience creators.
  3. Technical talent is “table stakes”. Great developers must be design and domain experts.
  4. Process is bankrupt without design. You get what you design so you better get the design right.
  5. Software is a creative endeavor, not an industrial process like building automobiles. The methodology is structured to support the creative talent.
I agree with all of these, although #4 concerns me.  What is Mike's definition of design?  What does it mean for a design to be 'right'?  How do you know when a design is right?  Is design ever done?  How much design is enough?

This segue's nicely into my growing interest in the Lean Startup community.  Lean Startup assumes you're going to use good engineering practices (many from the XP world) to build the software because they enable you to obtain the kind of rapid feedback required to refine a total product design into something for which people will actually pay money.  What I don't see in Mike's post is anything about that sort of validation - his process may ensure that you build some kick-ass, well-designed systems, but if you don't sell the product because you didn't realize that no one wanted it, have you really succeeded?

Mike has some very good ideas in his post.  I also saw some good ideas in a similar "post-agile" blog entry by Michael Dubakov.  While I don't agree with all of the points the respective authors make, I do think this sort of "what's next" thinking is healthy and will help make the second decade after the creation of the Agile Manifesto even better.

After all, reflection and continuous improvement are core concepts in Agile and the ideas put forth by Mike, Michael and Eric Ries of Lean Startup provide exactly that.

5 comments:

nosnhojn said...

Dave, I like your analysis. I read Mike's post earlier this morning and came to similar conclusions. I will say I felt mike countered hype with more hype, though I don't think that should raise any hackles. If the manifesto was as far as we could go, I think everyone would agree that'd be a little depressing regardless of how valuable its been. While agile has been a revolution for organizations that use it well, I'm sure there are more to come. That's a good thing :)

Chris R. Chapman said...

Dave; Really good rebuttal to Mike's post - you've given me some food-for-thought.

My initial takeaway from reading Mike's post was that he didn't really read the manifesto and understand the values and principles - he basically skimmed them.

For example, he calls out the seventh principle, "Working software is the primary measure of progress." as a cop-out because it isn't customer-centric enough, yet doesn't address the effect of principles #1 and #11:

"Our highest priority is to satisfy the customer through early and continuous delivery of valuable software."

"The best architectures, requirements, and designs emerge from self-organizing teams. "

Similarly, he diminishes the intent of the fourth principle, "Business people and developers must work together daily throughout the project" as motivated by laziness yet completely avoids discussing its natural counterpart two principles down, "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation" which also, IMHO, kneecaps the idea of having teams running in parallel isolation to come up with "the magic".

If there's anything I've come to understand over my ten years working with agile practices, it's the guys who lob the hand grenades at the manifesto who actually are seeking attention with their own manufactured hyperbole.

In short, despite Mike's impressive bio, it seems like he hasn't worked that seriously with agile practices - in fact, he seems outright hostile to them.

KeithC said...

Chris pointed out the agile principle "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation". This is, being blunt, stunningly naive, or, being rather kinder, only appropriate for certain types of communication. For a circular definition, it is appropriate only for that information that is conveyed best by face-to-face conversation. And that is information that is essentially ephemeral. If it is to be retained, then it needs to be recorded. If this is by the developers all making their own notes, well, guess what, they will not all record the same thing and will almost always be inconsistent. I posted a blog entry about this here: http://methods-and-music.blogspot.com/2011/10/documentation-and-collaboration.html

Dave Rooney said...

@KeithC,

I'll comment on your blog as well, but I don't recall anyone ever saying that you shouldn't record the outcome of a conversation if it was important. I coach that all the time, and encourage teams to document what they feel is important. I also coach them to be comfortable erring on the side of too much documentation until they understand how much they actually need to be able to work effectively.

What I do say they should not do is to attempt to document everything up front. First, it's virtually impossible to know everything up front that needs to be known. Second, locking in detailed requirements and design can multiply effects of any errors or omissions in those documents. Finally, I have directly observed the negative effect that 'completed' documentation has on the ability of teams to innovate - if something has been documented, then people are reticent to change it.

That still doesn't mean, though, that you don't need to document anything before starting. I've spent the last 19 months working with a large telecom client in the wireless space, and they have been able to reduce their up-front documentation requirements significantly, replacing them with 'live' documents on wikis.

So, I believe you've either created or bought into someone else's false dichotomy that you either have reams of up-front documentation or none whatsoever. That simply isn't the case.

What is provably true is that a face-to-face conversation is a much more effective form of communication than any document ever will be.

KeithC said...

Dave, I hope you didn't read my blog as suggesting that everything should be documented up front - if you did, I wrote it wrong! I am all in favour of documents growing as needed - Wikis are good, in some cases word processors or even formal tools are better (possible personal prejudice warning -I was a DOORS consultant for twelve years).
I agree that face-to-face conversations are very effective, and would be interested to see the research that proves this. I am not convinced that this is so for all communication and indeed I would contend that it is not the case for all communication. Especially for complex areas, where you need to digest the information, perhaps make annotations, etc., documents can be much better.