I consult mainly on Agile software development, with a special focus on how testing fits in. Contact me here: marick@exampler.com
My spam filter seems to be classifying everything as spam. I’ve unspammed the real comments that hadn’t been auto-deleted yet. If you were wondering why your comment never got posted, that’s why. Maybe I should go back to requiring registration.
For service, give me a rude but efficient New Englander over a friendly but slow Southerner any day. I’ve been made fun of for shutting the dishwasher door with my foot while simultaneously stretching the other way to grab something out of a cupboard, but it just makes sense to me to do things in parallel unless they have to be serial. I fume when behind people who wait until the cashier tells them the total before beginning to fumble for their money. So I ought to be all for one of the defining characteristics of Lean: driving out waste.
And I am, but… it’s a dangerous tool when used as an excuse by the inhumane. For an illustration, go to my favorite passage from my favorite Shakespeare play, King Lear. Lear has given over his power to two of his daughters, Goneril and Regan. He and a hundred rowdy knights are staying with Goneril, who wants him to dismiss half of them. He pitches a fit:
… thou are a boil,
A plague-sore, an embossed carbuncle,
In my corrupted blood. But I’ll not chide thee …
I can be patient; I can stay with Regan,
I and my hundred knights.
But Regan agrees with her sister:
… what, fifty followers?
Is it not well? What should you need of more?
… and then goes a step further:
… I entreat you
To bring but five and twenty: to no more
Will I give place or notice.
Lear is shocked, repudiates her, and decides to stay with Goneril, saying to her:
… I’ll go with thee:
Thy fifty yet doth double five and twenty,
And thou art twice her love.
But Goneril is remorseless:
… Hear me, my lord;
What need you five and twenty, ten, or five,
To follow in a house where twice so many
Have a command to tend you?
And, in what I consider one of the most devastating short lines ever written, Regan adds:
What need one?
Then comes Lear’s great ineffectual cry and descent into madness:
O reason not the need! Our basest beggars
Are in the poorest thing superfluous.
Allow not nature more than nature needs,
Man’s life is as cheap as beast’s. Thou art a lady:
If only to go warm were gorgeous,
Why, nature needs not what thou gorgeous wear’st,
Which scarcely keeps thee warm. […]… touch me with noble anger,
And let not women’s weapons, water drops,
Stain my man’s cheeks. No, you unnatural hags!
I will have such revenges on you both
That all the world shall–I will do such things–
What they are, yet I know not; but they shall be
The terrors of the earth. You think I’ll weep.
No, I’ll not weep.
Regan and Goneril were driving out waste. Those knights really were a rowdy, drunken gang of good-for-nothings. But waste was just an excuse. R&G really cared about personal power, not waste. And so will many people marching behind the Lean “banner with a strange device: muda!”
Now, as Jonathan Kohl would point out, many people marching behind the Agile banner do the same: they use Agile as another club with which to beat people. I’m less worried about Agile, though, because its base rhetoric is more explicitly humanist. Lean is more likely to be an attractive nuisance because the idea of driving out waste appeals to executives who find it less work to remove waste than to convert it into value—executives who get license to act sociopathic because they have a fiduciary duty to treat business as a machine for maximizing shareholder value, externalities be damned. I worry about Lean in a business culture where we are trained out of empathy for Lear, damned fool though he surely is.
Blurb for my Google Testapalooza keynote:
Four Ages of Testing
Just as biological species do, testing approaches change to fill new ecological niches. This talk covers four broad approaches to testing. It will spend most of its time on the third, an unfinished punctured equilibrium where testing is struggling to balance its traditional role — dispassionate judge of an end result — with a new demand for active help during design. It will also hint at a niche just opening up, one where technology allows testing to become a much more direct conduit for the will of the users.
Brian Marick (marick@exampler.com, www.exampler.com/blog, twitter.com/marick) was a programmer, tester, and team lead in the 80’s, a testing consultant in the 90’s, and is an Agile consultant this decade. He was one of the authors of the Manifesto for Agile Software Development and is the author of two books (The Craft of Software Testing and Everyday Scripting with Ruby) and a bunch of articles. He turned down a pre-IPO job offer from Google, in part because he expected the craze for its stock to have ended by the time his options vested, which ought to make you wonder about his insight.
Bruce Daley posts on how most humans are biased to think they’re less error-prone than they are. As far as I know, that’s a claim solidly based in empirical research. (See also Bruce Schneier’s The Psychology of Security.) From this, he concludes:
Given the nature of their work, software developers and software programmers suffer more from the illusion of knowledge and the illusion of control than most other professions, making them particularly subject to over-looking mistakes in their own code. Which is why software needs to be tested independently.
However. Consider the graph below.
Here, the programmer and independent tester start testing at the same time. (Bad programmer! Bad!) The programmer starts out with more knowledge of the app than the tester (the line marked P/+), but she also has a large amount of cognitive bias (P/-) and lacks testing skill. That makes her miss bugs her knowledge would otherwise allow her to find (the area under the red line). Moveover, her biases seem to be pretty impervious to evidence.
The tester starts out with less knowledge, but has no (relevant) cognitive biases at all. Also, his testing skill lets him ramp up his bug finding pretty fast—but it still takes him a while to overcome her advantage.
Which do you want doing the testing? If you’re shipping at time A, it looks like the programmer has the edge. (Compare the shaded areas under the curve.)
We could expect that advantage to erode over time. If the ship date is farther out, the independent tester would have an advantage, as this graph shows:
Even when all that matters is bug count, the decision is not straightforward, especially since it’s based on information you can’t know until after you’ve decided. (How long will it take the tester to get up to speed? How many and what kind of bugs will the programmer miss?)
On most projects, there are lots of other factors to consider.
So I encourage people not to make the assertion the post’s author does.
UPDATE: Turns out that what I want to do, modeled after something used for RubyConf, can’t be done in stock Twitter. Seeing if I can persuade the Twitter people to work the same magic for me.
Item: Richard P. Gabriel has this habit of making software people write or speak within artificial constraints.
For writers’ workshops (book-length PDF), he’s made reviewers write a summary exactly 29 words long.
In last OOPSLA’s “50 in 50” keynote, he and Guy Steele, Jr., covered the last five decades of programming languages in 50 segments, each exactly 50 words long (in a talk lasting, I believe, about 50 minutes).
The point of constraints is that they make you work: you can’t use the words that first come to mind. You have to struggle to say what you want while playing by the rules them—and sometimes that makes you realize you ought to be wanting to say something else. Constraints are a tool to make you think new thoughts.
Item: I’ve become strangely fond of Twitter. It’s a service that lets you send short (140 character) “tweets” out into the ether. Other people can subscribe to (”follow”) your tweets. They can see the tweets of everyone they follow by visiting their own twitter web page (here’s mine), subscribing to an RSS feed, or using a twitter-specific app to fetch tweets. (I use Twitterrific.)
That’s form: what about content? As Twitter user shalunov (Stanislav Shalunov) puts it (in a tweet):
Four main ways to tweet: ideas, news, @-chat, phatic coffee. The last is the original, rest invented by users.
Ideas are the tweets I’m most interested in. Slalunov’s is an example of an idea tweet.
News is my second interest. As a geographically isolated person, it’s one way of knowing what interesting people are chattering about.
bilkleb, for example, is fond of linking to both technical topics (“heard Paul Tuckfield talk about scaling YouTube MySQL, “mirror disk drives in hardware, stripe in software.” (http://tinyurl.com/277dvy)”, “watching Matz’s Google Talk: http://tinyurl.com/3bzbuu“) and not-technical ones (“@barbjordan http://kluster.com looks interesting.”)
Twitter’s also a good way for me to learn about new and interesting apps for the Mac.
…and I follow lrz because he’s Apple’s Ruby guy, so what he does will affect my next book.
…and there’s also the quick links to the absurd and humorous that’ve been around since the beginning of the web, like this from andytinkham: “This is just wrong, wrong, wrong (on so many levels): http://tinyurl.com/2zjpz5“ or this from objo: “hilarious http://rubyurl.com/DdyJ (via @timbray)”.
“@-chat” is a sort of person-to-person instant messaging. For example, cypher23 wrote “Stalker is a weird and wonderful film.” I replied: “@cypher23 Harrison’s new _Nova Swing_ is in the sub-sub-genre with Stalker, _Rogue Moon_, and _Roadside Picnic_. Liking it so far.” Anyone following cypher23 would see both his tweet and my reply. Someone following only me would see only my reply (but could click on the hyperlinked cypher23 to see all his recent tweets). Because of the one-sidedness, and because the topics tend to be less interesting than those in the first two categories, I tend not to follow people who have a high proportion of @-chat in their tweets.
“Phatic coffee” is just tweeting what you’re doing now, like avibryant’s recent “obsessively refreshing UPS tracking page for new laptop” Although I’m somewhat of a hermit and not much for social chit-chat, I’m not immune to phaticality. (I find chadfowler’s heavily phatic tweets appealing, oddly puckish, and somehow soothing.) But I likely won’t follow someone who’s predominantly phatic.
Item: While writing a book, I often find myself disinclined to spend spare time writing blog posts. Yet I continue to have ideas. I’m sure lots of other people do too.
Synthesis: I’ve created a fake twitter user named pithysoft. It’s for anyone’s pithy tweets about software development. When I finish this post, I’ll send the first one: “d pithysoft Business-facing tests are like personal ads: No matter how exact your description, the reality always tells you something new.” People following pithysoft will see it. If the pithy claim intrigues them, they can tweet pithysoft with something like “@marick More about tests and personal ads, plz”. That would encourage me to write it up on my blog. When I did that, I could tweet “@pithysoft Expanded on XYX here: XYX”
An experiment. Let’s see how it goes.
Since Dawn and I are effete, latte-swilling, Obama-supporting liberals,* we don’t have a television. We do, however, watch television series on a laptop.** We’re running out and need suggestions.
Dawn and I mix up series like The Wire and Deadwood with guilty pleasures like Veronica Mars and the first few seasons of 24. We’re starting on The Corner. Sopranos didn’t grab us. From that list, it looks like character-driven dramas with season-long story arcs are good. Depressing is certainly OK.
We also started watching Joan of Arcadia and Dead Like Me, both of which later migrated to whole-family viewing.
With the kids, we’ve watched Buffy, Angel, Dark Angel, Tru Calling, and Lost.
Sophie and I have watched Battlestar Galactica, but Dawn and Paul are not wild about outer-space SF.
Suggestions?
* Effete, latte-swilling liberals, but also salt of the earth Midwesterners who have both*** delivered calves by hauling on chains.
** Think of a depression-era family huddled around a radio. Salt of the earth, like I said.
*** Well, I’ve only done it once. Not so fun I’d make it a habit.
Cory Foy has started an Agile FAQ. His first question is What is Agile? Now, I’m notorious for wandering away from definitional arguments, and I like the answers Cory already has, but I think I have something to add. I have an incomplete and informal list of questions I ask myself about teams to gauge whether they really “get it”:
Is there spontaneous chatter? (Most often work-related, but a helping of casual chatter too.)
Is there hustle?
Are people afraid of being wrong?
Do people readily ask for help? Do people readily volunteer help? Even—especially—when they could say, “that’s not my job”?
Do I notice people giving in for the sake of the group? (Such as deciding to try something someone else’s way as a way to reduce tension.)
When people talk about solving problems, do they talk in terms of nudging something that’s wrong in the direction of rightness, or in terms of solving the problem once and for all?
Is their response to a problem to increase the visibility of information? Do they seem to think that if people know about a problem, and are continuously reminded of it, that they’re likely to just naturally act to solve it?
Are they a touch monomaniacal about getting working software out there, or at least being able to show someone something new that actually works?
Do they disparage the “business side of the house”, or do they have active sympathy with the people there?
Do they act helpless? Or as if they have power? Do they give up on problems because “they” will never let them be fixed? (”They” being management, the cubicle police, the configuration management board, etc.)
Do they want to be able to take pride in their work? (Or are they cynical or passive about whatever it is they do?) And do they take pride?
I don’t care if I know what Agile is if I know it when I see it. I don’t know to what extent certain values, practices, techniques, or tools influence my answer to the question “Is this team Agile?” To some extent, for sure.
Embracing Agile had me make some big changes, some fundamental changes. As a programmer, my role model changed from the lone genius with OCD to a gregarious social animal (but still hoping for just a touch of OCD). As a tester, I stopped thinking of myself as a dispassionate judge and began to consider myself an involved insider.
For the work-obsessed, such changes are more than just new roles: they’re changes in self-conception. I think it neither unfair nor insulting to observe that some people avoid Agile projects because they prefer not to change anything that close to their core identity.
Such big changes aren’t restricted to programmers and testers. Consider the team manager redesignated Scrum Master: she’s no longer The Boss.
People talk a lot about “the Agile Enterprise.” What big changes, fundamental changes, changes in self-conception would such a thing bring to the CxO? Will it? If not, why not?
In response to a potential client, I wrote something very like the following. The interesting thing is that I’m placing more emphasis on manual exploratory testing. It’s not so much that I suddenly realize its importance as that automated business-facing tests continue to be hard to implement and adopt. More on that anon.
A short sketch of a reasonable growth path would go like this:
Get the programmers sold on test-driven design. How difficult that is depends mainly on how much legacy code you have (where legacy code is, as Michael Feathers says, code without unit tests). Legacy code is hard to test, so programmers don’t see the benefits of testing as quickly, so it requires that much more discipline to get over what’s always a higher hump than with greenfield code. (Michael Feathers’ Working Effectively with Legacy Code is the gold standard book, though there’s an important strategy—”strangler applications“—that’s not covered in depth. Also, I’m the track chair for a new Legacy Code track at Agile2008, I just asked Feathers to give the keynote, and he says he has “a number of surprising proposals about how to make things better”.)
I’ve come to feel that the most important thing to get across to programmers is what it’s like to work with code built on a solid base of tests. If they understand that early on, they’ll have a clear idea of what to shoot for, which helps with the pain of legacy code. I wrote a workbook to that end.
At the same time, move testers away from scripted manual tests (if that’s what they’re doing) and toward a more exploratory style of manual testing. The people who are strongest on exploratory testing in Agile are Jonathan Kohl, Elisabeth Hendrickson, and Michael Bolton.
As programmers do more unit testing, they will become accustomed to changing their design and adding code in support of their own testing. It becomes more natural for them to do the same for the testers, allowing them to do “automation-assisted exploratory testing”. (Kohl writes about this.) I like to see some of the testers learn a scripting language to help with that. Ruby is my favorite, for a variety of reasons. I wrote a book to help testers learn it.
Over this period, the testers and programmers should shed most animosity or wariness they have toward each other. They’re working together and doing things to help each other. It helps a lot if they sit together.
Once the programmers are sold on test-driven design, they will start wishing that the product owners would supplement what they say about what they want with clear, concrete, executable examples of what they want. That is: tests, written in the language of the business. That isn’t as easy to do as we thought it would be five years ago, but it can be done more or less well. Often, the testers will find a new role as helpers to the product owners. For example, they should get involved early enough to ask questions that lead to tests that prevent bugs (which is better than discovering the bugs after you’ve paid some programmers to implement them).
Throughout this, some kinds of testing (like performance testing) don’t change all that much. For performance testing, I trust Scott Barber.
As a side note: I’m quite fond of the new The Art of Agile Development by Shore & Warden: enough to publicly declare that I’ll bring a copy to every team I work with. Lots of good from-the-trenches experience summarized there.
Nat Pryce wrote a nice little summary of what he’s learned about evolving an API.
An API is a user interface for programmers. That means you have to develop it like a user-interface: iteratively, adapting to how the users actually use and misuse the interface in real life […]
But… programmers focus on solutions, not goals, so their feature requests will be worded in terms of how they think they would change the API to support whatever it is they want to do. You have to engage in a lot of arduous back-and-forth to find out what they are trying to achieve […]
