Exploration Through Example

Example-driven development, Agile testing, context-driven testing, Agile programming, Ruby, and other things of interest to Brian Marick

191.8

⇒

167.2

⇒

186.2

183.6

184.0

183.2

184.6

Mon, 20 Feb 2006

I need a break

The Big Visible Chart of my weight at the top of the blog has been red for too long. This has been a stressful month and a lousy, lousy week, with stress coming in from multiple directions and time to exercise coming from nowhere. The added stress of falling short of the two-pounds-per-week goal every week is working against meeting it. It's the wrong kind of feedback. Therefore, from next week until things get better, green will mark any weight below 170, red any weight over 173, and grey the range between. I expect no better than grey.

## Posted at 18:11 in category /misc [permalink] [top]

End-to-end tests and the fear of truthiness

The American Dialect Society voted "truthiness" the 2005 word of the year. It "refers to the quality of preferring concepts or facts one wishes to be true, rather than concepts or facts known to be true." For me, 2006 is turning into the year of replacing end-to-end tests with unit tests. One risk to face is that unit tests can play into truthiness. This picture illustrates the problem:

Everything seems fine here. The tests all pass. What the picture doesn't show is that the Widget tests require the strings from the Wadget to be in ascending alphabetical order. The fake Wadget dutifully does that. The Wadget tests don't express that requirement, so the real Wadget isn't coded to satisfy it. The strings come back in any old order.

Truthiness would be wishing that unit tests add up to a working system. But the truth is that those two units would add up to a system like this:

We know that those sorts of mismatches happen in real life. So we should fear unit tests.

More tests are a proper response to fear. Hence the desire to wrap the entire chain above in an end-to-end test that 'sees what the user sees'. However, such tests tend to be slow, fragile, etc. So I want to replace them with smaller tests or other methods that are fast, robust, etc., thus reducing the need for end-to-end tests to a bare minimum.

Two such methods are:

1. Value Objects can be the authority over data format. Suppose neither the fake nor real Wadget returned an array of strings but rather an InventoryNames object. Before, both the real and fake Wadget were supposed to know about order. Now only one object needs to. The requirement on Wadget turns into two requirements: the first that it use InventoryNames, and the second that InventoryNames always yields names in the right order.

2. I earlier described one way for any code that generates <form action="want_new_case_form"...> to check at runtime whether there's any code corresponding to want_new_case_form. Because of that, a unit(ish) test that generates a page automatically checks link validity.

I expect there are a host of other tricks to learn (but I'm not at this moment aware of places where they're written down). What's seems to me key is to take the strategy of "something could go wrong somewhere, so here's a kind of test with a chance of stumbling over some wrongness" and replace it with (1) a host of tactics of the form "this could go wrong in places like that, so here's a specific kind of test or coding practice highly likely to prevent such bugs" and (2) a much more limited set of general tests (including especially manual exploratory testing).

P.S. I don't like the word "truthiness." It seems statements should have truthiness, not people. A question for you hepcats out there who are down with the happening slang: which is more copacetic, "that's a truthy statement" or "that's a truthish statement"?

## Posted at 18:11 in category /testing [permalink] [top]