Too much testing
Good points, entertainingly written, by Jason Gorman.
Since he’s a programmer enthused with testing, and allows exploratory testing in his worldview, and is in cahoots with Jonathan Kohl, I hope he figures out exploratory testing at the xunit level. The very flow that TDD provides acts as blinders on my ability to see anything other than my current path. I have to pull back and shift focus. I don’t know how to do that well.
For example, I just added groups to my fledgling Graffle parser. Previously, the object at the end-point of a line had to be at the same nesting level as the line itself (since there was only one level). When I wrote the code to find the end point in a multi-level document, a test that surprisingly didn’t pass after I wrote the code made me realize a null check needed to be a responds_to? check. I made the change and moved on.
Is that all I should have realized? Maybe the unexpected failure means that there’s something odd in the way I’m thinking about things that are Nestable and things that are Containers and things that are both. But how do I know, at a particular point, whether I should stop to think or move on? And how should I think when I stop? What’s the equivalent of the mnemonics and rules of thumb that James Bach and Michael Bolton are famous for?
More than a decade ago, I had some modest heuristics in my The Craft of Software Testing. But those were derived from looking at bugs in big chunks of C code, and they were concentrated on the question “did the programmer do what she intended?” Today, that question is (among better programmers) “yes” often enough that different ones have become more prominent: “has the programmer grasped the problem? has she intended to do the right thing? what has she overlooked?”
More research is needed. More discussion, too. Another excuse for a mini-conference.