Exploration Through Example
Example-driven development, Agile testing, context-driven testing, Agile programming, Ruby, and other things of interest to Brian Marick
Sat, 28 Jan 2006
Working your way out of the automated GUI testing tarpit (part 7)
part 1, part 2, part 3, part 4 part 5, part 6
Where do we stand?
I want to end this series by closing one important gap. We know that links go somewhere, but we don't know that they go to the right place, the place where the user can continue her task.
We could test that each link destination is as expected. But if following links is all about doing tasks, good link tests follow links along a path that demonstrates how a user would do her work. They are workflow tests or use-case tests. They are, in fact, the kind of design tests that Jeff Patton and I thought would be a communication tool between user experience designers and programmers. (At this point, you should wonder about hammers and nails.)
Here's a workflow test that shows a doctor entering a new case into the system.
I've written that with unusual messages, formatted oddly. Why?
Unlike this test, I think my final declarative tests really are unit tests. According to my definition, unit tests are ones written in the language of the implementation rather than the language of the business. My declarative tests are about what appears on a page and when, not about cases and cows and audits. They're unit tests, so I don't mind that they look geeky.
Workflow tests, however, are quintessential business-facing tests:
they're all about asserting that the app allows a doctor to perform
a key business task. So I'm trying to write them such that,
punctuational peculiarities aside, they're sentences someone calling
the support desk might speak.
I do that not so much because I expect a user to look at them
as because I want my perspective while writing them to be
outward-focused. That way, I'll stumble across more design
Sending all messages to an object representing a
Similarly, I'm using layout to emphasize what's most important. That's what the user can do and what, having done that, she can now do next. The actual checks that the action has landed her on the right page are less important—parenthetical—so I place them to the side. (Note also the nod to behavior-driven design.)
The methods that move around (like
As you can see, I don't check much about the page. I leave that to the declarative page tests.
That's it. I believe I have a strategy for transforming a tarpit of UI tests into (1) a small number of workflow tests that still go through the UI and (2) a larger number of unit tests of everything else.
Thanks for reading this far (supposing anyone has).
The tests I was transforming didn't do any checking of pure business logic, but in real life they probably would. They could be rewritten in the same way, though I'd prefer to have at least some such tests go below the presentation layer.
There are no browser compatibility tests. If the compatibility testing strategy is to run all the UI tests against different browsers, the transformation I advocate might well weaken it.
There are no tests of the Back button. Should they be part of workflow tests? Specialized? I don't know enough about how a well-behaved program deals with Back to speculate just now. (Hat tip to Seaside here (PDF).)
Can you do all this?
The transformation into unit tests depends on there being one place that receives HTTP requests (Webrick). Since Webrick is initialized in a single place, it's easy to find all the places that need to be changed to add a test-support feature. The same was true on the outgoing side, since there was a single renderer to make XHTML. So this isn't the holy grail—a test improvement strategy that can work with any old product code. Legacy desktop applications that have GUI code scattered everywhere are still going to be a mess.
See the code for complete details.