Mon, 09 Jan 2006
Working your way out of the automated GUI testing tarpit (part 6)
part
1, part
2,
part
3,
part
4
part
5
Here, I dispose of another reason to run tests through the GUI:
bad links and other ways of getting to pages. These bugs can be
found with unit tests instead. The mechanism fits in
well
with business-facing
test-driven design.
Let's start with a bug. In build 343, an Activity Summary
page is added to the app. Links to that
page are added to thirteen other pages. In build 582, someone
changes the URL of the Activity Summary page and dutifully changes
twelve of the thirteen pages that link to it. It's a user who finds
that the thirteenth link wasn't updated.
A link-checking program won't find all such bugs because it probably
can't get to all the pages of the program. So, the claim is, you
should have a GUI testing tool traverse every link. Here, I'll change the sample app to show a better way.
Because I was frightened
by DTML
as a small child, I lean away from template languages with embedded
code and toward code that generates XHTML. (We can argue the merits of the two
approaches another day.)
My Renderer class is nothing fancy. A bunch of core methods
generate simple XHTML. From them, I've built up more complicated
methods, such as the ones used here:
def case_display_page
case_record = @app.current_record
page("Case #{case_record.clinic_id}",
p("Owner: #{case_record.client}"),
visit_list(case_record.visits),
audit_list(case_record.audits),
add_visit_button,
add_audit_button)
end
|
Now suppose I want to add a help link to that page, using a method
called help_link_for(topic). Here's a simple
implementation of that method:
def help_link_for(topic)
%Q{<a href="javascript:standard_popup('help?topic=#{topic}')">Help</a>}
end
|
The method generates a link to a javascript popup, but I think it
should also check that the topic exists, like this:
def help_link_for(topic)
assert(@app.has_help_for?(topic), "Creating link to nonexistent link #{topic}.")
%Q{<a href="javascript:standard_popup('help?topic=#{topic}')">Help</a>}
end
|
has_help_for? checks that the topic exists, using the
same mechanism that the help action uses
to find the help to display. Therefore, you do not need
to follow the link to discover that it's bad, you merely need
to generate it. Which means generating the page that contains
it. Which we already do with a fast renderer unit test:
def test_typical_case_display_page
given_app_with {
case_record('clinic_id' => 19600219)
}
when_rendering(:case_display_page) {
assert_page_title_matches(/^Case 19600219/)
assert_page_has_action(:want_add_visit_form)
assert_page_has_action(:want_add_audit_form)
}
end
|
The test doesn't explicitly check the help link, but it doesn't
have to: the renderer assertion will nevertheless check it for
us. Here's what will happen if the link is bad:
2) Error:
test_typical_case_display_page(CaseDisplayPageTests):
StandardError: Programmer error. Creating link to nonexistent page "bogus_page". Please report this error to bugs@example.com.
./util.rb:3:in `assert'
./renderer.rb:106:in `help_link_for'
./renderer.rb:82:in `case_display_page'
|
(Note: I later added an explicit assertion that the help link
exists because I consider it an essential part of the page. The
implicit check only fails if the link exists but is bad; the
explicit assertion fails if it doesn't exist at all.)
The link-creation routine checks that the particular help topic exists, but it
doesn't check that "help" is the right action to get to the help
pages. It's easy to ask if
the app responds to an action named
help. Use this code: @app.respond_to?('help'). So I
could add another assertion to help_link_for, but I'd
like to handle the risk of an incomplete renaming in a different
way. To get there, let me start a seeming digression and fix that
long-standing bug
in our program (that it prompts you with a button to add an audit
even when no more audits are allowed).
Here's the code that adds the button to the page:
def add_audit_button
p(command_form('want_add_audit_form',
submit('Add an Audit Record')))
end
|
The renderer could ask the app before generating
the add_audit form, like this:
def add_audit_button
return unless @app.further_audits_allowed?
p(command_form('want_add_audit_form',
submit('Add an Audit Record')))
end
|
And, since I'm changing the method anyway, I might as well have it make sure
that want_add_audit_form is an action the app responds to:
def add_audit_action
assert(@app.respond_to?('want_add_audit_form'), ...)
return unless @app.further_audits_allowed?
p(command_form('want_add_audit_form',
submit('Add an Audit Record')))
end
|
But that's starting to bug
me. I'm asking
the App more and more, not telling it. Is this Feature
Envy? Do I want to worry that other methods that generate this
action will have
to duplicate
the knowledge of which checks are appropriate?
It seems to me that the renderer should hand a potential
presentation to the app and ask it to apply whatever rules are
relevant, but in a way that insulates the app from any knowledge of
the presentation (that it'll be in XHTML, etc.). That can be done using a closure as a callback:
def add_audit_button
@app.fill(:template_for_want_add_audit_form) { | action |
p(command_form(action,
submit('Add an Audit Record')))
}
end
|
The App would look
like this:
def fill(template_name, *args, &block)
self.send("fill_#{name}", *args, &block)
end
def fill_template_for_want_add_audit_form(&block)
return unless current_record.accepts_more_audits?
block.call(checked(:want_add_audit_form))
end
def checked(action_name)
assert(respond_to?(action_name),
"#{action_name} is not a defined action.")
action_name
end
|
fill bounces the work off to a particular
method. That checks whether the action is allowed by the business
presentation rules. If not, it
returns nil (which renders as nothing). Otherwise, it
passes the correct action name to the closure (after checking that
no one's renamed it out from under us) and lets that closure
render away.
(Note: the renderer could
call fill_template_for_want_add_audit_form directly—the
same knowledge is required—but this form seemed more
convenient for unit tests.)
This division of responsibility works well with test-driven design.
The customer says "You shouldn't be able to add any audits if
the last audit was nominal." After discussion, everyone agrees
the story is to leave the "Add Audit" button off the Case
Display page and update that page's help with an explanation.
There's an existing test that checks everything important about
the Case Display page. (It's test_typical_case_display_page, above.) A new test is written that claims the Add
Audit button is missing when the last audit is
nominal. Like test_typical_case_display_page, it
avoids fiddly details of XHTML structure.
Making the test pass is going to require some new business logic. That leads to
three unit tests describing how
fill
responds when client code asks it to fill in a
template_for_add_audit_form:
- if there are no audits, the template is filled in (with
the right action name),
- if there's an audit with nominal variance, nil is
returned instead of the filled-in template, and
- if there are n audit records (none nominal), the
template is filled in.
None of these tests refer to text at all, much less XHTML
text.
Those tests are made to pass.
The original test should now pass. If it doesn't, that means the
renderer doesn't call the app to judge the template. How is this
possible, since it's supposed to always use this mechanism to
get the action name? Bad
renderer! But easily fixed.
The story's not done until the Customer sees the new version of
the Case Display page, probably by walking through the workflow of creating a
nominal audit and then observing that there's no button to
create another. That might lead to tweaks of the presentation,
especially those aspects not important enough to be described in
a test.
If the Customer wants, the same business rule can be used to check incoming
actions. (Just because we don't provide a form to let
people add to nominal audits
doesn't mean that someone couldn't
send the
appropriate HTTP anyway.)
(As usual, I should note that I have not seen these ideas applied at
the scale of a real app. If I ever have time to create a Giant
Microbes fan site for my kids, I'll explore them further.)
At long last returning to the help popup, I can
change the code that
generates the link to this:
def help_link_for(topic)
@app.fill(:template_for_help_link, topic) { | action |
%Q{<a href="javascript:standard_popup('#{action}?topic=#{topic}')">Help</a>}
}
end
|
The App code that would rule on the template would be:
def fill_template_for_help_link(topic, &block)
assert(has_help_for?(topic),
"Creating link to nonexistent help topic '#{topic}'.")
block.call(checked(:help))
end
|
Any unit test that generated a help link would auto-check for a bad action or bad topic. It
would not check whether the
javascript standard_popup routine pops up a window,
pops up a reasonably-sized window, pops it up somewhere not
annoying, etc. That could be tested
with JsUnit, Watir,
or Selenium. Personally, I'd
just test it by hand and trust myself to retest it if I change it.
One final note: we are still working our way out of the
tarpit. I haven't stressed it in this installment, but both of the
old-format tests continue to work. As always, the goal is to
gradually reduce the need for
slow and fragile tests.
See
the code
for complete details.
## Posted at 11:51 in category /testing
[permalink]
[top]
|
