Outgrowing rhetoric-heavy business-facing tests
Tools like Fit/FitNesse, Cucumber, and my old Omnigraffle tests allow you to write what I call “rhetoric-heavy” business-facing tests. All business-facing tests should be written using nouns and verbs from the language of the business; rhetoric-heavy tests are specialized to situations where people from the business should be able to read (and, much more rarely, write) tests easily. Instead of a sentence like the following, friendly to a programming language interpreter:
given.default_payment_options_specified_for(:my_account)
… a test can use a line like this:
I have previously specified default payment options.
(The examples are from a description of Kookaburra, which looks to be a nice Gem for organizing the implementation of Cucumber tests.)
You could argue that rhetoric-heavy tests are useful even if no one from the business ever reads them. That argument might go something like this: “Perhaps it’s true that the programmers write the tests based on whiteboard conversations with product owners, and it’s certainly true that transcribing whiteboard examples into executable tests is easier if you go directly to code, but some of the value of testing is that it takes one out of a code-centric mindset to a what-would-the-user-expect mindset. In the latter mindset, you’re more likely to have a Eureka! moment and realize some important example had been overlooked. So even programmers should write Cucumber tests.”
I’m skeptical. I think even Cucumber scenarios or Fit tables are too constraining and formal to encourage Aha! moments the way that a whiteboard and conversation do. Therefore, I’m going to assume for this post that you wouldn’t do rhetoric-heavy tests unless someone from the business actually wants to read them.
Now: when would they most want to read them? In the beginning of the project, I think. That’s when the business representative is most likely to be skeptical of the team’s ability or desire to deliver business value. That’s when the programmers are most likely to misunderstand the business representative (because they haven’t learned the required domain knowledge). And it’s when the business representative is worst at explaining domain knowledge in ways that head off programmer misunderstandings. So that’s when it’d be most useful for the business representative to pore over the tests and say, “Wait! This test is wrong!”
Fine. But what about six months later? In my limited experience, the need to read tests will have declined greatly. (All of the reasons in the previous paragraph will have much less force.) If the business representative is still reading tests, it’s much more likely to be pro forma or a matter of habit: once-necessary scaffolding that hasn’t yet been taken down.
Because process is so hard to remove, I suspect that everyone involved will only tardily (or never) take the step of saying “We don’t need cucumber tests. We don’t need to write the fixturing code that adapts the rhetorically friendly test steps to actual test APIs called by actual test code.” People will keep writing this:
… when they could be writing something like this:
That latter (or something even more unit-test-like) makes tests into actual code that fits more nicely into the programmer’s usual flow.
How often is my suspicion true? How often do people keep doing rhetoric-heavy tests long after the rhetoric has ceased being useful?
February 13th, 2012 at 9:30 am
I guess we write rhetorical tests, in that we write a lot of FitNesse tests in tabular form or in the DoFixture, we haven’t done BDD given/then/when style (though I like that style).
Our customers really don’t want to look at the tests. We do use whiteboards and discussion to get the examples of desired/undesired behavior, user scenarios, biz rules. Then we turn them into tests that work for us.
The biggest value of the test is the collaboration between programmers and testers. We are now doing this approach with GUI tests as well. We are using the page object pattern with Se & Webdriver, the programmers write the page objects using the Geb framework. It’s a new experiment but so far, working really well. Before, the testers wrote all the GUI tests, that was ok, but they aren’t the best designed tests ever.
February 13th, 2012 at 11:19 am
I have had similar experiences. In the beginning of the project those rhetoric-heavy tests were looked together with PO. After few months everything went so smoothly that and in the planning sessions we just went through the examples with PO and wrote few bullet points to our backlog tool. The discussion and those bullet points were enough to get common understanding.
Still during the development we wrote down those test cases using Concordion. I had a discussion with a team about the situation. There would have been a lot of effort to convert existing cases to Java code. Also pure Java tools lacked lot of things that were possible in rhetoric-heavy tests (like parameterized tests). And for some reason all the developers had started to like nice reports we got from Concordion. So we decided to stick to use that even though it was mainly for developers anymore.
Looking back. If we would have had an option to use language that supports DSL better (like Scala or Ruby) things might have been different.
February 14th, 2012 at 11:28 am
We gave up on what you are calling “rhetoric heavy” tests a few years ago. We’ve found that folks willing to** read tests can read our ‘normal’ spec-style tests, and the folks that can’t read those aren’t willing to read through any tests. That said, by consolidating on specs, we do put a bit more work than normal into making sure that they are sufficiently readable to non-ruby programmers, and when practical non programmers.
February 14th, 2012 at 11:29 am
** = I’ve yet to have a customer that I didn’t need at least need to convince that it’s worth the time and effort to review our tests and/or execute code-level test suites.
March 7th, 2012 at 11:41 am
I’m a fan of so-called “business facing” tests. For me the value is not that the business reads them (I’ve seen them read them a few times, but most of the time not) but in encouraging the test authors to think from a business point of view and a test case point of view.
I have yet to find something better than tables to encourage people to think in terms of inputs/outputs/test cases. When I tell my team to use Fitnesse, and only use ColumnFixture, and leave out any data that isn’t relevant to what they’re testing, their tests come out looking like they were written by a tester. When I don’t do this, the tests come out looking like they were written by a developer, making it impossible to assess functional coverage.