|
Testing Foundations
|
an article by Marc Rettig, in Communications of the ACM, May
1991, Volume 34, Number 5
reviewed by Brian Marick on September 14, 1996
From the article:
"Testing is odious to most programmers. It is
tedious, never-ending, and just plain no fun. When our team
embarked on its current project, we decided to look testing full
in the face and try to implement a formal testing process. In my
opinion we have struck a pleasant balance between rigor and
tedium, making software testing both palatable and effective."
Over the past five years or so, I've seen more and more programmers testing their own code. Is that because programmers have developed a compulsion to test since Rettig wrote his article? Well, no. To explain the reason, I present the timeline below. It traces perhaps a day's worth of work for several people, spread over maybe a month of calendar time:
This is insane. The overhead of communication, task disruption, and getting back up to speed is ridiculous. (If you found it hard to keep context because there were too many interconnections and too small a window to view them in, think how the three people involved feel.) It's much cheaper to have the developer who created the bug find it soon thereafter (or, using similar techniques, find it before it's made). Independent testers should be reserved for where they can add unique value, doing the types of testing programmers rarely do well.
Arguing economics is all very well, but programmers still don't like to test. So one major job of the development lead is to motivate programmers to test. This paper can help. It's exactly what's needed to persuade a programmer: a testimonial from another programmer who reluctantly did something and was pleased to find that it worked.
One warning: the paper is eloquent about the benefits of automated testing, but it doesn't discuss the dangers. If you think developers hate to write tests, you should see how much they like to maintain them. Automated unit tests of the sort described by Rettig tend to disappear after a release or two. (It would be interesting to revisit his code and see how well the tests have survived. At the point the paper was written, the code had been through staged mini-releases, but hadn't yet been released to customers, much less made it to version 2.0.) Automated developer tests are usually a waste of time unless planning for maintainability is explicit and careful.
Click on the task name to see other works that address the same task.
Developer testing is not the only way to reduce the cost of bugs. The daily build process described in Microsoft Secrets will also help avoid the insanity of the diagram above.