This is a book review posted in comp.software.testing in November, 1994.
Brian Marick, The Craft of Software Testing, Prentice Hall, 1995, 553 pages, $47, ISBN 0-13-177411-5.
People on this list have seen occasional references to this book, which is at long last finished and available. I thought I would try to write a reasonably unbiased description. I'll mention weak points in the book as a way of forcing me to write -- and distribute -- improvements.
My main goal for the book was to solve a problem I saw with existing books: they were not specific enough. They required too much invention and "filling in the blanks" from the reader. I wanted to fix that, to present testing techniques in complete detail and to tie them together into a sensible "cookbook" process that would work reasonably well in most situations.
A secondary goal was to publicize two semi-original ideas of mine: making a clear distinction between deciding what needs testing (test requirements) and designing tests; and the use of test requirement catalogs.
Audience: the book is subtitled "subsystem testing". The subtitle is serious. This book does not attempt to describe all types of testing. It concentrates on "testing in the medium": moderate sized subsystems such as device drivers, class libraries, optimization phases in compilers, individual protocol modules, etc. A typical reader will be a developer testing his or her own code. Independent testers who can look at the code are a second audience. These techniques and this process can be adapted to system testing (I teach a course that does that), but there are many important blanks a system tester would have to fill in.
The book is divided into six parts:
This section describes the basic subsystem testing process in gory detail. It uses as a running example a single subroutine from L. Peter Deutsch's Ghostscript program. That's a small example, but it's at least real code, even somewhat tricky code, with a real bug.
This section describes how to adopt these techniques. The assumption is trying to adopt everything all at once is a mistake. You're better off starting small - spending a little effort/money, getting modest improvements, and being encouraged to continue - than trying to do everything all at once, getting frustrated, and giving up.
This section extends the basic technique in more realistic directions: testing big subsystems, what to do when you don't have all the information used in Part 1, testing bug fixes, and what to throw out when you don't have time to do everything.
A grab-bag of chapters extending the basic ideas to particular situations: syntax testing, testing consistency checkers, state machines and statecharts, and object-based and object-oriented software. Oops -- the publisher would want me to say, "testing object-based and ==>OBJECT-ORIENTED<== software".
Some more advanced wrinkles on the basic techniques. Optional.
These are the parts I use often: reference catalogs and checklists for day-to-day testing.
Does the book succeed at its goals? Reasonably well, I think, though better at the secondary goals. People who take my courses like the idea of test requirements and are usually enthusiastic about the idea of test requirement catalogs.
And the primary goal? Testing really is a craft, just like carpentry or mathematics. It's hard to learn a craft from a book. It's better to learn it in person, by watching someone else do it and then having that someone comment on your attempts. So, if you buy the book, you'll find the occasional blank to fill in, and it may take you some time to puzzle out something that I could clarify in a minute if I were sitting beside you.
The final test is whether whether the time spent reading the book pays off in smoother testing and bugs found earlier. I believe that, for most people, it will, so I'm content.
There are three larger blanks that I regret and promise to fill:
The chapter on test implementation has no discussion of maintainability of test suites, which is critically important.
I have some other minor quibbles.
I'm sure you'll have your own objections if you read the book. Let me know: firstname.lastname@example.org/testing-com