|
Testing Foundations
|
"Negotiating Testing Resources: A Collaborative Approach"
by Cem Kaner. A web
page, also in the Proceedings of Quality Week
1996
reviewed by Brian Marick on September 28, 1996
From the article:
"My objective is to facilitate a corporate consensus on testing tasks, priorities, and times. That is, I want to end up with:
a bottom-up task list of every test-related task or area to be tested that requires a day or more of testing;
corporate agreement on the priority of each task. I want to know the desired depth of testing;
a realistic assessment of the amount of time required for each task, or area of testing (at the agreed level of depth);
sufficient resources (time, money, people) to meet the testing requirements;
a method for tracking performance against the assessment, and a means of updating the assessment when estimates turn out to be imperfect."
This article provides some information on how to estimate a testing task, but Kaner's emphasis is on a different problem: you've estimated that you could do 60 tester years worth of work. You have three tester years budgeted. How do you decide what won't get done?
Kaner argues strongly that this is not a decision the testing group should make alone. It's a decision about where in the product bugs will be missed, and making it well requires predicting the consequences of those bugs. Testing staff can't predict as well as the staff who will be directly affected: the customer support people who will have to answer the resulting calls, the programmers who will have to fix the bugs later rather than sooner, the marketers who know what magazine reviewers will be looking for. Effective testing also means targeting effort to where bugs are most likely to be found; again, testers don't have complete information. Programmers know what parts of the design were hard or rushed, technical writers can explain what features were most confusing to explain (such features are often buggy), and customer support staff know something of how customers have found bugs in the past.
Kaner provides an admirably pragmatic step-by-step procedure for getting the required information and producing a consensus.
Click on the task name to see other works that address the same task.
"Practical Priorities in System Testing", by Nathan H. Petschenik, gives three rules for deciding where product testers should concentrate their effort. Petschenik's rules do not give as detailed a schedule as Kaner's process. Use them as a cross-check: if you follow Kaner's process and end up with a plan that violates Petschenik's rules, you should wonder why.