Mon, 28 Apr 2003
Imre Lakatos and persuasion
I was talking to Mark Miller last week. He's interested in
capability
security,
I'm interested in context-driven testing and agile methods, but we're
both interested in persuading people to change their minds. Mark quoted
a common summary of
Kuhn:
"Science advances one dead scientist at a time." The old guard
dies off, and the new guard takes their place.
Quite true, in the main, but some in the old guard have to change
their minds for the new guard to get started. Some people have to
switch from being Newtonians to being relativists, or from
being Ptolemaists to being Copernicans. Why and how
does that happen?
The philosopher Imre Lakatos's
methodology of scientific research programmes talks about why and
how scientists should change their minds. He extracts rules for
rationality from a somewhat idealized history of great scientific
advances. I'm going to do violence to his intentions by not caring
(here) about rationality. Instead, I'm going to build off my reaction
to Lakatos, which is "Those sure are nice rules. They capture a lot of
what I mean when I say, 'Those people who call themselves
the Fooists are really going places, and I want to go along." If
other software people respond to the same cues as I do, then we who
want people to go along with us might benefit in two ways from
careful study of Lakatos's rules:
- We'll explain what we're doing in a more appealing way.
- We'll think about what we're doing in a way tuned for
success. We'll make better choices.
But note well that I'm not claiming a complete solution to the problem of
persuasion.
So here goes. For skimmability, I'll indent the explanation of
Lakatos differently from some of my speculations about
various software movements.
At the heart of any research programme is a "hard core of
two, three, four or maximum five postulates. Consider Newton's theory:
its hard core is made up of three laws of dynamics plus his law of
gravitation." (Motterlini 1999, p. 103)
Agile software development has
four postulates. I might
be right in saying that capability security's hard core is also
concise. It seems to be a single diagram,
the
Granovetter Diagram.
Software postulates probably can't be as crisp as F=ma, but
I think the Agile Manifesto postulates are actually quite good as a
guide to behavior. If you have a problem in a project, you would
first try to solve it by thinking about the people involved and
their interactions, only secondarily thinking about
people-independent processes and tools. If getting working software soon is
in conflict with comprehensive documentation, you go with the
working software.
I've been critical (as an insider) of the
context-driven
testing principles, but they actually stack up reasonably
well. My objection has been that principles like "good software
testing is a challenging intellectual process" or "people, working
together, are the most important part of any project's context"
are too easy to agree to. I still think that's a problem, but they
are also - when taken literally - a good guide to behavior. The
first says that if you're trying to solve a problem by reducing
the amount of thinking testers have to do, you're solving it
wrong. The second has much the same import as the Agile Manifesto's
"individuals and interactions over processes and tools".
I'd recommend the context-driven principles be restated in more of an active voice. Make
them less statements of truth and more explicit heuristics for
behavior. And maybe knock off a few. (Though I'm skeptical of
Lakatos's "at most five", I don't know if I'll ever be able to
remember all seven.)
A set of postulates starts out inadequate. The common (Popperian)
wisdom about science is that scientists posit hypotheses, specify
refutable predictions that follow, and replace the hypotheses when
those predictions fail. That's not in fact what
happens. Counterexamples are shelved to be dealt with later.
For example, Newton's theory did not correctly predict the observed
movement of the moon, but he did not discard it. When Le Verrier
discovered the motion of Mercury's perihelion was faster than
predicted by Newton, people shrugged and waited for Einstein to
explain it.
Research programmes can proceed despite their obvious
falsity. Rutherford's model of the atom (mostly empty space, electrons
orbiting a nucleus) violated Maxwell's equations, which were believed
to be rock solid. They were certainly much more compelling than the
new evidence Rutherford's model was intended to explain. But
Rutherford's programme essentially said, "We'll figure out how to
reconcile with Maxwell later." (The solution was quantized orbits -
the "Bohr atom".)
How can we use this? I made a little splash in 1999 by
attacking
the popular V model for software development. I suggested
requirements for a better model. I now think of those requirements as groping toward
agile methods, particularly
Scrum's
management
style and
XP
release planning. As is typical, my presentations were well
received by those predisposed to agree with me, and thought
scandalous by those not. Defenders of the V model tended to list
problems that projects
using my alternative might have. Rather than argue with them, I should have
changed the subject (as Rutherford did): "Of course an early
version of a model or methodology will have problems. So?
Do you expect progress with no backward steps? Or can we
talk about something useful? "
Similarly, people attempt to discount XP by saying "XP is
crucially dependent on the myth of the on-site acceptance-test-writing
customer". That's a valid criticism in the sense of being a statement
of a limitation that XP should want to tackle someday. But it should
have no more innovation-stopping power than saying, early last
century, "Well, relativity has
problems too". (Lakatos said of Einstein:
"Einstein's theory inherits 99% of [the anomalies found with Newtonian
mechanics], eliminates only 1%... As a matter of fact, Einstein's
theory increases the number of anomalies; the sea of anomalies does
not diminish, it only shifts." (ibid, pp. 100-101))
The preface
to XP Explained does the right thing: "XP is designed to
work with projects that can be built by teams of two to ten
programmers, that aren't sharply constrained by the existing
computing environment, and where a reasonable job of executing
tests can be done in a fraction of a day." (p. xviii) Specifically
eschew universal claims, and don't let people tempt you into
claiming more than that you have a set of principles that work in
some cases and that you're going to work hard to make them work
in more.
A problem that context-driven testing has, I think, is that it's
not sharply enough delineated. It clearly comes from the
shrink-wrap product world, but it's hard to draw boundaries
around it. We "contexters" are always making nods toward
highly-regulated environments, talking about how we'd write
oodles of documents if the context demanded it. I think that
weakens our impact by lashing together the contexts where we
provide novelty and innovation and ferment with the contexts where
we don't say much (yet).
But surely factual evidence counts for something. Lakatos
says it does, in two ways.
First: while "theories grow in a sea of anomalies, and
counterexamples are merrily ignored (Motterlini, p. 99)," the same is not
true of dramatic ("novel") confirmations. What convinced scientists
of Newton's theory of gravitation? According to Lakatos, it was
Edmund Halley's successful prediction (to within a minute) of the
return date of the comet that now bears his name. What
"tipped"
scientific opinion toward Einstein's theory of general
relativity? The famous experiment in which the bending of light was
observed during a solar eclipse. (Interestingly, according to
Overbye's
Einstein
in Love, the observational evidence was actually shaky.)
Sometimes critics of agility will concede that it could be
appropriate for "completely chaotic, changing environments". The
implication is that a properly managed project, with customers
who knew their own minds and markets, could do better. But if
you're hopeless anyway, perhaps agility can salvage something.
The temptation is to argue that agility is not so narrowly
applicable, but I think that takes us off into unpersuasive
meta-arguments about what methodology would be best in the
abstract. Instead, rely on the fact that chaos exists,
that people on projects view it as a problem, and that
solving this problem serves as a novel confirmation of a
surprising prediction.
Seek out what's unlikely and demonstrate success at it. Much
credibility flows from that. That's part of the rhetorical power of
origin myth projects.
Something that many project members believe unlikely is job
satisfaction. Job satisfaction is thus both a novel and testable
prediction. For that reason, I think it's good that agile
advocates predict that agile methods lead to it. (See for
example, Lisa Crispin's
writing
and how Josh Kerievsky makes Enjoyment one of the five values of
Industrial XP.)
The second way that factual evidence counts is in the way proponents
respond to it. Lakatos pictures the core postulates as being surrounded
by a protective belt of auxiliary hypotheses that are used
to handle telling counterexamples.
Newton provides an important example of the right kind of
protective belt. He was informed that observations of the moon
refuted his theory. To protect it, he invented a new
theory of refraction that, together with his laws, did predict the
moon's movement. (Sort of - it was never right, because the moon's
center of mass isn't at the center of the sphere, which throws off the
calculations.)
His theory of optics not only corrects refuting
observations to make them match the theory. It is also a theory of
its own, makes new predictions, and had some of the new predictions
confirmed. Strictly, the observations that refuted Newton's theory
of gravitation served to surprisingly confirm his theory of
optics. He knew that there were refuting observations, but he
didn't have the values, so he couldn't "work backwards" from them
to a theory of optics that served no purpose other than to
protect his hard core.
It's that "working backwards" - fitting the protection to the
specifics - that distinguishes what Lakatos calls "ad hoc" protective
hypotheses from the good kind. "[Some proposed counterexample] was
never discussed before, but now you can account for this case too by
introducing an ad hoc auxiliary hypothesis. Everything can be
explained in this way, but there is never any prediction of a
corroborated novel fact." (ibid, p. 101)
The existence of ad hoc protective belts shows why my attempts to
take down the V model were pointless. For any criticism of
the use of the V model in a specific project, one can always
invent an ad hoc explanation. It needn't ever be the fault of the
methodology. It can always be the fault of inadequate training,
improperly motivated staff, a lack of management support,
upheaval due to an acquisition... None of these explanations
provide prescriptions about what to do and also make novel
predictions.
In contrast, I don't think Scrum's
approach
to management problems is ad hoc. It makes a specific prediction:
that if project management sees its role as facing outward from
the team (clearing roadblocks from their path) rather than
inward (telling them what to do) the team will perform
perfectly fine. Moreover, Scrum addresses the criticism that
"agile doesn't scale" with a specific mechanism (the
"
scrum of
scrums"), and predicts that this mechanism alone suffices for
teams of hundreds.
To be convincing, we must scrupulously avoid ad hoc,
after-the-fact explanations of why something didn't
work. Instead, we must either forthrightly admit that we don't
know how to deal with problem X, or we need to make
predictions much more specific than "get proper training".
And forget about flailing after things like the V model. While such attacks
do serve to cement the solidarity of those already on your side, they
are one of the less effective ways of convincing the unconvinced.
Research programmes, even ones as successful as Newton's, eventually
degenerate. A programme "is degenerating if ... (1) it does not lead
to stunning new predictions (at least occasionally...); (2) if all its
bold predictions are falsified; and (3) if it does not grow in steps
which follow the spirit of the programme." (ibid, p. 106) (This last seems
to mean avoiding the ad hoc additions mentioned above. But I think
there's also a less pin-down-able sense. It would not be in the spirit
of an agile method to extend itself by adding more and more examples
of written process documentation.)
I take this to mean that research programmes, like sharks, have to
keep moving or die. It would not be enough for agile methods to
confine themselves to small projects where there's rapid change,
else they would drift out of the software development
attention
space (see commandment 4). The agile methods have to keep
trying to push the boundaries of applicability. They need to
report success in new types of projects, with a reasonable
argument that the success is due to new practices or extensions
of old practices.
P.S.
The citations are from Lakatos's "Lectures on Scientific
Method", in
For and Against
Method, Matteo Motterlini (ed.).
I should be citing Lakatos's
Philosophical Papers,
Vol. 1: The Methodology of Scientific Research
Programmes. But when I wrote a long email on this same
topic, some years ago, I'd just happened to pick up Motterlini.
## Posted at 08:10 in category /misc
[permalink]
[top]
|
