Archive for the 'programming' Category

Fertile assets

One result of the technical debt workshop was that the attendees are more likely to talk about the code as an asset to preserve, improve, or destroy than as evidence of debt. See Mike Feathers on code as an asset or Chris McMahon on a project as an investment. Here’s my twist:

Consider a vegetable garden or a forest of pulpwood. The immediate value of either is what you get out of them at harvest time. That’s akin to the value a business gets when the deployed product gets used or a delivered product gets bought.

But there’ll be other harvests, and the way you prepare for this harvest affects the ones that follow. Over time, a well-tended garden gets better and better at growing vegetables (because the soil is looser, you’ve dug under organic matter, etc.). If you tend it poorly–just push seeds into the ground–it’ll never yield what it could.

Product code is like such a fertile asset: the programmers can tend it well or poorly as they prepare for release, and the result will only become obvious over time. The big difference is that the fertility of code varies a lot more than the fertility of a backyard garden. No matter how sloppily I treat my garden, I can’t ruin it as completely as sloppy, rushed coding can.

Barriers to acceptance-test driven design

At the AA Functional Test Tools workshop, we had a little session devoted to this question: Even where “ordinary” unit test-driven design (UTTD) works well, acceptance-test driven design (ATDD) is having more trouble getting traction. Why not?

My notes:

  1. Programmers miss the fun / aha! moments / benefits that they get from UTDD.

    1. Especially, there is a difference in scope and cadence of tests. (”Cadence” became a key word people kept coming back to.)
    2. Laborious fixturing, which doesn’t feel as valuable as “real programming”.
    3. No insight into structure of system.
  2. Business people don’t see the value (or ROI) from ATDD

    1. there’s not value for them personally (as perhaps opposed to the business)
    2. they are not used to working at that level of precision
    3. no time
    4. they prefer rules to examples
    5. tests are not replacing traditional specs, so they’re extra work.
  3. There is no “analyst type” or tester/analyst to do the work.

  4. There is an analyst type, but their separate existence (from programmers) leads to separate tools and hence general weakness, lack of coordination

  5. There’s no process/technique for doing ATDD, not like the one for UTDD.

  6. ATDD requires much more collaboration than UTDD (because the required knowledge and skills are dispersed among several people), but it is more fragile (because the benefit is distributed - perhaps unevenly - among those people).

  7. Programmers can be overloaded with masses of analyst- or tester-generated examples. The analyst or testers need to be viewed as teachers, teaching the programmers what they need to know to make right programming decisions. That means sequences of tests that teach, moving from simple-and-illustrative, to more complicated, with interesting-and-illuminating diversions along the way, etc.

What’s special about teams with low technical debt?

Notes from a session at the workshop on Technical Debt. It was organized around this question: Consider two sets of Agile teams. In the first set, the teams do a fine job at delivering new business value at frequent intervals, but their velocities are slowly decreasing (a sign of increasing technical debt or a declining code asset). The other teams also deliver, but their velocities are all increasing. What visible differences might there be between the two sets of teams?

The following are descriptions of what’s unique about a typical decreasing-debt team:

  • Most of the team produces 98% clean code all the time, but there is some untidiness (whether due to lack of knowledge, lack of effort, or whatever). However, one or two people take one or two hours a week doing a little extra to compensate. (Chet Hendrickson)

  • Behavior in relation to the code is not apprehensive. They’re not scared of the code they’re working on, they’re not afraid of making mistakes that break the code base. (Michael Feathers)

  • They’re frequently talking about the state of the code. There are many give-and-take technical discussions. They might be heated or not, but participants are always alert and engaged. The content is more important than the tone: they are discussing new things rather than rehashing old issues. The ongoing conversation is making progress. (Many)

  • There are no big refactorings; instead, there are many small ones. (Ron Jeffries?)

  • No code ownership. (I forget)

  • Time is spent on building affordances for the team’s own work. In Richard P. Gabriel’s terminology, they spend time making the code habitable: the code is where they live, so they want it to be livable, put things where they are easy to find, make them easier to use next thing. (Matt Heusser and others)

This isn’t a complete list, only the ones that struck me and that I remembered to write down.

Technical debt: debt and history

I’ve been beating my head against the technical debt metaphor a bit these days. Here’s one thing that oozed out.

Preface: I’m assuming there are two kinds of debt. One is frivolous debt; the other is debt-as-investment. When debt is an investment, you have a reasonable expectation that a bulk sum of money now will allow you to start something that will, at some future time, produce excess cash greater than the cumulative interest on the debt. For example, you borrow money to buy a car because it lets you choose from more—and more kinds of—jobs. More opportunity means greater lifetime earnings than if you rode the bus while saving for a car.

Business pressure to add technical debt to a code base usually is couched in terms of debt-as-investment: “If we miss that market window, we’re out of business, so it won’t matter how scalable the code is.” (Whether the expectations are reasonable, and whether the same parties pay for the debt as get the increased returns, I leave for another day.)

Frivolous debt is different. For example, John McCain and his wife recently had USD10,000-15,000 in credit card debt at 25.99% interest. There’s no reason for them to pay that stiff an interest rate, but what the hell: she made more than USD6,000,000 in 2006, so why sweat interest charges? More commonly, though, frivolous debt does matter, and it comes from ignorance. In code, for example, technical debt can accrue because people simply don’t know guidelines like “Tell, don’t ask” or “Don’t repeat yourself“.

In what follows, “debt” means “investment debt.”

The analogy to a growing GDP

We imagine that debt-free code is without history. Suppose that, during development, classes Foo and Bar split into Foo, Bar, and Baz, and some of Foo’s methods were merged into Bar. If you were coming on the project, I should be able to explain why Foo, Bar, and Baz make sense without referring to their history.

History only comes up where there’s technical debt. It makes you say things like “there’s duplication between Foo and Bar because we didn’t realize fact X until new story Y and we haven’t time to finish merging their behavior.”

I have a sense that telling that story isn’t just a way of making excuses: it’s actually important for being able to work with the code in the presence of debt.

But, because we have this history-free ideal, we don’t pay attention to that. Maybe embedding history in the code (somehow) is a way of increasing the debt load the team is capable of supporting in perpetuity.

So, where I had been thinking about how to reduce debt, now I’m thinking about how to make it matter less. Like there are two ways to solve the problem of an insupportable national debt: you can pay it down or you can grow the economy.

FlexMock and RubyCocoa

I struggled using mocks in RubyCocoa. Here’s what I learned, using FlexMock.

I’m writing some tests to understand and explain the RubyCocoa interface to key-value observing. Key-value observing is yet another way for one object to observe another. (I’ll call these two objects the @watcher and the @observed.)

If I were describing a key-value observing test over the phone, I’d describe it as having four steps:

  1. Set up the relationship between the two objects.

  2. Change an attribute of @observed.

  3. During that change, expect some @watcher method to be called. One of the arguments will be an NSDictionary that contains the values before and after the change.

  4. Also expect that the value really truly has been changed.

Here’s how I’d describe it in code:

  def test_setting_in_ruby_style
    newval = 5
    during {
      @observed.value = newval
    }.behold! {
      this_change(:old => @observed.value, :new => newval)
    }
    assert_equal(newval, @observed.value)
  end

I have an idiosyncratic way of using mocks in Ruby. Notice the order in my English description above. I say what the test is doing before I say what happens during and after that action. But mock expectations have to be set up before the action. In Java (etc.) that requires steps 2 and 3 to be reversed. In Ruby, I can use blocks to put the steps in the more normal order. (You can see the definitions of during and behold! in the complete code.)

Since each of the tests sets up almost the same expectations, I hived that work off into this_change:

  def this_change(expected)
    @watcher.should_receive(:observeValueForKeyPath_ofObject_change_context).
             once.
             with(’value‘,
                  @observed,
                  on { | actuals |
                       actuals[:old] == expected[:old] &&
                       actuals[:new] == expected[:new]
                  },
                  nil)
  end

That’s a fairly typical use of FlexMock. I specify three arguments exactly, and I use code to check the remaining one. (There are components in the actuals NSDictionary that I don’t care about.)

Now I have to start to do something special. Here’s setup:

  def setup
    super
    @observed = Observed.alloc.init
    @watcher = flexmock(Watcher.alloc.init)
    @observed.addObserver_forKeyPath_options_context(
                @watcher, value‘,
                OSX::NSKeyValueObservingOptionNew | OSX::NSKeyValueObservingOptionOld,
                nil)
  end

Normally, an argument to flexmock just provides a name for the mock, useful in debugging. However, key-value observing only works when the @watcher object inherits from NSObject. So I pass in an already-initialized object for FlexMock to “edit” and add mockish behavior to. It’s of class Watcher, which inherits from NSObject. But why not just use an NSObject?

I could, except for a helpful feature of key-value observing. Before each call into the @watcher, key-value observing checks if it responds to the method observeValueForKeyPath_ofObject_change_context. If not, it raises an exception.

If I just passed in an NSObject and gave FlexMock an expectation that it should_receive an observeValueForKeyPath_ofObject_change_context message, FlexMock would just attach that expectation to a list of expectations. It would not create any method of that name; rather, method_missing traps such calls to the method and then checks off the appropriate expectation.

Therefore, I have to create a Watcher class for no reason other than to contain the method:

class Watcher <  OSX::NSObject
  def observeValueForKeyPath_ofObject_change_context(
             keyPath, object, change, context)
  end
end

The method doesn’t have to actually do anything; it’s never called. Instead, FlexMock redefines it to trampoline a call to it over to message_missing.

You can find the rest of the code here: mock-example.rb.

Hope this helps.

Security mindset

A continual debate on the agile-testing mailing list is to what degree testers think differently than programmers and are therefore able to find bugs that programmers won’t. Without ever really resolving that question, the conversation usually moves onto whether the mental differences are innate or learnable.

I myself have no fixed opinion on the matter. That’s probably because, while vast numbers of testers are better than I am, I can imagine myself being like them and thinking like them. The same is true for programmers. In contrast, I simply can’t imagine being the sort of person who really cares who wins the World Cup or whether gay people I don’t know get married. (I’m not saying, “I couldn’t lower myself to their level” or anything stupid like that—I’m saying I can’t imagine what it would be like. It feels like trying to imagining what it is like to be a bat.)

However, I’ve long thought that security testers are a different breed, though I can’t articulate any way that they’re different in kind rather than degree. It’s just that the really good ones are transcendentally awesome at seeing how two disparate facts about a system can be combined and exploited. (A favorite example)

Bruce Schneier has an essay on security testers that I found interesting, though it doesn’t resolve any of my questions. Perhaps that’s because he said something I’ve been thinking for a while:

The designers are so busy making these systems work that they don’t stop to notice how they might fail or be made to fail, and then how those failures might be exploited. Teaching designers a security mindset will go a long way toward making future technological systems more secure.

The first sentence seems to make the second false. When I look back at the bugs I, acting as a programmer, fail to prevent and then fail to catch, an awful lot of the time their root cause wasn’t my knowledge. It’s that I have a compulsive personality and also habitually overcommit. As a result, there’s a lot of pressure to get done. The problem isn’t that I can’t flip into an adequate tester mindset, it’s that I don’t step back and take the time.

So, I suspect the interminable and seemingly irresolvable agile-testing debate should be shelved until we solve a more pressing problem: few teams have the discipline to adopt a sustainable pace, so few teams are even in a position to know if programmers could do as well as dedicated testers.

Project testing growth path

In response to a potential client, I wrote something very like the following. The interesting thing is that I’m placing more emphasis on manual exploratory testing. It’s not so much that I suddenly realize its importance as that automated business-facing tests continue to be hard to implement and adopt. More on that anon.

A short sketch of a reasonable growth path would go like this:

  1. Get the programmers sold on test-driven design. How difficult that is depends mainly on how much legacy code you have (where legacy code is, as Michael Feathers says, code without unit tests). Legacy code is hard to test, so programmers don’t see the benefits of testing as quickly, so it requires that much more discipline to get over what’s always a higher hump than with greenfield code. (Michael Feathers’ Working Effectively with Legacy Code is the gold standard book, though there’s an important strategy—”strangler applications“—that’s not covered in depth. Also, I’m the track chair for a new Legacy Code track at Agile2008, I just asked Feathers to give the keynote, and he says he has “a number of surprising proposals about how to make things better”.)

    I’ve come to feel that the most important thing to get across to programmers is what it’s like to work with code built on a solid base of tests. If they understand that early on, they’ll have a clear idea of what to shoot for, which helps with the pain of legacy code. I wrote a workbook to that end.

  2. At the same time, move testers away from scripted manual tests (if that’s what they’re doing) and toward a more exploratory style of manual testing. The people who are strongest on exploratory testing in Agile are Jonathan Kohl, Elisabeth Hendrickson, and Michael Bolton.

  3. As programmers do more unit testing, they will become accustomed to changing their design and adding code in support of their own testing. It becomes more natural for them to do the same for the testers, allowing them to do “automation-assisted exploratory testing”. (Kohl writes about this.) I like to see some of the testers learn a scripting language to help with that. Ruby is my favorite, for a variety of reasons. I wrote a book to help testers learn it.

  4. Over this period, the testers and programmers should shed most animosity or wariness they have toward each other. They’re working together and doing things to help each other. It helps a lot if they sit together.

  5. Once the programmers are sold on test-driven design, they will start wishing that the product owners would supplement what they say about what they want with clear, concrete, executable examples of what they want. That is: tests, written in the language of the business. That isn’t as easy to do as we thought it would be five years ago, but it can be done more or less well. Often, the testers will find a new role as helpers to the product owners. For example, they should get involved early enough to ask questions that lead to tests that prevent bugs (which is better than discovering the bugs after you’ve paid some programmers to implement them).

  6. Throughout this, some kinds of testing (like performance testing) don’t change all that much. For performance testing, I trust Scott Barber.

As a side note: I’m quite fond of the new The Art of Agile Development by Shore & Warden: enough to publicly declare that I’ll bring a copy to every team I work with. Lots of good from-the-trenches experience summarized there.

Evolving an API

Nat Pryce wrote a nice little summary of what he’s learned about evolving an API.

An API is a user interface for programmers. That means you have to develop it like a user-interface: iteratively, adapting to how the users actually use and misuse the interface in real life […]

But… programmers focus on solutions, not goals, so their feature requests will be worded in terms of how they think they would change the API to support whatever it is they want to do. You have to engage in a lot of arduous back-and-forth to find out what they are trying to achieve […]

Workshop on technical debt

From Matt Heusser:

Personally, I believe that the “technical debt” metaphor is compelling. It explains the tradeoffs involved in taking shortcuts in terms that management can understand, and can take the vague and undefined and turn it into something more concrete.

At the same time, it is not a perfect metaphor, it has weaknesses, and we have a lot more to explore. For example:
- How do you measure technical debt?
- How do you communicate it?
- How do you reverse it?
- How do you avoid it?
- Is reversing or avoiding technical debt always the right thing to do?

To get to some more satisfying answers, Steve Poling and I will be organizing a peer workshop on Technical Debt on August 14/15. The event is hosted by the Calvin College CS Department and free to participants. Seating is limited to fifteen (at most, twenty) people and is by application.

The Call for Participation is available on-line:
http://www.xndev.com/downloads/WOTD_2008_CFP.rtf

The announcement is available on my blog.

If you’ve got some experiences to share, some stories to relate, or you would just like to explore the issue some more, I would encourage you to apply.

A tagging meme reveals I short-change design

There’s one of those tagging memes going around. This one is: “grab the nearest book, open to page 123, go down to the 5th sentence, and type up the 3 following sentences.”

My first two books had pictures on p. 123.

The next three (Impro: Improvisation and the Theatre, AppleScript: the Definitive Guide, and Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life) didn’t have anything that was amusing, enlightening, or even comprehensible out of context. So I kept going, which is cheating I suppose. The last, How Designers Think, had this:

The designer’s job is never really done and it is probably always possible to do better. In this sense, designing is quite unlike puzzling. The solver of puzzles such as crosswords or mathematical problems can often recognize a correct answer and knows when the task is complete, but not so the designer.

That’s a hit. It made me realize a flaw in my thinking. You see, it reminded me of one of my early, semi-controversial papers, “Working Effectively With Developers” (referred to by one testing consultant as “the ‘how to suck up to programmers’ paper”). In its second section, “Explaining Your Job”, I explicitly liken programmers to problem solvers:

A legendary programmer would be one who was presented a large and messy problem, where simply understanding the problem required the mastery of a great deal of detail, boiled the problem down to its essential core, eliminated ambiguity, devised some simple operations that would allow the complexity to be isolated and tamed, demonstrated that all the detail could be handled by appropriate combinations of those operations, and produced the working system in a week.

Then I point out that this provides a way for testers to demonstrate value. I show a sample problem, then write:

Now, I’d expect any programmer to quickly solve this puzzle - they’re problem solvers, after all. But the key point is that someone had to create the puzzle before someone else could solve it. And problem creation is a different skill than problem solving.

Therefore, the tester’s role can be likened to the maker of a crossword or a mathematical problem: someone who presents a good, fully fleshed-out problem for the programmer to master and solve:

So what a tester does is help the programmer […] by presenting specific details (in the form of test cases) that otherwise would not come to her attention. Unfortunately, you often present this detail too late (after the code is written), so it reveals problems in the abstractions or their use. But that’s an unfortunate side-effect of putting testers on projects too late, and of the unfortunate notion that testing is all about running tests, rather than about designing them. If the programmer had had the detail earlier, the problems wouldn’t have happened.

Despite this weak 1998 gesture in the rough direction of TDD, I still have a rather waterfall conception of things: tester presents a problem, programmer solves it, we all go home.

But what that’s missing is my 2007 intellectual conception of a project as aiming to be less wrong than yesterday, to get progressively closer to a satisfactory answer that is discovered or refined along the way. In short—going back to the original quote—a conception of the project as a matter of design that’s at every level of detail and involves everyone. That whole-project design is something much trickier than mere puzzle-solving.

I used the word “intellectual” in the previous paragraph because I realize that I’m still rather emotionally attached to the idea of presenting a problem, solving it, and moving on. For example, I think of a test case as a matter of pushing us in a particular direction, only indirectly as a way of uncovering more questions. When I think about how testing+programming works, or about how product director + team conversations work, the learning is something of a side effect. I’m strong on doing the thing, weak on the mechanics of learning (a separate thing from the desire to learn).

That’s not entirely bad—I’m glad of my strong aversion to spending much time talking and re-talking about what we’ll build if we ever get around to building anything, of my preference for doing something and then taking stock once we have more concrete experience—but to the extent that it’s a habit rather than a conscious preference, it’s limiting. I’ll have to watch out for it.