Exploration Through Example

I’ll be giving one of the keynotes at XP2011 (Madrid, May 10-13). Here’s my tentative abstract. How do you like it?

What Forms of Work-and-Life Make Sense for Us?

It’s widely agreed that Agile has “crossed the chasm” to mainstream acceptance. Along the way, some of the more interesting bits have fallen off. This talk will be the latest in a multi-year effort to recover those bits, rehabilitate them, make them stranger, and encourage you to put them to work. There will be no overarching “Big Idea”, but some smaller ideas will include the distinction between what I call “the stance of rationality” and “the stance of reaction” (I favor the latter), the high cost of thinking, the virtues of habit, and some criticism of the optimistic ideas about teams, enterprises, and leaders that infest our field. There may also be a tango lesson.

The State monad is a big jump for the monad-learner. It jumps up a level of abstraction by using functions as values. That makes it confusing because you have functions that work on functions, and it can be hard to keep track of whether a particular function is a this-kind-of-function or a that-kind-of-function. I try to tease apart the Gordian Knot thusly:

1. I start by creating a logging monad that simply logs the value of each step. This is fairly straightforward.

2. Next, I decide I want only particular steps to be logged. Steps to be logged use a log function which returns either a plain value or a wrapped value. That leads to an if statement in the “decider”.

3. Then I raise the question: can we get rid of the if statement by pushing its work down into the log function? In this section of the tutorial, I do something like follow Beck’s rules of design: make a series of undirected local changes that arrive at something globally coherent. Specifically, that first simple decision of pushing work into one place forces us to implement the full State monad (though I only show it used for building a log).

4. Finally, I use that solution to illustrate some key concepts: base values, monadic functions, monadic values, and how they’re put together to make up a monad.

I have my doubts about how well this works, particularly the 3d step. I’d value your opinion.

http://www.vimeo.com/21307543

How the sequence monad works. The need for a “patcher” function to fix up results of the last function called by a chain of deciders. A better description of a monad: a decider and a patcher.

http://www.vimeo.com/20963938

[Update: I accidentally published my first draft of this.]

I’m reading that part of E.P. Thompson’s The Making of the English Working Class that’s about what happened before the industrial revolution. I’ve been struck by the food “riots” that were common until the 1840’s. I put “riots” in scare quotes because they were often (usually?) peaceful. Here, Thompson quotes John Wesley:

[The mob] had been in motion all day; but their business was only with the forestallers of the market, who had bought up all the corn far and near, to starve the poor, and load a Dutch ship, which lay at the quay; but the mob brought it all out into the market, and sold it for the owners at the common price. And this they did with all the calmness and composure imaginable, and without striking or hurting anyone. (p. 64)

Thompson gives another example:

In Honiton in 1766 lace-workers seized corn on the premises of the farmers, took it to market themselves, sold it, and returned the money and even the sacks back to the farmers. (p. 64)

Oddly similar to events in Madison, where most reports of the scene in the capital were of a surprisingly self-organizing, calm, and disciplined crowd.

What was the motivation for the bread riots? They were “legitimised by the assumptions of an older moral economy, which taught the immorality of any unfair method of forcing up the price of provisions by profiteering on the necessities of the people” (p. 65). Thompson also singles out the tradition of the free-born Englishman, which consisted of:

[…] a moral consensus in which authority at times shared, and of which at all times it was bound to take into account […] this question of the limits beyond which the Englishman was not prepared to be “pushed around,” and the limits beyond which authority did not dare to go, is crucial to an understanding of this period. The stance of the common Englishman was not so much democratic, in any positive sense, as anti-absolutist […] [The unwritten consensus stemming from the Glorious Revolution] might provide the propertied classes with a sanction for the most bloody code penalising offenders against property; but it provided no sanction for arbitrary authority. (pp. 79-80)

That resonates with me. I’m a comfortable white middle-class American. I have certain expectations that those with power will be… tactful. They won’t be absolutist (toward me). They won’t be arbitrary (toward me). They won’t nakedly use a crisis as an excuse to cut off their political enemies’ funding. They won’t be radical. Gov. Walker is radical in the same way that many Republicans have recently been, but their targets have been, well, poor people, brown people, people imprudent in their mortgage borrowing—people Not Much Like Me. Walker’s targets are (much more explicitly) teachers, middle-class workers, firefighters, policemen—people that are like me, not “others”.

That’s why I feel in my gut he’s gone too far. (My head thought all his sort had gone too far long ago.) That’s why I think many whitebread people like me feel he’s gone too far. We finally realize: there really is no social contract left that stops us from being next.

Building on the foundation of the Identity Decider from Part 1, I first introduce some notation to make a monadic computation look more like a `let`. Then I explain the Maybe monad and what it tells us about how monads are meant to be used. Finally, I show what the Sequence monad does as a teaser for Part 3.

http://vimeo.com/20798376

I’ve read or watched several tutorials on monads, and they left me confused. Thinking about why that was, I decided they emphasized a rather uninformative terminology too soon and obscured core ideas with surface syntax. So, following in the footsteps of countless others, I decided to teach monads right (dammit!). I’ve produced part 1 of a tutorial that ignores terminology and uses pictures to avoid the distractions of surface syntax. I’d like you to tell me if the approach is promising. If so, I’ll continue.

Monad Tutorial, Part 1 (Monads as Deciders / Identity Monad)

The tutorial assumes you know some Lisp. I use Clojure for examples.

I’ve recorded a 43-minute screencast. It’s a decent example of my current style of TDD for Clojure. It emphasizes:

• The use of Midje prerequisites (mocks) to support top-down development.

• Using metaconstants to make tests more abstract and readable.

There are two sources for the video:

• Vimeo: http://www.vimeo.com/19404746
• Download: https://github.com/marick/Midje-videos/raw/master/robozzle.mov (If that fails with “blob too big”, go here and select Downloads:

  

  You’ll get a zip file containing the video in its own directory.

UPDATE

Alert watcher Mark Derricutt notices that I got east and west mixed up. I, um, I… I did that deliberately (no, really!) to illustrate an important point about TDD. TDD helps you do what you intended. It is not particularly good at helping you intend the right thing. Indeed, the flow experience it can induce probably makes you worse at noticing you’re proceeding smoothly in the wrong direction.

For example, I had a niggling feeling throughout that I was getting the orientation wrong. That, plus general principle, made me shy away from just copying the answers from the tests into the rotation maps—instead, I recreated them. All for naught: the incorrect mental image I used when I calculated the first left turn had captured my mind.

This getting stuck in a rut is one of the reasons why you still need testing with a fresh mind (either someone else’s or your own, after a rest).

It’s interesting to speculate whether this bug would ever become visible. If, just after finishing the kata, I’d wired up a UI and chosen four ship images, I’d probably again get east and west flipped, meaning all would look well to the user. It’d be like using y as the variable representing horizontal position and x for vertical position: functionally correct if you do it consistently, but death to any sane mind reading the code.

If I chose ship images later, I’d be more likely to get them right (that is, wrong given the bug). I do at times know east from west.

For Midje, I need to take a data structure and produce a new data structure that’s = to the old one, but with all lazy sequences converted into lists. It was hard to get it right. I’m posting my solution here in the hopes that someone’s search engine saves them some work.

As I mentioned earlier this year, I’m looking to make one of my decadal career shifts. Since that decision, I’ve been doing part-time contract work on a RubyCocoa application, and I’ve found it satisfying to deliver working software to people who are happy to get it. It’s also helped with the nagging dread that—while I can talk the talk about programming, testing, refactoring, and all that—I wouldn’t be able to walk the walk. It turns out I can. Although I’m slower than I’d like, I do respectable work.

In my ideal contract, I’d:

• … code in Clojure, Ruby, or Javascript. Other than that, I don’t require super-advanced or cool technology, but I do have a hankering to work on something that could somehow be the inspiration for another book. I don’t care about the domain.

• … devote 1/2 to 3/4 of my time to a single project, working with a single team, over a period of months. Some portion of that—a week or two a month—would be spent onsite. (Chicago would be the best place because it’s easily accessible by train. I live in Central Illinois.)

• … work at a sustainable pace, and be given the leeway to do a good job by my standards. I’m trying to be artisanal about my code.

  (”I want to be artisanal” might raise red flags: will I decide I know what’s needed better than those who are paying for it? My saving grace is that I have a Labrador-like eagerness to please. I want product owners to smile when they think of me.)

• … be able to stretch by occasionally going slower while I experiment with techniques. (My work on outside-in TDD in Clojure is an example.) I’m willing to be paid less in order to improve faster.

• … be in frequent contact with the people who’ll viscerally appreciate the features they get for the money they spend. That given, I don’t care whether I am working directly for a product company or as a subcontractor on behalf of a contract programming company.

• … work in an Agile style. (I almost didn’t think to include this, since I assume anyone interested in hiring me would expect or accept that. I’m not interested in a job teaching the glories of continuous integration or TDD or refactoring. I’m interesting in learning how to do them ever better, and in working with people who have the same interests.)

However, there may not be an ideal, and I don’t intend to be rigid about opportunities. I could see, for example, working with several teams at once, being someone who helps convert a daily grind into an exploration of new techniques. That’d be more like my consulting past, but I’d be more hands-on than I have in the past, involved for longer, and feel more responsible for the product.

Also: although I listed Chicago as my desired location, it has drawbacks when it comes to (1) winter and (2) helping me with my (currently somewhat faltering) attempt to learn Spanish. I wouldn’t mind working in Costa Rica, Argentina, or elsewhere in Latin America (probably for a longer continuous chunk of time onsite).

I don’t have a huge portfolio of code to show you. What I have is on Github. Critter4Us shows my Cappuccino and Ruby code. My Clojure code is limited to Midje, which is a programmer’s tool rather than an end-user project.

My email address is marick@exampler.com.

I’ve created a mailing list for Midje, my Clojure TDD framework.