Backing the Truth into a Corner
Projects that fail do so mostly because they build the wrong thing, not because they do a bad job building whatever they do build. A big part of the difficulty is getting the users to tell us what they want. Traditional requirements engineering approaches have had mixed results, but requirements capture through “checked examples” (eg Fit -style tests) seems to work very well. Why should that be? Are there clues to be had from post-Aristotelian notions of “category” in the cognitive sciences? If so, how can that help find better examples?
This is a follow-on from a reasonably well-received talk given at Agile 07.
The session will be a work-torial. Or possibly tut-shop.
Definitions are nice: the individually necessary and jointly sufficient properties of the members of the set that’s represented by a concept map very nicely onto table signatures, or class definitions, or what have you. It’s damn hard to get trained engineers to do that right, though, and near impossible to get paying customers (aka “ordinary people”) to do it at all. Why?
The activity of “testing” (or, something that looks a lot like it) has become a primary one for developers in the Agile community. And it has spread away from the intimate action of coding test-first to become a requirements technique and project management tool. What has been discovered is the remarkable power of “checked examples” in programming, in requirements elicitation and gathering, in communication between project stakeholders, in measuring value.
All these benefits come a ridiculously low cost compared with other techniques that have similar effects. Why are collections of examples so fecund, so seminal, so protean?
If we (as system builders) could understand how and why example seem to be so much more comfortable a tool for ordinary people (as system users) than examples, maybe our example-enabled practices could be improved.
It turns out that there are some interesting parallels between what we have discovered in the realm of specifying systems using examples and what has been discovered by those who have chosen to explore by experiment how humans understand the world (rather than relying on the authority of long-dead Greeks). And with the work of those who try to explore how the mechanics of being in and understanding the world works. What could that possibly mean? How can we use these parallels to enhance our practice?
Talk
What definitions are. The classical view of category. Aristotle to Quine. Russell, his paradox. Diogenes and the chicken. It’s not news to anyone who’s been paying attention the last few millennia that there are issues around the use of definitions. And yet we still hear programmers banging on about “defining terms”. Why?
An motivating example from my practice whereby folks have struggled mightily to give a rule for some thing or other but proceeded very nicely with…examples. My current exemplar for this is do with the horrific difficulty of capturing how to avoid automatically generated FX trades trying to settle on a banking holiday when the currencies involved belong to cultures with different calendars. This example will be demonstrated: both the torturous (and unclear) rules and the limpid examples/tests will be shown. Aside: the release of the product that had these rules as its main feature was the first ever defect-free release made by that team, which they themselves attribute to having the “tests”.
Examples. As a whole, pick a domain and give lots of examples.
Examples are not created equal: If you wanted to explain to a Martian what a “bird” was, would you show them a penguin? Why not? An eagle? Why?
From the earlier examples, which seem more compelling, more central? Why? How could we make use of that when gathering checked examples?
Do techniques from the traditional, post-hoc testing world offer any help with finding exemplars? eg, partition into equivalence sets
Do
In groups, come up with examples, prototypes and exemplars for stuff. Explore how these might or might not help with systems work. Ways to do this that will (hopefully) break people out of their existing lines of thought on this:
The “game” game Game
Inspired by Aphorism 66.
- the discussion is seeded with a simple definition of the category of “games”
- each player in turn must name a game that clearly is a game but is not covered by the definition, and extend the definition to cover it
- once no-one can think of any more games, the resulting dog’s breakfast of a definition will be recorded for posterity
- now, the example games will be sorted in an affinity exercise, and the resulting graph mined for prototypical or exemplar games. (watch out for cultural differences here, should be interesting)
Example Examples
In groups, choose who has the most interesting (ie, obscure) domain.
- the non-experts in the groups question the expert on the interesting domain, in two modes:
- try to capture an aspect of the domain using examples and rules, but without giving any concrete examples
- try to capture a different aspect, using only examples
- which worked best? Which was easiest? Which gave the non-experts the greatest sense of confidence that they understood the domain?
My Exemplar Could Beat Up Your Example
In groups, pick a domain reasonably familiar to all. Each person in turn must give an example form the domain that seems more central the the previous one. (Watch out for people doing abstraction here, I suspect)
and more!
Ponder
Bringing it all back home: what does any of this suggest for the discover of really great examples to put into Fit (or whatever)?
Get
A handout containing an annotated bibliography and a summary of Rosch’s ideas