program selection system

There are two main approaches to testing a UI: “record-playback” through the user interface and “data-driven” bypassing it. The former are visual and user-friendly but have a reputation for being unmaintainable, so the Agile community has largely embraced the latter. But could we recover the advantages of record-playback without sacrificing maintainibility? This would mean moving beyond recording screen mechanics or GUI toolkit mechanics and actually trying to record the intent behind them. The approach presented achieves this via manually maintaining a mapping of UI actions to domain language statements.

The above paragraph is for the program: below follows the originally submitted description.

How do you test a user interface automatically? If you ask agilists that question the answers will vary, but a fairly frequent one is “you don’t, it’s too hard”. Standard approaches to GUI testing are deemed too brittle to work in an iterative manner over a long period of time. This is a shame, because there are clear advantages to having acceptance tests operate through the user interface ;

• Full system behaviour is tested and observed. A green box in a table can tell you that some rule is correct but it can’t tell you that you have anything approaching a usable system.
• Writing test scripts is pretty like programming - however much syntactic sugar you put in between the user and the actual code called. This raises the technical bar for being able to write tests.
• Tests through the UI can function as system demos and help with documentation of how to use the system.
• Writing tests by recording GUI usage can create a lot of tests pretty fast.

Which is all very well, but how can we get around the brittleness that has plagued automated UI tests in the past? A key is to address the form in which the tests are recorded - how they identify what is being done with the GUI. Once upon a time this was the screen mechanics - click on pixel 124,21 - and the tests broke as soon as you ran with different monitor settings. Now it’s usually the GUI toolkit mechanics - find this JTable and click in the third cell of the second row. But what if we ended up recording intent instead of mechanics - in a domain language? Then our test script might look like this:

wait for table to be loaded
select flight RA983
proceed to book
set customer name Geoff
set number of seats 2
confirm booking
quit

Then we can have our cake and eat it. We can write that as a test sketch without the system existing yet. We can record it (or some of it) if the system already exists. If we want to create a similar variant of it we probably won’t need to re-record it. It is essentially a description of a use case.

How does this work? Not by magic of course. It requires a specific effort to go through your GUI and map the controls to domain language statements. But once there you have the largest possible freedom to move the GUI around in the future. As an additional bonus we can handle multi-threading also, by identifying the point in the code where the table is loaded, we can ensure that when recording passes that point a wait is recorded, and when replaying gets to such a line in the script, it suspends until the same call is made.

Note the absence of assertions. The script doesn’t check anything about the state of the GUI at any point. This keeps it super-simple and stops it needing any concepts of variables, assertions etc, without which it would be more a script than a usecase. We verify behaviour by comparing plain text log files (see my other submitted session!).

We’ve done this with separate libraries for both PyGTK GUIs and Java Swing GUIs (“PyUseCase” and “JUseCase”). But we feel the ideas are more important than these tools, although either of these would be a decent starting point for anyone wanting to test another GUI with those toolkits.