Thursday, November 5, 2009

Creating Executable Specifications for Swing apps with Concordion and FEST

The FEST Swing Module provides an easy-to-use API for testing Java applications that use the Swing GUI toolkit.

For a recent project, we created executable specifications with Concordion that used FEST as a driver for the desktop Swing-based GUI. The combination was very powerful, resulting in readable specifications of the system behaviour that invoked end-to-end tests through the GUI.

Here are some tips on using FEST effectively with Concordion:

1) Use FEST's built-in checks that all of the GUI updates are performed in the Event Dispatch Thread (EDT):

@BeforeClass
public static void setUpEDTChecks() {
FailOnThreadViolationRepaintManager.install();
}


2) Install a big red switch. When running the tests locally, they're going to take over your desktop. Use FEST's EmergencyAbortListener to allow you to terminate the GUI using ctrl - shift - A:

@BeforeClass
public static void setUpAbortListener() {
listener = EmergencyAbortListener.registerInToolkit();
}

@AfterClass
public static void tearDownAbortListener() {
listener.unregister();
}


3) Create your user interface once for all tests:

public static UserInterface getInstance() {
if (instance == null) {
instance = new UserInterface();
}
return instance;
}


4)) Create your window within the Event Dispatching Thread (EDT)

private FrameFixture window;

private UserInterface() {
GuiActionRunner.execute(new GuiTask() {
public void executeInEDT() {
JFrame frame = ... // code to create main frame
window = new FrameFixture(frame);
setLookAndFeel();
}
});
window.show();


5) Slow the tests down when demonstrating to users, so they can see what's happening:

public void slowDown() {
window.robot.settings().delayBetweenEvents(500);
window.robot.settings().eventPostingDelay(500);
}


6) When using FEST's assertions, you can allow assertion failures to bubble up to Concordion as AssertionErrors. However, you will get more readable Concordion output if you translate them. For example, with the following Concordion specification:

<p>Only things of type 'Thing One' should be prioritised....</p>
<table concordion:execute="#prioritised =
isPrioritised(#thingType)">
<tr><th concordion:set="#thingType">Thing Type</th>
<th concordion:assertEquals="#prioritised">Prioritised</th></tr>
<tr><td>Thing One</td><td>Yes</td></tr>
<tr><td>Thing Two</td><td>No</td></tr>
</table>


, the corresponding fixture can be written as:

public String isPrioritised(String thingType) {
doStuffWithThing(thingType);
return userInterface.isPrioritised(request) ? "Yes" : "No";
}


, and integrated with Fest using:

public Boolean isPrioritised(Thing thing) {
int rowIndex = getRowIndex(thing);
try {
table.fontAt(row(rowIndex).column(col)).requireBold();
} catch (AssertionError e) {
return false;
}
return true;
}


7) Follow all of the usual hints and tips for creating Concordion specifications. In particular:

  • Write Specifications not Scripts.

  • Don't mention the GUI. Your specification should be independent of the implementation.

  • Evolve a domain-specific language



I'll write a follow-up to describe how we got these tests running in our Continuous Integration server.

Friday, July 3, 2009

Correction about Greenpepper

In my recent APN presentation on Agile Acceptance Testing, I stated that Atlassian had bought Greenpepper (the tool).

Well I was wrong. I showed some of the slides at CITCON ANZ last weekend, and the Atlassian guys corrected me that they had bought Greenhopper (the tool) from Greenpepper (the company), but not Greenpepper (the tool).

Wednesday, June 24, 2009

What types of application can I test with Concordion?

Following on from my Agile Acceptance Testing presentation at the APN on Monday, there were some questions about what types of application you could test with Concordion.

Concordion itself is a test framework which runs your executable specifications using test fixtures.

The executable specifications are written in HTML and are independent of any programming language. The same executable specifications can be used with Concordion for Java, or Concordion.NET. (and a subset of the syntax can be used with the Ruby and Python ports of Concordion).

The test fixtures are written in a programming language. For the Java version of Concordion, the test fixtures are written as JUnit classes. (However they don't include test methods and don't tend to include assertions, since these are defined in the executable specification.)

There are 2 main ways in which Concordion is used. Either through the User Interface, or "under the covers" to the application or services layer.

When testing "under the covers" of the application, the test fixture calls into the application code directly. In this case, the type of application is largely irrelevant.

When testing through the user interface, you need a Test Driver. Test drivers provide the functionality to find widgets, click buttons, enter text etc. There are a lot of high quality, open source test drivers available. Some of these, such as Selenium and SoapUI also come with their own test frameworks, but can be used purely as test drivers. The test drivers I have used with Concordion are:




















Application Type Test Driver
Web Application WebDriver and Selenium
Swing GUI Application Fest
Web Service SoapUI



Additionally there are a lot of test extension libraries available, such as DbUnit and XmlUnit that can be called from your test fixture.

Concordion is a great framework for running your executable specifications. When combined with the right driver you can test pretty much any type of application.