devxlogo

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Part 1 of this two-part series explained automated acceptance testing for Java Web applications using using JBehave, Thucydides and Selenium 2/Webdriver. This final installment demonstrates how to start implementing some automated acceptance criteria. To follow along, you can clone the github repository for the sample project.

Alternatively, if you want to create your own brand new Thucydides test suite and write your own tests, you can use the Maven archetype. From the command line, run mvn archetype:generate -Dfilter=thucydides-jbehave, as illlustrated below:

mvn archetype:generate -Dfilter=thucydides-jbehave...Choose archetype:1: remote -> net.thucydides:thucydides-jbehave-archetype 
     (Thucydides automated acceptance testing project using Selenium 2, JUnit and JBehave)Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1Choose net.thucydides:thucydides-jbehave-archetype version: 1: 0.9.872: 0.9.883: 0.9.894: 0.9.90Choose a number: 4: Define value for property 'groupId': : com.acme.myappDefine value for property 'artifactId': : mywebtestsDefine value for property 'version':  1.0-SNAPSHOT: : Define value for property 'package':  com.acme.myapp: : Confirm properties configuration:groupId: com.acme.myappartifactId: mywebtestsversion: 1.0-SNAPSHOTpackage: com.acme.myapp Y: :

In both cases, the project structure should look something like the one in Figure 1.

The sample project directory structure.
Figure 1: The sample project directory structure.

This directory structure follows standard Maven conventions. The JBehave stories are stored in the src/test/resources/stories directory, in a simple directory structure that reflects the organization of the requirements into capabilities and features. In the src/test/java directory, you will find the JBehave test harness (the AcceptanceTestSuite class) as well as any JBehave step implementation classes.

In the src/main/java directory, we find the Thucydides step classes and page objects. Placing these classes in the src/main/java is a relatively arbitrary choice, based on the following reasoning:

  • This maven project (or module) is dedicated to acceptance tests – if the acceptance tests are stored in the same project as the application code, this choice obviously makes less sense.
  • Page Objects and Thucydides steps are occasionally reused between related acceptance test projects, and can therefore be viewed as deliverables rather than test code: for example, page objects and steps related to integration with PayPal might be reused for several projects.

Automating Requirements and Acceptance Criteria

While JBehave helps document and automate the acceptance criteria themselves, Thucydides also reports on the features and capabilities being delivered. This helps give better context for the individual acceptance criteria as well as more relevant aggregate reporting. In this project, the capabilities are represented as directories underneath the src/test/resources/stories directory. The features are represented by JBehave stories directly in these directories; for a more complexe application, we might have a sub-directory for each feature, with JBehave stories in these sub-directories.

Inside each capability directory, a text file called narrative.txt describes the feature. The contents of this file are free text, with the first line serving as a title in the reports. An example is shown here:

Learn the meaning of a wordIn order to learn the meaning of a word that I don't knowAs an online readerI want to be able to find out the meaning of the word

Acceptance criteria in JBehave take the form of simple text files with the “.story” suffix. Each “.story” file contains the acceptance criteria for one feature or story, along with an optional narrative section that describes the feature or story in question. The examples above are all written in JBehave notation: you will find them in the src/test/resources/stories directory of the sample project. A simple example is shown here:

Narrative:In order to understand what I am reading aboutAs a readerI want to be able to look up definitions of words Scenario: Looking up the definition of a simple wordGiven the user does not know the meaning of the word 'banana'When the user looks up the definition of the word 'banana'Then they should obtain a definition containing the words 'An elongated curved fruit'

Once we have a set of “.story” files, we can automate them with JBehave. Thucydides provides a convenient test harness class, called ThucydidesJUnitStories, that you can extend to run all of the JBehave stories in in the src/test/resources/stories directory. Just create a class in your project that extends this class, like the following:

import net.thucydides.jbehave.ThucydidesJUnitStories;public class AcceptanceTestSuite extends ThucydidesJUnitStories {}

Implementing the Tests

Now that we have some acceptance criteria, we can start to automate them. When work starts on a feature or story, the acceptance criteria will be pending, with no implementation. As our understanding of the problem grows, we will be able to flesh out the implementation. We start off with a high-level sketch of what needs to be done to demonstrate the feature, without going into the details of how this will be done. This is done using a Thucydides Step class to add a layer of abstraction between the “how,” described in the JBehave steps, and the “what,” detailed inside the Thucydides step methods. An automated acceptance criterion is a guide for developers and living documentation for testers and other team members as much as it is a test, so it is important that at each stage we focus on clearly communicating what the test is trying to do. The following code illustrated one possible implementation of the JBehave story listed above using this approach:

public class DefinitionSteps {     @Steps    ReaderSteps reader;     @Given("the user does not know the meaning of the word '$word'")    public void givenTheUserDoesNotKnowAWord () {        reader.consults_the_online_dictionary();    }     @When("the user looks up the definition of the word '$word'")    public void whenTheUserLooksUpTheDefinitionOf(String word) {        reader.looks_up_the_definition_of(word);    }     @Then("they should obtain a definition containing the words '$definition'")    public void thenTheyShouldSeeADefinitionContainingTheWords(String definition) {        reader.should_see_a_definition_containing(definition);    }}

In the next step, we start to implement the Thucydides step methods, fleshing out the details of “how” this feature should be demonstrated. In the following example, we use Page Objects to isolate and centralize the UI logic from the business logic described in the test steps. Within these step methods, we focus on what each step is doing in logical terms, without committing too much to a particular UI implementation. Indeed, if this test involves the UI, we may well wait until the UI is reasonably stable before implementing these methods in order to avoid too much rework.

public class ReaderSteps extends ScenarioSteps {       DictionaryPage dictionaryPage;     public ReaderSteps(Pages pages) {        super(pages);        dictionaryPage = getPages().get(DictionaryPage.class);    }     @Step    public void consults_the_online_dictionary() {        dictionaryPage.open();    }     @Step    public void looks_up_the_definition_of(String term) {        enters(term);        starts_search();    }     @Step    public void should_see_a_definition_containing(String terms) {        List displayedDefinitions = (List)dictionaryPage.getDefinitions();        assertThat(displayedDefinitions, hasItem(containsString(terms)));    }     @Step    public void enters(String keyword) {        dictionaryPage.enter_keywords(keyword);    }     @Step    public void starts_search() {        dictionaryPage.lookup_terms();    }}

When the UI is stable enough to work with, or even once we have a reasonably stable HTML design, we can implement the Page Object for this web page. Page Objects isolate the HTML implementation details of a web page behind a friendlier and more implementation-neutral API, making the test code more reusable and easier to maintain. Selenium 2/WebDriver provides excellent support for the Page Object pattern out of the box. Here, we extend the Thucydides PageObject class, which provides a number of convenience methods, but the code should look reasonably familiar to any developer having worked with Selenium 2/WebDriver.

@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")public class DictionaryPage extends PageObject {     @FindBy(name="search")    private WebElement searchTerms;     @FindBy(name="go")    private WebElement lookupButton;     @FindBy(css="ol li")    private List definitionList;     public DictionaryPage(WebDriver driver) {        super(driver);    }     public void enter_keywords(String keyword) {        $(searchTerms).type(keyword);    }     public void lookup_terms() {        $(lookupButton).click();    }     public Iterable getDefinitions() {        return extract(definitionList, on(WebElement.class).getText());    }}

Reporting and Story Telling

Acceptance tests are as much about communication as they are about testing; indeed, testing is almost a secondary aspect of the whole BDD process. The Thucydides reports are naturally designed to record test results (see Figure 2), but they also provide more detailed reporting at other levels. Firstly, Thucydides records and reports on the steps that were executed during each test (see Figures 3 and 4), thus documenting how a particular acceptance criteria was demonstrated. If the test uses the web UI, screenshots are also recorded at each step (see Figure 5).

Secondly, it is important to know what features are to be implemented, and how many of these have currently been completed. Agile projects measure progress in terms of working software, and in a BDD project, working software can be demonstrated by a working acceptance test. Thucydides provides aggregate reporting that can help judge what features and capabilities are potentially ready for delivery (see Figure 6).

You can try this out for yourself by running the full test suite and generating the Thucydides reports in the demo project. To do this, just run the mvn verify in the project root directory. This will produce a set of reports in the target/site/thucydides directory.

A summary of test results
Figure 2: A summary of test results

A simple test report
Figure 3: A simple test report

A table-based test report
Figure 4: A table-based test report

Tests involving the UI record screenshots for each step
Figure 5: Tests involving the UI record screenshots for each step

What capabilities and features have been specified and tested?
Figure 6: What capabilities and features have been specified and tested?

Conclusion

Automated Acceptance Tests are an important part of the BDD arsenal. But to be a successful part of your project, Automated Acceptance Tests need to be simple, easy to maintain, and focused as much on communication as on testing. However this is too often not the case, especially when automating the UI is involved. Far from being merely a set of ad-hoc test scripts, acceptance criteria deserve the same respect as production code, and writing a good set of acceptance criteria requires all the discipline and skill that you need for your production code.

In this article, we have looked at how tools like JBehave, Thucydides and WebDriver can help in this regard. JBehave encourages teams to express requirements and acceptance criteria using a common vocabulary and structure. The Page Objects pattern promoted by Selenium 2/Webdriver helps make UI tests more stable and easier to maintain. And Thucydides uses a disciplined, layered approach to writing acceptance tests that is designed to encourage clean, maintainable, high quality test code with a high degree of reusability. In addition, the Thucydides test reports aim to deliver the BDD goal of “Living Documentation,” detailing not only how a feature is tested, but also where it fits into the application’s requirements as a whole.

devxblackblue

About Our Editorial Process

At DevX, we’re dedicated to tech entrepreneurship. Our team closely follows industry shifts, new products, AI breakthroughs, technology trends, and funding announcements. Articles undergo thorough editing to ensure accuracy and clarity, reflecting DevX’s style and supporting entrepreneurs in the tech sphere.

See our full editorial policy.

About Our Journalist