Back to overview

Event handlers in ReadyAPI (SOAPUI) clean up after test failure

Published on: 04 April 2016 Author: Richard Velden Category: Java & Web

In this article we’ll show why and how to use Event handlers in ReadyAPI. In our example we will use ‘event handlers’ to clean up JMS queues after a test-case has failed.

Scenario

Large integration project, with lots of test-scripts written in ReadyAPI
Test scripts are run after each build on our integration server
One test-case fails, possibly leaving the system in a uncertain state.
- Oracle BPEL processes still running, still waiting for a particular input
- Messages still left on queues, which might be used by the next test case
One of the following test-cases also fails because of the previous

Such a situation is undesirable. We do not want any dependency between test-cases. If one test fails, all the other tests should not be affected!

Goal

In this scenario we also have several cleanup scripts bundled in a separate ReadyAPI project. What we would like to do is run these generic cleanup scripts whenever one of the test-cases fails.

Options

To achieve this goal we first consider our options.

ReadyAPI structure

ReadyAPI structure starts with the workspace (see fig 1):

A workspace can contain multiple projects
Each project can contain multiple Test Suites
Test Suites can contain multiple Test Cases
A test case consists out of 1 or more test steps

Figure 1 - ReadyAPI workspace structure:

Given this structure, we would like to be able to check for failure after each individual test case.

Setup / Teardown scripts

Setup and Teardown scripts are used to perform pre- and post-testing actions. In a setup script one can create shared objects, which can be used by other test scripts Teardown scripts can be used for cleaning up, closing collections or connections.

In ReadyAPI these scripts can be defined per project, test suite, or test case.

Defining a teardown script for each separate test case could provide a solution for our problem [1].

Event Handlers

Event handlers in ReadyAPI are defined on project level. They enable us to perform various scripted tasks and modifications whenever a particular event occurs. A selection of events that can be used as a trigger are shown in figure 2.

Figure 2 - Events for which one can define an Event handler:

For our problem we could use the 'TestSuiteRunListener.afterTestCase' event. This enables us to perform our generic cleanup script after completion of each test case.

Similar to the teardown script, but with the difference that we only need to define the event handler once per project, instead of per test case [2,3].

Using event handlers

Let’s start with the implementation of our event handler. Event handlers can be defined in ReadyAPI by pressing the ‘Event’ icon (see fig 3).

Figure 3 - Event button in ReadyAPI Functional Testing perspective:

From here we can define event handlers, and add some Groovy script for handling these (see fig 4).

Figure 4 - Event Handler window defining event handler for event ‘afterTestCase’:

Groovy-script and ReadyAPI

Defining events is one, how to interact with ReadyAPI from Groovy script is the next challenge:

Which objects can I use to check the result of the last executed test case?
How can I start another test suite in a different project programmatically?

The first hint is directly shown in the right top corner of the code window (fig 5).

Figure 5 - Available objects in Groovy script:

The second hint is to use the CTRL-SPACE key combination for acquiring inline code completion (fig 6 and 7).

Figure 6 - Use CTRL-Space to see all directly available objects:

Figure 7 - CTRL-Space menu showing all available methods on object testRunner:

However, for actually doing something useful we recommend using the API documentation which can be found at [4]. It describes all objects available in ReadyAPI (see fig 8).

Figure 8 - ReadyAPI API docs:

Searching the web for example Groovy scripts showed little results. Especially for opening another projects, or ascertaining the previous test case outcome. Luckily the API Documentation website proved to be very useful.

Our assignment was two-fold:

Find out the outcome of the current test case (event = afterTestCase)
Run a test suite from a different ReadyAPI project, if the test failed (outcome = ‘FAILED’)

Groovy test case outcome

To check whether a test case failed or not, we quickly found some interesting methods on the testRunner object (see fig 9).

Figure 9 - testRunner.getResults():

testRunner.getResults() might be what we are looking for to ascertain the test case result. It returns a List. A list of what (see fig 10)?

Figure 10 - testRunner.getResults() contains object type?:

Let’s see, a list of Object? This is very useful information :( As mentioned earlier in this article, to actually do something useful we need the API documentation. For class TestSuiteRunner we found the following information (see fig 11) [5].

Figure 11 - API docs for TestSuiteRunner method getResults():

The list returned by getResults() contains TestCaseRunner objects. TestCaseRunner itself has(inherits) the method getStatus() which returns enumeration TestRunner.Status [6]. Possible values in this enumerations are shown in figure 12.

Figure 12 - Possible values from enumeration TestRunner.Status:

With this input we can construct an if-clause to detect failed tests (see fig 13). We have used context.getCurrentTestCaseIndex() to get the current test case index.

Figure 13 - if-clause to detect failed tests:

		if ( testRunner.getResults().get( context.getCurrentTestCaseIndex() ).getStatus() != FINISHED ) {
 	// Run Cleanup scripts
}

Groovy invoking test suite from a different project

To run a test suite from a different project we could use the already available variables to find what we needed. With some help of the API documentation we quickly found what we were looking for.

Obtaining current workspace was pretty straightforward:
Workspace workspace = testRunner.getTestSuite().getProject().getWorkspace();

Once we have a reference to the current workspace, we can try to find and open the ‘Cleanup’ project. With the project reference it is possible to find any test suite, and run it.

Project project = Workspace.getProjectByName( ‘Cleanup’ );
project.getTestSuiteAt(0).run( context.getProperties(), false );

This worked fine, testing from the ReadyAPI UI.

When testing from our integration server however, there was no workspace object. Our cleanup script failed!

The integration server did not use a workspace containing all the projects. It ran each ReadyAPI project separately from the others. With help of the API documentation we found out how to create a new workspace, import the ‘Cleanup’ project, and run it (see fig 14) [7,8]. Please check figure 15 for the combined code with some additional checks, and usage of projectRoot properties.

Figure 14 - Groovy script to create a workspace, import a project, and run the first Test Suite:

		WorkspaceFactory workspaceFactory = WorkspaceFactory.getInstance();
workspace = workspaceFactory.openWorkspace( "projects", null);
workspace.importProject( projectroot + "/Cleanup -project" );
Project project  = workspace.getProjectByName('Cleanup');
project.getTestSuiteAt(0).run( context.getProperties(),false);

	

Figure 15 - Groovy script detecting test failure and starting Cleanup project:

		import com.eviware.soapui.model.workspace.WorkspaceFactory
import com.eviware.soapui.model.workspace.Workspace
import com.eviware.soapui.support.types.*
import com.eviware.soapui.model.project.Project
import static com.eviware.soapui.model.testsuite.TestRunner.Status.*;
 
if ( testRunner.getResults().get( context.getCurrentTestCaseIndex() ).getStatus() != FINISHED ) {	
        log.info "Testcase failed: Cleaning up queues before proceeding";
	Workspace workspace = testRunner.getTestSuite().getProject().getWorkspace()
        if( workspace != null ) {
	        // Use currently open workspace to initialize (when run locally)	
                Project project = workspace.getProjectByName('CleanUp');
                project.getTestSuiteAt(0).run(context.getProperties(),false);
	} else {
	        // When run from scripts: create/open workspace and import Cleanup project			
                def projectroot = context.expand('${projectDir}'); 
 
		// open workspace for scripted testrun
		WorkspaceFactory workspaceFactory = WorkspaceFactory.getInstance();
		workspace = workspaceFactory.openWorkspace( "projects", null);
		if( workspace.getProjectCount() == 0 ) {
			workspace.importProject( projectroot + "/Cleanup -project" );
                }
                Project project  = workspace.getProjectByName('Cleanup');
                project.getTestSuiteAt(0).run( context.getProperties(),false);
		log.info "finished..." + workspace.getProjectAt(0).getName() + "." + workspace.getProjectAt(0).getTestSuiteAt(0).getName();
	}	
}
 

	

Passing additional project properties

In our particular case we experienced some missing / incorrect project properties. Our integration server script passed some additional parameters overwriting one of the seeded project properties. To pass these properties onto the newly opened project we have used the following code:

		TestProperty specialProjectProperty = testRunner.getTestSuite().getProject().getProperty("SpecialConfigPath");
project.setPropertyValue("SpecialConfigPath", specialProjectProperty.getValue());

An alternative is:

		def specialProjectProperty = context.expand( '${#Project#SpecialConfigPath }' );
project.setPropertyValue("SpecialConfigPath", specialProjectProperty);

References

About the author Richard Velden

Oracle Fusion Middleware Developer at Qualogy. Specializes in integration and cloud development using Oracle technologies such as: SOA Suite, Service Bus, Integration and Process Cloud.