Overview

Documents the approach to functional/integration testing for the application REST services layer.

Coverage

These service are currently covered by REST integration testing.

Complete coverage would address each mode of use of each method call of each service, and combinations of calls across them, as well as any ad hoc test cases for identified errors not covered by existing test cases.

Coverage Definition

The attached spreadsheet formally defines REST service layer coverage from an integration testing perspective. 

Assumptions

Coverage is defined in a multi-dimensional matrix that takes into account the services, the methods of the services, and the modes of use of the services (including normal use, degenerate use, edge case use, and role check). NOTE: in many cases there are many "normal" modes of use which may extend the number of columns in the spreadsheet.  For example, methods that take optional parameters (e.g. Pfs).

The following sample coverage spreadsheet is the current coverage definition.

Organization

Integration tests are organized under the integration-testing project.  Looking through the Java packages, you should find a "org.ihtsdo.otf.ts.test.rest" package that defines a number of test classes that represent the coverage spreadsheet.

Generally each column of each tab of the coverage spreadsheet is represented by a single testing class, where each row within that column corresponds to a particular method of that testing class.  The coverage spreadsheet defines both the class names used to implement tests as well as the method names (so they can be easily cross referenced).

The test names themselves are "test cases" that are organized into "test suites" according to the structure of the spreadsheet.

For each test suite and test case, there should also exist a documentation page in this wiki.  A test suite is a collection of test cases and a test case is a "script" detailing the actions involved and the expected outcomes with any other needed information.

Integration Test Suites

Test suites are organized by spreadsheet tab.

Degenerate Use Test Automation

For repetitive testing of degenerate use of methods with null and INVALID values, a class DegenerateUseMethodTestHelper is provided, which provides a mechanism for testing a method

The four methods available for degenerate use testing are:

Where:

Paging/Filtering/Sorting objects (PfsParameter) are explicitly handled with the following tests:

 

Example: 

// the class for which methods are to be tested, in this case SecurityClientRest
SecurityClientRest service = new SecurityClientRest(ConfigUtility.getConfigProperties());
 
// use reflection to get the desired method
Method method =
        service.getClass().getMethod(
			"authenticate", 								// method name and parameters
			new Class<?>[] {String.class, String.class});	// i.e. SecurityClientRest.authenticate(String username, String password)
 
// specify the valid parameters
Object[] parameters = new Object[] {
        new String("name"), new String("password")			// valid parameters that when passed to method produce successful outcome (e.g. authentication)
    };
 
// if all invalid and null parameters should throw exceptions, use this call
DegenerateUseMethodTestHelper.testDegenerateArguments(service, method,
        parameters);
 
// if different parameters cause different behavior, use this type of call
DegenerateUseMethodTestHelper.testDegenerateArguments(service, method,
        parameters, new ExpectedFailure[] {
            ExpectedFailure.STRING_INVALID_NO_RESULTS_NULL_EXCEPTION, // an INVALID string will result in no result, a NULL string will result in exception
			ExpectedFailure.EXCEPTION								  // both INVALID strings and NULL strings will result in exception
        });
 
// if specific invalid values are desired, use this type of call
DegenerateUseMethodTestHelper.testDegenerateArguments(service, method,
        parameters, new Object[] {						
			new String("Invalid value"),					// the first argument should test "Invalid value", and the method should throw an exception
			null											// no invalid value will be tested
		});

 

References/Links