Terms

The following terms are used in reference to GINT testing. Where possible, we tried to use standard terms. For convenience, sometimes the context will allow some alternative usages of terms .

Test

test or test script is a single collection of testcases that can be run. GINT tests are Groovy/GANT scripts.

Test run

A single instance of running a test. A test run produces results about each of the testcases included in the test.

Testcase

A single instance of a command to run or inline code to execute. This is the basic building block of a GINT test. A testcase is identifiable by name, can be run, and when run, either is considered to be successful or failed. Testcases can also be skipped during a test run based on parameters.

Depends testcase

A testcase that is required to be run and be successful prior to running another testcase.

Test report

A summary or detail report for a test run. A test report shows at least a summary of the number of testcases that were successful, failed, or skipped.

XML report

A test report in a JUnit-style XML format. This is the de facto standard for integrating test results with Continuous Integration (CI) tools. Reference.

Unit test

Reference - Wikipedia - Unit Test. Low level, white box, testing exercising code at the function/method call level.

Integration test

Reference - Wikipedia - Integration Test. Higher level testing exercising software at more of a user or external level.

Command generator

A Groovy closure that generates a command string based on a testcase. A command generator allows for command based testcases to be expressed in domain specific terminology appropriate for the environment being tested. Feature - Command Generators.

Testcase result

Each testcase that is part of a test run, will be associated with a result.

Success

A testcase result indicating that the testcase was run and behaved as expected. For instance, a testcase could expect the running of a command to be successful or expect the running of a command to error in a specific way.

Failed

A testcase result indicating that the testcase was run and some unexpected outcome occurred. For instance, incorrect data was produced or the testcase could not be completed correctly.

Skipped

A testcase result indicating that the testcase was not run due to conditions specified by the test, the testcase or other factors.

Successful test run

A test run is considered successful if no testcase had a failed result and the test run completed normally.

Failed test run

A test run is considered failed if at least one testcase had a failed result, the test had definitional errors, or the test run terminated unexpectedly.

SetUp

One or more actions required to be done prior to running testcases. For example, creating test data or setting up other external environments. For GINT tests, setUp actions can be done with regular Groovy code, however, it is preferred to use testcases defined as setUp testcases. SetUp testcases must be successful, but do not count in the standard summary results. SetUp testcase reporting only appears with verbose output or if there are failed setUp testcases.

TearDown

One or more actions that should be done after testcases have completed to remove artifacts of the test that are no longer needed. For GINT tests, tearDown actions can be done with regular Groovy code, however, it is preferred to use testcases defined as tearDown testcases. tearDown testcases may fail without causing a test run to fail. TearDown do not count in the standard summary results. TearDown testcase reporting only appears with verbose output.