Summary

GINT is a general test automation framework that is primarily targeted for integration and system level testing. This will discuss GINT as it relates to Wikipedia's definition of a test automation framework. Specifically, we will address the 4 elements a framework is responsible for:

1. Defining the format in which to express expectations
2. Creating a mechanism to hook into or drive the application under test
3. Executing the tests
4. Reporting results

Links

• Test automation framework - Wikipedia
• Software testing - Wikipedia
• Testcase parameters
• Feature - Automatic Comparisons
• Feature - Command Generators
• Feature - Atlassian Application Testing
• Feature - JUnit-style XML Output
• How to run Selenium as a GINT test
• How to use GINT for unit testing

Technique

GINT uses a procedural code based scripting approach with a declarative style with dependency driven execution. 

Expected results

Testing is all about making sure that code being tested produces the expected results. It is critical for the testcase author to know what is expected and to be able to express that in the framework efficiently. To be effective, the author must pick out the most important aspects that determine success or failure of the code without being too verbose. This is easier for white-box testing compared to black-box testing. GINT provides support to make the expression as simple as possible for most tests and customizable for more unique situations.

• expected parameter is the highest level way of determining success - a single value indicator 
• data parameter describes data is expected to be generated. This is the automatic comparisons feature. This is a very concise way of determining the success of a testcase.
• failData parameter describes data that must not be produced. This allows for quickly identifying failure situations.
• compare parameter just as easy to do deal with expected results in file based data including file diffs with previous results
• regular expressions also make expressing expected conditions concise and more flexible 
• custom scripts like resultClosure and similar make it possible to extend finding expected results to unique logic
• power asserts is a groovy capability supported in GINT for easily expressing expected conditions in logic terms. Groovy power asserts log information making it easier to debug. GINT handles failed asserts as a testcase failure.

Driving the application

GINT has 2 distinct ways to drive an application.

1. cmd parameter or equivalent via command generator. This is driving the application via a command line interface (CLI). In many cases, the application my be accessible indirectly via curl or tool providing a CLI. An example is: Atlassian application testing. 
2. inline parameter. This is driving the application via a groovy script. For example, running Selenium tests, groovy or java unit tests, or other unique ways to drive a test

Executing the test

The command line is the primary and most effective way to run tests. Test runs can be parameterized including running a subset of testcases, running last failed, and user controlled parameters. Logging provides feedback and includes recreate information and an explanation of what expected result condition failed. BGTP is an additional tool that makes running GINT tests in Atlassian Bamboo very easy.

Reporting results

Result reporting is extremely important especially for agile development. Both during testcase development and also for the entire team. GINT reports results in multiple ways.

1. Each test is logged as successful or failed.
2. The log contains summary statistics and an failure summary
3. JUnit style output file suitable for CI servers like Atlassian Bamboo and similar