Description

GINT provides considerable help for validating data produced by testcases matches expected results. A testcase simply needs to declare data expected in the output. Another useful technique is for a testcase to declare data that, if found in the output, indicates that the test has failed.

Capabilities

1. Data found in output of command - list of strings and/or regular expression patterns
2. Data that must NOT be found in output of command - list of strings and/or regular expression patterns
3. Output file that needs to be produced by the command or inline code
4. Output data that needs to be found in the file produced by the command or inline code - list of strings and/or regular expression patterns
5. Output data that must NOT be found in the file produced by the command or inline code - list of strings and/or regular expression patterns
6. File differences between current and expected output files are within tolerance condition
7. User defined closure that can do additional comparisons

Using

These are the Testcase parameters that control automatic comparisons:

1. data - a list of strings or regular expression patterns that must be found in the output produced by running the command or inline code
2. failData - a list of strings or regular expression patterns what will cause the testcase to fail if any of them are found in the output produced by running the command or inline code
3. patternFlag - used for string matching - use Pattern.CASE_INSENSITIVE so ignore case on string comparisons
4. output - a map that defines comparisons for an output file produced by running the command or inline code
   • output.file - the file name that is expected to be produced
   • output.data - similar to data above, except will be used for matching on the output file data
   • output.failData - similar to failData above, except will be used for matching on the output file data
   • output.patternFlag - similar to patternFlag above, except will be used for matching strings on the output file data
5. compare - true or a map that requests running a full file difference command on files. This capability requires an appropriate diff command be available on the system - most operating systems except Windows have a standard diff command available. Window's users may need to do an install - see Dependencies. Normally, this is used to compare the output of the testcase, but, can also be used in a separate testcase that is just comparing the output of other testcases or operations. Diff output explains the output when differences are detected.
   • compare: true - compare is done using standard defaults
   • compare.fileName - file name for comparison if either file1 or file2 is not specified explicitly, defaults to the testcase name
   • compare.asDir - true to treat a fileName with no extension as a directory name to allow for the entire directory to be compared. Otherwise the fileName will get extension '.txt' by default.
   • compare.file1 - usually the expected file output, defaults to the fileName in the the compare directory
   • compare.file2 - usually the file output just created, defaults to the fileName in the output directory
   • compare.expected - expected number of line differences between the files, default is 0. Comparison is considered successful if the number of differences is less than or equal to this number.
   • compare.fileDiff - either true or a file name. If specified, the output from the diff command will be redirected to a file. Default file name is the testcase name with '.diff' extension in the output directory. If not specified, the diff output is logged when verbose logging is requested.
   • compare.ignoreCase - true to ignore case in the comparison (case insensitive)
   • compare.ignoreWhiteSpaceChanges - true to ignore changes in white space in the comparison
   • compare.ignoreAllWhiteSpace - true to ignore all white space in the comparison
   • compare.ignoreBlankLines - true to ignore blank lines in the comparison
   • compare.options - other diff command line options for advanced users - diff reference

Regular expressions

• Regular expression patterns provide powerful matching capabilities when simple string matching is not sufficient
• Regular Expression Test Page is useful for testing regular expressions

• While patternFlag can be used for setting special matching characteristics, an alternative is to use embedded actions. A case insensitive pattern string example is: (?i)find this. Here are some:

  (?i) - case insensitive
  (?m) - multiline 
  (?s) - dot all, . matches line terminators
  (?d) - unix (\x) line endings 
  (?u) - unicode
  

Examples

Data and FailData

Logged output can be compared directly using the data or failData parameters.

gint.add(
    [name: 'dir', 
        cmd: 'dir c:\\.', 
        data: ['Program Files', 'Windows'], 
        failData: ['usr', 'bin'],
    ]
])

gint.add(
    [name: 'dirToFile', description: 'Note compare uses a case insensitive regex pattern',
        cmd: "dir c:\\. > ${gint.getOutputFile('dirToFile')}",
        output: [data: [~/(?i)program files/, 'Windows'], failData: ['usr', 'bin']],
    ]
])

File Compare

gant -f myTest -Dlabel=R1.0

gant -f myTest -Dlabel=R1.1 -DcompareLabel=R1.0

gint.add(
    [name: 'dirToFile',
        cmd: "dir c:\\. > ${gint.getOutputFile('dirToFile')}",
        output: [data: ['Program Files', 'Windows'], failData: ['usr', 'bin']],
        compare: true,
    ]
])