Feature - Interactive Testing

Owned by Bob Swift
2024-05-19



Description

In many cases, it is not practical to have fully automated tests. Manual and semi-automated testing is common practice. This can be resource intensive and be a significant time constraint in the software development lifecycle. Result reporting is either missing, manual, or poorly done. GINT interactive testing support is intended to:

  1. Help convert manual tests into semi-automated tests
  2. Help formalize and document manual or semi-automated tests
  3. Make it easier for less experienced resources to run manual tests in a structured way
  4. Provide standard reporting similar to automated tests

GINT uses an instant message (IM) service to allow tests to interact with a tester at appropriate times when manual intervention is required. Manual intervention could include:

  1. Running manual steps to setup for a testcase
  2. Determining if a testcase can be run at this time
  3. Verifying results after a testcase run
  4. Restarting a testcase after a failure

GINT allows for any testcase to define instructions and control interactions with a tester. Manual testcases can be inter-mixed with fully automated testcases in test scripts - subsets can be used to control specific runs to include or exclude interactive testcases.

Dependencies

  1. XMPP IM service with appropriate user accounts
    • At least 2 accounts, one for the automation to use and one for the user running the tests
    • Tested services
  2. Smack API jars - on classpath using similar techniques as described in Installation and Dependencies
    • Examples below are shown using Grape

Parameters

Testcase definitions can optionally specify the im parameter to make the testcase an interactive testcase. The following describes the parameters that control this behavior.




Example

interactiveExample.gant

/*
 * Interactive GINT test
 * - requires imConfig defined in a profile
 * - example config like in src/itest/resources/gintIm.properties
 *      gmailUser = sometester@gmail.com            (IM will be sent to this user)
 *      gmailUser2 = automationaccount@gmail.com    (GINT automation uses this account to send IM)
 *      gmailPassword = xxxxxx
 *      gmailPassword2 = xxxxxx
 *      imConfig_code  = [name: 'gtalk', to: '${gmailUser}', host: 'talk.google.com', \
 *                        service: 'gmail.com', userName: '${gmailUser2}', userPassword: '${gmailPassword2}' ]
 *
 * Run using: gant -f interactiveExample.gant
 */

/*
 * Needs access to smack api functions on classpath. This does it automatically, or you can use other classpath techniques.
 */
@Grapes([
    @Grab(group='jivesoftware', module='smack',  version='3.1.0'),
    @Grab(group='jivesoftware', module='smackx', version='3.1.0'),
    @GrabConfig(systemClassLoader=true),
])

import org.swift.tools.*

includeTool << Gint           // test framework

profiles = 'gintIm' // needs to define imConfig

gint.initialize(this) // required

gint.add('basic', [
    [name: 'simple', inline: { }, im: true],  // default content, before test is run
    [name: 'manualTest1', inline: { }, im: [content: "Instructions for running manualTest1"]],
    [name: 'manualVerify', description: 'Please verify GOOD appears in the log',
         inline: { println '>>>GOOD<<<'; return true },
         im: [after: true]],  // interact with user after the test code has run
])

gint.finalizeTest() // required - final preparations for running tests

Run

gant interactiveExample.gant

= = = = = =   interactiveexample started at Sun Oct 10 09:52:39 CDT 2010   = = = = = =

    [start] simple
 [response] s
    [start] manualTest1
 [response] s
    [start] manualVerify (Please verify GOOD appears in the log)
>>>GOOD<<<
   [ending] simple
 [complete] simple - 4.321 secs

   [ending] manualTest1
 [complete] manualTest1 - 2.216 secs

   [ending] manualVerify (Please verify GOOD appears in the log)
 [response] s
 [complete] manualVerify - 2.115 secs

     [info] Successful testcases  . . . . . . . . . . : 3    <<< TEST SUCCESSFUL
     [info] Total testcases . . . . . . . . . . . . . : 3
     [info] Elapsed run time  . . . . . . . . . . . . : 22.486 secs

= = = = = =   interactiveexample completed at Sun Oct 10 09:53:00 CDT 2010 = = = = = =

BUILD SUCCESSFUL
Total time: 23.95 seconds
imac:gint bob$

© 2005 -2024 gint.org