This software was produced by NIST, an agency of the U.S. government,
and by statute is not subject to copyright in the United States.
Recipients of this software assume all responsibilities associated
with its operation, modification and maintenance. However, to
facilitate maintenance we ask that before distributing modified
versions of this software, you first contact the authors at
oof_manager@nist.gov. 

------------------------------------------------------------------------------

This directory contains the GUI tests for oof2.  They tests ensure
that the user interface is working correctly: that the buttons,
sliders and other widgets are connected up properly, and that the
widgets are displayed and sensitized when they're supposed to be.
These tests generally do *not* do a thorough job of testing that the
underlying computations are correct.  Those tests are in the parent
directory, OOF2/TEST.

The individual tests are contained in the subdirectories whose names
begin with six integers (000000_tutorial_basics, etc.)  The names
vaguely indicate the purpose of the tests.  XXXXXX_tutorial_YYYYY runs
through the YYYYY tutorial; XXXXXX_page_ZZZZZ tests features of the
ZZZZZ page in the main OOF2 window.


RUNNING THE GUI TESTS

The main test script, oof2-guitest, resides in the "bin" subdirectory
of the installation (prefix) directory, which is what was specified by
CMAKE_INSTALL_PREFIX when oof2 was built.  When the instructions below
refer to <prefix>, they mean this directory.

To run all the tests, open a terminal window and type

   oof2-guitest

If <prefix>/bin is not in your shell's PATH, you can invoke the tests
like this:

   PATH=<prefix>/bin <prefix>/bin/oof2-guitest

To run a single test, append the name of the test directory to the
command line.  For example, typing

   oof2-guitest 000000_tutorial_basics

will run only the test in the 000000_tutorial_basics directory.
Running

   oof2-guitest --list

will print a list of all the tests.  You can run more
than one test, like this:

   oof2-guitest 000000_tutorial_basics 001000_page_micro

Running

   oof2-guitest --help

will list other ways of running a subset of the tests.

It is also possible to use Python regular expressions to run a subset
of tests, so it's not necessary to type the full names.  For example

  oof2-guitest "000000.*"

will run 000000_tutorial_basics because it is the only one that begins
with "000000".


WHAT THE OUTPUT MEANS

As the tests run, a lot of output appears in the terminal.  Most lines
are something like this:

000000_tutorial_basics (0) || 45/189 findWidget('Basics').resize(500, 518)

In this example

   '000000_tutorial_basics' is the name of the test.

   '(0)' indicates how many times this test has been repeated, if the
   --retries option was used.  Otherwise it's not shown.

   '||' is just a divider.  If the line is a comment from the test file,
   the divider is '##'.

   '45/189' is the number of the current line, and the total number of
   lines in the test script.

   The remainder of the line is the actual line from the test script.
   In this case, the script is changing the size of a widget named
   'Basics'.


WHAT TO DO WITH THE RESULTS

If the tests succeed, the line "<N> tests ran successfully!" will be
printed at the end of a long string of messages.

If a test fails, the program will either hang (output to the terminal
window will cease) or will bring up an OOF2 error window.  If the
program hangs, you should try to quit it with the File/Quit menu item.
If that doesn't work, you will have use the Unix 'kill' or 'killall'
command from a terminal window.  On some systems you may have to use
'kill -9' to kill all the oof2 threads.

If a test brings up the OOF2 error window, you can quit by pressing
the "Abort" button.  If you press the "OK" button, you may still be
told "<N> tests ran successfully!", but don't be misled by that.

If a test fails, please run it again.  We believe that there are still
some timing problems in the testing code, and some test failures do
not indicate actual errors in the oof2 code.  If you start
oof2-guitest with the --retries=10 option, it will repeat failed
tests up to 10 times before reporting the failure.

To re-run a failed test, get the test name from the output lines (you
may have to scroll up past error messages) and run

   oof2-guitest <name of failed test>

You can resume the full set of tests at that point by running

   oof2-guitest --from <name of failed test>

If a test fails repeatedly, please report it to oof_bugs@nist.gov.
Please tell us which test failed and include the last few lines of the
screen output (at least the last line beginning with the test name and
anything after that).  Also, please tell us what kind of computer
you're using and the version numbers of your operating system,
compiler, and python interpreter.

CAUTION: The tests work by simulating user interaction from the
keyboard and the mouse.  It is easy to confuse the testing machinery
by typing or using the mouse while the tests are in progress, since
the program can't distinguish real keypresses and clicks from
simulated ones.  If a test fails, start it again, go get a cup of
coffee, and don't touch the computer until the test is done.
