Reference
---------
Config File Helpers
~~~~~~~~~~~~~~~~~~~
Various helpers for more advanced config files.
.Helpers
[width="80%",cols="^,2m,2m,2m",frame="topbot",options="header"]
|======================
|Helper | Purpose | Valid values | Comments
| #{scheme (scheme code...)} | Execute arbitrary scheme code | Any valid scheme | Value returned from the call is converted to a string and processed as part of the config file
| #{system command} | Execute program, inserts exit code | Any valid Unix command | Discards the output from the program
| #{shell command} or #{sh ...} | Execute program, inserts result from stdout | Any valid Unix command | Value returned from the call is converted to a string and processed as part of the config file
| #{realpath path} or #{rp ...} | Replace with normalized path | Must be a valid path |
| #{getenv VAR} or #{gv VAR} | Replace with content of env variable | Must be a valid var |
| #{get s v} or #{g s v} | Replace with variable v from section s | Variable must be defined before use |
| #{rget v} | Replace with variable v from target or default of runconfigs file | |
| #{mtrah} | Replace with the path to the megatest testsuite area | |
|======================
Config File Settings
~~~~~~~~~~~~~~~~~~~~
Settings in megatest.config
Disk Space Checks
^^^^^^^^^^^^^^^^^
Some parameters you can put in the [setup] section of megatest.config:
-------------------
# minimum space required in a run disk
minspace 10000000
# minimum space required in dbdir:
dbdir-space-required 100000
# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh
-------------------
Trim trailing spaces
^^^^^^^^^^^^^^^^^^^^
------------------
[configf:settings trim-trailing-spaces yes]
------------------
Job Submission Control
^^^^^^^^^^^^^^^^^^^^^^
Submit jobs to Host Types based on Test Name
++++++++++++++++++++++++++++++++++++++++++++
.In megatest.config
------------------------
[host-types]
general nbfake
remote bsub
[launchers]
runfirst/sum% remote
% general
[jobtools]
launcher bsub
# if defined and not "no" flexi-launcher will bypass launcher unless
# there is no host-type match.
flexi-launcher yes
------------------------
host-types
++++++++++
List of host types and the commandline to run a job on that host type.
.host-type => launch command
------------
general nbfake
------------
launchers
+++++++++
.test/itempath => host-type
------------
runfirst/sum% remote
------------
Miscellaneous Setup Items
+++++++++++++++++++++++++
Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.
.In megatest.config
------------------
[setup]
reruns 5
------------------
Run time limit
++++++++++++++
-----------------
[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s
runtimelim 1h 2m 3s
-----------------
Tests browser view
~~~~~~~~~~~~~~~~~~
The tests browser (see the Run Control tab on the dashboard) has two views for displaying the tests.
. Dot (graphviz) based tree
. No dot, plain listing
The default is the graphviz based tree but if your tests don't view
well in that mode then use "nodot" to turn it off.
-----------------
[setup]
nodot
-----------------
Dashboard settings
~~~~~~~~~~~~~~~~~~
.Runs tab buttons, font and size
------------------
[dashboard]
btn-height x14
btn-fontsz 10
cell-width 60
------------------
Database settings
~~~~~~~~~~~~~~~~~
.Database config settings in [setup] section of megatest.config
[width="70%",cols="^,2m,2m,2m",frame="topbot",options="header"]
|======================
|Var | Purpose | Valid values | Comments
|delay-on-busy | Prevent concurrent access issues | yes\|no or not defined | Default=no, may help on some network file systems, may slow things down also.
|daemonize | Daemonize the server on start | yes\|no or not defined | Default=no
|faststart | All direct file access to sqlite db files | yes\|no or not defined | Default=yes, suggest no for central automated systems and yes for interactive use
|homehost | Start servers on this host | <hostname> | Defaults to local host
|hostname | Hostname to bind to | <hostname>\|- | On multi-homed hosts allows binding to specific hostname
|lowport | Start searching for a port at this portnum| 32768 |
|required | Server required | yes\|no or not defined | Default=no, force start of server always
|server-query-threshold | Start server when queries take longer than this | number in milliseconds | Default=300
|timeout | http api timeout | number in hours | Default is 1 minute, do not change
|======================
The testconfig File
-------------------
Setup section
~~~~~~~~~~~~~
Header
^^^^^^
-------------------
[setup]
-------------------
The runscript method is a brute force way to run scripts where the
user is responsible for setting STATE and STATUS
-------------------
runscript main.csh
-------------------
Requirements section
~~~~~~~~~~~~~~~~~~~~
.Header
-------------------
[requirements]
-------------------
Wait on Other Tests
^^^^^^^^^^^^^^^^^^^
-------------------
# A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2
-------------------
Mode
^^^^
The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start
-------------------
[requirements]
mode normal
-------------------
The toplevel mode requires only that the prior tests are COMPLETED.
-------------------
[requirements]
mode toplevel
-------------------
A item based waiton will start items in a test when the same-named
item is COMPLETED and PASS, CHECK or WAIVED in the prior test. This
was historically called "itemwait" mode. The terms "itemwait" and
"itemmatch" are synonyms.
-------------------
[requirements]
mode itemmatch
-------------------
Overriding Enviroment Variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Override variables before starting the test. Can include files (perhaps generated by megatest -envdelta or similar).
--------------------
[pre-launch-env-vars]
VAR1 value1
# Get some generated settings
[include ../generated-vars.config]
# Use this trick to unset variables
#{scheme (unsetenv "FOOBAR")}
--------------------
Itemmap Handling
~~~~~~~~~~~~~~~~
For cases were the dependent test has a similar but not identical
itempath to the downstream test an itemmap can allow for itemmatch
mode
-------------------
[requirements]
mode itemmatch
itemmap .*x/ y/
# ## pattern replacement notes
#
# ## Example
# ## Remove everything up to the last /
itemmap .*/
#
# ## Example
# ## Replace foo/ with bar/
itemmap foo/ bar/
# multi-line; matches are applied in the listed order
# The following would map:
# a123b321 to b321fooa123 then to 321fooa123p
#
itemmap (a\d+)(b\d+) \2foo\1
b(.*) \1p
-------------------
Complex mappings
^^^^^^^^^^^^^^^^
Complex mappings can be handled with the [itemmap] section
// image::itemmap.png[]
image::complex-itemmap.png[]
Example:
. Request to run D/1/res
. Megatest uses rule "(\d+)/res" -> "\1/aa" to create item C/1/aa from D/1/res
. Full list to be run is now: D/1/res, C/1/aa
. Megatest uses rule "(\d+)/aa" -> "aa/\1" to create item A/aa/1
. Full list to be run is now: D/1/res, C/1/aa, A/aa/1
.Testconfig for Test C
----------------------
[requirements]
waiton A B
[itemmap]
A (\d+)/aa aa/\1
B (\d+)/bb
----------------------
.Testconfig for Test D
----------------------
[requirements]
waiton C
itemmap (\d+)/res \1/aa
----------------------
.Testconfig for Test E
----------------------
[requirements]
waiton C
itemmap (\d+)/res \1/bb
----------------------
Dynamic Flow Dependency Tree
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.Autogeneration waiton list for dynamic flow dependency trees
-------------------
[requirements]
# With a toplevel test you may wish to generate your list
# of tests to run dynamically
#
waiton #{shell get-valid-tests-to-run.sh}
-------------------
Run time limit
^^^^^^^^^^^^^^
-----------------
[requirements]
runtimelim 1h 2m 3s # this will automatically kill the test if it runs for more than 1h 2m and 3s
-----------------
Skip
^^^^
A test with a skip section will conditional skip running.
.Skip section example
-----------------
[skip]
prevrunning x
# rundelay 30m 15s
-----------------
Skip on Still-running Tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^
-----------------
# NB// If the prevrunning line exists with *any* value the test will
# automatically SKIP if the same-named test is currently RUNNING. The
# "x" can be any string. Comment out the prevrunning line to turn off
# skip.
[skip]
prevrunning x
-----------------
Skip if a File Exists
^^^^^^^^^^^^^^^^^^^^^
-----------------
[skip]
fileexists /path/to/a/file # skip if /path/to/a/file exists
-----------------
Skip if test ran more recently than specified time
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.Skip if this test has been run in the past fifteen minutes and 15 seconds.
-----------------
[skip]
rundelay 15m 15s
-----------------
Disks
^^^^^
A disks section in testconfig will override the disks section in
megatest.config. This can be used to allocate disks on a per-test or per item
basis.
Controlled waiver propagation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED
Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)
-----------------
###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file rulename input_glob
waiver_1 logpro lookittmp.log
[waiver_rules]
# This builtin rule is the default if there is no <waivername>.logpro file
# diff diff %file1% %file2%
# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
-----------------
Ezsteps
~~~~~~~
.Example ezsteps with logpro rules
-----------------
[ezsteps]
lookittmp ls /tmp
[logpro]
lookittmp ;; Note: config file format supports multi-line entries where leading whitespace is removed from each line
;; a blank line indicates the end of the block of text
(expect:required in "LogFileBody" > 0 "A file name that should never exist!" #/This is a awfully stupid file name that should never be found in the temp dir/)
-----------------
To transfer the environment to the next step you can do the following:
----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}
----------------------------
Triggers
~~~~~~~~
In your testconfig triggers can be specified
-----------------
[triggers]
# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh
# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh
# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh
-----------------
Scripts called will have; test-id test-rundir trigger, added to the commandline.
HINT
To start an xterm (useful for debugging), use a command line like the following:
-----------------
[triggers]
COMPLETED/ xterm -e bash -s --
-----------------
NOTE: There is a trailing space after the --
Override the Toplevel HTML File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Megatest generates a simple html file summary for top level tests of
iterated tests. The generation can be overridden. NOTE: the output of
the script is captured from stdout to create the html.
.For test "runfirst" override the toplevel generation with a script "mysummary.sh"
-----------------
# Override the rollup for specific tests
[testrollup]
runfirst mysummary.sh
-----------------
Archiving Setup
---------------
In megatest.config add the following sections:
.megatest.config
--------------
[archive]
# where to get bup executable
# bup /path/to/bup
[archive-disks]
# Archives will be organised under these paths like this:
# <testsuite>/<creationdate>
# Within the archive the data is structured like this:
# <target>/<runname>/<test>/
archive0 /mfs/myarchive-data/adisk1
--------------
Handling Environment Variables
------------------------------
It is often necessary to capture and or manipulate environment
variables. Megatest has some facilities built in to help.
Capture variables
~~~~~~~~~~~~~~~~~
.Commands
------------------------------
# capture the current enviroment into a db called envdat.db under
# the context "before"
megatest -envcap before
# capture the current environment into a db called startup.db with
# context "after"
megatest -envcap after startup.db
# write the diff from before to after
megatest -envdelta before-after -dumpmode bash
------------------------------
Dump modes include bash, csh and config. You can include config data
into megatest.config or runconfigs.config.
.Example of generating and using config data
------------------------------
megatest -envcap original
# do some stuff here
megatest -envcap munged
megatest -envdelta original-munged -dumpmode ini -o modified.config
------------------------------
Then in runconfigs.config
.Example of using modified.config in a testconfig
------------------------------
cat testconfig
[pre-launch-env-vars]
[include modified.config]
------------------------------
Programming API
---------------
These routines can be called from the megatest repl.
.API Keys Related Calls
[width="70%",cols="^,2m,2m,2m",frame="topbot",options="header,footer"]
|======================
|API Call | Purpose comments | Returns | Comments
|(rmt:get-keys run-id) | | ( key1 key2 ... ) |
| (rmt:get-key-val-pairs run-id) | | #t=success/#f=fail | Works only if the server is still reachable
|======================
:numbered!: