// This file is part of Megatest.
//
// Megatest is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Megatest is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Megatest. If not, see <http://www.gnu.org/licenses/>.
Reference
---------
Megatest Use Modes
~~~~~~~~~~~~~~~~~~
.Base commands
[width="80%",cols="^,2m,2m",frame="topbot",options="header"]
|======================
|Use case | Megatest command | mtutil
|Start from scratch | -rerun-all | restart
|Rerun non-good completed | -rerun-clean | rerunclean
|Rerun all non-good and not completed yet | -set-state-status KILLREQ; -rerun-|clean | killrerun
|Continue run | -run | resume
|Remove run | -remove-runs | clean
|Lock run | -lock | lock
|Unlock run | -unlock | unlock
|killrun | -set-state-status KILLREQ; -kill-run | killrun
|======================
// Config File Helpers
// ~~~~~~~~~~~~~~~~~~~
//
// Various helpers for more advanced config files.
//
// .Helpers
// [width="80%",cols="^,2m,2m,2m",frame="topbot",options="header"]
// |======================
// |Helper | Purpose | Valid values | Comments
// | #{scheme (scheme code...)} | Execute arbitrary scheme code | Any valid scheme | Value returned from the call is converted to a string and processed as part of the config file
// | #{system command} | Execute program, inserts exit code | Any valid Unix command | Discards the output from the program
// | #{shell command} or #{sh ...} | Execute program, inserts result from stdout | Any valid Unix command | Value returned from the call is converted to a string and processed as part of the config file
// | #{realpath path} or #{rp ...} | Replace with normalized path | Must be a valid path |
// | #{getenv VAR} or #{gv VAR} | Replace with content of env variable | Must be a valid var |
// | #{get s v} or #{g s v} | Replace with variable v from section s | Variable must be defined before use |
// | #{rget v} | Replace with variable v from target or default of runconfigs file | |
// | #{mtrah} | Replace with the path to the megatest testsuite area | |
// |======================
//
// Config File Settings
// ~~~~~~~~~~~~~~~~~~~~
//
// Settings in megatest.config
//
// Config File Additional Features
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
//
// Including output from a script as if it was inline to the config file:
//
// -------------------------
// [scriptinc myscript.sh]
// -------------------------
//
// If the script outputs:
//
// -------------------------
// [items]
// A a b c
// B d e f
// -------------------------
//
// Then the config file would effectively appear to contain an items section
// exactly like the output from the script. This is useful when dynamically
// creating items, itemstables and other config structures. You can see the
// expansion of the call by looking in the cached files (look in your linktree
// for megatest.config and runconfigs.config cache files and in your test run
// areas for the expanded and cached testconfig).
//
// Wildcards and regexes in Targets
//
// -------------------------
// [a/2/b]
// VAR1 VAL1
//
// [a/%/b]
// VAR1 VAL2
// -------------------------
//
// Will result in:
//
// -------------------------
// [a/2/b]
// VAR1 VAL2
// -------------------------
//
// Can use either wildcard of "%" or a regular expression:
//
// -------------------------
// [/abc.*def/]
// -------------------------
//
// Disk Space Checks
// ^^^^^^^^^^^^^^^^^
//
// Some parameters you can put in the [setup] section of megatest.config:
//
// -------------------
// # minimum space required in a run disk
// minspace 10000000
//
// # minimum space required in dbdir:
// dbdir-space-required 100000
//
// # script that takes path as parameter and returns number of bytes available:
// free-space-script check-space.sh
// -------------------
//
// Trim trailing spaces
// ^^^^^^^^^^^^^^^^^^^^
//
// ------------------
// [configf:settings trim-trailing-spaces yes]
// ------------------
//
// Job Submission Control
// ^^^^^^^^^^^^^^^^^^^^^^
//
// Submit jobs to Host Types based on Test Name
// ++++++++++++++++++++++++++++++++++++++++++++
//
// .In megatest.config
// ------------------------
// [host-types]
// general nbfake
// remote bsub
//
// [launchers]
// runfirst/sum% remote
// % general
//
// [jobtools]
// launcher bsub
// # if defined and not "no" flexi-launcher will bypass launcher unless
// # there is no host-type match.
// flexi-launcher yes
// ------------------------
//
// host-types
// ++++++++++
//
// List of host types and the commandline to run a job on that host type.
//
// .host-type => launch command
// ------------
// general nbfake
// ------------
//
// launchers
// +++++++++
//
// .test/itempath => host-type
// ------------
// runfirst/sum% remote
// ------------
//
// Miscellaneous Setup Items
// +++++++++++++++++++++++++
//
// Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.
//
// .In megatest.config
// ------------------
// [setup]
// reruns 5
// ------------------
//
// Replace the default blacklisted environment variables with user supplied
// list.
//
// Default list: USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS MAKEF MAKEOVERRIDES
//
// .Add a "bad" variable "PROMPT" to the variables that will be commented out
// in the megatest.sh and megatest.csh files:
// -----------------
// [setup]
// blacklistvars USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS PROMPT
// -----------------
//
// Run time limit
// ++++++++++++++
//
// -----------------
// [setup]
// # this will automatically kill the test if it runs for more than 1h 2m and 3s
// runtimelim 1h 2m 3s
// -----------------
//
// Tests browser view
// ~~~~~~~~~~~~~~~~~~
//
// The tests browser (see the Run Control tab on the dashboard) has two views for displaying the tests.
//
// . Dot (graphviz) based tree
// . No dot, plain listing
//
// The default is the graphviz based tree but if your tests don't view
// well in that mode then use "nodot" to turn it off.
//
// -----------------
// [setup]
// nodot
// -----------------
//
// Capturing Test Data
// ~~~~~~~~~~~~~~~~~~~
//
// In a test you can capture arbitrary variables and roll them up in the
// megatest database for viewing on the dashboard or web app.
//
// .In a test as a script
// ------------------------
// $MT_MEGATEST -load-test-data << EOF
// foo,bar, 1.2, 1.9, >
// foo,rab, 1.0e9, 10e9, 1e9
// foo,bla, 1.2, 1.9, <
// foo,bal, 1.2, 1.2, < , ,Check for overload
// foo,alb, 1.2, 1.2, <= , Amps,This is the high power circuit test
// foo,abl, 1.2, 1.3, 0.1
// foo,bra, 1.2, pass, silly stuff
// faz,bar, 10, 8mA, , ,"this is a comment"
// EOF
// ------------------------
//
// Alternatively you can use logpro triggers to capture values and inject them
// into megatest using the -set-values mechanism:
//
// .Megatest help related to -set-values
// ------------------------
// Test data capture
// -set-values : update or set values in the testdata table
// :category : set the category field (optional)
// :variable : set the variable name (optional)
// :value : value measured (required)
// :expected : value expected (required)
// :tol : |value-expect| <= tol (required, can be <, >, >=, <= or number)
// :units : name of the units for value, expected_value etc. (optional)
// ------------------------
//
// Dashboard settings
// ~~~~~~~~~~~~~~~~~~
//
// .Runs tab buttons, font and size
// ------------------
// [dashboard]
// btn-height x14
// btn-fontsz 10
// cell-width 60
// ------------------
//
// Database settings
// ~~~~~~~~~~~~~~~~~
//
// .Database config settings in [setup] section of megatest.config
// [width="70%",cols="^,2m,2m,2m",frame="topbot",options="header"]
// |======================
// |Var | Purpose | Valid values | Comments
// |delay-on-busy | Prevent concurrent access issues | yes\|no or not defined | Default=no, may help on some network file systems, may slow things down also.
// |faststart | All direct file access to sqlite db files | yes\|no or not defined | Default=yes, suggest no for central automated systems and yes for interactive use
// |homehost | Start servers on this host | <hostname> | Defaults to local host
// |hostname | Hostname to bind to | <hostname>\|- | On multi-homed hosts allows binding to specific hostname
// |lowport | Start searching for a port at this portnum| 32768 |
// |required | Server required | yes\|no or not defined | Default=no, force start of server always
// |server-query-threshold | Start server when queries take longer than this | number in milliseconds | Default=300
// |timeout | http api timeout | number in hours | Default is 1 minute, do not change
// |======================
//
// The testconfig File
// -------------------
//
// Setup section
// ~~~~~~~~~~~~~
//
// Header
// ^^^^^^
//
// -------------------
// [setup]
// -------------------
//
// The runscript method is a brute force way to run scripts where the
// user is responsible for setting STATE and STATUS
//
// -------------------
// runscript main.csh
// -------------------
//
// Iteration
// ~~~~~~~~~
//
// .Sections for iteration
// ------------------
// # full combinations
// [items]
// A x y
// B 1 2
//
// # Yields: x/1 x/2 y/1 y/2
//
// # tabled
// [itemstable]
// A x y
// B 1 2
//
// # Yields x/1 y/2
// ------------------
//
//
// Requirements section
// ~~~~~~~~~~~~~~~~~~~~
//
// .Header
// -------------------
// [requirements]
// -------------------
//
// Wait on Other Tests
// ^^^^^^^^^^^^^^^^^^^
//
// -------------------
// # A normal waiton waits for the prior tests to be COMPLETED
// # and PASS, CHECK or WAIVED
// waiton test1 test2
// -------------------
//
// Mode
// ^^^^
//
// The default (i.e. if mode is not specified) is normal. All pre-dependent tests
// must be COMPLETED and PASS, CHECK or WAIVED before the test will start
//
// -------------------
// [requirements]
// mode normal
// -------------------
//
// The toplevel mode requires only that the prior tests are COMPLETED.
//
// -------------------
// [requirements]
// mode toplevel
// -------------------
//
// A item based waiton will start items in a test when the same-named
// item is COMPLETED and PASS, CHECK or WAIVED in the prior test. This
// was historically called "itemwait" mode. The terms "itemwait" and
// "itemmatch" are synonyms.
//
// -------------------
// [requirements]
// mode itemmatch
// -------------------
//
// Overriding Enviroment Variables
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
//
// Override variables before starting the test. Can include files (perhaps generated by megatest -envdelta or similar).
//
// --------------------
// [pre-launch-env-vars]
// VAR1 value1
//
// # Get some generated settings
// [include ../generated-vars.config]
//
// # Use this trick to unset variables
// #{scheme (unsetenv "FOOBAR")}
// --------------------
//
// Itemmap Handling
// ~~~~~~~~~~~~~~~~
//
// For cases were the dependent test has a similar but not identical
// itempath to the downstream test an itemmap can allow for itemmatch
// mode
//
// .example for removing part of itemmap for waiton test (eg: item +foo-x/bar+ depends on waiton's item +y/bar+)
// -------------------
// [requirements]
// mode itemwait
// # itemmap <item pattern for this test> <item replacement pattern for waiton test>
// itemmap .*x/ y/
//
// -------------------
//
// .example for removing part of itemmap for waiton test (eg: item +foo/bar/baz+ in this test depends on waiton's item +baz+)
// -------------------
//
// # ## pattern replacement notes
// #
// # ## Example
// # ## Remove everything up to the last /
// [requirements]
// mode itemwait
// # itemmap <item pattern for this test> <nothing here indicates removal>
// itemmap .*/
// -------------------
//
// .example replacing part of itemmap for (eg: item +foo/1234+ will imply waiton's item +bar/1234+)
// -------------------
//
// #
// # ## Example
// # ## Replace foo/ with bar/
// [requirements]
// mode itemwait
// # itemmap <item pattern for this test> <item replacement pattern for waiton test>
// itemmap foo/ bar/
//
// -------------------
//
// .example for backreference (eg: item +foo23/thud+ will imply waiton's item +num-23/bar/thud+
// -------------------
// #
// # ## Example
// # ## can use \{number} in replacement pattern to backreference a (capture) from matching pattern similar to sed or perl
// [requirements]
// mode itemwait
// # itemmap <item pattern for this test> <item replacement pattern for waiton test>
// itemmap foo(\d+)/ num-\1/bar/
//
// -------------------
//
// .example multiple itemmaps
// -------------------
//
// # multi-line; matches are applied in the listed order
// # The following would map:
// # a123b321 to b321fooa123 then to 321fooa123p
// #
// [requirements]
// itemmap (a\d+)(b\d+) \2foo\1
// b(.*) \1p
// -------------------
//
//
// Complex mapping
// ^^^^^^^^^^^^^^^
// Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section)
//
// Each line in an itemmap section starts with a waiton test name followed by an itemmap expression
//
// .eg: The following causes waiton test A item +bar/1234+ to run when our test's +foo/1234+ item is requested as well as causing waiton test B's +blah+ item to run when our test's +stuff/blah+ item is requested
// --------------
// [itemmap]
// A foo/ bar/
// B stuff/
// --------------
//
//
// Complex mapping example
// ^^^^^^^^^^^^^^^^^^^^^^^
//
//
//
// // image::itemmap.png[]
// image::complex-itemmap.png[]
//
//
// We accomplish this by configuring the testconfigs of our tests C D and E as follows:
//
// .Testconfig for Test E has
// ----------------------
// [requirements]
// waiton C
// itemmap (\d+)/res \1/bb
// ----------------------
//
// .Testconfig for Test D has
// ----------------------
// [requirements]
// waiton C
// itemmap (\d+)/res \1/aa
// ----------------------
//
// .Testconfig for Test C has
// ----------------------
// [requirements]
// waiton A B
//
// [itemmap]
// A (\d+)/aa aa/\1
// B (\d+)/bb bb/\1
// ----------------------
//
// .Testconfigs for Test B and Test A have no waiton or itemmap configured
// -------------------
// -------------------
//
// .Walk through one item -- we want the following to happen for testpatt +D/1/res+ (see blue boxes in complex itemmaping figure above):
//
// . eg from command line +megatest -run -testpatt D/1/res -target mytarget -runname myrunname+
// . Full list to be run is now: +D/1/res+
// . Test D has a waiton - test C. Test D's itemmap rule +itemmap (\d+)/res \1/aa+ -> causes +C/1/aa+ to run before +D/1/res+
// . Full list to be run is now: +D/1/res+, +C/1/aa+
// . Test C was a waiton - test A. Test C's rule +A (\d+)/aa aa/\1+ -> causes +A/aa/1+ to run before +C/1/aa+
// . Full list to be run is now: +D/1/res+, +C/1/aa+, +A/aa/1+
// . Test A has no waitons. All waitons of all tests in full list have been processed. Full list is finalized.
//
//
//
// itemstable
// ^^^^^^^^^^
// An alternative to defining items is the itemstable section. This lets you define the itempath in a table format rather than specifying components and relying on getting all permutations of those components.
//
//
//
//
//
// Dynamic Flow Dependency Tree
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//
// .Autogeneration waiton list for dynamic flow dependency trees
// -------------------
// [requirements]
// # With a toplevel test you may wish to generate your list
// # of tests to run dynamically
// #
// waiton #{shell get-valid-tests-to-run.sh}
// -------------------
//
// Run time limit
// ^^^^^^^^^^^^^^
//
// -----------------
// [requirements]
// runtimelim 1h 2m 3s # this will automatically kill the test if it runs for more than 1h 2m and 3s
// -----------------
//
// Skip
// ^^^^
//
// A test with a skip section will conditional skip running.
//
// .Skip section example
// -----------------
// [skip]
// prevrunning x
// # rundelay 30m 15s
// -----------------
//
// Skip on Still-running Tests
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^
//
// -----------------
// # NB// If the prevrunning line exists with *any* value the test will
// # automatically SKIP if the same-named test is currently RUNNING. The
// # "x" can be any string. Comment out the prevrunning line to turn off
// # skip.
//
// [skip]
// prevrunning x
// -----------------
//
// Skip if a File Exists
// ^^^^^^^^^^^^^^^^^^^^^
//
// -----------------
// [skip]
// fileexists /path/to/a/file # skip if /path/to/a/file exists
// -----------------
//
// Skip if test ran more recently than specified time
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//
// .Skip if this test has been run in the past fifteen minutes and 15 seconds.
// -----------------
// [skip]
// rundelay 15m 15s
// -----------------
//
// Disks
// ^^^^^
//
// A disks section in testconfig will override the disks section in
// megatest.config. This can be used to allocate disks on a per-test or per item
// basis.
//
// Controlled waiver propagation
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//
// If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
// If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED
//
// Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)
//
// -----------------
// ###### EXAMPLE FROM testconfig #########
// # matching file(s) will be diff'd with previous run and logpro applied
// # if PASS or WARN result from logpro then WAIVER state is set
// #
// [waivers]
// # logpro_file rulename input_glob
// waiver_1 logpro lookittmp.log
//
// [waiver_rules]
//
// # This builtin rule is the default if there is no <waivername>.logpro file
// # diff diff %file1% %file2%
//
// # This builtin rule is applied if a <waivername>.logpro file exists
// # logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
// -----------------
//
// Ezsteps
// ~~~~~~~
//
// .Example ezsteps with logpro rules
// -----------------
// [ezsteps]
// lookittmp ls /tmp
//
// [logpro]
// lookittmp ;; Note: config file format supports multi-line entries where leading whitespace is removed from each line
// ;; a blank line indicates the end of the block of text
// (expect:required in "LogFileBody" > 0 "A file name that should never exist!" #/This is a awfully stupid file name that should never be found in the temp dir/)
//
// -----------------
//
// To transfer the environment to the next step you can do the following:
//
// ----------------------------
// $MT_MEGATEST -env2file .ezsteps/${stepname}
// ----------------------------
//
// Triggers
// ~~~~~~~~
//
// In your testconfig or megatest.config triggers can be specified
//
// -----------------
// [triggers]
//
// # Call script running.sh when test goes to state=RUNNING, status=PASS
// RUNNING/PASS running.sh
//
// # Call script running.sh any time state goes to RUNNING
// RUNNING/ running.sh
//
// # Call script onpass.sh any time status goes to PASS
// PASS/ onpass.sh
// -----------------
//
// Scripts called will have; test-id test-rundir trigger test-name item-path state status event-time, added to the commandline.
//
// HINT
//
// To start an xterm (useful for debugging), use a command line like the following:
//
// -----------------
// [triggers]
// COMPLETED/ xterm -e bash -s --
// -----------------
//
// NOTE: There is a trailing space after the --
//
// There are a number of environment variables available to the trigger script
// but since triggers can be called in various contexts not all variables are
// available at all times. The trigger script should check for the variable and
// fail gracefully if it doesn't exist.
//
// .Environment variables visible to the trigger script
// [width="90%",cols="^,2m",frame="topbot",options="header"]
// |======================
// |Variable | Purpose
// | MT_TEST_RUN_DIR | The directory where Megatest ran this test
// | MT_CMDINFO | Encoded command data for the test
// | MT_DEBUG_MODE | Used to pass the debug mode to nested calls to Megatest
// | MT_RUN_AREA_HOME | Megatest home area
// | MT_TESTSUITENAME | The name of this testsuite or area
// | MT_TEST_NAME | The name of this test
// | MT_ITEM_INFO | The variable and values for the test item
// | MT_MEGATEST | Which Megatest binary is being used by this area
// | MT_TARGET | The target variable values, separated by '/'
// | MT_LINKTREE | The base of the link tree where all run tests can be found
// | MT_ITEMPATH | The values of the item path variables, separated by '/'
// | MT_RUNNAME | The name of the run
// |======================
//
//
// Override the Toplevel HTML File
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
//
// Megatest generates a simple html file summary for top level tests of
// iterated tests. The generation can be overridden. NOTE: the output of
// the script is captured from stdout to create the html.
//
//
// .For test "runfirst" override the toplevel generation with a script "mysummary.sh"
// -----------------
// # Override the rollup for specific tests
// [testrollup]
// runfirst mysummary.sh
// -----------------
//
// Archiving Setup
// ---------------
//
// In megatest.config add the following sections:
//
// .megatest.config
// --------------
// [archive]
// # where to get bup executable
// # bup /path/to/bup
//
// [archive-disks]
//
// # Archives will be organised under these paths like this:
// # <testsuite>/<creationdate>
// # Within the archive the data is structured like this:
// # <target>/<runname>/<test>/
// archive0 /mfs/myarchive-data/adisk1
// --------------
//
// Environment Variables
// ---------------------
//
// It is often necessary to capture and or manipulate environment
// variables. Megatest has some facilities built in to help.
//
// Capture variables
// ~~~~~~~~~~~~~~~~~
//
// .Commands
// ------------------------------
// # capture the current enviroment into a db called envdat.db under
// # the context "before"
// megatest -envcap before
//
// # capture the current environment into a db called startup.db with
// # context "after"
// megatest -envcap after startup.db
//
// # write the diff from before to after
// megatest -envdelta before-after -dumpmode bash
// ------------------------------
//
// Dump modes include bash, csh and config. You can include config data
// into megatest.config, runconfigs.config and testconfig files. This is
// useful for capturing a complex environment in a special-purpose test
// and then utilizing that environment in downstream tests.
//
// .Example of generating and using config data
// ------------------------------
// megatest -envcap original
// # do some stuff here
// megatest -envcap munged
// megatest -envdelta original-munged -dumpmode ini -o modified.config
// ------------------------------
//
// Then in runconfigs.config
//
// .Example of using modified.config in a testconfig
// ------------------------------
// [pre-launch-env-vars]
// [include modified.config]
// ------------------------------
//
// Managing Old Runs
// -----------------
//
// It is often desired to keep some older runs around but this must be balanced with the costs of disk space.
//
// . Use -remove-keep
// . Use -archive (can also be done from the -remove-keep interface)
// . use -remove-runs with -keep-records
//
// .For each target, remove all runs but the most recent 3 if they are over 1 week old
// ---------------------
// # use -precmd 'sleep 5;nbfake' to limit overloading the host computer but to allow the removes to run in parallel.
// megatest -actions print,remove-runs -remove-keep 3 -target %/%/%/% -runname % -age 1w -precmd 'sleep 5;nbfake'"
// ---------------------
//
// Nested Runs
// -----------
//
// A Megatest test can run a full Megatest run in either the same
// Megatest area or in another area. This is a powerful way of chaining
// complex suites of tests and or actions.
//
// If you are not using the current area you can use ezsteps to retrieve
// and setup the sub-Megatest run area.
//
// In the testconfig:
// ---------------
// [subrun]
//
// # Required: wait for the run or just launch it
// # if no then the run will be an automatic PASS irrespective of the actual result
// run-wait yes|no
//
// # Optional: where to execute the run. Default is the current runarea
// run-area /some/path/to/megatest/area
//
// # Optional: method to use to determine pass/fail status of the run
// # auto (default) - roll up the net state/status of the sub-run
// # logpro - use the provided logpro rules, happens automatically if there is a logpro section
// # passfail auto|logpro
// # Example of logpro:
// passfail logpro
//
// # Optional:
// logpro ;; if this section exists then logpro is used to determine pass/fail
// (expect:required in "LogFileBody" >= 1 "At least one pass" #/PASS/)
// (expect:error in "LogFileBody" = 0 "No FAILs allowed" #/FAIL/)
//
// # Optional: target translator, default is to use the parent target
// target #{shell somescript.sh}
//
// # Optional: runname translator/generator, default is to use the parent runname
// run-name #{somescript.sh}
//
// # Optional: testpatt spec, default is to first look for TESTPATT spec from runconfigs unless there is a contour spec
// test-patt %/item1,test2
//
// # Optional: contour spec, use the named contour from the megatest.config contour spec
// contour contourname ### NOTE: Not implemented yet! Let us know if you need this feature.
//
// # Optional: mode-patt, use this spec for testpatt from runconfigs
// mode-patt TESTPATT
//
// # Optional: tag-expr, use this tag-expr to select tests
// tag-expr quick
//
// # Optional: (not yet implemented, remove-runs is always propagated at this time), propagate these actions from the parent
// # test
// # Note// default is % for all
// propagate remove-runs archive ...
//
// ---------------
//
// Programming API
// ---------------
//
// These routines can be called from the megatest repl.
//
// .API Keys Related Calls
// [width="70%",cols="^,2m,2m,2m",frame="topbot",options="header,footer"]
// |======================
// |API Call | Purpose comments | Returns | Comments
// |(rmt:get-keys run-id) | | ( key1 key2 ... ) |
// | (rmt:get-key-val-pairs run-id) | | #t=success/#f=fail | Works only if the server is still reachable
// |======================
//
//
// :numbered!: