Index: docs/manual/howto.txt
==================================================================
--- docs/manual/howto.txt
+++ docs/manual/howto.txt
@@ -13,196 +13,213 @@
// You should have received a copy of the GNU General Public License
// along with Megatest. If not, see .
//
// Copyright 2006-2012, Matthew Welland.
-How To Do Things
-----------------
-
-Process Runs
-~~~~~~~~~~~~
-
-Remove Runs
-^^^^^^^^^^^
-
-From the dashboard click on the button (PASS/FAIL...) for one of the tests. From the test control panel that
-comes up push the clean test button. The command field will be prefilled with a template command for removing
-that test. You can edit the command, for example change the argument to -testpatt to "%" to remove all tests.
-
-.Remove the test diskperf and all it's items
-----------------
-megatest -remove-runs -target ubuntu/nfs/none -runname ww28.1a -testpatt diskperf/% -v
-----------------
-
-.Remove all tests for all runs and all targets
-----------------
-megatest -remove-runs -target %/%/% -runname % -testpatt % -v
-----------------
-
-Archive Runs
-^^^^^^^^^^^^
-
-Megatest supports using the bup backup tool (https://bup.github.io/) to archive your tests for efficient storage
-and retrieval. Archived data can be rapidly retrieved if needed. The metadata for the run (PASS/FAIL status, run
-durations, time stamps etc.) are all preserved in the megatest database.
-
-For setup information see the Archiving topic in the reference section of this manual.
-
-To Archive
-++++++++++
-
-Hint: use the test control panel to create a template command by pushing the "Archive Tests" button.
-
-.Archive a full run
-----------------
-megatest -target ubuntu/nfs/none -runname ww28.1a -archive save-remove -testpatt %
-----------------
-
-To Restore
-++++++++++
-
-.Retrieve a single test
-----------------
-megatest -target ubuntu/nfs/none -runname ww28.1a -archive restore -testpatt diskperf/%
-----------------
-
-Hint: You can browse the archive using bup commands directly.
-
-----------------
-bup -d /path/to/bup/archive ftp
-----------------
-
-Submit jobs to Host Types based on Test Name
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.In megatest.config
-------------------------
-[host-types]
-general ssh #{getbgesthost general}
-nbgeneral nbjob run JOBCOMMAND -log $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME.$MT_TESTNAME-$MT_ITEM_PATH.lgo
-
-[hosts]
-general cubian xena
-
-[launchers]
-envsetup general
-xor/%/n 4C16G
-% nbgeneral
-
-[jobtools]
-launcher bsub
-# if defined and not "no" flexi-launcher will bypass launcher unless there is no
-# match.
-flexi-launcher yes
-------------------------
-
-Tricks
-------
-
-This section is a compendium of a various useful tricks for debugging,
-configuring and generally getting the most out of Megatest.
-
-Limiting your running jobs
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.
-
-In your testconfig:
-
-----------------
-[test_meta]
-jobgroup group1
-----------------
-
-In your megatest.config:
-
----------------
-[jobgroups]
-group1 10
-custdes 4
----------------
-
-Debugging Tricks
-----------------
-
-Examining The Environment
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Test Control Panel - xterm
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the
-window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run
-scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate gvim or emacs window). This is the easiest way
-to debug your tests.
-
-During Config File Processing
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It is often helpful to know the content of variables in various
-contexts as Megatest does the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined.
-
-For example, if an item list is not being generated as expected you
-can inject the startup of an xterm as if it were an item:
-
-.Original items table
------------------
-[items]
-CELLNAME [system getcellname.sh]
------------------
-
-.Items table modified for debug
------------------
-[items]
-DEBUG [system xterm]
-CELLNAME [system getcellnames.sh]
------------------
-
-When this test is run an xterm will pop up. In that xterm the
-environment is exactly that in which the script "getcellnames.sh"
-would run. You can now debug the script to find out why it isn't
-working as expected.
-
-Organising Your Tests and Tasks
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The default location "tests" for storing tests can be extended by
-adding to your tests-paths section.
-
-----------------------------
-[misc]
-parent #{shell dirname $(readlink -f .)}
-
-[tests-paths]
-1 #{get misc parent}/simplerun/tests
-----------------------------
-
-The above example shows how you can use addition sections in your
-config file to do complex processing. By putting results of relatively
-slow operations into variables the processing of your configs can be
-kept fast.
-
-Alternative Method for Running your Job Script
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.Directly running job in testconfig
--------------------
-[setup]
-runscript main.csh
--------------------
-
-The runscript method is essentially a brute force way to run scripts where the
-user is responsible for setting STATE and STATUS and managing the details of running a test.
-
-Debugging Server Problems
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Some handy Unix commands to track down issues with servers not
-communicating with your test manager processes. Please put in tickets
-at https://www.kiatoa.com/fossils/megatest if you have problems with
-servers getting stuck.
-
-----------------
-sudo lsof -i
-sudo netstat -lptu
-sudo netstat -tulpn
+How To Do Things
+----------------
+
+Process Runs
+~~~~~~~~~~~~
+
+Remove Runs
+^^^^^^^^^^^
+
+From the dashboard click on the button (PASS/FAIL...) for one of the tests. From the test control panel that
+comes up push the clean test button. The command field will be prefilled with a template command for removing
+that test. You can edit the command, for example change the argument to -testpatt to "%" to remove all tests.
+
+.Remove the test diskperf and all it's items
+----------------
+megatest -remove-runs -target ubuntu/nfs/none -runname ww28.1a -testpatt diskperf/% -v
+----------------
+
+.Remove all tests for all runs and all targets
+----------------
+megatest -remove-runs -target %/%/% -runname % -testpatt % -v
+----------------
+
+Archive Runs
+^^^^^^^^^^^^
+
+Megatest supports using the bup backup tool (https://bup.github.io/) to archive your tests for efficient storage
+and retrieval. Archived data can be rapidly retrieved if needed. The metadata for the run (PASS/FAIL status, run
+durations, time stamps etc.) are all preserved in the megatest database.
+
+For setup information see the Archiving topic in the reference section of this manual.
+
+To Archive
+++++++++++
+
+Hint: use the test control panel to create a template command by pushing the "Archive Tests" button.
+
+.Archive a full run
+----------------
+megatest -target ubuntu/nfs/none -runname ww28.1a -archive save-remove -testpatt %
+----------------
+
+To Restore
+++++++++++
+
+.Retrieve a single test
+----------------
+megatest -target ubuntu/nfs/none -runname ww28.1a -archive restore -testpatt diskperf/%
+----------------
+
+Hint: You can browse the archive using bup commands directly.
+
+----------------
+bup -d /path/to/bup/archive ftp
+----------------
+
+Pass Data from Test to Test
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.To save the data call archive save within your test:
+----------------
+megatest -archive save
+----------------
+
+.To retrieve the data call archive get using patterns as needed
+----------------
+# Put the retrieved data into /tmp
+DESTPATH=/tmp/$USER/$MT_TARGET/$MT_RUN_NAME/$MT_TESTNAME/$MT_ITEMPATH/my_data
+mkdir -p $DESTPATH
+megatest -archive get -runname % -dest $DESTPATH
+----------------
+
+
+Submit jobs to Host Types based on Test Name
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.In megatest.config
+------------------------
+[host-types]
+general ssh #{getbgesthost general}
+nbgeneral nbjob run JOBCOMMAND -log $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME.$MT_TESTNAME-$MT_ITEM_PATH.lgo
+
+[hosts]
+general cubian xena
+
+[launchers]
+envsetup general
+xor/%/n 4C16G
+% nbgeneral
+
+[jobtools]
+launcher bsub
+# if defined and not "no" flexi-launcher will bypass launcher unless there is no
+# match.
+flexi-launcher yes
+------------------------
+
+Tricks
+------
+
+This section is a compendium of a various useful tricks for debugging,
+configuring and generally getting the most out of Megatest.
+
+Limiting your running jobs
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.
+
+In your testconfig:
+
+----------------
+[test_meta]
+jobgroup group1
+----------------
+
+In your megatest.config:
+
+---------------
+[jobgroups]
+group1 10
+custdes 4
+---------------
+
+Debugging Tricks
+----------------
+
+Examining The Environment
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Test Control Panel - xterm
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the
+window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run
+scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate gvim or emacs window). This is the easiest way
+to debug your tests.
+
+During Config File Processing
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is often helpful to know the content of variables in various
+contexts as Megatest does the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined.
+
+For example, if an item list is not being generated as expected you
+can inject the startup of an xterm as if it were an item:
+
+.Original items table
+-----------------
+[items]
+CELLNAME [system getcellname.sh]
+-----------------
+
+.Items table modified for debug
+-----------------
+[items]
+DEBUG [system xterm]
+CELLNAME [system getcellnames.sh]
+-----------------
+
+When this test is run an xterm will pop up. In that xterm the
+environment is exactly that in which the script "getcellnames.sh"
+would run. You can now debug the script to find out why it isn't
+working as expected.
+
+Organising Your Tests and Tasks
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The default location "tests" for storing tests can be extended by
+adding to your tests-paths section.
+
+----------------------------
+[misc]
+parent #{shell dirname $(readlink -f .)}
+
+[tests-paths]
+1 #{get misc parent}/simplerun/tests
+----------------------------
+
+The above example shows how you can use addition sections in your
+config file to do complex processing. By putting results of relatively
+slow operations into variables the processing of your configs can be
+kept fast.
+
+Alternative Method for Running your Job Script
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.Directly running job in testconfig
+-------------------
+[setup]
+runscript main.csh
+-------------------
+
+The runscript method is essentially a brute force way to run scripts where the
+user is responsible for setting STATE and STATUS and managing the details of running a test.
+
+Debugging Server Problems
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Some handy Unix commands to track down issues with servers not
+communicating with your test manager processes. Please put in tickets
+at https://www.kiatoa.com/fossils/megatest if you have problems with
+servers getting stuck.
+
+----------------
+sudo lsof -i
+sudo netstat -lptu
+sudo netstat -tulpn
----------------
Index: docs/manual/megatest_manual.html
==================================================================
--- docs/manual/megatest_manual.html
+++ docs/manual/megatest_manual.html
@@ -1452,10 +1452,26 @@
bup -d /path/to/bup/archive ftp
+
+
+
Pass Data from Test to Test
+
+
To save the data call archive save within your test:
+
+
megatest -archive save
+
+
+
To retrieve the data call archive get using patterns as needed
+
+
# Put the retrieved data into /tmp
+DESTPATH=/tmp/$USER/$MT_TARGET/$MT_RUN_NAME/$MT_TESTNAME/$MT_ITEMPATH/my_data
+mkdir -p $DESTPATH
+megatest -archive get -runname % -dest $DESTPATH