Index: docs/manual/Makefile
==================================================================
--- docs/manual/Makefile
+++ docs/manual/Makefile
@@ -30,15 +30,15 @@
# asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 design_spec.txt
#
all : server.ps megatest_manual.html client.ps complex-itemmap.png megatest_manual.pdf
-megatest_manual.html : megatest_manual.txt *.txt installation.txt *png
+megatest_manual.html : megatest_manual.txt *.txt installation.txt *png *.dot
asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 megatest_manual.txt
# dos2unix megatest_manual.html
-megatest_manual.pdf : megatest_manual.txt *.txt *png
+megatest_manual.pdf : megatest_manual.txt *.txt *png *.dot
a2x -a toc -f pdf megatest_manual.txt
server.ps : server.dot
dot -Tps server.dot > server.ps
ADDED docs/manual/bisecting.dot
Index: docs/manual/bisecting.dot
==================================================================
--- /dev/null
+++ docs/manual/bisecting.dot
@@ -0,0 +1,31 @@
+// Copyright 2021, Matthew Welland.
+//
+// This file is part of Megatest.
+//
+// Megatest is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// Megatest is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with Megatest. If not, see .
+//
+digraph G {
+ rankdir=LR
+ subgraph cluster_1 {
+ node [style=filled,shape=box];
+
+ B [label="B\nProblem is here"];
+ E [label="E\nProblem manifests here"];
+ A -> B;
+ B -> C;
+ C -> D;
+ D -> E;
+ }
+
+}
ADDED docs/manual/bisecting.png
Index: docs/manual/bisecting.png
==================================================================
--- /dev/null
+++ docs/manual/bisecting.png
cannot compute difference between binary files
ADDED docs/manual/debugging.txt
Index: docs/manual/debugging.txt
==================================================================
--- /dev/null
+++ docs/manual/debugging.txt
@@ -0,0 +1,110 @@
+// Copyright 2021, Matthew Welland.
+//
+// This file is part of Megatest.
+//
+// Megatest is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// Megatest is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with Megatest. If not, see .
+
+Debugging
+---------
+
+A word on Bisecting
+~~~~~~~~~~~~~~~~~~~
+
+Bisecting is a debug strategy intended to speed up finding the root
+cause.
+
+.A complex process with a problem found in stage "E"
+["graphviz", "bisecting.png"]
+----------------------------------------------------------------------
+include::bisecting.dot[]
+----------------------------------------------------------------------
+
+It is common to start debugging where the problem was observed and
+then work back. However by inspecting the output at stage "C" in the
+example above you would potentially save a lot of debug effort, this
+is similar to the feature in source control tools like git and fossil
+called biseceting.
+
+To know where to look for problems you will want to look at the log files at various stages in the execution process.
+
+.A simplified diagram of the stages Megatest goes through to run a test.
+["graphviz", "megatest-test-stages.png"]
+----------------------------------------------------------------------
+include::megatest-test-stages.dot[]
+----------------------------------------------------------------------
+
+.How to check variable values and inspect logs at each stage
+[width="80%",cols="<,2m,2m",frame="topbot",options="header"]
+|======================
+|Stage | How to inspect | Watch for
+|A: post config processing | megatest -show-config -target your/target | #f (failed var processing)
+|B: post runconfig | megatest -show-runconfig -target your/target |
+|C: processing testconfigs | inspect output from "megatest -run ..." | Messages indicating issues process configs, dependency problems.
+|D: process testconfig for test launch | inspect output from megatest runner | Zero items (items expansion yielded no items)
+|E,F: launching test | start test xterm, look at mt_launch.log | Did your batch system accept the job? Has the job landed on a machine?
+|G: starting test | look at your batch systems logs for the process | Did the megatest -execute process start and run?
+|H,H1,H2: step exectution | look at .log, .html and your own internal logs | Do you have sufficiently tight logpro rules? You must always have a "required" rule!
+|======================
+
+Examining The Test Logs and Environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Test Control Panel - xterm
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+From the dashboard click on a test PASS/FAIL button. This brings up a
+test control panel. Aproximately near the center left of the window
+there is a button "Start Xterm". Push this to get an xterm with the
+full context and environment loaded for that test. You can run scripts
+or ezsteps by copying from the testconfig (hint, load up the
+testconfig in a separate text editor window).
+
+With more recent versions of Megatest you can step through your test
+from the test control panel. Click on the cell labeled "rerun this
+step" to only rerun the step or click on "restart from here" to rerun
+that step and downstream steps.
+
+NOTE 1: visual feedback can take some time, give it a few seconds and
+you will see the step change color to blue as it starts running.
+
+NOTE 2: steping through only works if you are using ezsteps.
+
+Config File Processing
+^^^^^^^^^^^^^^^^^^^^^^
+
+As described above it is often helpful to know the content of
+variables in various contexts as Megatest works through the actions
+needed to run your tests. A handy technique is to force the startup of
+an xterm in the context being examined.
+
+For example, if an item list is not being generated as expected you
+can inject the startup of an xterm as if it were an item:
+
+.Original items table
+-----------------
+[items]
+CELLNAME [system getcellname.sh]
+-----------------
+
+.Items table modified for debug
+-----------------
+[items]
+DEBUG [system xterm]
+CELLNAME [system getcellnames.sh]
+-----------------
+
+When this test is run an xterm will pop up. In that xterm the
+environment is exactly that in which the script "getcellnames.sh"
+would run. You can now debug the script to find out why it isn't
+working as expected.
Index: docs/manual/howto.txt
==================================================================
--- docs/manual/howto.txt
+++ docs/manual/howto.txt
@@ -110,15 +110,15 @@
# if defined and not "no" flexi-launcher will bypass launcher unless there is no
# match.
flexi-launcher yes
------------------------
-Tricks
-------
+Tricks and Tips
+---------------
-This section is a compendium of a various useful tricks for debugging,
-configuring and generally getting the most out of Megatest.
+This section is a collection of a various useful tricks for that
+didn't quite fit elsewhere.
Limiting your running jobs
~~~~~~~~~~~~~~~~~~~~~~~~~~
The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.
@@ -136,50 +136,10 @@
[jobgroups]
group1 10
custdes 4
---------------
-Debugging Tricks
-----------------
-
-Examining The Environment
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Test Control Panel - xterm
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the
-window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run
-scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate gvim or emacs window). This is the easiest way
-to debug your tests.
-
-During Config File Processing
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It is often helpful to know the content of variables in various
-contexts as Megatest does the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined.
-
-For example, if an item list is not being generated as expected you
-can inject the startup of an xterm as if it were an item:
-
-.Original items table
------------------
-[items]
-CELLNAME [system getcellname.sh]
------------------
-
-.Items table modified for debug
------------------
-[items]
-DEBUG [system xterm]
-CELLNAME [system getcellnames.sh]
------------------
-
-When this test is run an xterm will pop up. In that xterm the
-environment is exactly that in which the script "getcellnames.sh"
-would run. You can now debug the script to find out why it isn't
-working as expected.
Organising Your Tests and Tasks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default location "tests" for storing tests can be extended by
ADDED docs/manual/megatest-test-stages.dot
Index: docs/manual/megatest-test-stages.dot
==================================================================
--- /dev/null
+++ docs/manual/megatest-test-stages.dot
@@ -0,0 +1,42 @@
+// Copyright 2021, Matthew Welland.
+//
+// This file is part of Megatest.
+//
+// Megatest is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// Megatest is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with Megatest. If not, see .
+//
+digraph G {
+ // rankdir=LR
+ subgraph cluster_1 {
+ node [style=filled,shape=box];
+ A [label="A: Process megatest.config"]
+ B [label="B: Process runconfig.config"]
+ A -> B [label="resolve vars"]
+ B2 [label="B2: Resolve variables"]
+ B -> B2 -> A
+ C [label="C: Process testconfigs (find tests to run)"]
+ D [label="D: Process testconfig for test of interest"]
+ E [label="E: Set vars for launching test"]
+ F [label="F: Launch into batch system, ssh, batch system\nand different hosts can all \nimpact variable values"]
+ G [label="G: Process testconfig again with all available variables"]
+ H [label="H: Start test"]
+ H1 [label="H1: Start step"]
+ H2 [label="H2: Execute step script"]
+ I [label="I: End step, process logfile with logpro"]
+ I -> H [label="Run remaining steps"]
+ H-> H1 -> H2 -> I
+ B->C->D->E->F->G->H
+ I->D
+ }
+
+}
ADDED docs/manual/megatest-test-stages.png
Index: docs/manual/megatest-test-stages.png
==================================================================
--- /dev/null
+++ docs/manual/megatest-test-stages.png
cannot compute difference between binary files
Index: docs/manual/megatest_manual.html
==================================================================
--- docs/manual/megatest_manual.html
+++ docs/manual/megatest_manual.html
@@ -1,10 +1,10 @@
-
+
The Megatest Users Manual