// Copyright 2021, Matthew Welland.
//
// This file is part of Megatest.
//
// Megatest is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Megatest is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Megatest. If not, see <http://www.gnu.org/licenses/>.
Debugging
---------
A word on Bisecting
~~~~~~~~~~~~~~~~~~~
Bisecting is a debug strategy intended to speed up finding the root
cause.
.A complex process with a problem found in stage "E"
["graphviz", "bisecting.png"]
----------------------------------------------------------------------
include::bisecting.dot[]
----------------------------------------------------------------------
It is common to start debugging where the problem was observed and
then work back. However by inspecting the output at stage "C" in the
example above you would potentially save a lot of debug effort, this
is similar to the feature in source control tools like git and fossil
called biseceting.
To know where to look for problems you will want to look at the log files at various stages in the execution process.
.A simplified diagram of the stages Megatest goes through to run a test.
["graphviz", "megatest-test-stages.png"]
----------------------------------------------------------------------
include::megatest-test-stages.dot[]
----------------------------------------------------------------------
.How to check variable values and inspect logs at each stage
[width="80%",cols="<,2m,2m",frame="topbot",options="header"]
|======================
|Stage | How to inspect | Watch for
|A: post config processing | megatest -show-config -target your/target | #f (failed var processing)
|B: post runconfig | megatest -show-runconfig -target your/target |
|C: processing testconfigs | inspect output from "megatest -run ..." | Messages indicating issues process configs, dependency problems.
|D: process testconfig for test launch | inspect output from megatest runner | Zero items (items expansion yielded no items)
|E,F: launching test | start test xterm, look at mt_launch.log | Did your batch system accept the job? Has the job landed on a machine?
|G: starting test | look at your batch systems logs for the process | Did the megatest -execute process start and run?
|H,H1,H2: step exectution | look at <stepname>.log, <stepname>.html and your own internal logs | Do you have sufficiently tight logpro rules? You must always have a "required" rule!
|======================
Examining The Test Logs and Environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Test Control Panel - xterm
^^^^^^^^^^^^^^^^^^^^^^^^^^
From the dashboard click on a test PASS/FAIL button. This brings up a
test control panel. Aproximately near the center left of the window
there is a button "Start Xterm". Push this to get an xterm with the
full context and environment loaded for that test. You can run scripts
or ezsteps by copying from the testconfig (hint, load up the
testconfig in a separate text editor window).
With more recent versions of Megatest you can step through your test
from the test control panel. Click on the cell labeled "rerun this
step" to only rerun the step or click on "restart from here" to rerun
that step and downstream steps.
NOTE 1: visual feedback can take some time, give it a few seconds and
you will see the step change color to blue as it starts running.
NOTE 2: steping through only works if you are using ezsteps.
Config File Processing
^^^^^^^^^^^^^^^^^^^^^^
As described above it is often helpful to know the content of
variables in various contexts as Megatest works through the actions
needed to run your tests. A handy technique is to force the startup of
an xterm in the context being examined.
For example, if an item list is not being generated as expected you
can inject the startup of an xterm as if it were an item:
.Original items table
-----------------
[items]
CELLNAME [system getcellname.sh]
-----------------
.Items table modified for debug
-----------------
[items]
DEBUG [system xterm]
CELLNAME [system getcellnames.sh]
-----------------
When this test is run an xterm will pop up. In that xterm the
environment is exactly that in which the script "getcellnames.sh"
would run. You can now debug the script to find out why it isn't
working as expected.