Megatest

</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="_preface">Preface</h2>
<div class="sectionbody">
<div class="paragraph"><p>This book is organised as three sub-books; getting started, writing tests and reference.</p></div>
<div class="listingblock">
<div class="title">License</div>
<div class="content monospaced">
<pre>    Copyright 2006-2017, Matthew Welland.

    This document is part of Megatest.

    Megatest is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    Megatest is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with Megatest.  If not, see &lt;http://www.gnu.org/licenses/&gt;.</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_why_megatest">Why Megatest?</h2>
<div class="sectionbody">
<div class="paragraph"><p>The Megatest project was started for two reasons, the first was an
immediate and pressing need for a generalized tool to manage a suite
of regression tests and the second was the fact that the author had
written or maintained several such tools at different companies over
the years and it seemed a good thing to have a single open source
tool, flexible enough to meet the needs of any team doing continuous
integrating and or running a complex suite of tests for release
qualification.</p></div>
of regression tests and the second was the fact that I had written or
maintained several such tools at different companies over the years. I
thought a single open source tool, flexible enough to meet the needs
of any team doing continuous integrating and or running a complex
suite of tests for release qualification would solve some problems for
me and for others.</p></div>
<div class="literalblock">
<div class="content monospaced">
<pre>-- Matt Welland, original author of the Megatest tool suite.</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_design_philosophy">Megatest Design Philosophy</h2>
<div class="sectionbody">
<div class="paragraph"><p>Megatest is intended to provide the minimum needed resources to make
writing a suite of tests and tasks for implementing continuous build
for software, design engineering or process control (via owlfs for
example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or
FAIL of a test or task. In most cases megatest is best used in
conjunction with logpro or a similar tool to parse, analyze and decide
<div class="paragraph"><p>Megatest is a distributed system intended to provide the minimum needed
resources to make writing a suite of tests and tasks for implementing
continuous build for software, design engineering or process control (via
owlfs for example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or FAIL
of a test or task. In most cases megatest is best used in conjunction with
logpro or a similar tool to parse, analyze and decide on the test outcome.</p></div>
on the test outcome.</p></div>
<div class="ulist"><ul>
<li>
<p>
Self-checking -Repeatable strive for directed or self-checking test
   as opposed to delta based tests
Self-checking - make it as easy as possible to write self-checking
   tests (as opposed to using deltas, i.e. tests that compare with a
   previous measurement to deterine PASS/FAIL).
</p>
</li>
<li>
<p>
Traceable - environment variables, host OS and other possibly influential
   variables are captured and kept recorded.
Traceable - environment variables, host OS and other possibly
   influential variables are captured and kept recorded.
</p>
</li>
<li>
<p>
Immutable - once a test is run it cannot be easily overwritten or
   modified accidentally.
</p>
</li>
<li>
<p>
Repeatable - test results can be recreated in the future using all
   the original variables.
</p>
</li>
<li>
<p>
Immutable - once this test is run it cannot be easily overwritten or
   accidentally modified.
</p>
</li>
<li>
<p>
Repeatable - this test result can be recreated in the future
</p>
</li>
<li>
<p>
Relocatable - the testsuite or automation area can be checked out and the tests run anywhere
Relocatable - the testsuite or automation area can be checked out
   and the tests run anywhere in the disk hierarchy.
</p>
</li>
<li>
<p>
Encapsulated - the tests run in self-contained directories and all inputs
   and outputs to the process can be found in the run areas.
Encapsulated - the tests run in self-contained directories and all
   inputs and outputs to the process can be found in the run areas.
</p>
</li>
<li>
<p>
Deployable - anyone on the team, at any site, at any time can run the flow
Deployable - a testsuite is self-contained and can be bundled with
   a software project and easily used by others with little to no
   setup burden.
</p>
</li>
</ul></div>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_architecture">Megatest Architecture</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_data_separation">Data separation</h3>
<div class="paragraph"><p>All data to specify the tests and configure the system is stored in
plain text files. All system state is stored in an sqlite3
database. Tests are launched using the launching system available for
the distributed compute platform in use. A template script is provided
plain text config files. All system state is stored in an sqlite3
database.</p></div>
</div>
<div class="sect2">
<h3 id="_distributed_compute">Distributed Compute</h3>
<div class="paragraph"><p>Tests are launched using the launching system available for the
distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database.</p></div>
sqlite3 database. Megatest has been used with the Intel Netbatch and
lsf (also known as openlava) batch systems and it should be
straightforward to use it with other similar systems.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_road_map">Road Map</h2>
<h2 id="_overview">Overview</h2>
<div class="sectionbody">
<div class="paragraph"><p>Note 1: This road-map is still evolving and subject to change without notice.</p></div>
<div class="sect2">
<h3 id="_architecture_refactor">Architecture Refactor</h3>
<div class="sect3">
<h3 id="_stand_alone_megatest_area">Stand-alone Megatest Area</h3>
<div class="paragraph"><p>A single, stand-alone, Megatest based testsuite or "area" is
sufficient for most validation, automation and build problems.</p></div>
<div class="imageblock">
<h4 id="_goals">Goals</h4>
<div class="olist arabic"><ol class="arabic">
<div class="content">
<li>
<p>
Reduce load on the file system. Sqlite3 files on network filesystem can be
  a burden.
<img src="megatest-stand-alone-area.png" alt="Static">
</p>
</li>
<li>
<p>
Reduce number of servers and frequency of start/stop. This is mostly an
  issue of clutter but also a reduction in "moving parts".
</p>
</li>
<li>
<p>
Coalesce activities to a single home host where possible. Give the user
  feedback that they have started the dashboard on a host other than the
  home host.
</p>
</li>
<li>
<p>
Reduce number of processes involved in managing running tests.
</p>
</li>
</ol></div>
</div>
</div>
<div class="paragraph"><p>Megatest is designed as a distributed or decoupled system. This means
you can run the areas stand-alone with no additional
infrastructure. I.e. there are no databases, web servers or other
centralized resources needed. However as your needs grow you can
integrate multiple areas into a bigger system.</p></div>
<div class="sect3">
<h4 id="_changes_needed">Changes Needed</h4>
<h4 id="_component_descriptions">Component Descriptions</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
ACID compliant db will be on /tmp and synced to megatest.db with a five
  second max delay.
Multi-area dashboard and xterm. A gui (the dashboard) is usually the
  best option for controlling and launching runs but all operations
  can also be done from the commandline. Note: The not yet released
  multi-area dashboard replaces the old dashboard for browsing and
  controlling runs but for managing a single area the old dashboard
  works very well.
</p>
</li>
<li>
<p>
Read/writes to db for processes on homehost will go direct to /tmp
  megatest.db file.
Area/testsuite. This is your testsuite or automation definition and
  consists of the information in megatest.config, runconfigs.config
  and your testconfigs along with any custom scripting that can&#8217;t be
  done with the native Megatest features.
</p>
</li>
<li>
<p>
If your testsuite or build automation is too large to run on a
Read/wites fron non-homehost processes will go through one server. Bulk
  reads (e.g. for dashboard or list-runs) will be cached on the current host
  single instance you can distribute your jobs into a compute server
  pool. The only current requirements are password-less ssh access and
  in /tmp and synced from the home megatest.db in the testsuite area.
</p>
</li>
<li>
<p>
Db syncs rely on the target db file timestame minus some margin.
</p>
</li>
<li>
<p>
Since bulk reads do not use the server we can switch to simple RPC for the
  network transport.
  a network filesystem.
</p>
</li>
<li>
<p>
Test running manager process extended to manage multiple running tests.
</p>
</li>
</ol></div>
</div>
</div>
<div class="sect2">
<h3 id="_current_items">Current Items</h3>
<div class="sect3">
<h3 id="_full_system_architecture">Full System Architecture</h3>
<div class="imageblock">
<h4 id="_ww05_migrate_to_inmem_db">ww05 - migrate to inmem-db</h4>
<div class="olist arabic"><ol class="arabic">
<div class="content">
<li>
<p>
Switch to inmem db with fast sync to on disk db&#8217;s [DONE]
</p>
</li>
<li>
<p>
Server polls tasks table for next action
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Task table used for tracking runner process [DONE]
</p>
</li>
<li>
<p>
Task table used for jobs to run
</p>
</li>
<li>
<p>
Task table used for queueing runner actions (remove runs, cleanRunExecute, etc)
</p>
</li>
</ol></div>
<img src="megatest-system-architecture.png" alt="Static">
</div>
</li>
</ol></div>
<div class="paragraph"><p>shifting, note that the preceding blank line is needed.</p></div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_installation">Installation</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_dependencies">Dependencies</h3>
<div class="paragraph"><p>Chicken scheme and a number of "eggs" are required for building
Megatest. See the script installall.sh in the utils directory of the
source distribution for an automated way to install everything
needed for building Megatest on Linux.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_getting_started">Getting Started</h2>
<div class="sectionbody">
<div class="openblock">
<div class="listingblock">
<div class="title">Getting started with Megatest</div>
<div class="content">
<div class="paragraph"><p>Creating a testsuite or flow and your first test or task.</p></div>
<div class="content monospaced">
<pre>Creating a testsuite or flow and your first test or task.</pre>
</div></div>
<div class="paragraph"><p>After installing Megatest you can create a flow or testsuite and add some
tests using the helpers. Here is a quickstart sequence to get you up and
running your first automated testsuite.</p></div>
<div class="sect2">
<h3 id="_creating_a_megatest_area">Creating a Megatest Area</h3>
<div class="sect3">

<div class="listingblock">
<div class="title">Starting dashboard</div>
<div class="content monospaced">
<pre>dashboard -rows 24</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_study_plan">Study Plan</h2>
<div class="sectionbody">
<div class="paragraph"><p>Megatest is an extensive program with a lot to learn. Following are some paths through the material to smooth the learning path.</p></div>
<div class="sect2">
<h3 id="_basic_concepts_suggest_you_pick_these_up_on_the_way">Basic Concepts (suggest you pick these up on the way)</h3>
<div class="ulist"><ul>
<li>
<p>
Components of automation; run, test, iteration
</p>
</li>
<li>
<p>
Selectors; target, runname, and testpatt
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_running_testsuites_or_automation">Running Testsuites or Automation</h3>
<div class="ulist"><ul>
<li>
<p>
Using the dashboard gui (recommended)
</p>
<div class="ulist"><ul>
<li>
<p>
Using the "Runs" panel.
</p>
</li>
<li>
<p>
Using the "Run Control" panel.
</p>
</li>
<li>
<p>
Using a test control panel
</p>
</li>
<li>
<p>
The Right Mouse Button menu
</p>
</li>
<li>
<p>
Debug features
</p>
<div class="ulist"><ul>
<li>
<p>
xterm
</p>
<div class="ulist"><ul>
<li>
<p>
pstree
</p>
</li>
<li>
<p>
log files; mt_copy.log, mt_launch.log
</p>
</li>
<li>
<p>
variables; megatest.csh, megatest.sh
</p>
</li>
<li>
<p>
testconfig dump, *testconfig
</p>
</li>
</ul></div>
</li>
<li>
<p>
State/status buttons
</p>
</li>
<li>
<p>
Run, Clean, KillReq
</p>
</li>
<li>
<p>
ReRunClean
</p>
</li>
</ul></div>
</li>
</ul></div>
</li>
<li>
<p>
Using the command line
</p>
<div class="ulist"><ul>
<li>
<p>
Getting help; megatest -h, megatest -manual
</p>
</li>
<li>
<p>
Starting runs; megatest -run
</p>
<div class="ulist"><ul>
<li>
<p>
Selection controls; -target, -runname and -testpatt
</p>
</li>
</ul></div>
</li>
</ul></div>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_writing_tests_and_flows">Writing Tests and Flows</h3>
<div class="ulist"><ul>
<li>
<p>
environment variables (table 5)
</p>
</li>
<li>
<p>
tests/<em>testname</em>/testconfig <a href="megatest_manual.html#_the_testconfig_file">testconfig details</a>
</p>
<div class="ulist"><ul>
<li>
<p>
ezsteps and logpro section
</p>
</li>
<li>
<p>
iteration (one test applied to many inputs), items, itemstable <a href="megatest_manual.html#_iteration">test iteration</a>
</p>
</li>
<li>
<p>
dependencies, waiton, itemmatch, itemwait <a href="megatest_manual.html#_requirements_section">test requirements</a>
</p>
</li>
<li>
<p>
miscellaneous; mode toplevel, runtimelim, skip on file, no file, script or on running, waiver propagation
</p>
</li>
</ul></div>
</li>
<li>
<p>
megatest areas
</p>
<div class="ulist"><ul>
<li>
<p>
megatest.config
</p>
</li>
<li>
<p>
runconfigs.config
</p>
</li>
<li>
<p>
config language features; include, shell, system, scheme, rp|realpath, getenv, get, rget, scriptinc <a href="megatest.html#_config_file_helpers">config file helpers</a>
</p>
</li>
</ul></div>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_advanced_topics">Advanced Topics</h3>
<div class="ulist"><ul>
<li>
<p>
Removing and keeping runs selectively <a href="megatest_manual.html#_managing_old_runs">managing runs</a>
</p>
</li>
<li>
<p>
Subruns <a href="megatest_manual.html#_nested_runs">nested runs</a>
</p>
</li>
<li>
<p>
Config file features <a href="megatest_manual.html#_config_file_helpers">config file features</a>
</p>
</li>
<li>
<p>
HTML output with -generate-html
</p>
</li>
<li>
<p>
Triggers, post run, state/status
</p>
</li>
<li>
<p>
MTLOWESTLOAD
</p>
</li>
<li>
<p>
flexilauncher
</p>
</li>
<li>
<p>
env delta and testconfig
</p>
</li>
<li>
<p>
capturing test data, extracting values from logpro and using them for pass/fail
</p>
</li>
<li>
<p>
mtutil, postgres connection, packets for cross-site/cross-user control (e.g. mcrun).
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_maintenance_and_troubleshooting">Maintenance and Troubleshooting</h3>
<div class="ulist"><ul>
<li>
<p>
cleanup-db, database structure of Megatest 1.6x
</p>
</li>
<li>
<p>
archiving
</p>
</li>
<li>
<p>
homehost management
</p>
</li>
<li>
<p>
show-runconfig
</p>
</li>
<li>
<p>
show-config
</p>
</li>
<li>
<p>
show with -debug 0,9
</p>
</li>
<li>
<p>
load management
</p>
</li>
</ul></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_writing_tests">Writing Tests</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_creating_a_new_test">Creating a new Test</h3>
<div class="paragraph"><p>The following steps will add a test "yourtestname" to your testsuite. This assumes

<div class="paragraph"><p>Hint: You can browse the archive using bup commands directly.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>bup -d /path/to/bup/archive ftp</pre>
</div></div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_pass_data_from_test_to_test">Pass Data from Test to Test</h3>
<div class="listingblock">
<div class="title">To save the data call archive save within your test:</div>
<div class="content monospaced">
<pre>megatest -archive save</pre>
</div></div>
<div class="listingblock">
<div class="title">To retrieve the data call archive get using patterns as needed</div>
<div class="content monospaced">
<pre># Put the retrieved data into /tmp
DESTPATH=/tmp/$USER/$MT_TARGET/$MT_RUN_NAME/$MT_TESTNAME/$MT_ITEMPATH/my_data
mkdir -p $DESTPATH
megatest -archive get -runname % -dest $DESTPATH</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_submit_jobs_to_host_types_based_on_test_name">Submit jobs to Host Types based on Test Name</h3>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[host-types]

</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_reference">Reference</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_megatest_use_modes">Megatest Use Modes</h3>
<table class="tableblock frame-topbot grid-all"
style="
width:80%;
">
<caption class="title">Table 2. Base commands</caption>
<col style="width:20%;">
<col style="width:40%;">
<col style="width:40%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >Use case                </th>
<th class="tableblock halign-left valign-top" > Megatest command     </th>
<th class="tableblock halign-left valign-top" >  mtutil</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">Start from scratch</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-rerun-all</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">restart</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">Rerun non-good completed</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-rerun-clean</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">rerunclean</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">Rerun all non-good and not completed yet</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-set-state-status KILLREQ; -rerun-</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">clean</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">killrerun</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Continue run</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-run</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">resume</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Remove run</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-remove-runs</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">clean</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Lock run</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-lock</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">lock</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Unlock run</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-unlock</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">unlock</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">killrun</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">-set-state-status KILLREQ; -kill-run</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="_config_file_helpers">Config File Helpers</h3>
<div class="paragraph"><p>Various helpers for more advanced config files.</p></div>
<table class="tableblock frame-topbot grid-all"
style="
width:80%;
">
<caption class="title">Table 2. Helpers</caption>
<caption class="title">Table 3. Helpers</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >Helper                      </th>

<div class="listingblock">
<div class="content monospaced">
<pre>[items]
A a b c
B d e f</pre>
</div></div>
<div class="paragraph"><p>Then the config file would effectively appear to contain an items section
exactly like the output from the script. This is extremely useful when
dynamically creating items, itemstables and other config structures. You can
see the expansion of the call by looking in the cached files (look in your
linktree for megatest.config and runconfigs.config cache files and in your
test run areas for the expanded and cached testconfig).</p></div>
<div class="sect3">
<h4 id="_disk_space_checks">Disk Space Checks</h4>
exactly like the output from the script. This is useful when dynamically
creating items, itemstables and other config structures. You can see the
expansion of the call by looking in the cached files (look in your linktree
for megatest.config and runconfigs.config cache files and in your test run
areas for the expanded and cached testconfig).</p></div>
<div class="paragraph"><p>Wildcards and regexes in Targets</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[a/2/b]
VAR1 VAL1

[a/%/b]
VAR1 VAL2</pre>
</div></div>
<div class="paragraph"><p>Will result in:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[a/2/b]
VAR1 VAL2</pre>
</div></div>
<div class="paragraph"><p>Can use either wildcard of "%" or a regular expression:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[/abc.*def/]</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_disk_space_checks">Disk Space Checks</h3>
<div class="paragraph"><p>Some parameters you can put in the [setup] section of megatest.config:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre># minimum space required in a run disk
minspace 10000000

# minimum space required in dbdir:
dbdir-space-required 100000

# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_trim_trailing_spaces">Trim trailing spaces</h4>
<div class="sect2">
<h3 id="_trim_trailing_spaces">Trim trailing spaces</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[configf:settings trim-trailing-spaces yes]</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_job_submission_control">Job Submission Control</h4>
<div class="sect4">
<h5 id="_submit_jobs_to_host_types_based_on_test_name_2">Submit jobs to Host Types based on Test Name</h5>
<div class="sect2">
<h3 id="_job_submission_control">Job Submission Control</h3>
<div class="sect3">
<h4 id="_submit_jobs_to_host_types_based_on_test_name_2">Submit jobs to Host Types based on Test Name</h4>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[host-types]
general   nbfake
remote    bsub

[launchers]
runfirst/sum% remote
% general

[jobtools]
launcher bsub
# if defined and not "no" flexi-launcher will bypass launcher unless
# there is no host-type match.
flexi-launcher yes</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_host_types">host-types</h5>
<div class="paragraph"><p>List of host types and the commandline to run a job on that host type.</p></div>
<div class="listingblock">
<div class="title">host-type &#8658; launch command</div>
<div class="content monospaced">
<pre>general nbfake</pre>

<h5 id="_miscellaneous_setup_items">Miscellaneous Setup Items</h5>
<div class="paragraph"><p>Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.</p></div>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[setup]
reruns 5</pre>
</div></div>
<div class="paragraph"><p>Replace the default blacklisted environment variables with user supplied
list.</p></div>
<div class="paragraph"><p>Default list: USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS MAKEF MAKEOVERRIDES</p></div>
<div class="paragraph"><div class="title">Add a "bad" variable "PROMPT" to the variables that will be commented out</div><p>in the megatest.sh and megatest.csh files:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]
blacklistvars USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS PROMPT</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_run_time_limit">Run time limit</h5>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]

<div class="paragraph"><p>The default is the graphviz based tree but if your tests don&#8217;t view
well in that mode then use "nodot" to turn it off.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]
nodot</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_capturing_test_data">Capturing Test Data</h3>
<div class="paragraph"><p>In a test you can capture arbitrary variables and roll them up in the
megatest database for viewing on the dashboard or web app.</p></div>
<div class="listingblock">
<div class="title">In a test as a script</div>
<div class="content monospaced">
<pre>$MT_MEGATEST -load-test-data &lt;&lt; EOF
foo,bar,   1.2,  1.9, &gt;
foo,rab, 1.0e9, 10e9, 1e9
foo,bla,   1.2,  1.9, &lt;
foo,bal,   1.2,  1.2, &lt;   ,     ,Check for overload
foo,alb,   1.2,  1.2, &lt;=  , Amps,This is the high power circuit test
foo,abl,   1.2,  1.3, 0.1
foo,bra,   1.2, pass, silly stuff
faz,bar,    10,  8mA,     ,     ,"this is a comment"
EOF</pre>
</div></div>
<div class="paragraph"><p>Alternatively you can use logpro triggers to capture values and inject them
into megatest using the -set-values mechanism:</p></div>
<div class="listingblock">
<div class="title">Megatest help related to -set-values</div>
<div class="content monospaced">
<pre>Test data capture
  -set-values             : update or set values in the testdata table
  :category               : set the category field (optional)
  :variable               : set the variable name (optional)
  :value                  : value measured (required)
  :expected               : value expected (required)
  :tol                    : |value-expect| &lt;= tol (required, can be &lt;, &gt;, &gt;=, &lt;= or number)
  :units                  : name of the units for value, expected_value etc. (optional)</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_dashboard_settings">Dashboard settings</h3>
<div class="listingblock">
<div class="title">Runs tab buttons, font and size</div>
<div class="content monospaced">
<pre>[dashboard]
btn-height x14
btn-fontsz 10
cell-width 60</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_database_settings">Database settings</h3>
<table class="tableblock frame-topbot grid-all"
style="
width:70%;
">
<caption class="title">Table 3. Database config settings in [setup] section of megatest.config</caption>
<caption class="title">Table 4. Database config settings in [setup] section of megatest.config</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >Var                       </th>
<th class="tableblock halign-left valign-top" > Purpose            </th>
<th class="tableblock halign-left valign-top" > Valid values            </th>
<th class="tableblock halign-left valign-top" > Comments</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">delay-on-busy</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Prevent concurrent access issues</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">yes|no or not defined</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Default=no, may help on some network file systems, may slow things down also.</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">daemonize</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Daemonize the server on start</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">yes|no or not defined</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Default=no</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">faststart</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">All direct file access to sqlite db files</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">yes|no or not defined</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Default=yes, suggest no for central automated systems and yes for interactive use</p></td>
</tr>
<tr>
<td class="tableblock halign-center valign-top" ><p class="tableblock">homehost</p></td>

<div class="paragraph"><p>The runscript method is a brute force way to run scripts where the
user is responsible for setting STATE and STATUS</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>runscript main.csh</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_iteration">Iteration</h3>
<div class="listingblock">
<div class="title">Sections for iteration</div>
<div class="content monospaced">
<pre># full combinations
[items]
A x y
B 1 2

# Yields: x/1 x/2 y/1 y/2

# tabled
[itemstable]
A x y
B 1 2

# Yields x/1 y/2</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_requirements_section">Requirements section</h3>
<div class="listingblock">
<div class="title">Header</div>
<div class="content monospaced">
<pre>[requirements]</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_wait_on_other_tests">Wait on Other Tests</h4>
<div class="sect2">
<h3 id="_wait_on_other_tests">Wait on Other Tests</h3>
<div class="listingblock">
<div class="content monospaced">
<pre># A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_mode">Mode</h4>
<div class="sect2">
<h3 id="_mode">Mode</h3>
<div class="paragraph"><p>The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[requirements]
mode   normal</pre>
</div></div>

"itemmatch" are synonyms.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[requirements]
mode itemmatch</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_overriding_enviroment_variables">Overriding Enviroment Variables</h3>
<div class="paragraph"><p>Override variables before starting the test. Can include files (perhaps generated by megatest -envdelta or similar).</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[pre-launch-env-vars]
VAR1 value1

</div>
<div class="sect2">
<h3 id="_itemmap_handling">Itemmap Handling</h3>
<div class="paragraph"><p>For cases were the dependent test has a similar but not identical
itempath to the downstream test an itemmap can allow for itemmatch
mode</p></div>
<div class="listingblock">
<div class="title">example for removing part of itemmap for waiton test (eg: item <span class="monospaced">foo-x/bar</span> depends on waiton&#8217;s item <span class="monospaced">y/bar</span>)</div>
<div class="content monospaced">
<pre>[requirements]
mode itemmatch
itemmap .*x/ y/

# ## pattern replacement notes
mode itemwait
# itemmap &lt;item pattern for this test&gt;  &lt;item replacement pattern for waiton test&gt;
itemmap .*x/ y/</pre>
</div></div>
<div class="listingblock">
<div class="title">example for removing part of itemmap for waiton test (eg: item <span class="monospaced">foo/bar/baz</span> in this test depends on waiton&#8217;s item <span class="monospaced">baz</span>)</div>
<div class="content monospaced">
<pre># ## pattern replacement notes
#
# ## Example
# ## Remove everything up to the last /
[requirements]
mode itemwait
# itemmap &lt;item pattern for this test&gt; &lt;nothing here indicates removal&gt;
itemmap .*/
itemmap .*/</pre>
#
</div></div>
<div class="listingblock">
<div class="title">example replacing part of itemmap for (eg: item <span class="monospaced">foo/1234</span> will imply waiton&#8217;s item <span class="monospaced">bar/1234</span>)</div>
<div class="content monospaced">
<pre>#
# ## Example
# ## Replace foo/ with bar/
[requirements]
mode itemwait
# itemmap &lt;item pattern for this test&gt;  &lt;item replacement pattern for waiton test&gt;
itemmap foo/ bar/

# multi-line; matches are applied in the listed order
itemmap foo/ bar/</pre>
</div></div>
<div class="listingblock">
<div class="title">example for backreference (eg: item <span class="monospaced">foo23/thud</span> will imply waiton&#8217;s item <span class="monospaced">num-23/bar/thud</span></div>
<div class="content monospaced">
<pre>#
# ## Example
# ## can use \{number} in replacement pattern to backreference a (capture) from matching pattern similar to sed or perl
[requirements]
mode itemwait
# itemmap &lt;item pattern for this test&gt;  &lt;item replacement pattern for waiton test&gt;
itemmap foo(\d+)/ num-\1/bar/</pre>
</div></div>
<div class="listingblock">
<div class="title">example multiple itemmaps</div>
<div class="content monospaced">
<pre># multi-line; matches are applied in the listed order
# The following would map:
#   a123b321 to b321fooa123 then to 321fooa123p
#
[requirements]
itemmap (a\d+)(b\d+) \2foo\1
  b(.*) \1p</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_complex_mappings">Complex mappings</h4>
<div class="paragraph"><p>Complex mappings can be handled with the [itemmap] section</p></div>
<div class="imageblock">
<div class="content">
<img src="complex-itemmap.png" alt="complex-itemmap.png">
</div>
<div class="sect2">
<h3 id="_complex_mapping">Complex mapping</h3>
<div class="paragraph"><p>Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section)</p></div>
<div class="paragraph"><p>Each line in an itemmap section starts with a waiton test name followed by an itemmap expression</p></div>
<div class="listingblock">
<div class="title">eg: The following causes waiton test A item <span class="monospaced">bar/1234</span> to run when our test&#8217;s <span class="monospaced">foo/1234</span> item is requested as well as causing waiton test B&#8217;s <span class="monospaced">blah</span> item to run when our test&#8217;s <span class="monospaced">stuff/blah</span> item is requested</div>
<div class="content monospaced">
<pre>[itemmap]
A foo/ bar/
B stuff/</pre>
</div></div>
</div>
<div class="paragraph"><p>Example:</p></div>
<div class="olist arabic"><ol class="arabic">
<div class="sect2">
<h3 id="_complex_mapping_example">Complex mapping example</h3>
<div class="imageblock">
<div class="content">
<li>
<p>
Request to run D/1/res
</p>
</li>
<li>
<p>
Megatest uses rule "(\d+)/res" &#8594; "\1/aa" to create item C/1/aa from D/1/res
</p>
<img src="complex-itemmap.png" alt="complex-itemmap.png">
</div>
</li>
<li>
<p>
Full list to be run is now: D/1/res, C/1/aa
</p>
</li>
<li>
<p>
Megatest uses rule "(\d+)/aa" &#8594; "aa/\1" to create item A/aa/1
</p>
</li>
<li>
<p>
Full list to be run is now: D/1/res, C/1/aa, A/aa/1
</p>
</li>
</ol></div>
</div>
<div class="paragraph"><p>We accomplish this by configuring the testconfigs of our tests C D and E as follows:</p></div>
<div class="listingblock">
<div class="title">Testconfig for Test C</div>
<div class="title">Testconfig for Test E has</div>
<div class="content monospaced">
<pre>[requirements]
waiton A B
waiton C

[itemmap]
itemmap (\d+)/res \1/bb</pre>
A (\d+)/aa aa/\1
B (\d+)/bb</pre>
</div></div>
<div class="listingblock">
<div class="title">Testconfig for Test D</div>
<div class="title">Testconfig for Test D has</div>
<div class="content monospaced">
<pre>[requirements]
waiton C
itemmap (\d+)/res \1/aa</pre>
</div></div>
<div class="listingblock">
<div class="title">Testconfig for Test E</div>
<div class="title">Testconfig for Test C has</div>
<div class="content monospaced">
<pre>[requirements]
waiton C
itemmap (\d+)/res \1/bb</pre>
waiton A B

[itemmap]
A (\d+)/aa aa/\1
B (\d+)/bb bb/\1</pre>
</div></div>
<div class="listingblock">
<div class="title">Testconfigs for Test B and Test A have no waiton or itemmap configured</div>
<div class="content monospaced">
<pre></pre>
</div></div>
<div class="olist arabic"><div class="title">Walk through one item&#8201;&#8212;&#8201;we want the following to happen for testpatt <span class="monospaced">D/1/res</span> (see blue boxes in complex itemmaping figure above):</div><ol class="arabic">
<li>
<p>
eg from command line <span class="monospaced">megatest -run -testpatt D/1/res -target mytarget -runname myrunname</span>
</p>
</li>
<li>
<p>
Full list to be run is now: <span class="monospaced">D/1/res</span>
</p>
</li>
<li>
<p>
Test D has a waiton - test C.  Test D&#8217;s itemmap rule <span class="monospaced">itemmap (\d&plus;)/res \1/aa</span> &#8594;  causes <span class="monospaced">C/1/aa</span> to run before <span class="monospaced">D/1/res</span>
</p>
</li>
<li>
<p>
Full list to be run is now: <span class="monospaced">D/1/res</span>, <span class="monospaced">C/1/aa</span>
</p>
</li>
<li>
<p>
Test C was a waiton - test A. Test C&#8217;s rule <span class="monospaced">A (\d&plus;)/aa aa/\1</span> &#8594; causes <span class="monospaced">A/aa/1</span> to run before <span class="monospaced">C/1/aa</span>
</p>
</li>
<li>
<p>
Full list to be run is now: <span class="monospaced">D/1/res</span>, <span class="monospaced">C/1/aa</span>, <span class="monospaced">A/aa/1</span>
</p>
</li>
<li>
<p>
Test A has no waitons.  All waitons of all tests in full list have been processed.  Full list is finalized.
</p>
</li>
</ol></div>
</div>
<div class="sect3">
<h4 id="_dynamic_flow_dependency_tree">Dynamic Flow Dependency Tree</h4>
<div class="sect2">
<h3 id="_itemstable">itemstable</h3>
<div class="paragraph"><p>An alternative to defining items is the itemstable section.  This lets you define the itempath in a table format rather than specifying components and relying on getting all permutations of those components.</p></div>
</div>
<div class="sect2">
<h3 id="_dynamic_flow_dependency_tree">Dynamic Flow Dependency Tree</h3>
<div class="listingblock">
<div class="title">Autogeneration waiton list for dynamic flow dependency trees</div>
<div class="content monospaced">
<pre>[requirements]
# With a toplevel test you may wish to generate your list
# of tests to run dynamically
#
waiton #{shell get-valid-tests-to-run.sh}</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_run_time_limit_2">Run time limit</h4>
<div class="sect2">
<h3 id="_run_time_limit_2">Run time limit</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[requirements]
runtimelim 1h 2m 3s  # this will automatically kill the test if it runs for more than 1h 2m and 3s</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip">Skip</h4>
<div class="sect2">
<h3 id="_skip">Skip</h3>
<div class="paragraph"><p>A test with a skip section will conditional skip running.</p></div>
<div class="listingblock">
<div class="title">Skip section example</div>
<div class="content monospaced">
<pre>[skip]
prevrunning x
# rundelay 30m 15s</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip_on_still_running_tests">Skip on Still-running Tests</h4>
<div class="sect2">
<h3 id="_skip_on_still_running_tests">Skip on Still-running Tests</h3>
<div class="listingblock">
<div class="content monospaced">
<pre># NB// If the prevrunning line exists with *any* value the test will
# automatically SKIP if the same-named test is currently RUNNING. The
# "x" can be any string. Comment out the prevrunning line to turn off
# skip.

[skip]
prevrunning x</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip_if_a_file_exists">Skip if a File Exists</h4>
<div class="sect2">
<h3 id="_skip_if_a_file_exists">Skip if a File Exists</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[skip]
fileexists /path/to/a/file # skip if /path/to/a/file exists</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip_if_test_ran_more_recently_than_specified_time">Skip if test ran more recently than specified time</h4>
<div class="sect2">
<h3 id="_skip_if_a_file_does_not_exist">Skip if a File Does not Exist</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[skip]
filenotexists /path/to/a/file # skip if /path/to/a/file does not exist</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_skip_if_a_script_completes_with_0_status">Skip if a script completes with 0 status</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[skip]
script /path/to/a/script # skip if /path/to/a/script completes with 0 status</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_skip_if_test_ran_more_recently_than_specified_time">Skip if test ran more recently than specified time</h3>
<div class="listingblock">
<div class="title">Skip if this test has been run in the past fifteen minutes and 15 seconds.</div>
<div class="content monospaced">
<pre>[skip]
rundelay 15m 15s</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_disks">Disks</h4>
<div class="sect2">
<h3 id="_disks">Disks</h3>
<div class="paragraph"><p>A disks section in testconfig will override the disks section in
megatest.config. This can be used to allocate disks on a per-test or per item
basis.</p></div>
</div>
<div class="sect3">
<h4 id="_controlled_waiver_propagation">Controlled waiver propagation</h4>
<div class="sect2">
<h3 id="_controlled_waiver_propagation">Controlled waiver propagation</h3>
<div class="paragraph"><p>If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED</p></div>
<div class="paragraph"><p>Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_ezsteps">Ezsteps</h3>
<div class="listingblock">
<div class="title">Example ezsteps with logpro rules</div>
<div class="content monospaced">
<pre>[ezsteps]
lookittmp   ls /tmp

[logpro]
lookittmp ;; Note: config file format supports multi-line entries where leading whitespace is removed from each line
  ;;     a blank line indicates the end of the block of text
  (expect:required in "LogFileBody" &gt; 0 "A file name that should never exist!" #/This is a awfully stupid file name that should never be found in the temp dir/)</pre>
</div></div>
<div class="paragraph"><p>To transfer the environment to the next step you can do the following:</p></div>
<div class="listingblock">
<div class="title">Propagate environment to next step</div>
<div class="content monospaced">
<pre>$MT_MEGATEST -env2file .ezsteps/${stepname}</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_scripts">Scripts</h3>
<div class="listingblock">
<div class="title">Specifying scripts inline (best used for only simple scripts)</div>
<div class="content monospaced">
<pre>[scripts]
loaddb #!/bin/bash
  sqlite3 $1 &lt;&lt;EOF
  .mode tabs
  .import $2 data
  .q
  EOF</pre>
</div></div>
<div class="paragraph"><p>The above snippet results in the creation of an executable script
called "loaddb" in the test directory. NOTE: every line in the script
must be prefixed with the exact same number of spaces. Lines beginning
with a # will not work as expected. Currently you cannot indent
intermediate lines.</p></div>
<div class="listingblock">
<div class="title">Full example with ezsteps, logpro rules, scripts etc.</div>
<div class="content monospaced">
<pre># You can include a common file
#
[include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc]

# Use "var" for a scratch pad
#
[var]
dumpsql select * from data;
sepstr .....................................

# NOT IMPLEMENTED YET!
#
[ezsteps-addendum]
prescript something.sh
postscript something2.sh

# Add additional steps here. Format is "stepname script"
[ezsteps]
importdb loaddb prod.db prod.sql
dumpprod dumpdata prod.db "#{get var dumpsql}"
diff (echo "prod#{get var sepstr}test";diff --side-by-side \
     dumpprod.log reference.log ;echo DIFFDONE)

[scripts]
loaddb #!/bin/bash
  sqlite3 $1 &lt;&lt;EOF
  .mode tabs
  .import $2 data
  .q
  EOF

dumpdata #!/bin/bash
  sqlite3 $1 &lt;&lt;EOF
  .separator ,
  $2
  .q
  EOF

# Test requirements are specified here
[requirements]
waiton setup
priority 0

# Iteration for your test is controlled by the items section
# The complicated if is needed to allow processing of the config for the dashboard when there are no actual runs.
[items]
THINGNAME [system generatethings.sh | sort -u]

# Logpro rules for each step can be captured here in the testconfig
# note: The ;; after the stepname and the leading whitespace are required
#
[logpro]
inputdb ;;
  (expect:ignore   in "LogFileBody"  &lt; 99 "Ignore error in comments"      #/^\/\/.*error/)
  (expect:warning  in "LogFileBody"  = 0 "Any warning"                    #/warn/)
  (expect:required in "LogFileBody"  &gt; 0 "Some data found"                #/^[a-z]{3,4}[0-9]+_r.*/)

diff ;;
  (expect:ignore   in "LogFileBody"  &lt; 99 "Ignore error in comments"      #/^\/\/.*error/)
  (expect:warning  in "LogFileBody"  = 0 "Any warning"                    #/warn/)
  (expect:error    in "LogFileBody"  = 0 "&lt; or &gt; indicate missing entry"  (list #/(&lt;|&gt;)/   #/error/i))
  (expect:error    in "LogFileBody"  = 0 "Difference in data"             (list #/\s+\|\s+/ #/error/i))
  (expect:required in "LogFileBody"  &gt; 0 "DIFFDONE Marker found"          #/DIFFDONE/)
  (expect:required in "LogFileBody"  &gt; 0 "Some things found"              #/^[a-z]{3,4}[0-9]+_r.*/)

# NOT IMPLEMENTED YET!
#
## Also: enhance logpro to take list of command files: file1,file2...
[waivers]
createprod{target=%78/%/%/%} ;;
  (disable:required "DIFFDONE Marker found")
  (disable:error    "Some error")
  (expect:waive  in "LogFileBody" &lt; 99 "Waive if failed due to version" #/\w+3\.6.*/)

# test_meta is a section for storing additional data on your test
[test_meta]
author matt
owner  matt
description Compare things
tags tagone,tagtwo
reviewed never</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_triggers">Triggers</h3>
<div class="paragraph"><p>In your testconfig triggers can be specified</p></div>
<div class="paragraph"><p>In your testconfig or megatest.config triggers can be specified</p></div>
<div class="listingblock">
<div class="title">Triggers spec</div>
<div class="content monospaced">
<pre>[triggers]

# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh

# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh

# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh</pre>
</div></div>
<div class="paragraph"><p>Scripts called will have; test-id test-rundir trigger, added to the commandline.</p></div>
<div class="paragraph"><p>Scripts called will have; test-id test-rundir trigger test-name item-path state status event-time, added to the commandline.</p></div>
<div class="paragraph"><p>HINT</p></div>
<div class="paragraph"><p>To start an xterm (useful for debugging), use a command line like the following:</p></div>
<div class="listingblock">
<div class="title">Start an xterm using a trigger for test completed.</div>
<div class="content monospaced">
<pre>[triggers]
COMPLETED/ xterm -e bash -s --</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">There is a trailing space after the --</td>
<td class="content">There is a trailing space after the double-dash</td>
</tr></table>
</div>
<div class="paragraph"><p>There are a number of environment variables available to the trigger script
but since triggers can be called in various contexts not all variables are
available at all times. The trigger script should check for the variable and
fail gracefully if it doesn&#8217;t exist.</p></div>
<table class="tableblock frame-topbot grid-all"
style="
width:90%;
">
<caption class="title">Table 5. Environment variables visible to the trigger script</caption>
<col style="width:50%;">
<col style="width:50%;">
<thead>
<tr>
<th class="tableblock halign-left valign-top" > Variable            </th>
<th class="tableblock halign-left valign-top" > Purpose</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TEST_RUN_DIR</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The directory where Megatest ran this test</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_CMDINFO</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Encoded command data for the test</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_DEBUG_MODE</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Used to pass the debug mode to nested calls to Megatest</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_RUN_AREA_HOME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Megatest home area</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TESTSUITENAME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The name of this testsuite or area</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TEST_NAME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The name of this test</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_ITEM_INFO</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The variable and values for the test item</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_MEGATEST</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Which Megatest binary is being used by this area</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TARGET</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The target variable values, separated by <em>/</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_LINKTREE</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The base of the link tree where all run tests can be found</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_ITEMPATH</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The values of the item path variables, separated by <em>/</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_RUNNAME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The name of the run</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="_override_the_toplevel_html_file">Override the Toplevel HTML File</h3>
<div class="paragraph"><p>Megatest generates a simple html file summary for top level tests of
iterated tests. The generation can be overridden. NOTE: the output of
the script is captured from stdout to create the html.</p></div>
<div class="listingblock">

# Within the archive the data is structured like this:
#  <target>/<runname>/<test>/
archive0 /mfs/myarchive-data/adisk1</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_handling_environment_variables">Handling Environment Variables</h2>
<h2 id="_environment_variables">Environment Variables</h2>
<div class="sectionbody">
<div class="paragraph"><p>It is often necessary to capture and or manipulate environment
variables. Megatest has some facilities built in to help.</p></div>
<div class="sect2">
<h3 id="_capture_variables">Capture variables</h3>
<div class="listingblock">
<div class="title">Commands</div>
<div class="content monospaced">
<pre># capture the current enviroment into a db called envdat.db under
# the context "before"
megatest -envcap before

# capture the current environment into a db called startup.db with
# context "after"
megatest -envcap after startup.db

# write the diff from before to after
megatest -envdelta before-after -dumpmode bash</pre>
</div></div>
<div class="paragraph"><p>Dump modes include bash, csh and config. You can include config data
into megatest.config or runconfigs.config.</p></div>
into megatest.config, runconfigs.config and testconfig files. This is
useful for capturing a complex environment in a special-purpose test
and then utilizing that environment in downstream tests.</p></div>
<div class="listingblock">
<div class="title">Example of generating and using config data</div>
<div class="content monospaced">
<pre>megatest -envcap original
# do some stuff here
megatest -envcap munged
megatest -envdelta original-munged -dumpmode ini -o modified.config</pre>
</div></div>
<div class="paragraph"><p>Then in runconfigs.config</p></div>
<div class="listingblock">
<div class="title">Example of using modified.config in a testconfig</div>
<div class="content monospaced">
<pre>cat testconfig

[pre-launch-env-vars]
<pre>[pre-launch-env-vars]
[include modified.config]</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_managing_old_runs">Managing Old Runs</h2>
<div class="sectionbody">
<div class="paragraph"><p>It is often desired to keep some older runs around but this must be balanced with the costs of disk space.</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Use -remove-keep
</p>
</li>
<li>
<p>
Use -archive (can also be done from the -remove-keep interface)
</p>
</li>
<li>
<p>
use -remove-runs with -keep-records
</p>
</li>
</ol></div>
<div class="listingblock">
<div class="title">For each target, remove all runs but the most recent 3 if they are over 1 week old</div>
<div class="content monospaced">
<pre># use -precmd 'sleep 5;nbfake' to limit overloading the host computer but to allow the removes to run in parallel.
megatest -actions print,remove-runs -remove-keep 3 -target %/%/%/% -runname % -age 1w -precmd 'sleep 5;nbfake'"</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_nested_runs">Nested Runs</h2>
<div class="sectionbody">
<div class="paragraph"><p>A Megatest test can run a full Megatest run in either the same
Megatest area or in another area. This is a powerful way of chaining
complex suites of tests and or actions.</p></div>
<div class="paragraph"><p>If you are not using the current area you can use ezsteps to retrieve
and setup the sub-Megatest run area.</p></div>
<div class="paragraph"><p>In the testconfig:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[subrun]

# Required: wait for the run or just launch it
#           if no then the run will be an automatic PASS irrespective of the actual result
run-wait yes|no

# Optional: where to execute the run. Default is the current runarea
run-area /some/path/to/megatest/area

# Optional: method to use to determine pass/fail status of the run
#   auto (default) - roll up the net state/status of the sub-run
#   logpro         - use the provided logpro rules, happens automatically if there is a logpro section
# passfail auto|logpro
# Example of logpro:
passfail logpro

# Optional:
logpro ;; if this section exists then logpro is used to determine pass/fail
  (expect:required in "LogFileBody" &gt;= 1 "At least one pass" #/PASS/)
  (expect:error     in "LogFileBody"  = 0 "No FAILs allowed"  #/FAIL/)

# Optional: target translator, default is to use the parent target
target #{shell somescript.sh}

# Optional: runname translator/generator, default is to use the parent runname
run-name #{somescript.sh}

# Optional: testpatt spec, default is to first look for TESTPATT spec from runconfigs unless there is a contour spec
test-patt %/item1,test2

# Optional: contour spec, use the named contour from the megatest.config contour spec
contour contourname ### NOTE: Not implemented yet! Let us know if you need this feature.

# Optional: mode-patt, use this spec for testpatt from runconfigs
mode-patt TESTPATT

# Optional: tag-expr, use this tag-expr to select tests
tag-expr quick

# Optional: (not yet implemented, remove-runs is always propagated at this time), propagate these actions from the parent
#           test
#   Note// default is % for all
propagate remove-runs archive ...</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_programming_api">Programming API</h2>
<div class="sectionbody">
<div class="paragraph"><p>These routines can be called from the megatest repl.</p></div>
<table class="tableblock frame-topbot grid-all"
style="
width:70%;
">
<caption class="title">Table 4. API Keys Related Calls</caption>
<caption class="title">Table 6. API Keys Related Calls</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >API Call                        </th>

<div class="content">
<img src="server.png" alt="server.png">
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_3rd_party_integrations"> Third Party plugins</h2>
<div class="sectionbody">
<div class="paragraph"><p>Megatest can be intigrated with following tools</p></div>
<div class="sect2">
<h3 id="_team_city">Team City</h3>
<p>TBD</p>
<h3 id="_jenkins_junit_xml">Jenkins - JUnit Realtime Test Reporter</h3>
<div class="listingblock">
<h2 id="_example_index">Example Index</h2>

<div class="content monospaced">
<pre>[runs]
junit-test-reporter-xml yes
junit-test-report-dir /path/to/store/xml/file
</pre>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_road_map">Road Map</h2>
<div class="sectionbody">
<div class="paragraph"><p>Note 1: This road-map continues to evolve and subject to change without notice.</p></div>
<div class="sect2">
<h3 id="_here_is_a_smattering_of_ideas_for_megatest_2_0">Here is a smattering of ideas for Megatest 2.0</h3>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Add variable $MT_RUNPATH = $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME
</p>
</li>
</ol></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[db]
api legacy|new</pre>
</div></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
One big lesson from the 1.63-1.65 generation was that the main.db, 1.db … model was really good at scaling. I’d like to combine that model with the current also-very-good model. Obviously this is a disruptive change. I think making the old model the default and the new model an option for at least one generation would be fair.
</p>
</li>
<li>
<p>
Rigorous megatest.config and runconfig.config caching.
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Cache the configs in $MT_RUNPATH
</p>
</li>
<li>
<p>
Following invocations of –run, -rerun* will calculate the new config but only overwrite the cached file IF changed
</p>
</li>
</ol></div>
</li>
<li>
<p>
If the cached file changes ALL existing tests go from COMPLETED &#8594; STALE, I’m not sure what to do about RUNNING tests
</p>
</li>
<li>
<p>
!VARS in runconfigs are not exported to the environment. They are accessed via rget as if the ! was not there.
</p>
</li>
<li>
<p>
Per test copy commands (crude example below is not correct).
</p>
</li>
</ol></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[testcopy]
%/iind% unison SRC DEST
% cp –r SRC DEST</pre>
</div></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Test management via pkts (optional?)
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Control pkt types: run, kill, rerunclean, clean, archive, status?
</p>
</li>
<li>
<p>
Status pkt types: ack, step, status_change
</p>
</li>
</ol></div>
</li>
<li>
<p>
Add nanomsg as a transport option
.
</p>
</li>
</ol></div>
</div>
<div class="sect2">
<h3 id="_rfc_m01_add_ability_to_move_runs_to_other_areas">RFC M01: Add ability to move runs to other Areas</h3>
<div class="paragraph"><p><strong>Purpose</strong>: allow shrinking megatest.db data by moving runs to an alternate
 Megatest area with same keys.</p></div>
<div class="paragraph"><p><strong>Method</strong>: extend db sync to take a different megatest area as a destination.</p></div>
<div class="paragraph"><p><strong>Design</strong>:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
add param -destination [area|path]. when specified runs are copied to new
  area and removed from local db.
</p>
</li>
<li>
<p>
the data move would involve these steps
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
copy the run data to destination area megatest.db
</p>
</li>
<li>
<p>
mark the run records as deleted, do not remove the run data on disk
</p>
</li>
</ol></div>
</li>
<li>
<p>
accessing the data would be by running dashboard in the satellite area
</p>
</li>
<li>
<p>
future versions of Megatest dashboard should support displaying areas in a
  merged way.
</p>
</li>
<li>
<p>
some new controls would be supported in the config
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
[setup] &#8658; allow-runs [no|yes]  &#8656;= used to disallow runs
</p>
</li>
<li>
<p>
[setup] &#8658; auto-migrate=[areaname|path]  &#8656;= used to automatically
   migrate data to a satellite area.
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p><strong>Branch</strong>: This work is taking place on branch v1.65-reduce-records</p></div>
</div>
<div class="sect2">
<h3 id="_rfc_m02_move_data_into_completed_runs_db">RFC M02: Move data into completed-runs.db</h3>
<div class="paragraph"><p><strong>Purpose</strong>: shrink megatest.db data to enable lower load and higher performance.</p></div>
<div class="paragraph"><p><strong>Method</strong>: add a completed-runs.db and automatically move runs data from megatest.db to that db</p></div>
<div class="paragraph"><p><strong>Design</strong>:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
completed-runs.db is a full megatest database with complete schema
</p>
</li>
<li>
<p>
the data move would involve these steps
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
copy the run data to completed-runs.db
</p>
</li>
<li>
<p>
remove the run data, first from /tmp/…/megatest.db and /tmp/…/megatest_ref.db, followed by megatest.db
</p>
</li>
</ol></div>
</li>
<li>
<p>
accessing the data would be unchanged for most operations.
</p>
</li>
<li>
<p>
a mode -full-db will be added which when specified would attach the completed-runs.db to megatest.db before doing the query
</p>
</li>
<li>
<p>
mechanisms for moving runs to/from the megatest.db would be added
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
-reduce-records &#8658; move runs to completed-runs.db
</p>
</li>
<li>
<p>
-restore-records &#8658; move runs from completed-runs.db to megatest.db
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p><strong>Branch</strong>: This work is taking place on branch v1.65-reduce-records</p></div>
</div>
<div class="sect2">
<h3 id="_rfc_m03_automatic_homehost_migrations">RFC M03: Automatic homehost migrations</h3>
<div class="paragraph"><p><strong>Purpose</strong>: Automatically migrate homehost.</p></div>
<div class="paragraph"><p><strong>Method</strong>: Check that there are no tests running, launched or remotehoststart in past ½ hour then if not on homehost migrate the db to current host</p></div>
<div class="paragraph"><p><strong>Design</strong>:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Check that the system is quiescent, i.e. that there are no runs in flight or recently run
</p>
</li>
<li>
<p>
Create a lock
</p>
</li>
<li>
<p>
Migrate the /tmp cache db to the current host
</p>
</li>
<li>
<p>
Update the .homehost file
</p>
</li>
<li>
<p>
Remove the lock
</p>
</li>
</ol></div>
<div class="paragraph"><p><strong>Branch</strong>: This work not yet started</p></div>
</div>
<div class="sect2">
<h3 id="_architecture_refactor">Architecture Refactor</h3>
<div class="sect3">
<h4 id="_goals">Goals</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Reduce load on the file system. Sqlite3 files on network filesystem can be
  a burden. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Reduce number of servers and frequency of start/stop. This is mostly an
  issue of clutter but also a reduction in "moving parts". <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Coalesce activities to a single home host where possible. Give the user
  feedback that they have started the dashboard on a host other than the
  home host. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Reduce number of processes involved in managing running tests.
</p>
</li>
</ol></div>
</div>
<div class="sect3">
<h4 id="_changes_needed">Changes Needed</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
ACID compliant db will be on /tmp and synced to megatest.db with a five
  second max delay. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Read/writes to db for processes on homehost will go direct to /tmp
  megatest.db file. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Read/wites fron non-homehost processes will go through one server. Bulk
  reads (e.g. for dashboard or list-runs) will be cached on the current host
  in /tmp and synced from the home megatest.db in the testsuite area. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Db syncs rely on the target db file timestame minus some margin. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Since bulk reads do not use the server we can switch to simple RPC for the
  network transport. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Test running manager process extended to manage multiple running tests.
</p>
</li>
</ol></div>
</div>
</div>
<div class="sect2">
<h3 id="_current_items">Current Items</h3>
<div class="sect3">
<h4 id="_ww05_migrate_to_inmem_db">ww05 - migrate to inmem-db</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Switch to inmem db with fast sync to on disk db&#8217;s <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Server polls tasks table for next action
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Task table used for tracking runner process <span class="red">[Replaced by mtutil]</span>
</p>
</li>
<li>
<p>
Task table used for jobs to run <span class="red">[Replaced by mtutil]</span>
</p>
</li>
<li>
<p>
Task table used for queueing runner actions (remove runs,
   cleanRunExecute, etc)  <span class="red">[Replaced by mtutil</span>]
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p>shifting, note that the preceding blank line is needed.</p></div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_index">Index</h2>
<div class="sectionbody">
</div>
</div>
</div>
<div id="footnotes"><hr></div>
<div id="footer">
<div id="footer-text">
Version 1.0<br>
Last updated 2016-12-12 13:03:08 PST
Last updated 2020-02-24 08:51:51 PST
</div>
</div>
</body>
</html>