Megatest

<div id="globalWrapper">
<div class="Standard">

</div>
<div class="Standard">

</div>
<div class="Standard">

</div>
<h1 class="title">
Megatest
</h1>
<h2 class="author">
Matthew Welland
</h2>
<h2 class="Date">
Sept. 20, 
</h2>

</div>
<div class="Standard">
<p><br/>
</p>

</div>
<h1 class="Section-">
<a class="toc" name="toc-Section--1"></a>Megatest/document Revision History
</h1>
<div class="Standard">
<i>Notable revisions of the software are occasionally documented here</i>.
</div>
<div class="Standard">
<div class="center">
<table>

<td align="center" valign="top">
v1.25
</td>
<td align="center" valign="top">
matt
</td>
<td align="justify" valign="top">
converted to new document template
</td>
<td align="center" valign="top">
<div class="PlainVisible">

</div>

</td>

</div>
<div class="fulltoc">
<div class="tocheader">
Table of Contents
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section--1">Section: Megatest/document Revision History</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-1">Section 1: Introduction</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-1.1">Subsection 1.1: Megatest design philosophy</a>

</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.2">Subsection 6.2: Rewrite your main.csh for iteration</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.3">Subsection 6.3: Tests with Inter-test dependencies</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.4">Subsection 6.4: Rolling up Miscellaneous Data</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.5">Subsection 6.5: Rolling up Runs</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-7">Section 7: Dashboard</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-8">Section 8: Generating an OpenDocument Spreadsheet from the Database</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-9">Section 9: Reference</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-9.1">Subsection 9.1: Configuration file Syntax</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.1.1">Subsubsection 9.1.1: Sections</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.1.2">Subsubsection 9.1.2: Variables</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.1.3">Subsubsection 9.1.3: Includes</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.1.4">Subsubsection 9.1.4: Setting a variable by running a command</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.1.5">Subsubsection 9.1.5: Notes</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-9.2">Subsection 9.2: Environment variables</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-9.3">Subsection 9.3: megatest.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-9.4">Subsection 9.4: runconfigs.config file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-9.5">Subsection 9.5: Writing tests</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.5.1">Subsubsection 9.5.1: testconfig file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-9.5.2">Subsubsection 9.5.2: Command line</a>
</div>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-A">Section A: Data</a>
</div>
<div class="toc">

waiton cpu mem
</div>

</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.4">6.4</a> Rolling up Miscellaneous Data
</h2>
<div class="Standard">
Use the -load-test-data switch to roll up arbitrary data from a test into the test_data table.
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"># Fields are:
# category,variable,value,expected,tol,units,comment,status
​
$MT_MEGATEST -load-test-data &lt;&lt; EOF
foo,bar,1.2,1.9,&gt;
foo,rab,1.0e9,10e9,1e9
foo,bla,1.2,1.9,&lt;
foo,bal,1.2,1.2,&lt;,,Check for overload
foo,alb,1.2,1.2,&lt;=,Amps,This is the high power circuit test
foo,abl,1.2,1.3,0.1
foo,bra,1.2,pass,silly stuff
faz,bar,10,8mA,,,"this is a comment"
EOF
</pre>
</div>

</div>
<div class="Standard">
New entries are keyed on the category and variable. If a new record is inserted with a category and variable that have already been used the new record will replace the old record.
</div>
<div class="Standard">
Where value, expected and tol are specified the behavior is as follows.
</div>
<ul>
<li>
If value, expected and tol are numbers then status is calculated as PASS if (expected-tol) &lt;= value &lt;= (expected+tol)
</li>
<li>
If value and expected are numbers and tol is &gt;, &lt;, &gt;= or &lt;= then value is compared with expected using the operator given by tol
</li>
<li>
If status is specified its value overrides the above calculations.
</li>

</ul>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.5">6.5</a> Rolling up Runs
</h2>
<div class="Standard">
To roll up a number of tests in a sequence of runs to a single run use the -rollup command. 
</div>
<div class="Standard">
<div class="listing">
<pre class="listing">megatest -rollup :sysname ubuntu :fsname nfs :datapath none :runname rollup_ww38
</pre>
</div>

</div>
<div class="Standard">
All keys must be specified and the runname is the name of the run that will be created. All paths are kept original inside the database. When -remove-runs is used to delete runs the data is not deleted if there are rollups that refer to the data. 
</div>
<h1 class="Section">
<a class="toc" name="toc-Section-7">7</a> Dashboard
</h1>
<div class="Standard">
<div class="listing">
<pre class="listing">&gt; dashboard &amp;
</pre>
</div>

</div>
<div class="Standard">
<img class="embedded" src="dashboard.png" alt="figure dashboard.png" style="width: 730px; max-width: 913px; height: 626px; max-height: 783px; "/>

</div>
<div class="Standard">
Pushing one of the buttons on the main dashboard will bring up the test specific dashboard. Values are updated in semi-real time as the test runs.
</div>
<div class="Standard">
<img class="embedded" src="dashboard-test.png" alt="figure dashboard-test.png" style="width: 643px; max-width: 804px; height: 571px; max-height: 714px; "/>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-8">8</a> Generating an OpenDocument Spreadsheet from the Database
</h1>
<div class="Standard">
And OpenDocument multi-paned spreadsheet can be generated from the megatest.db file by running -extract-ods
</div>
<div class="Standard">
<div class="listing">
<pre class="listing">megatest -extract-ods results.ods :runname % 
</pre>
</div>

</div>
<div class="Standard">
You can optionally specify the keys for your database to limit further the runs to extract into the spreadsheet. The first sheet contains all the run data and subsequent sheets contain data rolled up for the individual tests.
</div>
<h1 class="Section">
<a class="toc" name="toc-Section-9">9</a> Reference
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-9.1">9.1</a> Configuration file Syntax
</h2>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.1.1">9.1.1</a> Sections
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">[section name]
</pre>
</div>

</div>
<div class="Standard">
This creates a section named “section name”
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.1.2">9.1.2</a> Variables
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">VARX has this value
</pre>
</div>

</div>
<div class="Standard">
The variable “VARX” will have the value “has this value”
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.1.3">9.1.3</a> Includes
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">[include filename]
</pre>
</div>

</div>
<div class="Standard">
The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.1.4">9.1.4</a> Setting a variable by running a command
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">VARNAME [system ls /tmp]
</pre>
</div>

</div>
<div class="Standard">
The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.1.5">9.1.5</a> Notes
</h3>
<ul>
<li>
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
</li>
<li>
Comments (lines starting with #) and blank lines are ignored.
</li>

</ul>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-9.2">9.2</a> Environment variables
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
Variable
</td>

</tr>

</table>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-9.3">9.3</a> megatest.config
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>

</tr>

</table>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-9.4">9.4</a> runconfigs.config file
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>

<div class="Code">
ENCRYPTION true
</div>
<div class="Code">
TESTPATH /nfs/testing/megacorp_runs
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-9.5">9.5</a> Writing tests
</h2>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.5.1">9.5.1</a> testconfig file
</h3>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>

</tr>

</table>

</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-9.5.2">9.5.2</a> Command line
</h3>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
switch or param
</td>

</h1>
<h1 class="Section">
<a class="toc" name="toc-Section-B">B</a> References
</h1>

<hr class="footer"/>
<div class="footer" id="generated-by">
Document generated by <a href="http://elyxer.nongnu.org/">eLyXer 1.0.0 (2010-07-21)</a> on <span class="create-date">2011-09-20T23:44:22.141636</span>
</div>
</div>