Index: docs/manual/megatest_manual.html
==================================================================
--- docs/manual/megatest_manual.html
+++ docs/manual/megatest_manual.html
@@ -900,10 +900,69 @@
sqlite3 database. Megatest has been used with the Intel Netbatch and
lsf (also known as openlava) batch systems and it should be
straightforward to use it with other similar systems.</p></div>
</div>
</div>
+</div>
+<div class="sect1">
+<h2 id="_overview">Overview</h2>
+<div class="sectionbody">
+<div class="sect2">
+<h3 id="_stand_alone_megatest_area">Stand-alone Megatest Area</h3>
+<div class="paragraph"><p>A single, stand-alone, Megatest based testsuite or "area" is
+sufficient for most validation, automation and build problems.</p></div>
+<div class="imageblock">
+<div class="content">
+<img src="megatest-stand-alone-area.png" alt="Static">
+</div>
+</div>
+<div class="paragraph"><p>Megatest is designed as a distributed or decoupled system. This means
+you can run the areas stand-alone with no additional
+infrastructure. I.e. there are no databases, web servers or other
+centralized resources needed. However as your needs grow you can
+integrate multiple areas into a bigger system.</p></div>
+<div class="sect3">
+<h4 id="_component_descriptions">Component Descriptions</h4>
+<div class="olist arabic"><ol class="arabic">
+<li>
+<p>
+Multi-area dashboard and xterm. A gui (the dashboard) is usually the
+ best option for controlling and launching runs but all operations
+ can also be done from the commandline. Note: The not yet released
+ multi-area dashboard replaces the old dashboard for browsing and
+ controlling runs but for managing a single area the old dashboard
+ works very well.
+</p>
+</li>
+<li>
+<p>
+Area/testsuite. This is your testsuite or automation definition and
+ consists of the information in megatest.config, runconfigs.config
+ and your testconfigs along with any custom scripting that can’t be
+ done with the native Megatest features.
+</p>
+</li>
+<li>
+<p>
+If your testsuite or build automation is too large to run on a
+ single instance you can distribute your jobs into a compute server
+ pool. The only current requirements are password-less ssh access and
+ a network filesystem.
+</p>
+</li>
+</ol></div>
+</div>
+</div>
+<div class="sect2">
+<h3 id="_full_system_architecture">Full System Architecture</h3>
+<div class="imageblock">
+<div class="content">
+<img src="megatest-system-architecture.png" alt="Static">
+</div>
+</div>
+</div>
+</div>
</div>
<div class="sect1">
<h2 id="_installation">Installation</h2>
<div class="sectionbody">
<div class="sect2">
@@ -2307,10 +2366,95 @@
<div class="listingblock">
<div class="title">Propagate environment to next step</div>
<div class="content monospaced">
<pre>$MT_MEGATEST -env2file .ezsteps/${stepname}</pre>
</div></div>
+<div class="listingblock">
+<div class="title">Full example with ezsteps, logpro rules, scripts etc.</div>
+<div class="content monospaced">
+<pre># You can include a common file
+#
+[include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc]
+
+# Use "var" for a scratch pad
+#
+[var]
+dumpsql select * from data;
+sepstr .....................................
+
+# NOT IMPLEMENTED YET!
+#
+[ezsteps-addendum]
+prescript something.sh
+postscript something2.sh
+
+# Add additional steps here. Format is "stepname script"
+[ezsteps]
+importdb loaddb prod.db prod.sql
+dumpprod dumpdata prod.db "#{get var dumpsql}"
+diff (echo "prod#{get var sepstr}test";diff --side-by-side \
+ dumpprod.log reference.log ;echo DIFFDONE)
+
+[scripts]
+loaddb #!/bin/bash
+ sqlite3 $1 <<EOF
+ .mode tabs
+ .import $2 data
+ .q
+ EOF
+
+dumpdata #!/bin/bash
+ sqlite3 $1 <<EOF
+ .separator ,
+ $2
+ .q
+ EOF
+
+# Test requirements are specified here
+[requirements]
+waiton setup
+priority 0
+
+# Iteration for your test is controlled by the items section
+# The complicated if is needed to allow processing of the config for the dashboard when there are no actual runs.
+[items]
+THINGNAME [system generatethings.sh | sort -u]
+
+# Logpro rules for each step can be captured here in the testconfig
+# note: The ;; after the stepname and the leading whitespace are required
+#
+[logpro]
+inputdb ;;
+ (expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
+ (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
+ (expect:required in "LogFileBody" > 0 "Some data found" #/^[a-z]{3,4}[0-9]+_r.*/)
+
+diff ;;
+ (expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
+ (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
+ (expect:error in "LogFileBody" = 0 "< or > indicate missing entry" (list #/(<|>)/ #/error/i))
+ (expect:error in "LogFileBody" = 0 "Difference in data" (list #/\s+\|\s+/ #/error/i))
+ (expect:required in "LogFileBody" > 0 "DIFFDONE Marker found" #/DIFFDONE/)
+ (expect:required in "LogFileBody" > 0 "Some things found" #/^[a-z]{3,4}[0-9]+_r.*/)
+
+# NOT IMPLEMENTED YET!
+#
+## Also: enhance logpro to take list of command files: file1,file2...
+[waivers]
+createprod{target=%78/%/%/%} ;;
+ (disable:required "DIFFDONE Marker found")
+ (disable:error "Some error")
+ (expect:waive in "LogFileBody" < 99 "Waive if failed due to version" #/\w+3\.6.*/)
+
+# test_meta is a section for storing additional data on your test
+[test_meta]
+author matt
+owner matt
+description Compare things
+tags tagone,tagtwo
+reviewed never</pre>
+</div></div>
</div>
<div class="sect2">
<h3 id="_triggers">Triggers</h3>
<div class="paragraph"><p>In your testconfig or megatest.config triggers can be specified</p></div>
<div class="listingblock">
@@ -2990,10 +3134,10 @@
</div>
<div id="footnotes"><hr></div>
<div id="footer">
<div id="footer-text">
Version 1.0<br>
-Last updated 2019-07-09 14:27:38 PDT
+Last updated 2020-01-02 13:39:49 PST
</div>
</div>
</body>
</html>
ADDED docs/manual/overview.txt
Index: docs/manual/overview.txt
==================================================================
--- /dev/null
+++ docs/manual/overview.txt
@@ -0,0 +1,43 @@
+
+Overview
+--------
+
+Stand-alone Megatest Area
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A single, stand-alone, Megatest based testsuite or "area" is
+sufficient for most validation, automation and build problems.
+
+image::megatest-stand-alone-area.png[Static,300]
+
+Megatest is designed as a distributed or decoupled system. This means
+you can run the areas stand-alone with no additional
+infrastructure. I.e. there are no databases, web servers or other
+centralized resources needed. However as your needs grow you can
+integrate multiple areas into a bigger system.
+
+Component Descriptions
+^^^^^^^^^^^^^^^^^^^^^^
+
+. Multi-area dashboard and xterm. A gui (the dashboard) is usually the
+ best option for controlling and launching runs but all operations
+ can also be done from the commandline. Note: The not yet released
+ multi-area dashboard replaces the old dashboard for browsing and
+ controlling runs but for managing a single area the old dashboard
+ works very well.
+
+. Area/testsuite. This is your testsuite or automation definition and
+ consists of the information in megatest.config, runconfigs.config
+ and your testconfigs along with any custom scripting that can't be
+ done with the native Megatest features.
+
+. If your testsuite or build automation is too large to run on a
+ single instance you can distribute your jobs into a compute server
+ pool. The only current requirements are password-less ssh access and
+ a network filesystem.
+
+Full System Architecture
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+image::megatest-system-architecture.png[Static,300]
+
Index: docs/manual/reference.txt
==================================================================
--- docs/manual/reference.txt
+++ docs/manual/reference.txt
@@ -651,10 +651,95 @@
.Propagate environment to next step
----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}
----------------------------
+
+.Full example with ezsteps, logpro rules, scripts etc.
+-----------------
+# You can include a common file
+#
+[include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc]
+
+# Use "var" for a scratch pad
+#
+[var]
+dumpsql select * from data;
+sepstr .....................................
+
+# NOT IMPLEMENTED YET!
+#
+[ezsteps-addendum]
+prescript something.sh
+postscript something2.sh
+
+# Add additional steps here. Format is "stepname script"
+[ezsteps]
+importdb loaddb prod.db prod.sql
+dumpprod dumpdata prod.db "#{get var dumpsql}"
+diff (echo "prod#{get var sepstr}test";diff --side-by-side \
+ dumpprod.log reference.log ;echo DIFFDONE)
+
+[scripts]
+loaddb #!/bin/bash
+ sqlite3 $1 <<EOF
+ .mode tabs
+ .import $2 data
+ .q
+ EOF
+
+dumpdata #!/bin/bash
+ sqlite3 $1 <<EOF
+ .separator ,
+ $2
+ .q
+ EOF
+
+# Test requirements are specified here
+[requirements]
+waiton setup
+priority 0
+
+# Iteration for your test is controlled by the items section
+# The complicated if is needed to allow processing of the config for the dashboard when there are no actual runs.
+[items]
+THINGNAME [system generatethings.sh | sort -u]
+
+# Logpro rules for each step can be captured here in the testconfig
+# note: The ;; after the stepname and the leading whitespace are required
+#
+[logpro]
+inputdb ;;
+ (expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
+ (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
+ (expect:required in "LogFileBody" > 0 "Some data found" #/^[a-z]{3,4}[0-9]+_r.*/)
+
+diff ;;
+ (expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
+ (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
+ (expect:error in "LogFileBody" = 0 "< or > indicate missing entry" (list #/(<|>)/ #/error/i))
+ (expect:error in "LogFileBody" = 0 "Difference in data" (list #/\s+\|\s+/ #/error/i))
+ (expect:required in "LogFileBody" > 0 "DIFFDONE Marker found" #/DIFFDONE/)
+ (expect:required in "LogFileBody" > 0 "Some things found" #/^[a-z]{3,4}[0-9]+_r.*/)
+
+# NOT IMPLEMENTED YET!
+#
+## Also: enhance logpro to take list of command files: file1,file2...
+[waivers]
+createprod{target=%78/%/%/%} ;;
+ (disable:required "DIFFDONE Marker found")
+ (disable:error "Some error")
+ (expect:waive in "LogFileBody" < 99 "Waive if failed due to version" #/\w+3\.6.*/)
+
+# test_meta is a section for storing additional data on your test
+[test_meta]
+author matt
+owner matt
+description Compare things
+tags tagone,tagtwo
+reviewed never
+-----------------
Triggers
~~~~~~~~
In your testconfig or megatest.config triggers can be specified