1
2
3
4
5
6
7
8
9
10
11
12
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="generator" content="AsciiDoc 8.6.9">
<title>The Megatest Users Manual</title>
<style type="text/css">
/* Shared CSS for AsciiDoc xhtml11 and html5 backends */
/* Default font. */
body {
font-family: Georgia,serif;
|
|
|
1
2
3
4
5
6
7
8
9
10
11
12
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="generator" content="AsciiDoc 8.6.7">
<title>The Megatest Users Manual</title>
<style type="text/css">
/* Shared CSS for AsciiDoc xhtml11 and html5 backends */
/* Default font. */
body {
font-family: Georgia,serif;
|
| ︙ | | | ︙ | |
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
|
ul, ol, li > p {
margin-top: 0;
}
ul > li { color: #aaa; }
ul > li > * { color: black; }
.monospaced, code, pre {
font-family: "Courier New", Courier, monospace;
font-size: inherit;
color: navy;
pre {
white-space: pre-wrap;
}
|
|
<
<
<
<
<
<
|
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
|
ul, ol, li > p {
margin-top: 0;
}
ul > li { color: #aaa; }
ul > li > * { color: black; }
pre {
padding: 0;
margin: 0;
}
#author {
color: #527bbd;
font-weight: bold;
font-size: 1.1em;
}
#email {
|
| ︙ | | | ︙ | |
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
|
div.exampleblock > div.content {
border-left: 3px solid #dddddd;
padding-left: 0.5em;
}
div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; vertical-align: text-bottom; }
a.image:visited { color: white; }
dl {
margin-top: 0.8em;
margin-bottom: 0.8em;
}
dt {
|
|
|
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
|
div.exampleblock > div.content {
border-left: 3px solid #dddddd;
padding-left: 0.5em;
}
div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; }
a.image:visited { color: white; }
dl {
margin-top: 0.8em;
margin-bottom: 0.8em;
}
dt {
|
| ︙ | | | ︙ | |
415
416
417
418
419
420
421
422
423
424
425
426
427
428
|
div.unbreakable { page-break-inside: avoid; }
/*
* xhtml11 specific
*
* */
div.tableblock {
margin-top: 1.0em;
margin-bottom: 1.5em;
}
div.tableblock > table {
border: 3px solid #527bbd;
|
>
>
>
>
>
>
|
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
|
div.unbreakable { page-break-inside: avoid; }
/*
* xhtml11 specific
*
* */
tt {
font-family: "Courier New", Courier, monospace;
font-size: inherit;
color: navy;
}
div.tableblock {
margin-top: 1.0em;
margin-bottom: 1.5em;
}
div.tableblock > table {
border: 3px solid #527bbd;
|
| ︙ | | | ︙ | |
448
449
450
451
452
453
454
455
456
457
458
459
460
461
|
}
/*
* html5 specific
*
* */
table.tableblock {
margin-top: 1.0em;
margin-bottom: 1.5em;
}
thead, p.tableblock.header {
font-weight: bold;
|
>
>
>
>
>
>
|
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
|
}
/*
* html5 specific
*
* */
.monospaced {
font-family: "Courier New", Courier, monospace;
font-size: inherit;
color: navy;
}
table.tableblock {
margin-top: 1.0em;
margin-bottom: 1.5em;
}
thead, p.tableblock.header {
font-weight: bold;
|
| ︙ | | | ︙ | |
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
|
body.manpage div.sectionbody {
margin-left: 3em;
}
@media print {
body.manpage div#toc { display: none; }
}
|
<
<
|
534
535
536
537
538
539
540
541
542
543
544
545
546
547
|
body.manpage div.sectionbody {
margin-left: 3em;
}
@media print {
body.manpage div#toc { display: none; }
}
@media screen {
body {
max-width: 50em; /* approximately 80 characters wide */
margin-left: 16em;
}
#toc {
|
| ︙ | | | ︙ | |
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
|
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="_preface">Preface</h2>
<div class="sectionbody">
<div class="paragraph"><p>This book is organised as three sub-books; getting started, writing tests and reference.</p></div>
<div class="sect2">
<h3 id="_why_megatest">Why Megatest?</h3>
<div class="paragraph"><p>The Megatest project was started for two reasons, the first was an
immediate and pressing need for a generalized tool to manage a suite
of regression tests and the second was the fact that the author had
written or maintained several such tools at different companies over
the years and it seemed a good thing to have a single open source
tool, flexible enough to meet the needs of any team doing continuous
integrating and or running a complex suite of tests for release
qualification.</p></div>
</div>
<div class="sect2">
<h3 id="_megatest_design_philosophy">Megatest Design Philosophy</h3>
<div class="paragraph"><p>Megatest is intended to provide the minimum needed resources to make
writing a suite of tests and tasks for implementing continuous build
for software, design engineering or process control (via owlfs for
example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or
FAIL of a test or task. In most cases megatest is best used in
conjunction with logpro or a similar tool to parse, analyze and decide
|
>
>
|
|
>
>
|
|
>
|
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
|
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="_preface">Preface</h2>
<div class="sectionbody">
<div class="paragraph"><p>This book is organised as three sub-books; getting started, writing tests and reference.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_why_megatest">Why Megatest?</h2>
<div class="sectionbody">
<div class="paragraph"><p>The Megatest project was started for two reasons, the first was an
immediate and pressing need for a generalized tool to manage a suite
of regression tests and the second was the fact that the author had
written or maintained several such tools at different companies over
the years and it seemed a good thing to have a single open source
tool, flexible enough to meet the needs of any team doing continuous
integrating and or running a complex suite of tests for release
qualification.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_design_philosophy">Megatest Design Philosophy</h2>
<div class="sectionbody">
<div class="paragraph"><p>Megatest is intended to provide the minimum needed resources to make
writing a suite of tests and tasks for implementing continuous build
for software, design engineering or process control (via owlfs for
example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or
FAIL of a test or task. In most cases megatest is best used in
conjunction with logpro or a similar tool to parse, analyze and decide
|
| ︙ | | | ︙ | |
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
|
<li>
<p>
Relocatable - the testsuite or automation area can be checked out and the tests run anywhere
</p>
</li>
<li>
<p>
Encapsulated - the area where the tests run are self-contained and all inputs
and outputs to the process can be found in the run areas.
</p>
</li>
<li>
<p>
Deployable - anyone on the team, at any site, at any time can run the flow
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_megatest_architecture">Megatest Architecture</h3>
<div class="paragraph"><p>All data to specify the tests and configure the system is stored in
plain text files. All system state is stored in an sqlite3
database. Tests are launched using the launching system available for
the distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database.</p></div>
</div>
</div>
</div>
<h1 id="_road_map">Road Map</h1>
<div class="paragraph"><p>Note 1: This road-map is tentative and subject to change without notice.</p></div>
<div class="paragraph"><p>Note 2: Starting over. Old plan is commented out.</p></div>
<div class="sect1">
<h2 id="_current_items">Current Items</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_ww05_migrate_to_inmem_db">ww05 - migrate to inmem-db</h3>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Switch to inmem db with fast sync to on disk db’s [DONE]
</p>
</li>
<li>
|
|
>
|
|
>
|
|
>
|
|
>
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
|
<li>
<p>
Relocatable - the testsuite or automation area can be checked out and the tests run anywhere
</p>
</li>
<li>
<p>
Encapsulated - the tests run in self-contained directories and all inputs
and outputs to the process can be found in the run areas.
</p>
</li>
<li>
<p>
Deployable - anyone on the team, at any site, at any time can run the flow
</p>
</li>
</ul></div>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_architecture">Megatest Architecture</h2>
<div class="sectionbody">
<div class="paragraph"><p>All data to specify the tests and configure the system is stored in
plain text files. All system state is stored in an sqlite3
database. Tests are launched using the launching system available for
the distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_road_map">Road Map</h2>
<div class="sectionbody">
<div class="paragraph"><p>Note 1: This road-map is still evolving and subject to change without notice.</p></div>
<div class="sect2">
<h3 id="_architecture_refactor">Architecture Refactor</h3>
<div class="sect3">
<h4 id="_goals">Goals</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Reduce load on the file system. Sqlite3 files on network filesystem can be
a burden.
</p>
</li>
<li>
<p>
Reduce number of servers and frequency of start/stop. This is mostly an
issue of clutter but also a reduction in "moving parts".
</p>
</li>
<li>
<p>
Coalesce activities to a single home host where possible. Give the user
feedback that they have started the dashboard on a host other than the
home host.
</p>
</li>
<li>
<p>
Reduce number of processes involved in managing running tests.
</p>
</li>
</ol></div>
</div>
<div class="sect3">
<h4 id="_changes_needed">Changes Needed</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
ACID compliant db will be on /tmp and synced to megatest.db with a five
second max delay.
</p>
</li>
<li>
<p>
Read/writes to db for processes on homehost will go direct to /tmp
megatest.db file.
</p>
</li>
<li>
<p>
Read/wites fron non-homehost processes will go through one server. Bulk
reads (e.g. for dashboard or list-runs) will be cached on the current host
in /tmp and synced from the home megatest.db in the testsuite area.
</p>
</li>
<li>
<p>
Db syncs rely on the target db file timestame minus some margin.
</p>
</li>
<li>
<p>
Since bulk reads do not use the server we can switch to simple RPC for the
network transport.
</p>
</li>
<li>
<p>
Test running manager process extended to manage multiple running tests.
</p>
</li>
</ol></div>
</div>
</div>
<div class="sect2">
<h3 id="_current_items">Current Items</h3>
<div class="sect3">
<h4 id="_ww05_migrate_to_inmem_db">ww05 - migrate to inmem-db</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Switch to inmem db with fast sync to on disk db’s [DONE]
</p>
</li>
<li>
|
| ︙ | | | ︙ | |
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
|
<p>
Task table used for queueing runner actions (remove runs, cleanRunExecute, etc)
</p>
</li>
</ol></div>
</li>
</ol></div>
</div>
</div>
</div>
<h1 id="_getting_started">Getting Started</h1>
<div class="openblock">
<div class="title">Getting started with Megatest</div>
<div class="content">
<div class="paragraph"><p>How to install Megatest and set it up for running your regressions and continuous integration process.</p></div>
</div></div>
Megatest. See the script installall.sch in the utils directory of the
distribution for a mostly automated way to install everything needed
for building Megatest on Linux.</p></div>
<div class="paragraph"><p><span class="footnote"><br>[An example footnote.]<br></span></p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_writing_tests">Writing Tests</h2>
<div class="sectionbody">
<div class="sect2">
|
>
<
<
<
<
<
|
|
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
|
<p>
Task table used for queueing runner actions (remove runs, cleanRunExecute, etc)
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p>shifting, note that the preceding blank line is needed.</p></div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_installation">Installation</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_dependencies">Dependencies</h3>
<div class="paragraph"><p>Chicken scheme and a number of "eggs" are required for building
Megatest. See the script installall.sh in the utils directory of the
source distribution for an automated way to install everything
needed for building Megatest on Linux.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_getting_started">Getting Started</h2>
<div class="sectionbody">
<div class="openblock">
<div class="title">Getting started with Megatest</div>
<div class="content">
<div class="paragraph"><p>Creating a testsuite or flow and your first test or task.</p></div>
</div></div>
<div class="paragraph"><p>After installing Megatest you can create a flow or testsuite and add some
tests using the helpers. Here is a quickstart sequence to get you up and
running your first automated testsuite.</p></div>
<div class="sect2">
<h3 id="_creating_a_megatest_area">Creating a Megatest Area</h3>
<div class="sect3">
<h4 id="_choose_target_keys">Choose Target Keys</h4>
<div class="paragraph"><p>First choose your "target" keys. These are used to organise your runs in a
way that is meaningful to your project. If you are unsure about what to use
for keys just use a single generic key such as "RUNTYPE". These keys will be
used to hand values to your tests via environment variables so ensure they
are unique. Prefixing them with something such as PROJKEYS_ is a good
strategy.</p></div>
<div class="paragraph"><p>Examples of keys:</p></div>
<table class="tableblock frame-all grid-all"
style="
width:60%;
">
<caption class="title">Table 1. Example keys</caption>
<col style="width:50%;">
<col style="width:50%;">
<thead>
<tr>
<th class="tableblock halign-left valign-top" > Option </th>
<th class="tableblock halign-left valign-top" > Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">RELEASE/ITERATION</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">This example is used by Megatest for its internal QA.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">ARCH/OS/RELEASE</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">For a software project targeting multiple platforms</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">UCTRLR/NODETYPE</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Microcontroller project with different controllers
running same software</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect3">
<h4 id="_create_area_config_files">Create Area Config Files</h4>
<div class="paragraph"><p>You will need to choose locations for your runs (the data generated every
time you run the testsuite) and link tree. For getting started answer the
prompts with "runs" and "links". We use the Unix editor "vi" in the examples
below but you can use any plain text editor.</p></div>
<div class="listingblock">
<div class="title">Using the helper to create a Megatest area</div>
<div class="content monospaced">
<pre>megatest -create-megatest-area
# optional: verify that the settings are ok
vi megatest.config
vi runconfigs.config</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_creating_a_test">Creating a Test</h3>
<div class="paragraph"><p>Choose the test name for your first test and run the helper. You can edit
the files after the initial creation. You will need to enter names and
scripts for the steps to be run and then edit the
tests/<testname>/testconfig file and modify the logpro rules to properly
process the log output from your steps. For your first test just hit enter
for the "waiton", "priority" and iteration variable prompts.</p></div>
<div class="paragraph"><p>Hint: for geting started make your logpro rules very liberal. expect:error
patterns should match nothing and comment out expect:required rules.</p></div>
<div class="listingblock">
<div class="title">Using the helper to create a Megatest test</div>
<div class="content monospaced">
<pre>megatest -create-test myfirsttest
# then edit the generated config
vi tests/myfirsttest/testconfig</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_running_your_test">Running your test</h3>
<div class="paragraph"><p>First choose a target and runname. If you have a two-place target such as
RELEASE/ITERATION a target would look like v1.0/aff3 where v1.0 is the
RELEASE and aff3 is the ITERATION. For a run name just use something like
run1.</p></div>
<div class="listingblock">
<div class="title">Running all tests (testpatt of "%" matches all tests)</div>
<div class="content monospaced">
<pre>megatest -run -target v1.0/aff3 -runname run1 -testpatt % -log run1.log</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_viewing_the_results">Viewing the results</h3>
<div class="paragraph"><p>Start the dashboard and browse your run in the "Runs" tab.</p></div>
<div class="listingblock">
<div class="title">Starting dashboard</div>
<div class="content monospaced">
<pre>dashboard -rows 24</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_writing_tests">Writing Tests</h2>
<div class="sectionbody">
<div class="sect2">
|
| ︙ | | | ︙ | |
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
|
</div></div>
<div class="paragraph"><p>This test runs a single step called "stepname1" which runs a script
"stepname.sh". Note that although it is common to put the actions
needed for a test step into a script it is not necessary.</p></div>
</div>
</div>
</div>
<h1 id="_how_to_do_things">How To Do Things</h1>
<div class="sect1">
<h2 id="_process_runs">Process Runs</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_remove_runs">Remove Runs</h3>
<div class="paragraph"><p>From the dashboard click on the button (PASS/FAIL…) for one of the tests. From the test control panel that
comes up push the clean test button. The command field will be prefilled with a template command for removing
that test. You can edit the command, for example change the argument to -testpatt to "%" to remove all tests.</p></div>
<div class="listingblock">
<div class="title">Remove the test diskperf and all it’s items</div>
<div class="content monospaced">
<pre>megatest -remove-runs -target ubuntu/nfs/none -runname ww28.1a -testpatt diskperf/% -v</pre>
</div></div>
<div class="listingblock">
<div class="title">Remove all tests for all runs and all targets</div>
<div class="content monospaced">
<pre>megatest -remove-runs -target %/%/% -runname % -testpatt % -v</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_archive_runs">Archive Runs</h3>
<div class="paragraph"><p>Megatest supports using the bup backup tool (<a href="https://bup.github.io/">https://bup.github.io/</a>) to archive your tests for efficient storage
and retrieval. Archived data can be rapidly retrieved if needed. The metadata for the run (PASS/FAIL status, run
durations, time stamps etc.) are all preserved in the megatest database.</p></div>
<div class="paragraph"><p>For setup information see the Archiving topic in the reference section of this manual.</p></div>
<div class="sect3">
<h4 id="_to_archive">To Archive</h4>
<div class="paragraph"><p>Hint: use the test control panel to create a template command by pushing the "Archive Tests" button.</p></div>
<div class="listingblock">
<div class="title">Archive a full run</div>
<div class="content monospaced">
<pre>megatest -target ubuntu/nfs/none -runname ww28.1a -archive save-remove -testpatt %</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_to_restore">To Restore</h4>
<div class="listingblock">
<div class="title">Retrieve a single test</div>
<div class="content monospaced">
<pre>megatest -target ubuntu/nfs/none -runname ww28.1a -archive restore -testpatt diskperf/%</pre>
</div></div>
<div class="paragraph"><p>Hint: You can browse the archive using bup commands directly.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>bup -d /path/to/bup/archive ftp</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_submit_jobs_to_host_types_based_on_test_name">Submit jobs to Host Types based on Test Name</h3>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
|
>
|
|
<
|
>
|
|
|
|
|
|
|
|
>
|
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
|
</div></div>
<div class="paragraph"><p>This test runs a single step called "stepname1" which runs a script
"stepname.sh". Note that although it is common to put the actions
needed for a test step into a script it is not necessary.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_how_to_do_things">How To Do Things</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_process_runs">Process Runs</h3>
<div class="sect3">
<h4 id="_remove_runs">Remove Runs</h4>
<div class="paragraph"><p>From the dashboard click on the button (PASS/FAIL…) for one of the tests. From the test control panel that
comes up push the clean test button. The command field will be prefilled with a template command for removing
that test. You can edit the command, for example change the argument to -testpatt to "%" to remove all tests.</p></div>
<div class="listingblock">
<div class="title">Remove the test diskperf and all it’s items</div>
<div class="content monospaced">
<pre>megatest -remove-runs -target ubuntu/nfs/none -runname ww28.1a -testpatt diskperf/% -v</pre>
</div></div>
<div class="listingblock">
<div class="title">Remove all tests for all runs and all targets</div>
<div class="content monospaced">
<pre>megatest -remove-runs -target %/%/% -runname % -testpatt % -v</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_archive_runs">Archive Runs</h4>
<div class="paragraph"><p>Megatest supports using the bup backup tool (<a href="https://bup.github.io/">https://bup.github.io/</a>) to archive your tests for efficient storage
and retrieval. Archived data can be rapidly retrieved if needed. The metadata for the run (PASS/FAIL status, run
durations, time stamps etc.) are all preserved in the megatest database.</p></div>
<div class="paragraph"><p>For setup information see the Archiving topic in the reference section of this manual.</p></div>
<div class="sect4">
<h5 id="_to_archive">To Archive</h5>
<div class="paragraph"><p>Hint: use the test control panel to create a template command by pushing the "Archive Tests" button.</p></div>
<div class="listingblock">
<div class="title">Archive a full run</div>
<div class="content monospaced">
<pre>megatest -target ubuntu/nfs/none -runname ww28.1a -archive save-remove -testpatt %</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_to_restore">To Restore</h5>
<div class="listingblock">
<div class="title">Retrieve a single test</div>
<div class="content monospaced">
<pre>megatest -target ubuntu/nfs/none -runname ww28.1a -archive restore -testpatt diskperf/%</pre>
</div></div>
<div class="paragraph"><p>Hint: You can browse the archive using bup commands directly.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>bup -d /path/to/bup/archive ftp</pre>
</div></div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_submit_jobs_to_host_types_based_on_test_name">Submit jobs to Host Types based on Test Name</h3>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
|
| ︙ | | | ︙ | |
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
|
# if defined and not "no" flexi-launcher will bypass launcher unless there is no
# match.
flexi-launcher yes</pre>
</div></div>
</div>
</div>
</div>
<h1 id="_tricks">Tricks</h1>
<div class="paragraph"><p>This section is a compendium of a various useful tricks for debugging,
configuring and generally getting the most out of Megatest.</p></div>
<div class="sect1">
<h2 id="_limiting_your_running_jobs">Limiting your running jobs</h2>
<div class="sectionbody">
<div class="paragraph"><p>The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.</p></div>
<div class="paragraph"><p>In your testconfig:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[test_meta]
jobgroup group1</pre>
</div></div>
<div class="paragraph"><p>In your megatest.config:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[jobgroups]
group1 10
custdes 4</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_debugging_tricks">Debugging Tricks</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_examining_the_environment">Examining The Environment</h3>
|
>
|
>
|
|
<
>
|
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
|
# if defined and not "no" flexi-launcher will bypass launcher unless there is no
# match.
flexi-launcher yes</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_tricks">Tricks</h2>
<div class="sectionbody">
<div class="paragraph"><p>This section is a compendium of a various useful tricks for debugging,
configuring and generally getting the most out of Megatest.</p></div>
<div class="sect2">
<h3 id="_limiting_your_running_jobs">Limiting your running jobs</h3>
<div class="paragraph"><p>The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.</p></div>
<div class="paragraph"><p>In your testconfig:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[test_meta]
jobgroup group1</pre>
</div></div>
<div class="paragraph"><p>In your megatest.config:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[jobgroups]
group1 10
custdes 4</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_debugging_tricks">Debugging Tricks</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_examining_the_environment">Examining The Environment</h3>
|
| ︙ | | | ︙ | |
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
|
<pre>sudo lsof -i
sudo netstat -lptu
sudo netstat -tulpn</pre>
</div></div>
</div>
</div>
</div>
<h1 id="_reference">Reference</h1>
<div class="sect1">
<h2 id="_megatest_config_file_settings">Megatest Config File Settings</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_disk_space_checks">Disk Space Checks</h3>
<div class="paragraph"><p>Some parameters you can put in the [setup] section of megatest.config:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre># minimum space required in a run disk
minspace 10000000
# minimum space required in dbdir:
dbdir-space-required 100000
# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_trim_trailing_spaces">Trim trailing spaces</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[configf:settings trim-trailing-spaces yes]</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_job_submission_control">Job Submission Control</h3>
<div class="sect3">
<h4 id="_submit_jobs_to_host_types_based_on_test_name_2">Submit jobs to Host Types based on Test Name</h4>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[host-types]
general nbfake
remote bsub
[launchers]
runfirst/sum% remote
% general
[jobtools]
launcher bsub
# if defined and not "no" flexi-launcher will bypass launcher unless
# there is no host-type match.
flexi-launcher yes</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_host_types">host-types</h4>
<div class="paragraph"><p>List of host types and the commandline to run a job on that host type.</p></div>
<div class="listingblock">
<div class="title">host-type ⇒ launch command</div>
<div class="content monospaced">
<pre>general nbfake</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_launchers">launchers</h4>
<div class="listingblock">
<div class="title">test/itempath ⇒ host-type</div>
<div class="content monospaced">
<pre>runfirst/sum% remote</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_miscellaneous_setup_items">Miscellaneous Setup Items</h3>
<div class="sect3">
<h4 id="_run_time_limit">Run time limit</h4>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s
runtimelim 1h 2m 3s</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_tests_browser_view">Tests browser view</h4>
<div class="paragraph"><p>The tests browser (see the Run Control tab on the dashboard) has two views for displaying the tests.</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Dot (graphviz) based tree
</p>
</li>
|
>
|
|
>
|
|
<
|
|
|
|
|
|
|
|
|
|
|
<
|
|
>
|
|
>
>
|
|
|
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
|
<pre>sudo lsof -i
sudo netstat -lptu
sudo netstat -tulpn</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_reference">Reference</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_megatest_config_file_settings">Megatest Config File Settings</h3>
<div class="sect3">
<h4 id="_disk_space_checks">Disk Space Checks</h4>
<div class="paragraph"><p>Some parameters you can put in the [setup] section of megatest.config:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre># minimum space required in a run disk
minspace 10000000
# minimum space required in dbdir:
dbdir-space-required 100000
# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_trim_trailing_spaces">Trim trailing spaces</h4>
<div class="listingblock">
<div class="content monospaced">
<pre>[configf:settings trim-trailing-spaces yes]</pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_job_submission_control">Job Submission Control</h4>
<div class="sect4">
<h5 id="_submit_jobs_to_host_types_based_on_test_name_2">Submit jobs to Host Types based on Test Name</h5>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[host-types]
general nbfake
remote bsub
[launchers]
runfirst/sum% remote
% general
[jobtools]
launcher bsub
# if defined and not "no" flexi-launcher will bypass launcher unless
# there is no host-type match.
flexi-launcher yes</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_host_types">host-types</h5>
<div class="paragraph"><p>List of host types and the commandline to run a job on that host type.</p></div>
<div class="listingblock">
<div class="title">host-type ⇒ launch command</div>
<div class="content monospaced">
<pre>general nbfake</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_launchers">launchers</h5>
<div class="listingblock">
<div class="title">test/itempath ⇒ host-type</div>
<div class="content monospaced">
<pre>runfirst/sum% remote</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_miscellaneous_setup_items">Miscellaneous Setup Items</h5>
<div class="paragraph"><p>Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.</p></div>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[setup]
reruns 5</pre>
</div></div>
</div>
<div class="sect4">
<h5 id="_run_time_limit">Run time limit</h5>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s
runtimelim 1h 2m 3s</pre>
</div></div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_tests_browser_view">Tests browser view</h3>
<div class="paragraph"><p>The tests browser (see the Run Control tab on the dashboard) has two views for displaying the tests.</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Dot (graphviz) based tree
</p>
</li>
|
| ︙ | | | ︙ | |
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
|
well in that mode then use "nodot" to turn it off.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]
nodot</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_database_settings">Database settings</h3>
<table class="tableblock frame-topbot grid-all"
style="
width:70%;
">
<caption class="title">Table 1. Database config settings in [setup] section of megatest.config</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >Var </th>
|
>
>
>
>
>
>
>
>
>
>
|
|
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
|
well in that mode then use "nodot" to turn it off.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[setup]
nodot</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_dashboard_settings">Dashboard settings</h3>
<div class="listingblock">
<div class="title">Runs tab buttons, font and size</div>
<div class="content monospaced">
<pre>[dashboard]
btn-height x14
btn-fontsz 10
cell-width 60</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_database_settings">Database settings</h3>
<table class="tableblock frame-topbot grid-all"
style="
width:70%;
">
<caption class="title">Table 2. Database config settings in [setup] section of megatest.config</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >Var </th>
|
| ︙ | | | ︙ | |
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
|
<div class="content monospaced">
<pre>[triggers]
COMPLETED/ xterm -e bash -s --</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">There is a trailing space after the --</td>
</tr></table>
</div>
</div>
<div class="sect2">
<h3 id="_override_the_toplevel_html_file">Override the Toplevel HTML File</h3>
|
|
|
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
|
<div class="content monospaced">
<pre>[triggers]
COMPLETED/ xterm -e bash -s --</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">There is a trailing space after the --</td>
</tr></table>
</div>
</div>
<div class="sect2">
<h3 id="_override_the_toplevel_html_file">Override the Toplevel HTML File</h3>
|
| ︙ | | | ︙ | |
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
|
<h2 id="_programming_api">Programming API</h2>
<div class="sectionbody">
<div class="paragraph"><p>These routines can be called from the megatest repl.</p></div>
<table class="tableblock frame-topbot grid-all"
style="
width:70%;
">
<caption class="title">Table 2. API Keys Related Calls</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >API Call </th>
|
|
|
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
|
<h2 id="_programming_api">Programming API</h2>
<div class="sectionbody">
<div class="paragraph"><p>These routines can be called from the megatest repl.</p></div>
<table class="tableblock frame-topbot grid-all"
style="
width:70%;
">
<caption class="title">Table 3. API Keys Related Calls</caption>
<col style="width:14%;">
<col style="width:28%;">
<col style="width:28%;">
<col style="width:28%;">
<thead>
<tr>
<th class="tableblock halign-center valign-top" >API Call </th>
|
| ︙ | | | ︙ | |
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
|
<td class="tableblock halign-center valign-top" ><p class="tableblock">(rmt:get-keys run-id)</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced"></p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">( key1 key2 … )</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced"></p></td>
</tr>
</tbody>
</table>
<div class="sect2">
<h3 id="_megatest_internals">Megatest Internals</h3>
<div class="imageblock graphviz">
<div class="content">
<img src="server.png" alt="server.png">
</div>
</div>
</div>
<h2 id="_example_appendix">Appendix A: Example Appendix</h2>
<div class="sectionbody">
<div class="paragraph"><p>One or more optional appendixes go here at section level zero.</p></div>
<div class="sect2">
<h3 id="_appendix_sub_section">Appendix Sub-section</h3>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">Preface and appendix subsections start out of sequence at level
2 (level 1 is skipped). This only applies to multi-part book
documents.</td>
</tr></table>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_example_bibliography">Example Bibliography</h2>
<div class="sectionbody">
<div class="paragraph"><p>The bibliography list is a style of AsciiDoc bulleted list.</p></div>
<div class="ulist bibliography"><ul>
<li>
<p>
<a id="taoup"></a>[taoup] Eric Steven Raymond. <em>The Art of Unix
Programming</em>. Addison-Wesley. ISBN 0-13-142901-9.
</p>
</li>
<li>
<p>
<a id="walsh-muellner"></a>[walsh-muellner] Norman Walsh & Leonard Muellner.
<em>DocBook - The Definitive Guide</em>. O’Reilly & Associates. 1999.
ISBN 1-56592-580-7.
</p>
</li>
</ul></div>
</div>
</div>
<div class="sect1">
<h2 id="_example_glossary">Example Glossary</h2>
<div class="sectionbody">
<div class="paragraph"><p>Glossaries are optional. Glossaries entries are an example of a style
of AsciiDoc labeled lists.</p></div>
<div class="dlist glossary"><dl>
<dt>
A glossary term
</dt>
<dd>
<p>
The corresponding (indented) definition.
</p>
</dd>
<dt>
A second glossary term
</dt>
<dd>
<p>
The corresponding (indented) definition.
</p>
</dd>
</dl></div>
<div class="sect1">
<h2 id="_example_colophon">Example Colophon</h2>
<div class="sectionbody">
<div class="paragraph"><p>Text at the end of a book describing facts about its production.</p></div>
Last updated 2016-08-20 20:32:22 MST
</div>
</div>
</body>
</html>
|
<
<
<
<
<
<
<
<
|
<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
|
|
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
|
<td class="tableblock halign-center valign-top" ><p class="tableblock">(rmt:get-keys run-id)</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced"></p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">( key1 key2 … )</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock monospaced"></p></td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_internals">Megatest Internals</h2>
<div class="sectionbody">
<div class="imageblock graphviz">
<div class="content">
<img src="server.png" alt="server.png">
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_example_index">Example Index</h2>
<div class="sectionbody">
</div>
</div>
</div>
<div id="footnotes"><hr></div>
<div id="footer">
<div id="footer-text">
Version 1.0<br>
Last updated 2016-10-19 10:23:07 PDT
</div>
</div>
</body>
</html>
|