Megatest

Check-in [384db8d983]
Login
☰Home Timeline Files Branches Tags Tickets Wiki Login
Overview
Comment:Started hacking the User Manual. Big mess :(
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | dev
Files: files | file ages | folders
SHA1: 384db8d983d1554c81d9128423b10686a733f119
User & Date: mrwellan on 2013-07-11 17:44:23
Other Links: branch diff | manifest | tags
Context
2013-07-12
11:35
Phase 1 of test search path implementation check-in: 16e44a18cb user: mrwellan tags: dev
2013-07-11
17:44
Started hacking the User Manual. Big mess :( check-in: 384db8d983 user: mrwellan tags: dev
13:03
Added runtimelim, changed kill to simply use built-in process-signal on top process, several minor bug fixes, changed run browser for summary page to start collapsed check-in: 1695ea179a user: mrwellan tags: dev
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Modified docs/manual/megatest_manual.html from [2055d691a1] to [e15bf233d1].

︙︙
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770

<h1>The Megatest Users Manual</h1>
<span id="author">Matt Welland</span><br />
<span id="email"><tt>&lt;<a href="mailto:matt@kiatoa.com">matt@kiatoa.com</a>&gt;</tt></span><br />
<span id="revnumber">version 1.0,</span>
<span id="revdate">April 2012</span>
</div>
<div id="content">
<div class="sect1">
<h2 id="_dedication">Dedication</h2>
<div class="sectionbody">
<div class="paragraph"><p>Dedicated to my wife Joanna who has kindly supported my working on various projects over the years.</p></div>
<div class="sect1">
<h2 id="_thanks">Thanks</h2>
<div class="sectionbody">
<div class="paragraph"><p>Thank you the many people I&#8217;ve worked over the years who have
shared their knowledge and insights with me.</p></div>
<div class="paragraph"><p>Thanks also to the creators of the various open source projects that
Megatest is built on. These include Linux, xemacs, chicken scheme,
fossil and asciidoc. Without these projects something like Megatest
would be difficult or impossible to do.</p></div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_preface">Preface</h2>
<div class="sectionbody">
<div class="paragraph"><p>This book is organised as three sub-books; getting started, writing tests and reference.</p></div>
<div class="sect2">
<h3 id="_why_megatest">Why Megatest?</h3>
<div class="paragraph"><p>The Megatest project was started for two reasons, the first was an








<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<








740
741
742
743
744
745
746

















747
748
749
750
751
752
753

<h1>The Megatest Users Manual</h1>
<span id="author">Matt Welland</span><br />
<span id="email"><tt>&lt;<a href="mailto:matt@kiatoa.com">matt@kiatoa.com</a>&gt;</tt></span><br />
<span id="revnumber">version 1.0,</span>
<span id="revdate">April 2012</span>
</div>
<div id="content">

















<div class="sect1">
<h2 id="_preface">Preface</h2>
<div class="sectionbody">
<div class="paragraph"><p>This book is organised as three sub-books; getting started, writing tests and reference.</p></div>
<div class="sect2">
<h3 id="_why_megatest">Why Megatest?</h3>
<div class="paragraph"><p>The Megatest project was started for two reasons, the first was an
︙︙
895
896
897
898
899
900
901




902
903
904















905
906





907
908
909
910














911
912
913
914


915
916


917
918
919
920
921




























































922
923
924
925
926
927
928

<div class="paragraph"><p>Chapters grouped into book parts are at level 1 and can contain
sub-sections.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_the_testconfig_file">The testconfig File</h2>
<div class="sectionbody">




<div class="listingblock">
<div class="content">
<pre><tt># testconfig















[requirements]






# A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2















# A item based waiton will start items in a test when the
# same-named item is COMPLETED and PASS, CHECK or WAIVED
# in the prior test
#


mode   normal | toplevel



# With a toplevel test you may wish to generate your list
# of tests to run dynamically
#
# waiton #{shell get-valid-tests-to-run.sh}</tt></pre>
</div></div>




























































</div>
</div>
<div class="sect1">
<h2 id="_example_appendix">Appendix A: Example Appendix</h2>
<div class="sectionbody">
<div class="paragraph"><p>One or more optional appendixes go here at section level zero.</p></div>
<div class="sect2">








>
>
>
>


|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
>
>
>
>
>
|

|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
|
<
>
>
|
|
>
>
|




>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934

935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012

<div class="paragraph"><p>Chapters grouped into book parts are at level 1 and can contain
sub-sections.</p></div>
</div>
</div>
<div class="sect1">
<h2 id="_the_testconfig_file">The testconfig File</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_setup_section">Setup section</h3>
<div class="sect3">
<h4 id="_header">Header</h4>
<div class="listingblock">
<div class="content">
<pre><tt>[setup]</tt></pre>
</div></div>
<div class="paragraph"><p>The runscript method is a brute force way to run scripts where the
user is responsible for setting STATE and STATUS</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>runscript main.csh</tt></pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_requirements_section">Requirements section</h3>
<div class="sect3">
<h4 id="_header_2">Header</h4>
<div class="listingblock">
<div class="content">
<pre><tt>[requirements]</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_wait_on_other_tests">Wait on Other Tests</h4>
<div class="listingblock">
<div class="content">
<pre><tt># A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_mode">Mode</h4>
<div class="paragraph"><p>The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>mode   normal</tt></pre>
</div></div>
<div class="paragraph"><p>The toplevel mode requires only that the prior tests are COMPLETED.</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>mode toplevel</tt></pre>
</div></div>
<div class="paragraph"><p>A item based waiton will start items in a test when the
same-named item is COMPLETED and PASS, CHECK or WAIVED
in the prior test</p></div>

<div class="listingblock">
<div class="content">
<pre><tt>mode itemmatch</tt></pre>
</div></div>
<div class="listingblock">
<div class="content">
<pre><tt># With a toplevel test you may wish to generate your list
# of tests to run dynamically
#
# waiton #{shell get-valid-tests-to-run.sh}</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_run_time_limit">Run time limit</h4>
<div class="listingblock">
<div class="content">
<pre><tt>runtimelim 1h 2m 3s  # this will automatically kill the test if it runs for more than 1h 2m and 3s</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip">Skip</h4>
</div>
<div class="sect3">
<h4 id="_header_3">Header</h4>
<div class="listingblock">
<div class="content">
<pre><tt>[skip]</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip_on_still_running_tests">Skip on Still-running Tests</h4>
<div class="listingblock">
<div class="content">
<pre><tt># NB// If the prevrunning line exists with *any* value the test will
# automatically SKIP if the same-named test is currently RUNNING

prevrunning x</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_skip_if_a_file_exists">Skip if a File Exists</h4>
<div class="listingblock">
<div class="content">
<pre><tt>fileexists /path/to/a/file # skip if /path/to/a/file exists</tt></pre>
</div></div>
</div>
<div class="sect3">
<h4 id="_controlled_waiver_propagation">Controlled waiver propagation</h4>
<div class="paragraph"><p>If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED</p></div>
<div class="paragraph"><p>Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)</p></div>
<div class="listingblock">
<div class="content">
<pre><tt>###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file    rulename      input_glob
waiver_1         logpro        lookittmp.log

[waiver_rules]

# This builtin rule is the default if there is no &lt;waivername&gt;.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a &lt;waivername&gt;.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html</tt></pre>
</div></div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_example_appendix">Appendix A: Example Appendix</h2>
<div class="sectionbody">
<div class="paragraph"><p>One or more optional appendixes go here at section level zero.</p></div>
<div class="sect2">
︙︙
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009

</div>
</div>
</div>
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
Version 1.0<br />
Last updated 2013-03-25 09:17:02 MST
</div>
</div>
</body>
</html>








|





1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093

</div>
</div>
</div>
<div id="footnotes"><hr /></div>
<div id="footer">
<div id="footer-text">
Version 1.0<br />
Last updated 2013-07-11 16:20:42 MST
</div>
</div>
</body>
</html>

Modified docs/manual/megatest_manual.txt from [6b638a28bf] to [5c050691d8].

︙︙
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

the distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database.

include::getting_started.txt[] include::writing_tests.txt[]
include::reference.txt[]


Controlled waiver propagation
=============================
If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED

Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)

=========================================================
###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file    rulename      input_glob
waiver_1         logpro        lookittmp.log

[waiver_rules]

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
=========================================================



[appendix]
Example Appendix
================
One or more optional appendixes go here at section level zero.

Appendix Sub-section








<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<








42
43
44
45
46
47
48




























49
50
51
52
53
54
55

the distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database.

include::getting_started.txt[] include::writing_tests.txt[]
include::reference.txt[]





























[appendix]
Example Appendix
================
One or more optional appendixes go here at section level zero.

Appendix Sub-section
︙︙

Modified docs/manual/reference.txt from [1a25745098] to [09090d6f26]. 

1
2
3
4
5
6
7
8
9
10
11
12






13


14














15

16




17
18
19

20
















21
22
23
24



25

26
27
28
29
30
31































































32
33
34


Reference
=========

The First Chapter of the Second Part
------------------------------------
Chapters grouped into book parts are at level 1 and can contain
sub-sections.

The testconfig File
-------------------







-------------------


# testconfig














[requirements]






# A normal waiton waits for the prior tests to be COMPLETED 
# and PASS, CHECK or WAIVED
waiton test1 test2


















# A item based waiton will start items in a test when the
# same-named item is COMPLETED and PASS, CHECK or WAIVED
# in the prior test
#



mode   normal | toplevel


# With a toplevel test you may wish to generate your list 
# of tests to run dynamically
#
# waiton #{shell get-valid-tests-to-run.sh}
------------------
































































:numbered!:














>
>
>
>
>
>

>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>

>

>
>
>
>



>

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
|
|
>
>
>
|
>






>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>




1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145


Reference
=========

The First Chapter of the Second Part
------------------------------------
Chapters grouped into book parts are at level 1 and can contain
sub-sections.

The testconfig File
-------------------

Setup section
~~~~~~~~~~~~~

Header
^^^^^^

-------------------
[setup]
-------------------

The runscript method is a brute force way to run scripts where the
user is responsible for setting STATE and STATUS

-------------------
runscript main.csh
-------------------

Requirements section
~~~~~~~~~~~~~~~~~~~~

Header
^^^^^^

-------------------
[requirements]
-------------------

Wait on Other Tests
^^^^^^^^^^^^^^^^^^^

-------------------
# A normal waiton waits for the prior tests to be COMPLETED 
# and PASS, CHECK or WAIVED
waiton test1 test2
-------------------

Mode
^^^^

The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start

-------------------
mode   normal
-------------------

The toplevel mode requires only that the prior tests are COMPLETED.

-------------------
mode toplevel
-------------------

A item based waiton will start items in a test when the
same-named item is COMPLETED and PASS, CHECK or WAIVED
in the prior test

-------------------
mode itemmatch
-------------------

-------------------

# With a toplevel test you may wish to generate your list 
# of tests to run dynamically
#
# waiton #{shell get-valid-tests-to-run.sh}
------------------



Run time limit
^^^^^^^^^^^^^^

-----------------
runtimelim 1h 2m 3s  # this will automatically kill the test if it runs for more than 1h 2m and 3s
-----------------

Skip
^^^^

Header
^^^^^^

-----------------
[skip]
-----------------

Skip on Still-running Tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^

-----------------
# NB// If the prevrunning line exists with *any* value the test will 
# automatically SKIP if the same-named test is currently RUNNING

prevrunning x
-----------------

Skip if a File Exists
^^^^^^^^^^^^^^^^^^^^^

-----------------
fileexists /path/to/a/file # skip if /path/to/a/file exists
-----------------

Controlled waiver propagation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED

Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)

-----------------
###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file    rulename      input_glob
waiver_1         logpro        lookittmp.log

[waiver_rules]

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
-----------------


:numbered!:

This page was generated in about 0.132s by Fossil 2.24 [452f8e1ffc] 2024-03-26 15:36:00