Changes In Branch v1.65-debugging-update-orig Through [392f89b8e5] Excluding Merge-Ins
This is equivalent to a diff from 206d14bb44 to 392f89b8e5
2022-02-17
| ||
09:25 | Merged the debugging updates to the manual Leaf check-in: c59e09e91a user: mrwellan tags: v1.65-defunct | |
09:22 | Squashed branch for manual updates Closed-Leaf check-in: e5d45e029c user: mrwellan tags: v1.65-debugging-update | |
2021-09-10
| ||
11:41 | Updates to debugging section of manual check-in: 43b9e9cf05 user: mrwellan tags: v1.65-debugging-update-orig | |
10:20 | stuff check-in: 392f89b8e5 user: mrwellan tags: v1.65-debugging-update-orig | |
05:54 | Merged couple changes made accidentally on trunk check-in: ee69c12599 user: matt tags: v1.65-debugging-update-orig | |
2021-09-03
| ||
08:30 | Updates to debugging section in manual check-in: 8e72fb284e user: matt tags: v1.65-debugging-update-orig | |
2021-02-25
| ||
11:08 | Create new branch named "nada" Closed-Leaf check-in: 2381efdb48 user: mrwellan tags: nada | |
2021-01-11
| ||
16:18 | backed out 21849054cacf4bd4d07ebc04019fba05ec6f5fd4, which was causing DEAD tests Leaf check-in: 1449ea317b user: mmgraham tags: v1.65-backout-2148 | |
2021-01-07
| ||
04:18 | Merged v1.65 into rerun-fixes Closed-Leaf check-in: 7310bcd03f user: matt tags: v1.65-rerun-fixes | |
2021-01-05
| ||
21:57 | Merged v1.65-dashboard-prefilter Closed-Leaf check-in: 206d14bb44 user: mrwellan tags: v1.65 | |
21:56 | Launching subrun dashboard prefiltered to only show the run of interest now works Leaf check-in: e16b8946d5 user: mrwellan tags: v1.65-dashboard-prefilter | |
2020-12-30
| ||
08:41 | Improved the makefile hacks for installing some needed .so files. check-in: 03539b7fce user: matt tags: v1.65 | |
Modified docs/manual/Makefile from [ec9633c3d9] to [759e8c25e6].
︙ | ︙ | |||
28 29 30 31 32 33 34 | # design_spec.html : $(SRCFILES) $(CSVFILES) # asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 design_spec.txt # all : server.ps megatest_manual.html client.ps complex-itemmap.png megatest_manual.pdf | | | | 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | # design_spec.html : $(SRCFILES) $(CSVFILES) # asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 design_spec.txt # all : server.ps megatest_manual.html client.ps complex-itemmap.png megatest_manual.pdf megatest_manual.html : megatest_manual.txt *.txt installation.txt *png *.dot asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 megatest_manual.txt # dos2unix megatest_manual.html megatest_manual.pdf : megatest_manual.txt *.txt *png *.dot a2x -a toc -f pdf megatest_manual.txt server.ps : server.dot dot -Tps server.dot > server.ps client.ps : client.dot dot -Tps client.dot > client.ps |
︙ | ︙ |
Added docs/manual/bisecting.dot version [01396be470].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | // Copyright 2021, Matthew Welland. // // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // Megatest is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with Megatest. If not, see <http://www.gnu.org/licenses/>. // digraph G { rankdir=LR subgraph cluster_1 { node [style=filled,shape=box]; B [label="B\nProblem is here"]; E [label="E\nProblem manifests here"]; A -> B; B -> C; C -> D; D -> E; } } |
Added docs/manual/bisecting.png version [81573d185b].
cannot compute difference between binary files
Added docs/manual/debugging.txt version [9d2c5b9df9].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | // Copyright 2021, Matthew Welland. // // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // Megatest is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with Megatest. If not, see <http://www.gnu.org/licenses/>. Debugging --------- Well Written Tests ~~~~~~~~~~~~~~~~~~ Test Design and Surfacing ^^^^^^^^^^^^^^^^^^^^^^^^^ Design your tests to surface errors. Ensure that all logs are processed by logpro and can be reached by a mouse click or two from the test control panel. using the tool to make life easier layers - less is often more Clicks-to-error A word on Bisecting ~~~~~~~~~~~~~~~~~~~ Bisecting is a debug strategy intended to speed up finding the root cause. .A complex process with a problem found in stage "E" ["graphviz", "bisecting.png"] ---------------------------------------------------------------------- include::bisecting.dot[] ---------------------------------------------------------------------- It is common to start debugging where the problem was observed and then work back. However by inspecting the output at stage "C" in the example above you would potentially save a lot of debug effort, this is similar to the feature in source control tools like git and fossil called biseceting. To know where to look for problems you will want to look at the log files at various stages in the execution process. .A simplified diagram of the stages Megatest goes through to run a test. ["graphviz", "megatest-test-stages.png"] ---------------------------------------------------------------------- include::megatest-test-stages.dot[] ---------------------------------------------------------------------- .How to check variable values and inspect logs at each stage [width="80%",cols="<,2m,2m",frame="topbot",options="header"] |====================== |Stage | How to inspect | Watch for |A: post config processing | megatest -show-config -target your/target | #f (failed var processing) |B: post runconfig | megatest -show-runconfig -target your/target | |C: processing testconfigs | inspect output from "megatest -run ..." | Messages indicating issues process configs, dependency problems. |D: process testconfig for test launch | inspect output from megatest runner | Zero items (items expansion yielded no items) |E,F: launching test | start test xterm, look at mt_launch.log | Did your batch system accept the job? Has the job landed on a machine? |G: starting test | look at your batch systems logs for the process | Did the megatest -execute process start and run? |H,H1,H2: step exectution | look at <stepname>.log, <stepname>.html and your own internal logs | Do you have sufficiently tight logpro rules? You must always have a "required" rule! |====================== Examining The Test Logs and Environment ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Test Control Panel - xterm ^^^^^^^^^^^^^^^^^^^^^^^^^^ From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate text editor window). With more recent versions of Megatest you can step through your test from the test control panel. Click on the cell labeled "rerun this step" to only rerun the step or click on "restart from here" to rerun that step and downstream steps. NOTE 1: visual feedback can take some time, give it a few seconds and you will see the step change color to blue as it starts running. NOTE 2: steping through only works if you are using ezsteps. Bisecting megatest.csh/sh ^^^^^^^^^^^^^^^^^^^^^^^^^ Sometimes finding the environment variable that is causing the problem can be very difficult. Bisection can be applied. Edit the megatest.csh or megatest.sh file and comment out 50% per round, source in fresh xterm and run the test. This idea can also be applied to your .cshrc, .bashrc, .aliases and other similar files. csh and -f ^^^^^^^^^^ A common issue when tcsh or csh shells are used for scripting is to forget or choose to not use -f in your #! line . Config File Processing ^^^^^^^^^^^^^^^^^^^^^^ As described above it is often helpful to know the content of variables in various contexts as Megatest works through the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined. For example, if an item list is not being generated as expected you can inject the startup of an xterm as if it were an item: .Original items table ----------------- [items] CELLNAME [system getcellname.sh] ----------------- .Items table modified for debug ----------------- [items] DEBUG [system xterm] CELLNAME [system getcellnames.sh] ----------------- When this test is run an xterm will pop up. In that xterm the environment is exactly that in which the script "getcellnames.sh" would run. You can now debug the script to find out why it isn't working as expected. |
Modified docs/manual/howto.txt from [5266978039] to [1b987427fd].
︙ | ︙ | |||
108 109 110 111 112 113 114 | [jobtools] launcher bsub # if defined and not "no" flexi-launcher will bypass launcher unless there is no # match. flexi-launcher yes ------------------------ | | | | | | 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | [jobtools] launcher bsub # if defined and not "no" flexi-launcher will bypass launcher unless there is no # match. flexi-launcher yes ------------------------ Tricks and Tips --------------- This section is a collection of a various useful tricks for that didn't quite fit elsewhere. Limiting your running jobs ~~~~~~~~~~~~~~~~~~~~~~~~~~ The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously. In your testconfig: |
︙ | ︙ | |||
134 135 136 137 138 139 140 | --------------- [jobgroups] group1 10 custdes 4 --------------- | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | --------------- [jobgroups] group1 10 custdes 4 --------------- Organising Your Tests and Tasks ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default location "tests" for storing tests can be extended by adding to your tests-paths section. |
︙ | ︙ |
Added docs/manual/megatest-test-stages.dot version [41d7d6e43f].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | // Copyright 2021, Matthew Welland. // // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // Megatest is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with Megatest. If not, see <http://www.gnu.org/licenses/>. // digraph G { // rankdir=LR subgraph cluster_1 { node [style=filled,shape=box]; A [label="A: Process megatest.config"] B [label="B: Process runconfig.config"] A -> B [label="resolve vars"] B2 [label="B2: Resolve variables"] B -> B2 -> A C [label="C: Process testconfigs (find tests to run)"] D [label="D: Process testconfig for test of interest"] E [label="E: Set vars for launching test"] F [label="F: Launch into batch system, ssh, batch system\nand different hosts can all \nimpact variable values"] G [label="G: Process testconfig again with all available variables"] H [label="H: Start test"] H1 [label="H1: Start step"] H2 [label="H2: Execute step script"] I [label="I: End step, process logfile with logpro"] I -> H [label="Run remaining steps"] H-> H1 -> H2 -> I B->C->D->E->F->G->H I->D } } |
Added docs/manual/megatest-test-stages.png version [e53ee8abbe].
cannot compute difference between binary files
Modified docs/manual/megatest_manual.html from [cce838ed12] to [23e5f0702c].
︙ | ︙ | |||
806 807 808 809 810 811 812 | along with Megatest. If not, see <http://www.gnu.org/licenses/>.</pre> </div></div> </div> </div> <div class="sect1"> <h2 id="_why_megatest">Why Megatest?</h2> <div class="sectionbody"> | | < | > | | | < | < < < < | 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 | along with Megatest. If not, see <http://www.gnu.org/licenses/>.</pre> </div></div> </div> </div> <div class="sect1"> <h2 id="_why_megatest">Why Megatest?</h2> <div class="sectionbody"> <div class="paragraph"><p>Megatest was created to provide a generalized tool for managing suites of regression tests and to provide a multi-host, distributed alternative to "make". The EDA world is littered with proprietory, company-specific tools for this purpose and by going open source and keeping the tool flexible the hope is that Megatest could be useful to any team at any company for continuous integration and almost any other general automation tasks.</p></div> </div> </div> <div class="sect1"> <h2 id="_megatest_design_philosophy">Megatest Design Philosophy</h2> <div class="sectionbody"> <div class="paragraph"><p>Megatest is a distributed system intended to provide the minimum needed resources to make writing a suite of tests and tasks for implementing |
︙ | ︙ | |||
957 958 959 960 961 962 963 | <div class="imageblock"> <div class="content"> <img src="megatest-system-architecture.png" alt="Static"> </div> </div> </div> </div> | < < < < < | 952 953 954 955 956 957 958 959 960 961 962 963 964 965 | <div class="imageblock"> <div class="content"> <img src="megatest-system-architecture.png" alt="Static"> </div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_todo_road_map">TODO / Road Map</h2> <div class="sectionbody"> <div class="paragraph"><p>Note: This road-map is a wish list and not a formal plan. Items are in rough priority but are subject to change. Development is driven by user requests, developer "itch" and bug reports. Please contact |
︙ | ︙ | |||
1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 | reviewed never</pre> </div></div> <div class="paragraph"><p>This test runs a single step called "stepname1" which runs a script "stepname.sh". Note that although it is common to put the actions needed for a test step into a script it is not necessary.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_how_to_do_things">How To Do Things</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_process_runs">Process Runs</h3> <div class="sect3"> | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 | reviewed never</pre> </div></div> <div class="paragraph"><p>This test runs a single step called "stepname1" which runs a script "stepname.sh". Note that although it is common to put the actions needed for a test step into a script it is not necessary.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_debugging">Debugging</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_well_written_tests">Well Written Tests</h3> <div class="sect3"> <h4 id="_test_design_and_surfacing">Test Design and Surfacing</h4> <div class="paragraph"><p>Design your tests to surface errors. Ensure that all logs are processed by logpro and can be reached by a mouse click or two from the test control panel.</p></div> <div class="paragraph"><p>using the tool to make life easier</p></div> <div class="paragraph"><p>layers - less is often more</p></div> <div class="paragraph"><p>Clicks-to-error</p></div> </div> </div> <div class="sect2"> <h3 id="_a_word_on_bisecting">A word on Bisecting</h3> <div class="paragraph"><p>Bisecting is a debug strategy intended to speed up finding the root cause.</p></div> <div class="imageblock graphviz"> <div class="content"> <img src="bisecting.png" alt="bisecting.png"> </div> <div class="title">Figure 1. A complex process with a problem found in stage "E"</div> </div> <div class="paragraph"><p>It is common to start debugging where the problem was observed and then work back. However by inspecting the output at stage "C" in the example above you would potentially save a lot of debug effort, this is similar to the feature in source control tools like git and fossil called biseceting.</p></div> <div class="paragraph"><p>To know where to look for problems you will want to look at the log files at various stages in the execution process.</p></div> <div class="imageblock graphviz"> <div class="content"> <img src="megatest-test-stages.png" alt="megatest-test-stages.png"> </div> <div class="title">Figure 2. A simplified diagram of the stages Megatest goes through to run a test.</div> </div> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> <caption class="title">Table 2. How to check variable values and inspect logs at each stage</caption> <col style="width:20%;"> <col style="width:40%;"> <col style="width:40%;"> <thead> <tr> <th class="tableblock halign-left valign-top" >Stage </th> <th class="tableblock halign-left valign-top" > How to inspect </th> <th class="tableblock halign-left valign-top" > Watch for</th> </tr> </thead> <tbody> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">A: post config processing</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">megatest -show-config -target your/target</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">#f (failed var processing)</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">B: post runconfig</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">megatest -show-runconfig -target your/target</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced"></p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">C: processing testconfigs</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">inspect output from "megatest -run …"</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Messages indicating issues process configs, dependency problems.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">D: process testconfig for test launch</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">inspect output from megatest runner</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Zero items (items expansion yielded no items)</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">E,F: launching test</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">start test xterm, look at mt_launch.log</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Did your batch system accept the job? Has the job landed on a machine?</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">G: starting test</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">look at your batch systems logs for the process</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Did the megatest -execute process start and run?</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">H,H1,H2: step exectution</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">look at <stepname>.log, <stepname>.html and your own internal logs</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Do you have sufficiently tight logpro rules? You must always have a "required" rule!</p></td> </tr> </tbody> </table> </div> <div class="sect2"> <h3 id="_examining_the_test_logs_and_environment">Examining The Test Logs and Environment</h3> <div class="sect3"> <h4 id="_test_control_panel_xterm">Test Control Panel - xterm</h4> <div class="paragraph"><p>From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate text editor window).</p></div> <div class="paragraph"><p>With more recent versions of Megatest you can step through your test from the test control panel. Click on the cell labeled "rerun this step" to only rerun the step or click on "restart from here" to rerun that step and downstream steps.</p></div> <div class="paragraph"><p>NOTE 1: visual feedback can take some time, give it a few seconds and you will see the step change color to blue as it starts running.</p></div> <div class="paragraph"><p>NOTE 2: steping through only works if you are using ezsteps.</p></div> </div> <div class="sect3"> <h4 id="_config_file_processing">Config File Processing</h4> <div class="paragraph"><p>As described above it is often helpful to know the content of variables in various contexts as Megatest works through the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined.</p></div> <div class="paragraph"><p>For example, if an item list is not being generated as expected you can inject the startup of an xterm as if it were an item:</p></div> <div class="listingblock"> <div class="title">Original items table</div> <div class="content monospaced"> <pre>[items] CELLNAME [system getcellname.sh]</pre> </div></div> <div class="listingblock"> <div class="title">Items table modified for debug</div> <div class="content monospaced"> <pre>[items] DEBUG [system xterm] CELLNAME [system getcellnames.sh]</pre> </div></div> <div class="paragraph"><p>When this test is run an xterm will pop up. In that xterm the environment is exactly that in which the script "getcellnames.sh" would run. You can now debug the script to find out why it isn’t working as expected.</p></div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_how_to_do_things">How To Do Things</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_process_runs">Process Runs</h3> <div class="sect3"> |
︙ | ︙ | |||
2034 2035 2036 2037 2038 2039 2040 | # match. flexi-launcher yes</pre> </div></div> </div> </div> </div> <div class="sect1"> | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 | # match. flexi-launcher yes</pre> </div></div> </div> </div> </div> <div class="sect1"> <h2 id="_tricks_and_tips">Tricks and Tips</h2> <div class="sectionbody"> <div class="paragraph"><p>This section is a collection of a various useful tricks for that didn’t quite fit elsewhere.</p></div> <div class="sect2"> <h3 id="_limiting_your_running_jobs">Limiting your running jobs</h3> <div class="paragraph"><p>The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.</p></div> <div class="paragraph"><p>In your testconfig:</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[test_meta] jobgroup group1</pre> </div></div> <div class="paragraph"><p>In your megatest.config:</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[jobgroups] group1 10 custdes 4</pre> </div></div> <div class="sect3"> <h4 id="_organising_your_tests_and_tasks">Organising Your Tests and Tasks</h4> <div class="paragraph"><p>The default location "tests" for storing tests can be extended by adding to your tests-paths section.</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[misc] |
︙ | ︙ | |||
2146 2147 2148 2149 2150 2151 2152 | <div class="sectionbody"> <div class="sect2"> <h3 id="_megatest_use_modes">Megatest Use Modes</h3> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> | | | 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 | <div class="sectionbody"> <div class="sect2"> <h3 id="_megatest_use_modes">Megatest Use Modes</h3> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> <caption class="title">Table 3. Base commands</caption> <col style="width:20%;"> <col style="width:40%;"> <col style="width:40%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >Use case </th> <th class="tableblock halign-left valign-top" > Megatest command </th> |
︙ | ︙ | |||
2208 2209 2210 2211 2212 2213 2214 | <div class="sect2"> <h3 id="_config_file_helpers">Config File Helpers</h3> <div class="paragraph"><p>Various helpers for more advanced config files.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> | | | 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 | <div class="sect2"> <h3 id="_config_file_helpers">Config File Helpers</h3> <div class="paragraph"><p>Various helpers for more advanced config files.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> <caption class="title">Table 4. Helpers</caption> <col style="width:14%;"> <col style="width:28%;"> <col style="width:28%;"> <col style="width:28%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >Helper </th> |
︙ | ︙ | |||
2508 2509 2510 2511 2512 2513 2514 | </div> <div class="sect2"> <h3 id="_database_settings">Database settings</h3> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> | | | 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 | </div> <div class="sect2"> <h3 id="_database_settings">Database settings</h3> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> <caption class="title">Table 5. Database config settings in [setup] section of megatest.config</caption> <col style="width:14%;"> <col style="width:28%;"> <col style="width:28%;"> <col style="width:28%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >Var </th> |
︙ | ︙ | |||
3240 3241 3242 3243 3244 3245 3246 | but since triggers can be called in various contexts not all variables are available at all times. The trigger script should check for the variable and fail gracefully if it doesn’t exist.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:90%; "> | | | 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 | but since triggers can be called in various contexts not all variables are available at all times. The trigger script should check for the variable and fail gracefully if it doesn’t exist.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:90%; "> <caption class="title">Table 6. Environment variables visible to the trigger script</caption> <col style="width:50%;"> <col style="width:50%;"> <thead> <tr> <th class="tableblock halign-left valign-top" > Variable </th> <th class="tableblock halign-left valign-top" > Purpose</th> </tr> |
︙ | ︙ | |||
3474 3475 3476 3477 3478 3479 3480 | <h2 id="_programming_api">Programming API</h2> <div class="sectionbody"> <div class="paragraph"><p>These routines can be called from the megatest repl.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> | | | 3561 3562 3563 3564 3565 3566 3567 3568 3569 3570 3571 3572 3573 3574 3575 | <h2 id="_programming_api">Programming API</h2> <div class="sectionbody"> <div class="paragraph"><p>These routines can be called from the megatest repl.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> <caption class="title">Table 7. API Keys Related Calls</caption> <col style="width:14%;"> <col style="width:28%;"> <col style="width:28%;"> <col style="width:28%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >API Call </th> |
︙ | ︙ | |||
3564 3565 3566 3567 3568 3569 3570 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.5<br> | | | 3651 3652 3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.5<br> Last updated 2021-09-10 06:09:41 PDT </div> </div> </body> </html> |
Added docs/manual/megatest_manual.pdf version [774cf56d82].
cannot compute difference between binary files
Modified docs/manual/megatest_manual.txt from [cb5cc67576] to [5b54398387].
︙ | ︙ | |||
31 32 33 34 35 36 37 | along with Megatest. If not, see <http://www.gnu.org/licenses/>. ---------------------------- Why Megatest? ------------- | < | | > | | | < < | < | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | along with Megatest. If not, see <http://www.gnu.org/licenses/>. ---------------------------- Why Megatest? ------------- Megatest was created to provide a generalized tool for managing suites of regression tests and to provide a multi-host, distributed alternative to "make". The EDA world is littered with proprietory, company-specific tools for this purpose and by going open source and keeping the tool flexible the hope is that Megatest could be useful to any team at any company for continuous integration and almost any other general automation tasks. Megatest Design Philosophy -------------------------- Megatest is a distributed system intended to provide the minimum needed resources to make writing a suite of tests and tasks for implementing continuous build for software, design engineering or process control (via |
︙ | ︙ | |||
96 97 98 99 100 101 102 103 104 105 106 | Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems. include::overview.txt[] include::plan.txt[] | > > | | | | 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems. // :leveloffset: 0 include::overview.txt[] include::plan.txt[] include::installation.txt[] include::getting_started.txt[] include::study_plan.txt[] include::writing_tests.txt[] include::debugging.txt[] include::howto.txt[] include::reference.txt[] include::testplan.txt[] |
︙ | ︙ |
Modified docs/manual/plan.txt from [84407a87ee] to [5f519e210f].
|
| < < < | 1 2 3 4 5 6 7 | // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // |
︙ | ︙ |
Modified docs/manual/reference.txt from [e5228eb513] to [624b8fd08d].
︙ | ︙ | |||
761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 | [setup] ezpropvars sh --------------------------- .Write your ezsteps. The loadenv.csh step will use /bin/csh as its shell, other steps will use bash. --------------------------- [ezsteps] loadenv.csh source $REF/ourenviron.csh compile make install make install --------------------------- Bash and csh are supported. You can override the shell binary location from the default /bin/bash and /bin/csh if needed. | > > > > | 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 | [setup] ezpropvars sh --------------------------- .Write your ezsteps. The loadenv.csh step will use /bin/csh as its shell, other steps will use bash. --------------------------- [ezsteps] # if your upstream file is csh you can force csh like this loadenv.csh source $REF/ourenviron.csh # if your upstream is bash loadenv source $REF/ourenviron.sh compile make install make install --------------------------- Bash and csh are supported. You can override the shell binary location from the default /bin/bash and /bin/csh if needed. |
︙ | ︙ |
Added docs/megatest-debug-tutorial.odp version [e618e203db].
cannot compute difference between binary files
Modified http-transport.scm from [8b77952676] to [92b19b8dad].
︙ | ︙ | |||
38 39 40 41 42 43 44 | (declare (uses portlogger)) (declare (uses rmt)) (include "common_records.scm") (include "db_records.scm") (include "js-path.scm") | | | 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | (declare (uses portlogger)) (declare (uses rmt)) (include "common_records.scm") (include "db_records.scm") (include "js-path.scm") (use stml2) (define (http-transport:make-server-url hostport) (if (not hostport) #f (conc "http://" (car hostport) ":" (cadr hostport)))) (define *server-loop-heart-beat* (current-seconds)) |
︙ | ︙ |
Modified mtut.scm from [413cf26858] to [cc88551c71].
︙ | ︙ | |||
33 34 35 36 37 38 39 | (declare (uses configf)) ;; (declare (uses rmt)) (use ducttape-lib) (include "megatest-fossil-hash.scm") | | | 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | (declare (uses configf)) ;; (declare (uses rmt)) (use ducttape-lib) (include "megatest-fossil-hash.scm") (use stml2) ;; stuff for the mapper and checker functions ;; (define *target-mappers* (make-hash-table)) (define *runname-mappers* (make-hash-table)) (define *area-checkers* (make-hash-table)) |
︙ | ︙ |
Modified tests.scm from [698654fba2] to [8b856ef5fd].
︙ | ︙ | |||
32 33 34 35 36 37 38 | (declare (uses runconfig)) ;; (declare (uses sdb)) (declare (uses server)) ;;(declare (uses stml2)) (use sqlite3 srfi-1 posix regex regex-case srfi-69 dot-locking tcp directory-utils) (import (prefix sqlite3 sqlite3:)) | | | 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | (declare (uses runconfig)) ;; (declare (uses sdb)) (declare (uses server)) ;;(declare (uses stml2)) (use sqlite3 srfi-1 posix regex regex-case srfi-69 dot-locking tcp directory-utils) (import (prefix sqlite3 sqlite3:)) (use stml2) (include "common_records.scm") (include "key_records.scm") (include "db_records.scm") (include "run_records.scm") (include "test_records.scm") (include "js-path.scm") |
︙ | ︙ |
Modified tests/mintest/megatest.config from [bfa94e44af] to [444a72500f].
︙ | ︙ | |||
17 18 19 20 21 22 23 | [fields] X TEXT [setup] max_concurrent_jobs 50 linktree #{getenv MT_RUN_AREA_HOME}/linktree | < | | 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | [fields] X TEXT [setup] max_concurrent_jobs 50 linktree #{getenv MT_RUN_AREA_HOME}/linktree [server] port 8090 [jobtools] useshell yes launcher nbfake [disks] disk0 #{getenv PWD}/runs |
Modified tests/mintest/tests/a/testconfig from [7f9803b9fe] to [61be51e47c].
︙ | ︙ | |||
14 15 16 17 18 19 20 21 22 23 | # # You should have received a copy of the GNU General Public License # along with Megatest. If not, see <http://www.gnu.org/licenses/>. # Add steps here. Format is "stepname script" [ezsteps] step1 echo SUCCESS [requirements] waiton b | > > > > > > > | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | # # You should have received a copy of the GNU General Public License # along with Megatest. If not, see <http://www.gnu.org/licenses/>. # Add steps here. Format is "stepname script" [ezsteps] step1 echo SUCCESS # step1 ((xterm&);echo SUCCESS) # step2 xterm_and_success [scripts] xterm_and_success #!/bin/bash xterm -T step2 & echo SUCCESS [requirements] waiton b |