Changes In Branch v1.65-debugging-update-orig Through [21da809595] Excluding Merge-Ins
This is equivalent to a diff from 206d14bb44 to 21da809595
2022-02-17
| ||
09:25 | Merged the debugging updates to the manual Leaf check-in: c59e09e91a user: mrwellan tags: v1.65-defunct | |
09:22 | Squashed branch for manual updates Closed-Leaf check-in: e5d45e029c user: mrwellan tags: v1.65-debugging-update | |
2021-09-08
| ||
05:48 | Updates to manual on debugging check-in: 3080c40f63 user: matt tags: v1.65-debugging-update-orig | |
2021-09-07
| ||
21:01 | Added more debugging info check-in: 21da809595 user: matt tags: v1.65-debugging-update-orig | |
19:49 | Added missing graphviz file check-in: fd375bd1a1 user: matt tags: v1.65-debugging-update-orig | |
2021-09-03
| ||
08:30 | Updates to debugging section in manual check-in: 8e72fb284e user: matt tags: v1.65-debugging-update-orig | |
2021-02-25
| ||
11:08 | Create new branch named "nada" Closed-Leaf check-in: 2381efdb48 user: mrwellan tags: nada | |
2021-01-11
| ||
16:18 | backed out 21849054cacf4bd4d07ebc04019fba05ec6f5fd4, which was causing DEAD tests Leaf check-in: 1449ea317b user: mmgraham tags: v1.65-backout-2148 | |
2021-01-07
| ||
04:18 | Merged v1.65 into rerun-fixes Closed-Leaf check-in: 7310bcd03f user: matt tags: v1.65-rerun-fixes | |
2021-01-05
| ||
21:57 | Merged v1.65-dashboard-prefilter Closed-Leaf check-in: 206d14bb44 user: mrwellan tags: v1.65 | |
21:56 | Launching subrun dashboard prefiltered to only show the run of interest now works Leaf check-in: e16b8946d5 user: mrwellan tags: v1.65-dashboard-prefilter | |
2020-12-30
| ||
08:41 | Improved the makefile hacks for installing some needed .so files. check-in: 03539b7fce user: matt tags: v1.65 | |
Modified docs/manual/Makefile from [ec9633c3d9] to [759e8c25e6].
︙ | ︙ | |||
28 29 30 31 32 33 34 | # design_spec.html : $(SRCFILES) $(CSVFILES) # asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 design_spec.txt # all : server.ps megatest_manual.html client.ps complex-itemmap.png megatest_manual.pdf | | | | 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | # design_spec.html : $(SRCFILES) $(CSVFILES) # asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 design_spec.txt # all : server.ps megatest_manual.html client.ps complex-itemmap.png megatest_manual.pdf megatest_manual.html : megatest_manual.txt *.txt installation.txt *png *.dot asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 megatest_manual.txt # dos2unix megatest_manual.html megatest_manual.pdf : megatest_manual.txt *.txt *png *.dot a2x -a toc -f pdf megatest_manual.txt server.ps : server.dot dot -Tps server.dot > server.ps client.ps : client.dot dot -Tps client.dot > client.ps |
︙ | ︙ |
Added docs/manual/bisecting.dot version [01396be470].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | // Copyright 2021, Matthew Welland. // // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // Megatest is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with Megatest. If not, see <http://www.gnu.org/licenses/>. // digraph G { rankdir=LR subgraph cluster_1 { node [style=filled,shape=box]; B [label="B\nProblem is here"]; E [label="E\nProblem manifests here"]; A -> B; B -> C; C -> D; D -> E; } } |
Added docs/manual/bisecting.png version [6161dfa9e5].
cannot compute difference between binary files
Added docs/manual/debugging.txt version [8c97f79f3e].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | // Copyright 2021, Matthew Welland. // // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // Megatest is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with Megatest. If not, see <http://www.gnu.org/licenses/>. Debugging --------- A word on Bisecting ~~~~~~~~~~~~~~~~~~~ Bisecting is a debug strategy intended to speed up finding the root cause. ["graphviz", "bisecting.png"] ---------------------------------------------------------------------- include::bisecting.dot[] ---------------------------------------------------------------------- It is common to start debugging where the problem was observed and then work back. However by inspecting the output at stage "C" in the example above you would potentially save a lot of debug effort, this is similar to the feature in source control tools like git and fossil called biseceting. To know where to look for problems you will want to look at the log files at various stages in the execution process. ["graphviz", "megatest-test-stages.png"] ---------------------------------------------------------------------- include::megatest-test-stages.dot[] ---------------------------------------------------------------------- .How to check variable values at each stage [width="80%",cols="<,2m,2m",frame="topbot",options="header"] |====================== |Stage | How to inspect | Watch for |A: post config processing | megatest -show-config -target your/target | #f (failed var processing) |B: post runconfig | megatest -show-runconfig -target your/target | |C: processing testconfigs | inspect output from "megatest -run ..." | Messages indicating issues process configs, dependency problems. |D: process testconfig for test launch | inspect output from megatest runner | Zero items (items expansion yielded no items) |E,F: launching test | start test xterm, look at mt_launch.log | Did your batch system accept the job? Has the job landed on a machine? |G: starting test | look at your batch systems logs for the process | Did the megatest -execute process start and run? |H,H1,H2: step exectution | look at <stepname>.log, <stepname>.html and your own internal logs | Do you have sufficiently tight logpro rules? You must always have a "required" rule! |====================== Examining The Environment ~~~~~~~~~~~~~~~~~~~~~~~~~ Test Control Panel - xterm ^^^^^^^^^^^^^^^^^^^^^^^^^^ From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate text editor window). This is the easiest way to debug your tests. During Config File Processing ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It is often helpful to know the content of variables in various contexts as Megatest does the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined. For example, if an item list is not being generated as expected you can inject the startup of an xterm as if it were an item: .Original items table ----------------- [items] CELLNAME [system getcellname.sh] ----------------- .Items table modified for debug ----------------- [items] DEBUG [system xterm] CELLNAME [system getcellnames.sh] ----------------- When this test is run an xterm will pop up. In that xterm the environment is exactly that in which the script "getcellnames.sh" would run. You can now debug the script to find out why it isn't working as expected. |
Modified docs/manual/howto.txt from [5266978039] to [1b987427fd].
︙ | ︙ | |||
108 109 110 111 112 113 114 | [jobtools] launcher bsub # if defined and not "no" flexi-launcher will bypass launcher unless there is no # match. flexi-launcher yes ------------------------ | | | | | | 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | [jobtools] launcher bsub # if defined and not "no" flexi-launcher will bypass launcher unless there is no # match. flexi-launcher yes ------------------------ Tricks and Tips --------------- This section is a collection of a various useful tricks for that didn't quite fit elsewhere. Limiting your running jobs ~~~~~~~~~~~~~~~~~~~~~~~~~~ The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously. In your testconfig: |
︙ | ︙ | |||
134 135 136 137 138 139 140 | --------------- [jobgroups] group1 10 custdes 4 --------------- | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | --------------- [jobgroups] group1 10 custdes 4 --------------- Organising Your Tests and Tasks ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The default location "tests" for storing tests can be extended by adding to your tests-paths section. |
︙ | ︙ |
Added docs/manual/megatest-test-stages.dot version [41d7d6e43f].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | // Copyright 2021, Matthew Welland. // // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // Megatest is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // // You should have received a copy of the GNU General Public License // along with Megatest. If not, see <http://www.gnu.org/licenses/>. // digraph G { // rankdir=LR subgraph cluster_1 { node [style=filled,shape=box]; A [label="A: Process megatest.config"] B [label="B: Process runconfig.config"] A -> B [label="resolve vars"] B2 [label="B2: Resolve variables"] B -> B2 -> A C [label="C: Process testconfigs (find tests to run)"] D [label="D: Process testconfig for test of interest"] E [label="E: Set vars for launching test"] F [label="F: Launch into batch system, ssh, batch system\nand different hosts can all \nimpact variable values"] G [label="G: Process testconfig again with all available variables"] H [label="H: Start test"] H1 [label="H1: Start step"] H2 [label="H2: Execute step script"] I [label="I: End step, process logfile with logpro"] I -> H [label="Run remaining steps"] H-> H1 -> H2 -> I B->C->D->E->F->G->H I->D } } |
Added docs/manual/megatest-test-stages.png version [6b9ff2543b].
cannot compute difference between binary files
Modified docs/manual/megatest_manual.html from [cce838ed12] to [3ae4795548].
1 2 3 4 | <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> | | | 1 2 3 4 5 6 7 8 9 10 11 12 | <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta name="generator" content="AsciiDoc 8.6.10"> <title>The Megatest Users Manual</title> <style type="text/css"> /* Shared CSS for AsciiDoc xhtml11 and html5 backends */ /* Default font. */ body { font-family: Georgia,serif; |
︙ | ︙ | |||
82 83 84 85 86 87 88 | ul, ol, li > p { margin-top: 0; } ul > li { color: #aaa; } ul > li > * { color: black; } | | > > > > > > | 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | ul, ol, li > p { margin-top: 0; } ul > li { color: #aaa; } ul > li > * { color: black; } .monospaced, code, pre { font-family: "Courier New", Courier, monospace; font-size: inherit; color: navy; padding: 0; margin: 0; } pre { white-space: pre-wrap; } #author { color: #527bbd; font-weight: bold; font-size: 1.1em; } #email { |
︙ | ︙ | |||
214 215 216 217 218 219 220 | div.exampleblock > div.content { border-left: 3px solid #dddddd; padding-left: 0.5em; } div.imageblock div.content { padding-left: 0; } | | | 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 | div.exampleblock > div.content { border-left: 3px solid #dddddd; padding-left: 0.5em; } div.imageblock div.content { padding-left: 0; } span.image img { border-style: none; vertical-align: text-bottom; } a.image:visited { color: white; } dl { margin-top: 0.8em; margin-bottom: 0.8em; } dt { |
︙ | ︙ | |||
410 411 412 413 414 415 416 | /* * xhtml11 specific * * */ | < < < < < < | 416 417 418 419 420 421 422 423 424 425 426 427 428 429 | /* * xhtml11 specific * * */ div.tableblock { margin-top: 1.0em; margin-bottom: 1.5em; } div.tableblock > table { border: 3px solid #527bbd; } |
︙ | ︙ | |||
449 450 451 452 453 454 455 | /* * html5 specific * * */ | < < < < < < | 449 450 451 452 453 454 455 456 457 458 459 460 461 462 | /* * html5 specific * * */ table.tableblock { margin-top: 1.0em; margin-bottom: 1.5em; } thead, p.tableblock.header { font-weight: bold; color: #527bbd; |
︙ | ︙ | |||
534 535 536 537 538 539 540 541 542 543 544 545 546 547 | body.manpage div.sectionbody { margin-left: 3em; } @media print { body.manpage div#toc { display: none; } } @media screen { body { max-width: 50em; /* approximately 80 characters wide */ margin-left: 16em; } #toc { | > > | 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 | body.manpage div.sectionbody { margin-left: 3em; } @media print { body.manpage div#toc { display: none; } } @media screen { body { max-width: 50em; /* approximately 80 characters wide */ margin-left: 16em; } #toc { |
︙ | ︙ | |||
806 807 808 809 810 811 812 | along with Megatest. If not, see <http://www.gnu.org/licenses/>.</pre> </div></div> </div> </div> <div class="sect1"> <h2 id="_why_megatest">Why Megatest?</h2> <div class="sectionbody"> | | < | > | | | < | < < < < | 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 | along with Megatest. If not, see <http://www.gnu.org/licenses/>.</pre> </div></div> </div> </div> <div class="sect1"> <h2 id="_why_megatest">Why Megatest?</h2> <div class="sectionbody"> <div class="paragraph"><p>Megatest was created to provide a generalized tool for managing suites of regression tests and to provide a multi-host, distributed alternative to "make". The EDA world is littered with proprietory, company-specific tools for this purpose and by going open source and keeping the tool flexible the hope is that Megatest could be useful to any team at any company for continuous integration and almost any other general automation tasks.</p></div> </div> </div> <div class="sect1"> <h2 id="_megatest_design_philosophy">Megatest Design Philosophy</h2> <div class="sectionbody"> <div class="paragraph"><p>Megatest is a distributed system intended to provide the minimum needed resources to make writing a suite of tests and tasks for implementing |
︙ | ︙ | |||
957 958 959 960 961 962 963 | <div class="imageblock"> <div class="content"> <img src="megatest-system-architecture.png" alt="Static"> </div> </div> </div> </div> | < < < < < | 948 949 950 951 952 953 954 955 956 957 958 959 960 961 | <div class="imageblock"> <div class="content"> <img src="megatest-system-architecture.png" alt="Static"> </div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_todo_road_map">TODO / Road Map</h2> <div class="sectionbody"> <div class="paragraph"><p>Note: This road-map is a wish list and not a formal plan. Items are in rough priority but are subject to change. Development is driven by user requests, developer "itch" and bug reports. Please contact |
︙ | ︙ | |||
1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 | reviewed never</pre> </div></div> <div class="paragraph"><p>This test runs a single step called "stepname1" which runs a script "stepname.sh". Note that although it is common to put the actions needed for a test step into a script it is not necessary.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_how_to_do_things">How To Do Things</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_process_runs">Process Runs</h3> <div class="sect3"> | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 | reviewed never</pre> </div></div> <div class="paragraph"><p>This test runs a single step called "stepname1" which runs a script "stepname.sh". Note that although it is common to put the actions needed for a test step into a script it is not necessary.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_debugging">Debugging</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_a_word_on_bisecting">A word on Bisecting</h3> <div class="paragraph"><p>Bisecting is a debug strategy intended to speed up finding the root cause.</p></div> <div class="imageblock graphviz"> <div class="content"> <img src="bisecting.png" alt="bisecting.png"> </div> </div> <div class="paragraph"><p>It is common to start debugging where the problem was observed and then work back. However by inspecting the output at stage "C" in the example above you would potentially save a lot of debug effort, this is similar to the feature in source control tools like git and fossil called biseceting.</p></div> <div class="paragraph"><p>To know where to look for problems you will want to look at the log files at various stages in the execution process.</p></div> <div class="imageblock graphviz"> <div class="content"> <img src="megatest-test-stages.png" alt="megatest-test-stages.png"> </div> </div> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> <caption class="title">Table 2. How to check variable values at each stage</caption> <col style="width:20%;"> <col style="width:40%;"> <col style="width:40%;"> <thead> <tr> <th class="tableblock halign-left valign-top" >Stage </th> <th class="tableblock halign-left valign-top" > How to inspect </th> <th class="tableblock halign-left valign-top" > Watch for</th> </tr> </thead> <tbody> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">A: post config processing</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">megatest -show-config -target your/target</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">#f (failed var processing)</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">B: post runconfig</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">megatest -show-runconfig -target your/target</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced"></p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">C: processing testconfigs</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">inspect output from "megatest -run …"</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Messages indicating issues process configs, dependency problems.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">D: process testconfig for test launch</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">inspect output from megatest runner</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Zero items (items expansion yielded no items)</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">E,F: launching test</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">start test xterm, look at mt_launch.log</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Did your batch system accept the job? Has the job landed on a machine?</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">G: starting test</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">look at your batch systems logs for the process</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Did the megatest -execute process start and run?</p></td> </tr> <tr> <td class="tableblock halign-left valign-top" ><p class="tableblock">H,H1,H2: step exectution</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">look at <stepname>.log, <stepname>.html and your own internal logs</p></td> <td class="tableblock halign-left valign-top" ><p class="tableblock monospaced">Do you have sufficiently tight logpro rules? You must always have a "required" rule!</p></td> </tr> </tbody> </table> </div> <div class="sect2"> <h3 id="_examining_the_environment">Examining The Environment</h3> <div class="sect3"> <h4 id="_test_control_panel_xterm">Test Control Panel - xterm</h4> <div class="paragraph"><p>From the dashboard click on a test PASS/FAIL button. This brings up a test control panel. Aproximately near the center left of the window there is a button "Start Xterm". Push this to get an xterm with the full context and environment loaded for that test. You can run scripts or ezsteps by copying from the testconfig (hint, load up the testconfig in a separate text editor window). This is the easiest way to debug your tests.</p></div> </div> <div class="sect3"> <h4 id="_during_config_file_processing">During Config File Processing</h4> <div class="paragraph"><p>It is often helpful to know the content of variables in various contexts as Megatest does the actions needed to run your tests. A handy technique is to force the startup of an xterm in the context being examined.</p></div> <div class="paragraph"><p>For example, if an item list is not being generated as expected you can inject the startup of an xterm as if it were an item:</p></div> <div class="listingblock"> <div class="title">Original items table</div> <div class="content monospaced"> <pre>[items] CELLNAME [system getcellname.sh]</pre> </div></div> <div class="listingblock"> <div class="title">Items table modified for debug</div> <div class="content monospaced"> <pre>[items] DEBUG [system xterm] CELLNAME [system getcellnames.sh]</pre> </div></div> <div class="paragraph"><p>When this test is run an xterm will pop up. In that xterm the environment is exactly that in which the script "getcellnames.sh" would run. You can now debug the script to find out why it isn’t working as expected.</p></div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_how_to_do_things">How To Do Things</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_process_runs">Process Runs</h3> <div class="sect3"> |
︙ | ︙ | |||
2034 2035 2036 2037 2038 2039 2040 | # match. flexi-launcher yes</pre> </div></div> </div> </div> </div> <div class="sect1"> | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 | # match. flexi-launcher yes</pre> </div></div> </div> </div> </div> <div class="sect1"> <h2 id="_tricks_and_tips">Tricks and Tips</h2> <div class="sectionbody"> <div class="paragraph"><p>This section is a collection of a various useful tricks for that didn’t quite fit elsewhere.</p></div> <div class="sect2"> <h3 id="_limiting_your_running_jobs">Limiting your running jobs</h3> <div class="paragraph"><p>The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.</p></div> <div class="paragraph"><p>In your testconfig:</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[test_meta] jobgroup group1</pre> </div></div> <div class="paragraph"><p>In your megatest.config:</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[jobgroups] group1 10 custdes 4</pre> </div></div> <div class="sect3"> <h4 id="_organising_your_tests_and_tasks">Organising Your Tests and Tasks</h4> <div class="paragraph"><p>The default location "tests" for storing tests can be extended by adding to your tests-paths section.</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[misc] |
︙ | ︙ | |||
2146 2147 2148 2149 2150 2151 2152 | <div class="sectionbody"> <div class="sect2"> <h3 id="_megatest_use_modes">Megatest Use Modes</h3> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> | | | 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 | <div class="sectionbody"> <div class="sect2"> <h3 id="_megatest_use_modes">Megatest Use Modes</h3> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> <caption class="title">Table 3. Base commands</caption> <col style="width:20%;"> <col style="width:40%;"> <col style="width:40%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >Use case </th> <th class="tableblock halign-left valign-top" > Megatest command </th> |
︙ | ︙ | |||
2208 2209 2210 2211 2212 2213 2214 | <div class="sect2"> <h3 id="_config_file_helpers">Config File Helpers</h3> <div class="paragraph"><p>Various helpers for more advanced config files.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> | | | 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 | <div class="sect2"> <h3 id="_config_file_helpers">Config File Helpers</h3> <div class="paragraph"><p>Various helpers for more advanced config files.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:80%; "> <caption class="title">Table 4. Helpers</caption> <col style="width:14%;"> <col style="width:28%;"> <col style="width:28%;"> <col style="width:28%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >Helper </th> |
︙ | ︙ | |||
2338 2339 2340 2341 2342 2343 2344 | </div></div> </div> <div class="sect2"> <h3 id="_trim_trailing_spaces">Trim trailing spaces</h3> <div class="admonitionblock"> <table><tr> <td class="icon"> | | | 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 | </div></div> </div> <div class="sect2"> <h3 id="_trim_trailing_spaces">Trim trailing spaces</h3> <div class="admonitionblock"> <table><tr> <td class="icon"> <img src="/usr/images/icons/note.png" alt="Note"> </td> <td class="content">As of Megatest version v1.6548 trim-trailing-spaces defaults to yes.</td> </tr></table> </div> <div class="listingblock"> <div class="content monospaced"> <pre>[configf:settings trim-trailing-spaces no] |
︙ | ︙ | |||
2508 2509 2510 2511 2512 2513 2514 | </div> <div class="sect2"> <h3 id="_database_settings">Database settings</h3> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> | | | 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 | </div> <div class="sect2"> <h3 id="_database_settings">Database settings</h3> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> <caption class="title">Table 5. Database config settings in [setup] section of megatest.config</caption> <col style="width:14%;"> <col style="width:28%;"> <col style="width:28%;"> <col style="width:28%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >Var </th> |
︙ | ︙ | |||
2655 2656 2657 2658 2659 2660 2661 | <pre># A normal waiton waits for the prior tests to be COMPLETED # and PASS, CHECK or WAIVED waiton test1 test2</pre> </div></div> <div class="admonitionblock"> <table><tr> <td class="icon"> | | | 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 | <pre># A normal waiton waits for the prior tests to be COMPLETED # and PASS, CHECK or WAIVED waiton test1 test2</pre> </div></div> <div class="admonitionblock"> <table><tr> <td class="icon"> <img src="/usr/images/icons/note.png" alt="Note"> </td> <td class="content">Dynamic waiton lists must be capable of being calculated at the beginning of a run. This is because Megatest walks the tree of waitons to create the list of tests to execute.</td> </tr></table> </div> <div class="listingblock"> |
︙ | ︙ | |||
3014 3015 3016 3017 3018 3019 3020 | <div class="sect2"> <h3 id="_ezsteps">Ezsteps</h3> <div class="paragraph"><p>Ezsteps is the recommended way to implement tests and automation in Megatest.</p></div> <div class="admonitionblock"> <table><tr> <td class="icon"> | | | 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 | <div class="sect2"> <h3 id="_ezsteps">Ezsteps</h3> <div class="paragraph"><p>Ezsteps is the recommended way to implement tests and automation in Megatest.</p></div> <div class="admonitionblock"> <table><tr> <td class="icon"> <img src="/usr/images/icons/note.png" alt="Note"> </td> <td class="content">Each ezstep must be a single line. Use the [scripts] mechanism to create multiline scripts (see example below).</td> </tr></table> </div> <div class="listingblock"> <div class="title">Example ezsteps with logpro rules</div> |
︙ | ︙ | |||
3039 3040 3041 3042 3043 3044 3045 | <h4 id="_automatic_environment_propagation_with_ezsteps">Automatic environment propagation with Ezsteps</h4> <div class="paragraph"><p>Turn on ezpropvars and environment variables will be propagated from step to step. Use this to source script files that modify the envionment where the modifications are needed in subsequent steps.</p></div> <div class="admonitionblock"> <table><tr> <td class="icon"> | | | 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 | <h4 id="_automatic_environment_propagation_with_ezsteps">Automatic environment propagation with Ezsteps</h4> <div class="paragraph"><p>Turn on ezpropvars and environment variables will be propagated from step to step. Use this to source script files that modify the envionment where the modifications are needed in subsequent steps.</p></div> <div class="admonitionblock"> <table><tr> <td class="icon"> <img src="/usr/images/icons/note.png" alt="Note"> </td> <td class="content">aliases and variables with strange whitespace or characters will not propagate correctly. Put in a ticket on the <a href="http://www.kiatoa.com/fossils/megatest">http://www.kiatoa.com/fossils/megatest</a> site if you need support for a specific strange character combination.</td> </tr></table> </div> |
︙ | ︙ | |||
3227 3228 3229 3230 3231 3232 3233 | <div class="content monospaced"> <pre>[triggers] COMPLETED/ xterm -e bash -s --</pre> </div></div> <div class="admonitionblock"> <table><tr> <td class="icon"> | | | | 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 | <div class="content monospaced"> <pre>[triggers] COMPLETED/ xterm -e bash -s --</pre> </div></div> <div class="admonitionblock"> <table><tr> <td class="icon"> <img src="/usr/images/icons/note.png" alt="Note"> </td> <td class="content">There is a trailing space after the double-dash</td> </tr></table> </div> <div class="paragraph"><p>There are a number of environment variables available to the trigger script but since triggers can be called in various contexts not all variables are available at all times. The trigger script should check for the variable and fail gracefully if it doesn’t exist.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:90%; "> <caption class="title">Table 6. Environment variables visible to the trigger script</caption> <col style="width:50%;"> <col style="width:50%;"> <thead> <tr> <th class="tableblock halign-left valign-top" > Variable </th> <th class="tableblock halign-left valign-top" > Purpose</th> </tr> |
︙ | ︙ | |||
3474 3475 3476 3477 3478 3479 3480 | <h2 id="_programming_api">Programming API</h2> <div class="sectionbody"> <div class="paragraph"><p>These routines can be called from the megatest repl.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> | | | 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 | <h2 id="_programming_api">Programming API</h2> <div class="sectionbody"> <div class="paragraph"><p>These routines can be called from the megatest repl.</p></div> <table class="tableblock frame-topbot grid-all" style=" width:70%; "> <caption class="title">Table 7. API Keys Related Calls</caption> <col style="width:14%;"> <col style="width:28%;"> <col style="width:28%;"> <col style="width:28%;"> <thead> <tr> <th class="tableblock halign-center valign-top" >API Call </th> |
︙ | ︙ | |||
3564 3565 3566 3567 3568 3569 3570 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.5<br> | | > | 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.5<br> Last updated 2021-09-07 17:58:32 MST </div> </div> </body> </html> |
Added docs/manual/megatest_manual.pdf version [0043fc7813].
cannot compute difference between binary files
Modified docs/manual/megatest_manual.txt from [cb5cc67576] to [5b54398387].
︙ | ︙ | |||
31 32 33 34 35 36 37 | along with Megatest. If not, see <http://www.gnu.org/licenses/>. ---------------------------- Why Megatest? ------------- | < | | > | | | < < | < | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | along with Megatest. If not, see <http://www.gnu.org/licenses/>. ---------------------------- Why Megatest? ------------- Megatest was created to provide a generalized tool for managing suites of regression tests and to provide a multi-host, distributed alternative to "make". The EDA world is littered with proprietory, company-specific tools for this purpose and by going open source and keeping the tool flexible the hope is that Megatest could be useful to any team at any company for continuous integration and almost any other general automation tasks. Megatest Design Philosophy -------------------------- Megatest is a distributed system intended to provide the minimum needed resources to make writing a suite of tests and tasks for implementing continuous build for software, design engineering or process control (via |
︙ | ︙ | |||
96 97 98 99 100 101 102 103 104 105 106 | Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems. include::overview.txt[] include::plan.txt[] | > > | | | | 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems. // :leveloffset: 0 include::overview.txt[] include::plan.txt[] include::installation.txt[] include::getting_started.txt[] include::study_plan.txt[] include::writing_tests.txt[] include::debugging.txt[] include::howto.txt[] include::reference.txt[] include::testplan.txt[] |
︙ | ︙ |
Modified docs/manual/plan.txt from [84407a87ee] to [5f519e210f].
|
| < < < | 1 2 3 4 5 6 7 | // This file is part of Megatest. // // Megatest is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // |
︙ | ︙ |
Modified docs/manual/server.png from [ae7d7ee58e] to [43882638fe].
cannot compute difference between binary files
Modified http-transport.scm from [8b77952676] to [92b19b8dad].
︙ | ︙ | |||
38 39 40 41 42 43 44 | (declare (uses portlogger)) (declare (uses rmt)) (include "common_records.scm") (include "db_records.scm") (include "js-path.scm") | | | 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | (declare (uses portlogger)) (declare (uses rmt)) (include "common_records.scm") (include "db_records.scm") (include "js-path.scm") (use stml2) (define (http-transport:make-server-url hostport) (if (not hostport) #f (conc "http://" (car hostport) ":" (cadr hostport)))) (define *server-loop-heart-beat* (current-seconds)) |
︙ | ︙ |
Modified mtut.scm from [413cf26858] to [cc88551c71].
︙ | ︙ | |||
33 34 35 36 37 38 39 | (declare (uses configf)) ;; (declare (uses rmt)) (use ducttape-lib) (include "megatest-fossil-hash.scm") | | | 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | (declare (uses configf)) ;; (declare (uses rmt)) (use ducttape-lib) (include "megatest-fossil-hash.scm") (use stml2) ;; stuff for the mapper and checker functions ;; (define *target-mappers* (make-hash-table)) (define *runname-mappers* (make-hash-table)) (define *area-checkers* (make-hash-table)) |
︙ | ︙ |
Modified tests.scm from [698654fba2] to [8b856ef5fd].
︙ | ︙ | |||
32 33 34 35 36 37 38 | (declare (uses runconfig)) ;; (declare (uses sdb)) (declare (uses server)) ;;(declare (uses stml2)) (use sqlite3 srfi-1 posix regex regex-case srfi-69 dot-locking tcp directory-utils) (import (prefix sqlite3 sqlite3:)) | | | 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | (declare (uses runconfig)) ;; (declare (uses sdb)) (declare (uses server)) ;;(declare (uses stml2)) (use sqlite3 srfi-1 posix regex regex-case srfi-69 dot-locking tcp directory-utils) (import (prefix sqlite3 sqlite3:)) (use stml2) (include "common_records.scm") (include "key_records.scm") (include "db_records.scm") (include "run_records.scm") (include "test_records.scm") (include "js-path.scm") |
︙ | ︙ |