Overview
| Comment: | Added full example testconfig with ezsteps, scripts section etc. |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | v1.65 |
| Files: | files | file ages | folders |
| SHA1: |
0193399945554a426feec8595486cbc2 |
| User & Date: | mrwellan on 2020-03-03 13:25:37 |
| Other Links: | branch diff | manifest | tags |
Context
|
2020-03-10
| ||
| 15:14 | Updated Version check-in: e1e57863ea user: jmoon18 tags: v1.65 | |
|
2020-03-05
| ||
| 16:45 | Fixed the removal of test and run directories. Leaf check-in: e24a447e39 user: mmgraham tags: v1.65-martins-stuff | |
|
2020-03-03
| ||
| 13:25 | Added full example testconfig with ezsteps, scripts section etc. check-in: 0193399945 user: mrwellan tags: v1.65 | |
|
2020-03-02
| ||
| 09:17 | Add fix to start server after 15 attempts check-in: 37d16ef457 user: jmoon18 tags: v1.65, v1.6543, v1.6544 | |
Changes
Modified docs/manual/megatest_manual.html from [3974acbb23] to [2cca53be9f].
| ︙ | ︙ | |||
898 899 900 901 902 903 904 905 906 907 908 909 910 911 | which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_installation">Installation</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_dependencies">Dependencies</h3> <div class="paragraph"><p>Chicken scheme and a number of "eggs" are required for building | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 | which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_overview">Overview</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_stand_alone_megatest_area">Stand-alone Megatest Area</h3> <div class="paragraph"><p>A single, stand-alone, Megatest based testsuite or "area" is sufficient for most validation, automation and build problems.</p></div> <div class="imageblock"> <div class="content"> <img src="megatest-stand-alone-area.png" alt="Static"> </div> </div> <div class="paragraph"><p>Megatest is designed as a distributed or decoupled system. This means you can run the areas stand-alone with no additional infrastructure. I.e. there are no databases, web servers or other centralized resources needed. However as your needs grow you can integrate multiple areas into a bigger system.</p></div> <div class="sect3"> <h4 id="_component_descriptions">Component Descriptions</h4> <div class="olist arabic"><ol class="arabic"> <li> <p> Multi-area dashboard and xterm. A gui (the dashboard) is usually the best option for controlling and launching runs but all operations can also be done from the commandline. Note: The not yet released multi-area dashboard replaces the old dashboard for browsing and controlling runs but for managing a single area the old dashboard works very well. </p> </li> <li> <p> Area/testsuite. This is your testsuite or automation definition and consists of the information in megatest.config, runconfigs.config and your testconfigs along with any custom scripting that can’t be done with the native Megatest features. </p> </li> <li> <p> If your testsuite or build automation is too large to run on a single instance you can distribute your jobs into a compute server pool. The only current requirements are password-less ssh access and a network filesystem. </p> </li> </ol></div> </div> </div> <div class="sect2"> <h3 id="_full_system_architecture">Full System Architecture</h3> <div class="imageblock"> <div class="content"> <img src="megatest-system-architecture.png" alt="Static"> </div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_installation">Installation</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_dependencies">Dependencies</h3> <div class="paragraph"><p>Chicken scheme and a number of "eggs" are required for building |
| ︙ | ︙ | |||
2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 |
</div></div>
<div class="paragraph"><p>To transfer the environment to the next step you can do the following:</p></div>
<div class="listingblock">
<div class="title">Propagate environment to next step</div>
<div class="content monospaced">
<pre>$MT_MEGATEST -env2file .ezsteps/${stepname}</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_triggers">Triggers</h3>
<div class="paragraph"><p>In your testconfig or megatest.config triggers can be specified</p></div>
<div class="listingblock">
<div class="title">Triggers spec</div>
<div class="content monospaced">
| > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 |
</div></div>
<div class="paragraph"><p>To transfer the environment to the next step you can do the following:</p></div>
<div class="listingblock">
<div class="title">Propagate environment to next step</div>
<div class="content monospaced">
<pre>$MT_MEGATEST -env2file .ezsteps/${stepname}</pre>
</div></div>
<div class="listingblock">
<div class="title">Full example with ezsteps, logpro rules, scripts etc.</div>
<div class="content monospaced">
<pre># You can include a common file
#
[include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc]
# Use "var" for a scratch pad
#
[var]
dumpsql select * from data;
sepstr .....................................
# NOT IMPLEMENTED YET!
#
[ezsteps-addendum]
prescript something.sh
postscript something2.sh
# Add additional steps here. Format is "stepname script"
[ezsteps]
importdb loaddb prod.db prod.sql
dumpprod dumpdata prod.db "#{get var dumpsql}"
diff (echo "prod#{get var sepstr}test";diff --side-by-side \
dumpprod.log reference.log ;echo DIFFDONE)
[scripts]
loaddb #!/bin/bash
sqlite3 $1 <<EOF
.mode tabs
.import $2 data
.q
EOF
dumpdata #!/bin/bash
sqlite3 $1 <<EOF
.separator ,
$2
.q
EOF
# Test requirements are specified here
[requirements]
waiton setup
priority 0
# Iteration for your test is controlled by the items section
# The complicated if is needed to allow processing of the config for the dashboard when there are no actual runs.
[items]
THINGNAME [system generatethings.sh | sort -u]
# Logpro rules for each step can be captured here in the testconfig
# note: The ;; after the stepname and the leading whitespace are required
#
[logpro]
inputdb ;;
(expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
(expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
(expect:required in "LogFileBody" > 0 "Some data found" #/^[a-z]{3,4}[0-9]+_r.*/)
diff ;;
(expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
(expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
(expect:error in "LogFileBody" = 0 "< or > indicate missing entry" (list #/(<|>)/ #/error/i))
(expect:error in "LogFileBody" = 0 "Difference in data" (list #/\s+\|\s+/ #/error/i))
(expect:required in "LogFileBody" > 0 "DIFFDONE Marker found" #/DIFFDONE/)
(expect:required in "LogFileBody" > 0 "Some things found" #/^[a-z]{3,4}[0-9]+_r.*/)
# NOT IMPLEMENTED YET!
#
## Also: enhance logpro to take list of command files: file1,file2...
[waivers]
createprod{target=%78/%/%/%} ;;
(disable:required "DIFFDONE Marker found")
(disable:error "Some error")
(expect:waive in "LogFileBody" < 99 "Waive if failed due to version" #/\w+3\.6.*/)
# test_meta is a section for storing additional data on your test
[test_meta]
author matt
owner matt
description Compare things
tags tagone,tagtwo
reviewed never</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_triggers">Triggers</h3>
<div class="paragraph"><p>In your testconfig or megatest.config triggers can be specified</p></div>
<div class="listingblock">
<div class="title">Triggers spec</div>
<div class="content monospaced">
|
| ︙ | ︙ | |||
2988 2989 2990 2991 2992 2993 2994 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.0<br> | | | 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.0<br> Last updated 2020-01-02 13:39:49 PST </div> </div> </body> </html> |
Added docs/manual/overview.txt version [79d741067f].
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | Overview -------- Stand-alone Megatest Area ~~~~~~~~~~~~~~~~~~~~~~~~~ A single, stand-alone, Megatest based testsuite or "area" is sufficient for most validation, automation and build problems. image::megatest-stand-alone-area.png[Static,300] Megatest is designed as a distributed or decoupled system. This means you can run the areas stand-alone with no additional infrastructure. I.e. there are no databases, web servers or other centralized resources needed. However as your needs grow you can integrate multiple areas into a bigger system. Component Descriptions ^^^^^^^^^^^^^^^^^^^^^^ . Multi-area dashboard and xterm. A gui (the dashboard) is usually the best option for controlling and launching runs but all operations can also be done from the commandline. Note: The not yet released multi-area dashboard replaces the old dashboard for browsing and controlling runs but for managing a single area the old dashboard works very well. . Area/testsuite. This is your testsuite or automation definition and consists of the information in megatest.config, runconfigs.config and your testconfigs along with any custom scripting that can't be done with the native Megatest features. . If your testsuite or build automation is too large to run on a single instance you can distribute your jobs into a compute server pool. The only current requirements are password-less ssh access and a network filesystem. Full System Architecture ~~~~~~~~~~~~~~~~~~~~~~~~ image::megatest-system-architecture.png[Static,300] |
Modified docs/manual/reference.txt from [ae6c9c2e3a] to [41ce966733].
| ︙ | ︙ | |||
649 650 651 652 653 654 655 656 657 658 659 660 661 662 |
To transfer the environment to the next step you can do the following:
.Propagate environment to next step
----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}
----------------------------
Triggers
~~~~~~~~
In your testconfig or megatest.config triggers can be specified
.Triggers spec
| > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 |
To transfer the environment to the next step you can do the following:
.Propagate environment to next step
----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}
----------------------------
.Full example with ezsteps, logpro rules, scripts etc.
-----------------
# You can include a common file
#
[include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc]
# Use "var" for a scratch pad
#
[var]
dumpsql select * from data;
sepstr .....................................
# NOT IMPLEMENTED YET!
#
[ezsteps-addendum]
prescript something.sh
postscript something2.sh
# Add additional steps here. Format is "stepname script"
[ezsteps]
importdb loaddb prod.db prod.sql
dumpprod dumpdata prod.db "#{get var dumpsql}"
diff (echo "prod#{get var sepstr}test";diff --side-by-side \
dumpprod.log reference.log ;echo DIFFDONE)
[scripts]
loaddb #!/bin/bash
sqlite3 $1 <<EOF
.mode tabs
.import $2 data
.q
EOF
dumpdata #!/bin/bash
sqlite3 $1 <<EOF
.separator ,
$2
.q
EOF
# Test requirements are specified here
[requirements]
waiton setup
priority 0
# Iteration for your test is controlled by the items section
# The complicated if is needed to allow processing of the config for the dashboard when there are no actual runs.
[items]
THINGNAME [system generatethings.sh | sort -u]
# Logpro rules for each step can be captured here in the testconfig
# note: The ;; after the stepname and the leading whitespace are required
#
[logpro]
inputdb ;;
(expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
(expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
(expect:required in "LogFileBody" > 0 "Some data found" #/^[a-z]{3,4}[0-9]+_r.*/)
diff ;;
(expect:ignore in "LogFileBody" < 99 "Ignore error in comments" #/^\/\/.*error/)
(expect:warning in "LogFileBody" = 0 "Any warning" #/warn/)
(expect:error in "LogFileBody" = 0 "< or > indicate missing entry" (list #/(<|>)/ #/error/i))
(expect:error in "LogFileBody" = 0 "Difference in data" (list #/\s+\|\s+/ #/error/i))
(expect:required in "LogFileBody" > 0 "DIFFDONE Marker found" #/DIFFDONE/)
(expect:required in "LogFileBody" > 0 "Some things found" #/^[a-z]{3,4}[0-9]+_r.*/)
# NOT IMPLEMENTED YET!
#
## Also: enhance logpro to take list of command files: file1,file2...
[waivers]
createprod{target=%78/%/%/%} ;;
(disable:required "DIFFDONE Marker found")
(disable:error "Some error")
(expect:waive in "LogFileBody" < 99 "Waive if failed due to version" #/\w+3\.6.*/)
# test_meta is a section for storing additional data on your test
[test_meta]
author matt
owner matt
description Compare things
tags tagone,tagtwo
reviewed never
-----------------
Triggers
~~~~~~~~
In your testconfig or megatest.config triggers can be specified
.Triggers spec
|
| ︙ | ︙ |