Changes In Branch v1.65-cpick01 Through [f471242dd5] Excluding Merge-Ins
This is equivalent to a diff from 367ffc5bdf to f471242dd5
2020-05-31
| ||
21:19 | Cherrypick 0495fb 2a858 from v1.65-broken. Better support for utilizing MT_ vars to fill defaults when -target, -testpatt, etc. are not specified check-in: 74a0e98868 user: mrwellan tags: v1.65 | |
21:16 | Cherrypick 0a9e69 aa2998 63e558 b487e8 e28be4 50237f 64aa934, passes all. Closed-Leaf check-in: ac1b58436b user: mrwellan tags: v1.65-cpick01 | |
21:01 | Cherrypick ea8c3e 5b7f0 7ff937 18da8b, cpick03, all pass. check-in: f471242dd5 user: mrwellan tags: v1.65-cpick01 | |
20:27 | Cherry pick 5de2f 06451 0364b 07ff7 a374d and full pass on acatqa and ext-tests check-in: 181a8b6b1c user: mrwellan tags: v1.65-cpick01 | |
19:49 | Cherrypick 0495fb 2a858 from v1.65-broken. check-in: fb39cefbf1 user: mrwellan tags: v1.65-cpick01 | |
16:48 | Pulled in changes only related to fixes from newview branch Closed-Leaf check-in: 4d6ba57051 user: matt tags: v1.65-oops-n | |
2020-05-27
| ||
12:46 | Create new branch named "v1.65-junit-xml" check-in: 473d6eaf82 user: pjhatwal tags: v1.65-junit-xml | |
2020-04-20
| ||
21:49 | Merged newview work into v1.65 to minimize divergences check-in: 3b86fd8d4c user: mrwellan tags: v1.65-broken | |
2020-04-06
| ||
16:21 | Merged in v1.65 check-in: db55d34798 user: mrwellan tags: v1.65-newview | |
2020-03-10
| ||
15:14 | merged branch check-in: 367ffc5bdf user: mmgraham tags: v1.65, v1.6545 | |
15:14 | Updated Version check-in: e1e57863ea user: jmoon18 tags: v1.65 | |
2020-03-05
| ||
16:45 | Fixed the removal of test and run directories. Leaf check-in: e24a447e39 user: mmgraham tags: v1.65-martins-stuff | |
Modified archive.scm from [618f9a591e] to [80337cecee].
︙ | |||
384 385 386 387 388 389 390 | 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 | - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + | (debug:print-info 0 *default-log-port* "Restoring archived data to " new-test-physical-path " from archive in " archive-path " ... " archive-internal-path) ;; (mutex-lock! bup-mutex) (run-n-wait bup-exe params: bup-restore-params print-cmd: #f) ;; (mutex-unlock! bup-mutex) (mt:test-set-state-status-by-id run-id test-id "COMPLETED" #f #f))) (debug:print-error 0 *default-log-port* "No archive path in the record for run-id=" run-id " test-id=" test-id)))) (filter vector? tests)))) |
Modified common.scm from [4d4a2441c8] to [dc9844e378].
︙ | |||
1195 1196 1197 1198 1199 1200 1201 | 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 | - + + + + + + + + + + + + + | ;; use bash to expand a glob. Does NOT handle paths with spaces! ;; (define (common:bash-glob instr) (string-split (with-input-from-pipe (conc "/bin/bash -c \"echo " instr "\"") |
︙ |
Modified docs/manual/howto.txt from [b66065dad3] to [5266978039].
︙ | |||
66 67 68 69 70 71 72 73 74 75 76 77 78 79 | 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | + + + + + + + + + + + + + + + + + | ---------------- Hint: You can browse the archive using bup commands directly. ---------------- bup -d /path/to/bup/archive ftp ---------------- Pass Data from Test to Test ~~~~~~~~~~~~~~~~~~~~~~~~~~~ .To save the data call archive save within your test: ---------------- megatest -archive save ---------------- .To retrieve the data call archive get using patterns as needed ---------------- # Put the retrieved data into /tmp DESTPATH=/tmp/$USER/$MT_TARGET/$MT_RUN_NAME/$MT_TESTNAME/$MT_ITEMPATH/my_data mkdir -p $DESTPATH megatest -archive get -runname % -dest $DESTPATH ---------------- Submit jobs to Host Types based on Test Name ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .In megatest.config ------------------------ [host-types] |
︙ |
Modified docs/manual/megatest_manual.html from [2cca53be9f] to [dd1dac404b].
︙ | |||
1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 | 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 | + + + + + + + + + + + + + + + + | <div class="paragraph"><p>Hint: You can browse the archive using bup commands directly.</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>bup -d /path/to/bup/archive ftp</pre> </div></div> </div> </div> </div> <div class="sect2"> <h3 id="_pass_data_from_test_to_test">Pass Data from Test to Test</h3> <div class="listingblock"> <div class="title">To save the data call archive save within your test:</div> <div class="content monospaced"> <pre>megatest -archive save</pre> </div></div> <div class="listingblock"> <div class="title">To retrieve the data call archive get using patterns as needed</div> <div class="content monospaced"> <pre># Put the retrieved data into /tmp DESTPATH=/tmp/$USER/$MT_TARGET/$MT_RUN_NAME/$MT_TESTNAME/$MT_ITEMPATH/my_data mkdir -p $DESTPATH megatest -archive get -runname % -dest $DESTPATH</pre> </div></div> </div> <div class="sect2"> <h3 id="_submit_jobs_to_host_types_based_on_test_name">Submit jobs to Host Types based on Test Name</h3> <div class="listingblock"> <div class="title">In megatest.config</div> <div class="content monospaced"> <pre>[host-types] |
︙ | |||
2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 | 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 | + + + + + + + + + + + + + + + + + + + | </div></div> <div class="paragraph"><p>To transfer the environment to the next step you can do the following:</p></div> <div class="listingblock"> <div class="title">Propagate environment to next step</div> <div class="content monospaced"> <pre>$MT_MEGATEST -env2file .ezsteps/${stepname}</pre> </div></div> </div> <div class="sect2"> <h3 id="_scripts">Scripts</h3> <div class="listingblock"> <div class="title">Specifying scripts inline (best used for only simple scripts)</div> <div class="content monospaced"> <pre>[scripts] loaddb #!/bin/bash sqlite3 $1 <<EOF .mode tabs .import $2 data .q EOF</pre> </div></div> <div class="paragraph"><p>The above snippet results in the creation of an executable script called "loaddb" in the test directory. NOTE: every line in the script must be prefixed with the exact same number of spaces. Lines beginning with a # will not work as expected. Currently you cannot indent intermediate lines.</p></div> <div class="listingblock"> <div class="title">Full example with ezsteps, logpro rules, scripts etc.</div> <div class="content monospaced"> <pre># You can include a common file # [include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc] |
︙ | |||
3132 3133 3134 3135 3136 3137 3138 | 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 | - + | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.0<br> |
Modified docs/manual/reference.txt from [41ce966733] to [332bf8caed].
︙ | |||
462 463 464 465 466 467 468 | 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 | - - | -------------- Complex mapping example ~~~~~~~~~~~~~~~~~~~~~~~ |
︙ | |||
512 513 514 515 516 517 518 | 510 511 512 513 514 515 516 517 518 519 520 521 522 523 | - - - - | . Test A has no waitons. All waitons of all tests in full list have been processed. Full list is finalized. itemstable ~~~~~~~~~~ An alternative to defining items is the itemstable section. This lets you define the itempath in a table format rather than specifying components and relying on getting all permutations of those components. |
︙ | |||
650 651 652 653 654 655 656 657 658 659 660 661 662 663 | 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 | + + + + + + + + + + + + + + + + + + + + | To transfer the environment to the next step you can do the following: .Propagate environment to next step ---------------------------- $MT_MEGATEST -env2file .ezsteps/${stepname} ---------------------------- Scripts ~~~~~~~ .Specifying scripts inline (best used for only simple scripts) ---------------------------- [scripts] loaddb #!/bin/bash sqlite3 $1 <<EOF .mode tabs .import $2 data .q EOF ---------------------------- The above snippet results in the creation of an executable script called "loaddb" in the test directory. NOTE: every line in the script must be prefixed with the exact same number of spaces. Lines beginning with a # will not work as expected. Currently you cannot indent intermediate lines. .Full example with ezsteps, logpro rules, scripts etc. ----------------- # You can include a common file # [include #{getenv MT_RUN_AREA_HOME}/global-testconfig.inc] # Use "var" for a scratch pad |
︙ |
Modified launch.scm from [6b5c8d69de] to [76b25bc200].
︙ | |||
621 622 623 624 625 626 627 | 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 | + - + + + + + + + + | (set-signal-handler! signal/int sighand) (set-signal-handler! signal/term sighand) ) ;; (set-signal-handler! signal/stop sighand) ;; Do not run the test if it is REMOVING, RUNNING, KILLREQ or REMOTEHOSTSTART, ;; Mark the test as REMOTEHOSTSTART *IMMEDIATELY* ;; (let* ((test-info (let loop ((tries 0)) |
︙ | |||
1122 1123 1124 1125 1126 1127 1128 | 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 | - + + | (set! toppath *toppath*) (if (not *toppath*) (begin (debug:print-error 0 *default-log-port* "you are not in a megatest area!") (exit 1))) (setenv "MT_RUN_AREA_HOME" *toppath*) ;; the seed read is done, now read runconfigs, cache it then read megatest.config one more time and cache it |
︙ |
Modified megatest.scm from [7e6fcbd15f] to [c4f77722c1].
︙ | |||
224 225 226 227 228 229 230 | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 | - + | -refdb2dat refdb : convert refdb to sexp or to format specified by s-dumpmode formats: perl, ruby, sqlite3, csv (for csv the -o param will substitute %s for the sheet name in generating multiple sheets) -o : output file for refdb2dat (defaults to stdout) -archive cmd : archive runs specified by selectors to one of disks specified in the [archive-disks] section. |
︙ | |||
297 298 299 300 301 302 303 304 305 306 307 308 309 310 | 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | + | "-logpro" "-m" "-rerun" "-days" "-rename-run" "-to" "-dest" ;; values and messages ":category" ":variable" ":value" ":expected" ":tol" ":units" |
︙ | |||
1039 1040 1041 1042 1043 1044 1045 | 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 | - + + + + + + + + + - + + - + + - - - + + + - - + + + - - - + + + | ;;====================================================================== ;; Remove old run(s) ;;====================================================================== ;; since several actions can be specified on the command line the removal ;; is done first |
︙ | |||
1862 1863 1864 1865 1866 1867 1868 | 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 | + + + + + + + - - - - - + + + + + | ;;====================================================================== ;; Archive tests ;;====================================================================== ;; Archive tests matching target, runname, and testpatt (if (args:get-arg "-archive") ;; else do a general-run-call (begin ;; for the archive get we need to preserve the starting dir as part of the target path (if (and (args:get-arg "-dest") (not (equal? (substring (args:get-arg "-dest") 0 1) "/"))) (let ((newpath (conc (current-directory) "/" (args:get-arg "-dest")))) (debug:print-info 1 *default-log-port* "Preserving original path to destination, was " (args:get-arg "-dest") ", now " newpath) (hash-table-set! args:arg-hash "-dest" newpath))) |
︙ |
Modified runs.scm from [6cea658ad9] to [ebe409b112].
︙ | |||
1279 1280 1281 1282 1283 1284 1285 | 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 | + - - - - - - - - + + + + + + + + + + + + + + + | (let ((curr-sec (current-seconds))) (if (> (- curr-sec (runs:gendat-inc-results-last-update *runs:general-data*)) 5) ;; at least five seconds since last update (let* ((run-dat (or (runs:gendat-run-info *runs:general-data*)(rmt:get-run-info run-id))) (runname (or (runs:gendat-runname *runs:general-data*) (db:get-value-by-header (db:get-rows run-dat) (db:get-header run-dat) "runname"))) (target (or (runs:gendat-target *runs:general-data*)(rmt:get-target run-id))) (testsdat (let ((res (rmt:get-tests-for-run |
︙ | |||
2103 2104 2105 2106 2107 2108 2109 | 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 | - + + | (header (vector-ref rundat 0)) (runs (vector-ref rundat 1)) (states (if state (string-split state ",") '())) (statuses (if status (string-split status ",") '())) (state-status (if (string? new-state-status) (string-split new-state-status ",") '(#f #f))) (rp-mutex (make-mutex)) (bup-mutex (make-mutex)) |
︙ | |||
2137 2138 2139 2140 2141 2142 2143 | 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 | - + | (let* ((run-id (db:get-value-by-header run header "id")) (run-state (db:get-value-by-header run header "state")) (run-name (db:get-value-by-header run header "runname")) (tests (if (not (equal? run-state "locked")) (proc-get-tests run-id) '())) (lasttpath "/does/not/exist/I/hope") |
︙ | |||
2164 2165 2166 2167 2168 2169 2170 | 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 | + - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + | ((print-run) (debug:print 1 *default-log-port* "Printing info for run " runkey ", run=" run ", tests=" tests ", header=" header) action) ((run-wait) (debug:print 1 *default-log-port* "Waiting for run " runkey ", run=" runnamepatt " to complete")) ((archive) (debug:print 1 *default-log-port* "Archiving/restoring (" (args:get-arg "-archive") ") data for run: " runkey " " (db:get-value-by-header run header "runname")) (let ((op (string->symbol (args:get-arg "-archive")))) |
︙ | |||
2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 | 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 | + | (debug:print-info 0 *default-log-port* "Estimating disk space usage for " test-fulln ": " (common:get-disk-space-used ddir))))))) (if (not (null? tal)) (loop (car tal)(cdr tal)))) ))) ) (if worker-thread (thread-join! worker-thread))) (common:join-backgrounded-threads)))) ;; remove the run if zero tests remain (if (eq? action 'remove-runs) (let* ((run-id (db:get-value-by-header run header "id")) ;; NB// masks run-id from above? (remtests (mt:get-tests-for-run run-id #f '("DELETED") '("n/a") not-in: #t))) (if (null? remtests) ;; no more tests remaining ;; Remove the last dir from the path. ;; And same for the link-resolved path |
︙ | |||
2421 2422 2423 2424 2425 2426 2427 | 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 | - - - - + + + + + + | (debug:print 1 *default-log-port* "Recursively removing real dir " realpath) (runs:recursive-delete-with-error-msg realpath) ))))) )) runs) |
︙ | |||
2498 2499 2500 2501 2502 2503 2504 | 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 | - + | ;;====================================================================== ;; Routines for manipulating runs ;;====================================================================== ;; Since many calls to a run require pretty much the same setup ;; this wrapper is used to reduce the replication of code (define (general-run-call switchname action-desc proc) |
︙ |