Overview
Comment: | updated itemmap section in manual; hopefully it is clearer now |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
d2c0ce97e35fc7f363a4b5115c82b0c7 |
User & Date: | bjbarcla on 2017-12-12 14:36:45 |
Other Links: | manifest | tags |
Context
2017-12-12
| ||
15:31 | updated manual check-in: 828825ea53 user: bjbarcla tags: trunk | |
14:36 | updated itemmap section in manual; hopefully it is clearer now check-in: d2c0ce97e3 user: bjbarcla tags: trunk | |
14:34 | updated itemmap section in manual; hopefully it is clearer now check-in: c607976150 user: bjbarcla tags: v1.64 | |
2017-12-01
| ||
14:54 | Merge v1.65 to trunk to bring online manual up to date. check-in: fa8b9592a7 user: mrwellan tags: trunk | |
Changes
Modified common.scm from [3e2ab1e8be] to [113790f9bc].
︙ | ︙ | |||
517 518 519 520 521 522 523 524 525 526 527 528 529 530 | (3 "KILLED") (4 "NOT_STARTED") (5 "COMPLETED") (6 "LAUNCHED") (7 "REMOTEHOSTSTART") (8 "RUNNING") )) ;; BBnote: *common:std-statuses* dashboard filter control and test control status buttons defined here; used in set-fields-panel and dboard:make-controls ;; note these statuses are sorted from better to worse. ;; This sort order is important to dcommon:status-compare3 and db:set-state-status-and-roll-up-items (define *common:std-statuses* '(;; (0 "DELETED") (1 "n/a") | > > > > > > > | 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 | (3 "KILLED") (4 "NOT_STARTED") (5 "COMPLETED") (6 "LAUNCHED") (7 "REMOTEHOSTSTART") (8 "RUNNING") )) (define *common:dont-roll-up-states* '("DELETED" "REMOVING" "CLEANING" "ARCHIVE_REMOVING" )) ;; BBnote: *common:std-statuses* dashboard filter control and test control status buttons defined here; used in set-fields-panel and dboard:make-controls ;; note these statuses are sorted from better to worse. ;; This sort order is important to dcommon:status-compare3 and db:set-state-status-and-roll-up-items (define *common:std-statuses* '(;; (0 "DELETED") (1 "n/a") |
︙ | ︙ | |||
554 555 556 557 558 559 560 561 562 563 564 565 566 567 | '("RUNNING" "REMOTEHOSTSTART" "LAUNCHED" "STARTED")) (define *common:cant-run-states* ;; These are stopping conditions that prevent a test from being run '("COMPLETED" "KILLED" "UNKNOWN" "INCOMPLETE" "ARCHIVED")) (define *common:not-started-ok-statuses* ;; if not one of these statuses when in not_started state treat as dead '("n/a" "na" "PASS" "FAIL" "WARN" "CHECK" "WAIVED" "DEAD" "SKIP")) (define (common:special-sort items order comp) (let ((items-order (map reverse order)) (acomp (or comp >))) (sort items (lambda (a b) (let ((a-num (cadr (or (assoc a items-order) '(0 0)))) | > > > > > > | 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 | '("RUNNING" "REMOTEHOSTSTART" "LAUNCHED" "STARTED")) (define *common:cant-run-states* ;; These are stopping conditions that prevent a test from being run '("COMPLETED" "KILLED" "UNKNOWN" "INCOMPLETE" "ARCHIVED")) (define *common:not-started-ok-statuses* ;; if not one of these statuses when in not_started state treat as dead '("n/a" "na" "PASS" "FAIL" "WARN" "CHECK" "WAIVED" "DEAD" "SKIP")) ;; group tests into buckets corresponding to rollup ;;; Running, completed-pass, completed-non-pass + worst status, not started. ;; filter out ;(define (common:categorize-items-for-rollup in-tests) ; ( (define (common:special-sort items order comp) (let ((items-order (map reverse order)) (acomp (or comp >))) (sort items (lambda (a b) (let ((a-num (cadr (or (assoc a items-order) '(0 0)))) |
︙ | ︙ |
Modified dashboard.scm from [62334ab065] to [22bb220403].
︙ | ︙ | |||
109 110 111 112 113 114 115 | (print "Failed to find megatest.config, exiting") (exit 1))) ;; deal with RH 5.11 gtk lib or iup lib missing detachbox feature ;; first check for the switch ;; (if (or (args:get-arg "-rh5.11") | | > | 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | (print "Failed to find megatest.config, exiting") (exit 1))) ;; deal with RH 5.11 gtk lib or iup lib missing detachbox feature ;; first check for the switch ;; (if (or (args:get-arg "-rh5.11") (configf:lookup *configdat* "dashboard" "no-detachbox") (not (file-exists? "/etc/os-release"))) (set! iup:detachbox iup:vbox)) (if (not (common:on-homehost?)) (begin (debug:print 0 *default-log-port* "WARNING: Current policy requires running dashboard on homehost: " (common:get-homehost)))) ;; RA => Might require revert for filters |
︙ | ︙ |
Modified db.scm from [20d0d3c079] to [1866550cfc].
︙ | ︙ | |||
3607 3608 3609 3610 3611 3612 3613 | *common:not-started-ok-statuses*)))) state-status-counts))) ;; (non-completes (filter (lambda (x) ;; (not (equal? (dbr:counts-state x) "COMPLETED"))) ;; state-status-counts)) (all-curr-states (common:special-sort ;; worst -> best (sort of) (delete-duplicates | | | | | 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 | *common:not-started-ok-statuses*)))) state-status-counts))) ;; (non-completes (filter (lambda (x) ;; (not (equal? (dbr:counts-state x) "COMPLETED"))) ;; state-status-counts)) (all-curr-states (common:special-sort ;; worst -> best (sort of) (delete-duplicates (if (not (member state *common:dont-roll-up-states*)) (cons state (map dbr:counts-state state-status-counts)) (map dbr:counts-state state-status-counts))) *common:std-states* >)) (all-curr-statuses (common:special-sort ;; worst -> best (delete-duplicates (if (not (member state *common:dont-roll-up-states*)) (cons status (map dbr:counts-status state-status-counts)) (map dbr:counts-status state-status-counts))) *common:std-statuses* >)) (non-completes (filter (lambda (x) (not (member x (cons "COMPLETED" *common:dont-roll-up-states*)))) all-curr-states)) (preq-fails (filter (lambda (x) (equal? x "PREQ_FAIL")) all-curr-statuses)) (num-non-completes (length non-completes)) (newstate (cond ((> running 0) "RUNNING") ;; anything running, call the situation running |
︙ | ︙ | |||
4337 4338 4339 4340 4341 4342 4343 | (if (and waiton-is-toplevel ;; if upstream rollup test is completed, parent-waiton-met is set (or waiton-is-completed waiton-is-running)) (set! parent-waiton-met #t))) ;; normal checking of parent items, any parent or parent item not ok blocks running ((and waiton-is-completed (or waiton-is-ok (member 'toplevel mode)) ;; toplevel does not block on FAIL | | | 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 | (if (and waiton-is-toplevel ;; if upstream rollup test is completed, parent-waiton-met is set (or waiton-is-completed waiton-is-running)) (set! parent-waiton-met #t))) ;; normal checking of parent items, any parent or parent item not ok blocks running ((and waiton-is-completed (or waiton-is-ok (member 'toplevel mode)) ;; toplevel does not block on FAIL (and waiton-is-ok (member 'itemmatch mode) ;; itemmatch blocks on not ok ;; TODO: THIS IS PROBABLY A BUG. ITEMMATCH AND ITEMWAIT ARE SYNONYMS!! WHAT HAPPENED OT ITEMWAIT??? )) ;;(BB> "cond4") (set! item-waiton-met #t)) ((and waiton-is-completed waiton-is-ok same-itempath) ;;(BB> "cond5") (set! item-waiton-met #t)) |
︙ | ︙ |
Modified docs/manual/complex-itemmap.dot from [1ce1e37196] to [29ad0d06b8].
︙ | ︙ | |||
36 37 38 39 40 41 42 | subgraph cluster_6 { node [style=filled]; label = "Test E"; "C/1/bb" -> "E/1/res"; "C/2/bb" -> "E/2/res"; } | | | 36 37 38 39 40 41 42 43 44 45 46 47 | subgraph cluster_6 { node [style=filled]; label = "Test E"; "C/1/bb" -> "E/1/res"; "C/2/bb" -> "E/2/res"; } label = "Complex Itemmapping (arrows indicate order of execution)"; color=green; } } |
Modified docs/manual/complex-itemmap.png from [d2bdff7bfb] to [e000b2b035].
cannot compute difference between binary files
Modified docs/manual/megatest_manual.html from [9ee0d26d38] to [acbcdce45d].
︙ | ︙ | |||
1724 1725 1726 1727 1728 1729 1730 1731 1732 | </div> <div class="sect2"> <h3 id="_itemmap_handling">Itemmap Handling</h3> <div class="paragraph"><p>For cases were the dependent test has a similar but not identical itempath to the downstream test an itemmap can allow for itemmatch mode</p></div> <div class="listingblock"> <div class="content monospaced"> <pre>[requirements] | > | > | | > > > | > > > | < > > > > > > > > | | > > > > > > > > > > > > > > | > | | > | > | | > > | | > > | < < < < < < | < | < < < < < < < < < < < < < < < < | > | 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 | </div> <div class="sect2"> <h3 id="_itemmap_handling">Itemmap Handling</h3> <div class="paragraph"><p>For cases were the dependent test has a similar but not identical itempath to the downstream test an itemmap can allow for itemmatch mode</p></div> <div class="listingblock"> <div class="title">example for removing part of itemmap for waiton test (eg: item <span class="monospaced">foo-x/bar</span> depends on waiton’s item <span class="monospaced">y/bar</span>)</div> <div class="content monospaced"> <pre>[requirements] mode itemwait # itemmap <item pattern for this test> <item replacement pattern for waiton test> itemmap .*x/ y/</pre> </div></div> <div class="listingblock"> <div class="title">example for removing part of itemmap for waiton test (eg: item <span class="monospaced">foo/bar/baz</span> in this test depends on waiton’s item <span class="monospaced">baz</span>)</div> <div class="content monospaced"> <pre># ## pattern replacement notes # # ## Example # ## Remove everything up to the last / [requirements] mode itemwait # itemmap <item pattern for this test> <nothing here indicates removal> itemmap .*/</pre> </div></div> <div class="listingblock"> <div class="title">example replacing part of itemmap for (eg: item <span class="monospaced">foo/1234</span> will imply waiton’s item <span class="monospaced">bar/1234</span>)</div> <div class="content monospaced"> <pre># # ## Example # ## Replace foo/ with bar/ [requirements] mode itemwait # itemmap <item pattern for this test> <item replacement pattern for waiton test> itemmap foo/ bar/</pre> </div></div> <div class="listingblock"> <div class="title">example for backreference (eg: item <span class="monospaced">foo23/thud</span> will imply waiton’s item <span class="monospaced">num-23/bar/thud</span></div> <div class="content monospaced"> <pre># # ## Example # ## can use \{number} in replacement pattern to backreference a (capture) from matching pattern similar to sed or perl [requirements] mode itemwait # itemmap <item pattern for this test> <item replacement pattern for waiton test> itemmap foo(\d+)/ num-\1/bar/</pre> </div></div> <div class="listingblock"> <div class="title">example multiple itemmaps</div> <div class="content monospaced"> <pre># multi-line; matches are applied in the listed order # The following would map: # a123b321 to b321fooa123 then to 321fooa123p # [requirements] itemmap (a\d+)(b\d+) \2foo\1 b(.*) \1p</pre> </div></div> <div class="sect3"> <h4 id="_complex_mapping">Complex mapping</h4> <div class="paragraph"><p>Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section)</p></div> <div class="paragraph"><p>Each line in an itemmap section starts with a waiton test name followed by an itemmap expression</p></div> <div class="listingblock"> <div class="title">eg: The following causes waiton test A item <span class="monospaced">bar/1234</span> to run when our test’s <span class="monospaced">foo/1234</span> item is requested as well as causing waiton test B’s <span class="monospaced">blah</span> item to run when our test’s <span class="monospaced">stuff/blah</span> item is requested</div> <div class="content monospaced"> <pre>[itemmap] A foo/ bar/ B stuff/</pre> </div></div> </div> <div class="sect3"> <h4 id="_complex_mapping_example">Complex mapping example</h4> <div class="imageblock"> <div class="content"> <img src="complex-itemmap.png" alt="complex-itemmap.png"> </div> </div> <div class="paragraph"><p>We accomplish this by configuring the testconfigs of our tests C D and E as follows:</p></div> <div class="listingblock"> <div class="title">Testconfig for Test C</div> <div class="content monospaced"> <pre>[requirements] waiton A B [itemmap] |
︙ | ︙ | |||
1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 | <div class="listingblock"> <div class="title">Testconfig for Test E</div> <div class="content monospaced"> <pre>[requirements] waiton C itemmap (\d+)/res \1/bb</pre> </div></div> </div> <div class="sect3"> <h4 id="_dynamic_flow_dependency_tree">Dynamic Flow Dependency Tree</h4> <div class="listingblock"> <div class="title">Autogeneration waiton list for dynamic flow dependency trees</div> <div class="content monospaced"> <pre>[requirements] | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 | <div class="listingblock"> <div class="title">Testconfig for Test E</div> <div class="content monospaced"> <pre>[requirements] waiton C itemmap (\d+)/res \1/bb</pre> </div></div> <div class="paragraph"><p>Example from image just above, we want the following to occur:</p></div> <div class="olist arabic"><ol class="arabic"> <li> <p> We want the above to execute when we request pattern <span class="monospaced">D/1/res</span>, eg from command line <span class="monospaced">megatest -run -testpatt D/1/res -target … -runname …</span> </p> </li> <li> <p> "(\d+)/res" → "\1/aa" to require <span class="monospaced">C/1/aa</span> be executed before <span class="monospaced">D/1/res</span> </p> </li> <li> <p> Full list to be run is now: D/1/res, C/1/aa </p> </li> <li> <p> "(\d+)/aa" → "aa/\1" to create item <span class="monospaced">A/aa/1</span> </p> </li> <li> <p> Full list to be run is now: <span class="monospaced">D/1/res</span>, <span class="monospaced">C/1/aa</span>, <span class="monospaced">A/aa/1</span> </p> </li> </ol></div> </div> <div class="sect3"> <h4 id="_dynamic_flow_dependency_tree">Dynamic Flow Dependency Tree</h4> <div class="listingblock"> <div class="title">Autogeneration waiton list for dynamic flow dependency trees</div> <div class="content monospaced"> <pre>[requirements] |
︙ | ︙ | |||
2243 2244 2245 2246 2247 2248 2249 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.0<br> | | | 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 | </div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.0<br> Last updated 2017-07-17 14:05:11 PDT </div> </div> </body> </html> |
Modified docs/manual/reference.txt from [3fc5197c1a] to [39fa0839a8].
︙ | ︙ | |||
298 299 300 301 302 303 304 305 306 | Itemmap Handling ~~~~~~~~~~~~~~~~ For cases were the dependent test has a similar but not identical itempath to the downstream test an itemmap can allow for itemmatch mode ------------------- [requirements] | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > > > > > > | > > > < < < < < < > | 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 | Itemmap Handling ~~~~~~~~~~~~~~~~ For cases were the dependent test has a similar but not identical itempath to the downstream test an itemmap can allow for itemmatch mode .example for removing part of itemmap for waiton test (eg: item +foo-x/bar+ depends on waiton's item +y/bar+) ------------------- [requirements] mode itemwait # itemmap <item pattern for this test> <item replacement pattern for waiton test> itemmap .*x/ y/ ------------------- .example for removing part of itemmap for waiton test (eg: item +foo/bar/baz+ in this test depends on waiton's item +baz+) ------------------- # ## pattern replacement notes # # ## Example # ## Remove everything up to the last / [requirements] mode itemwait # itemmap <item pattern for this test> <nothing here indicates removal> itemmap .*/ ------------------- .example replacing part of itemmap for (eg: item +foo/1234+ will imply waiton's item +bar/1234+) ------------------- # # ## Example # ## Replace foo/ with bar/ [requirements] mode itemwait # itemmap <item pattern for this test> <item replacement pattern for waiton test> itemmap foo/ bar/ ------------------- .example for backreference (eg: item +foo23/thud+ will imply waiton's item +num-23/bar/thud+ ------------------- # # ## Example # ## can use \{number} in replacement pattern to backreference a (capture) from matching pattern similar to sed or perl [requirements] mode itemwait # itemmap <item pattern for this test> <item replacement pattern for waiton test> itemmap foo(\d+)/ num-\1/bar/ ------------------- .example multiple itemmaps ------------------- # multi-line; matches are applied in the listed order # The following would map: # a123b321 to b321fooa123 then to 321fooa123p # [requirements] itemmap (a\d+)(b\d+) \2foo\1 b(.*) \1p ------------------- Complex mapping ^^^^^^^^^^^^^^^ Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section) Each line in an itemmap section starts with a waiton test name followed by an itemmap expression .eg: The following causes waiton test A item +bar/1234+ to run when our test's +foo/1234+ item is requested as well as causing waiton test B's +blah+ item to run when our test's +stuff/blah+ item is requested -------------- [itemmap] A foo/ bar/ B stuff/ -------------- Complex mapping example ^^^^^^^^^^^^^^^^^^^^^^^ // image::itemmap.png[] image::complex-itemmap.png[] We accomplish this by configuring the testconfigs of our tests C D and E as follows: .Testconfig for Test C ---------------------- [requirements] waiton A B [itemmap] |
︙ | ︙ | |||
360 361 362 363 364 365 366 367 368 369 370 371 372 373 | .Testconfig for Test E ---------------------- [requirements] waiton C itemmap (\d+)/res \1/bb ---------------------- Dynamic Flow Dependency Tree ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .Autogeneration waiton list for dynamic flow dependency trees ------------------- [requirements] | > > > > > > > > > | 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 | .Testconfig for Test E ---------------------- [requirements] waiton C itemmap (\d+)/res \1/bb ---------------------- Example from image just above, we want the following to occur: . We want the above to execute when we request pattern +D/1/res+, eg from command line +megatest -run -testpatt D/1/res -target ... -runname ...+ . "(\d+)/res" -> "\1/aa" to require +C/1/aa+ be executed before +D/1/res+ . Full list to be run is now: D/1/res, C/1/aa . "(\d+)/aa" -> "aa/\1" to create item +A/aa/1+ . Full list to be run is now: +D/1/res+, +C/1/aa+, +A/aa/1+ Dynamic Flow Dependency Tree ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .Autogeneration waiton list for dynamic flow dependency trees ------------------- [requirements] |
︙ | ︙ |
Modified launch.scm from [13e1eee06d] to [ab11d5875b].
︙ | ︙ | |||
1531 1532 1533 1534 1535 1536 1537 | (list "MT_ITEMPATH" item-path) ) itemdat))) (testprevvals (alist->env-vars (hash-table-ref/default tconfig "pre-launch-env-overrides" '()))) ;; Launchwait defaults to true, must override it to turn off wait (launchwait (if (equal? (configf:lookup *configdat* "setup" "launchwait") "no") #f #t)) | | | 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 | (list "MT_ITEMPATH" item-path) ) itemdat))) (testprevvals (alist->env-vars (hash-table-ref/default tconfig "pre-launch-env-overrides" '()))) ;; Launchwait defaults to true, must override it to turn off wait (launchwait (if (equal? (configf:lookup *configdat* "setup" "launchwait") "no") #f #t)) (launch-results (apply (if launchwait ;; BB: TODO: refactor this to examine return code of launcher, if nonzero, set state to launch failed. process:cmd-run-with-stderr->list process-run) (if useshell (let ((cmdstr (string-intersperse fullcmd " "))) (if launchwait cmdstr (conc cmdstr " >> mt_launch.log 2>&1 &"))) |
︙ | ︙ |
Modified runs.scm from [dcb3d16ebc] to [b0c63a44c5].
︙ | ︙ | |||
308 309 310 311 312 313 314 | (debug:print 0 *default-log-port* "ERROR: failed to run post-hook " run-post-hook ", check the log " log-file)) (debug:print-info 0 *default-log-port* "running run-post-hook: \"" run-post-hook "\", log is " actual-logf) (system (conc run-post-hook " >> " actual-logf " 2>&1")) (debug:print-info 0 *default-log-port* "post-hook \"" run-post-hook "\" took " (- (current-seconds) start-time) " seconds to run.")))))) ;; return #t when all items in waitors-upon list are represented in test-patt, #f otherwise. (define (runs:testpatts-mention-waitors-upon? test-patt waitors-upon) | < < < < < < | | 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 | (debug:print 0 *default-log-port* "ERROR: failed to run post-hook " run-post-hook ", check the log " log-file)) (debug:print-info 0 *default-log-port* "running run-post-hook: \"" run-post-hook "\", log is " actual-logf) (system (conc run-post-hook " >> " actual-logf " 2>&1")) (debug:print-info 0 *default-log-port* "post-hook \"" run-post-hook "\" took " (- (current-seconds) start-time) " seconds to run.")))))) ;; return #t when all items in waitors-upon list are represented in test-patt, #f otherwise. (define (runs:testpatts-mention-waitors-upon? test-patt waitors-upon) (null? (tests:filter-test-names-not-matched waitors-upon test-patt))) ;; test-names: Comma separated patterns same as test-patts but used in selection ;; of tests to run. The item portions are not respected. ;; FIXME: error out if /patt specified ;; (define (runs:run-tests target runname test-patts user flags #!key (run-count 1)) ;; test-names (let* ((keys (keys:config-get-fields *configdat*)) |
︙ | ︙ | |||
424 425 426 427 428 429 430 431 | ;; NEW STRATEGY HERE: ;; 1. fill required tests with test-patts ;; 2. scan testconfigs and if waitons, itemwait, itempatt calc prior test test-patt ;; 3. repeat until all deps propagated ;; any tests with direct mention in test-patts can be added to required ;; | > > < | 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 | ;; NEW STRATEGY HERE: ;; 1. fill required tests with test-patts ;; 2. scan testconfigs and if waitons, itemwait, itempatt calc prior test test-patt ;; 3. repeat until all deps propagated ;; any tests with direct mention in test-patts can be added to required ;;(set! required-tests (lset-intersection equal? (string-split test-patts ",") all-test-names)) (set! required-tests (tests:filter-test-names all-test-names test-patts)) ;; ;; (set! required-tests (lset-intersection equal? test-names all-test-names)) ;; look up all tests matching the comma separated list of globs in ;; test-patts (using % as wildcard) ;; (set! test-names (delete-duplicates (tests:get-valid-tests *toppath* test-patts))) (debug:print-info 0 *default-log-port* "tests search path: " (string-intersperse (tests:get-tests-search-path *configdat*) " ")) |
︙ | ︙ | |||
483 484 485 486 487 488 489 | (if (not (null? test-names)) ;; BEGIN test-names loop (let loop ((hed (car test-names)) ;; NOTE: This is the main loop that iterates over the test-names (tal (cdr test-names))) ;; 'return-procs tells the config reader to prep running system but return a proc (debug:print-info 4 *default-log-port* "\n\ntestpatt elaboration loop => hed="hed " tal="tal" test-patts="test-patts" test-names="test-names) (change-directory *toppath*) ;; PLEASE OPTIMIZE ME!!! I think this should be a no-op but there are several places where change-directories could be happening. (setenv "MT_TEST_NAME" hed) ;; | | | > > > > > | 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 | (if (not (null? test-names)) ;; BEGIN test-names loop (let loop ((hed (car test-names)) ;; NOTE: This is the main loop that iterates over the test-names (tal (cdr test-names))) ;; 'return-procs tells the config reader to prep running system but return a proc (debug:print-info 4 *default-log-port* "\n\ntestpatt elaboration loop => hed="hed " tal="tal" test-patts="test-patts" test-names="test-names) (change-directory *toppath*) ;; PLEASE OPTIMIZE ME!!! I think this should be a no-op but there are several places where change-directories could be happening. (setenv "MT_TEST_NAME" hed) ;; (let*-values (((waitons waitors config) (tests:get-waitons hed all-tests-registry)) ((hed-mode) (let ((m (config-lookup config "requirements" "mode"))) (if m (map string->symbol (string-split m)) '(normal)))) ((hed-itemized-waiton) ;; are items in hed waiting on items of waiton? (not (null? (lset-intersection eq? hed-mode '(itemmatch itemwait))))) ) (debug:print-info 8 *default-log-port* "waitons: " waitons) ;; check for hed in waitons => this would be circular, remove it and issue an ;; error (if (or (member hed waitons) (member hed waitors)) (begin (debug:print-error 0 *default-log-port* "test " hed " has listed itself as a waiton or waitor, please correct this!") |
︙ | ︙ | |||
527 528 529 530 531 532 533 | (let* ((waitors-in-testpatt (runs:testpatts-mention-waitors-upon? test-patts (hash-table-ref/default waitors-upon waiton '()))) (waiton-record (hash-table-ref/default test-records waiton #f)) (waiton-tconfig (if waiton-record (vector-ref waiton-record 1) #f)) (waiton-itemized (and waiton-tconfig (or (hash-table-ref/default waiton-tconfig "items" #f) (hash-table-ref/default waiton-tconfig "itemstable" #f)))) (itemmaps (tests:get-itemmaps config)) ;; (configf:lookup config "requirements" "itemmap")) | | | 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 | (let* ((waitors-in-testpatt (runs:testpatts-mention-waitors-upon? test-patts (hash-table-ref/default waitors-upon waiton '()))) (waiton-record (hash-table-ref/default test-records waiton #f)) (waiton-tconfig (if waiton-record (vector-ref waiton-record 1) #f)) (waiton-itemized (and waiton-tconfig (or (hash-table-ref/default waiton-tconfig "items" #f) (hash-table-ref/default waiton-tconfig "itemstable" #f)))) (itemmaps (tests:get-itemmaps config)) ;; (configf:lookup config "requirements" "itemmap")) (new-test-patts (tests:extend-test-patts test-patts hed waiton itemmaps hed-itemized-waiton))) (debug:print-info 0 *default-log-port* "Test " waiton " has " (if waiton-record "a" "no") " waiton-record and" (if waiton-itemized " " " no ") "items") ;; need to account for test-patt here, if I am test "a", selected with a test-patt of "hed/b%" ;; and we are waiting on "waiton" we need to add "waiton/,waiton/b%" to test-patt ;; is this satisfied by merely appending "/" to the waiton name added to the list? ;; ;; This approach causes all of the items in an upstream test to be run |
︙ | ︙ | |||
1486 1487 1488 1489 1490 1491 1492 | " ") "\n")) items))) (let* ((items-in-testpatt (filter (lambda (my-itemdat) | | > | 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 | " ") "\n")) items))) (let* ((items-in-testpatt (filter (lambda (my-itemdat) (tests:match test-patts hed (item-list->path my-itemdat) )) ;; was: (tests:match test-patts hed (item-list->path my-itemdat) required: required-tests)) items) )) (if (null? items-in-testpatt) (let ((test-id (rmt:get-test-id run-id test-name ""))) (debug:print-info 0 *default-log-port* "Test " (tests:testqueue-get-testname test-record) " is itemized but has no items matching test pattern -- marking status ZERO_ITEMS") (if test-id (mt:test-set-state-status-by-id run-id test-id "NOT_STARTED" "ZERO_ITEMS" "This test has no items which match test pattern."))) |
︙ | ︙ |
Modified tests.scm from [333db4fd76] to [936f9af395].
︙ | ︙ | |||
72 73 74 75 76 77 78 79 80 81 82 83 84 85 | (if (and (not (hash-table-ref/default test-registry tname #f)) (common:file-exists? tconfig)) (hash-table-set! test-registry tname test-path)))) (glob (conc hed "/*")))) (if (null? tal) test-registry (loop (car tal)(cdr tal)))))) (define (tests:filter-test-names test-names test-patts) (delete-duplicates (filter (lambda (testname) (tests:match test-patts testname #f)) test-names))) | > > > > > > > | 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | (if (and (not (hash-table-ref/default test-registry tname #f)) (common:file-exists? tconfig)) (hash-table-set! test-registry tname test-path)))) (glob (conc hed "/*")))) (if (null? tal) test-registry (loop (car tal)(cdr tal)))))) (define (tests:filter-test-names-not-matched test-names test-patts) (delete-duplicates (filter (lambda (testname) (not (tests:match test-patts testname #f))) test-names))) (define (tests:filter-test-names test-names test-patts) (delete-duplicates (filter (lambda (testname) (tests:match test-patts testname #f)) test-names))) |
︙ | ︙ | |||
205 206 207 208 209 210 211 | ;; ;; [requirements] [requirements] ;; mode itemwait ;; # trim off the cell to determine what to run for genlib ;; itemmap /.* ;; ;; waiting-test is waiting on waiton-test so we need to create a pattern for waiton-test given waiting-test and itemmap | > > > > > > > > | > > | | | | | | | | | | | | | | | | | | | | | | | | | | | > > > > > > | 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 | ;; ;; [requirements] [requirements] ;; mode itemwait ;; # trim off the cell to determine what to run for genlib ;; itemmap /.* ;; ;; waiting-test is waiting on waiton-test so we need to create a pattern for waiton-test given waiting-test and itemmap ;; BB> (tests:extend-test-patts "normal-second/2" "normal-second" "normal-first" '()) ;; observed -> "normal-first/2,normal-first/,normal-second/2,normal-second/" ;; expected -> "normal-first,normal-second/2,normal-second/" ;; testpatt = normal-second/2 ;; waiting-test = normal-second ;; waiton-test = normal-first ;; itemmaps = () (define (tests:extend-test-patts test-patt waiting-test waiton-test itemmaps itemized-waiton) (cond (itemized-waiton (let* ((itemmap (tests:lookup-itemmap itemmaps waiton-test)) (patts (string-split test-patt ",")) (waiting-test-len (+ (string-length waiting-test) 1)) (patts-waiton (map (lambda (x) ;; for each incoming patt that matches the waiting test (let* ((modpatt (if itemmap (db:convert-test-itempath x itemmap) x)) (newpatt (conc waiton-test "/" (substring modpatt waiting-test-len (string-length modpatt))))) ;; (conc waiting-test "/," waiting-test "/" (substring modpatt waiton-test-len (string-length modpatt))))) ;; (print "in map, x=" x ", newpatt=" newpatt) newpatt)) (filter (lambda (x) (eq? (substring-index (conc waiting-test "/") x) 0)) ;; is this patt pertinent to the waiting test patts))) (extended-test-patt (append patts (if (null? patts-waiton) (list (conc waiton-test "/%")) ;; really shouldn't add the waiton forcefully like this patts-waiton))) (extended-test-patt-with-toplevels (fold (lambda (testpatt-item accum ) (let ((my-match (string-match "^([^%\\/]+)\\/.+$" testpatt-item))) (cons testpatt-item (if my-match (cons (conc (cadr my-match) "/") accum) accum)))) '() extended-test-patt))) (string-intersperse (delete-duplicates extended-test-patt-with-toplevels) ","))) (else ;; not waiting on items, waiting on entire waiton test. (let* ((patts (string-split test-patt ",")) (new-patts (if (member waiton-test patts) patts (cons waiton-test patts)))) (string-intersperse (delete-duplicates new-patts) ","))))) ;; tests:glob-like-match (define (tests:glob-like-match patt str) (let ((like (substring-index "%" patt))) (let* ((notpatt (equal? (substring-index "~" patt) 0)) |
︙ | ︙ |
Modified utils/installall.sh from [4b2e44cf40] to [3528934c37].
︙ | ︙ | |||
20 21 22 23 24 25 26 | echo You may need to do the following first: echo sudo apt install libreadline-dev echo sudo apt install libwebkitgtk-dev echo sudo apt install libpangox-1.0-0 zlib1g-dev libfreetype6-dev cmake echo sudo apt install libssl-dev uuid-dev echo sudo apt install libmotif3 -OR- set KTYPE=26g4 | | > > > > > > > > | | | | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 | echo You may need to do the following first: echo sudo apt install libreadline-dev echo sudo apt install libwebkitgtk-dev echo sudo apt install libpangox-1.0-0 zlib1g-dev libfreetype6-dev cmake echo sudo apt install libssl-dev uuid-dev echo sudo apt install libmotif3 -OR- set KTYPE=26g4 echo sudo apt install cmake curl ruby wget echo echo Set OPTION to std, currently OPTION=$OPTION echo echo Additionally, if you want mysql-client, you will need to make sure echo mysql_config is in your path echo for postgres to install dbi libpq-dev echo echo You are using PREFIX=$PREFIX echo You are using proxy="$proxy" echo echo "Set additional_libpath to help find gtk or other libraries, don't forget a leading :" SYSTEM_TYPE=$(lsb_release -irs |tr ' ' '_' |tr '\n' '-')$(uname -i)-$OPTION CHICKEN_VERSION=4.10.0 CHICKEN_BASEVER=4.10.0 # Set up variables # case $SYSTEM_TYPE in Ubuntu-17.10-x86_64-std) KTYPE=32 CDVER=5.11.1 IUPVER=3.22 IMVER=3.12 # CHICKEN_VERSION=4.12.0 # CHICKEN_BASEVER=4.12.0 ;; Ubuntu-17.04-x86_64-std) KTYPE=32 CDVER=5.11.1 IUPVER=3.22 IMVER=3.12 # CHICKEN_VERSION=4.12.0 # CHICKEN_BASEVER=4.12.0 ;; Ubuntu-16.04-x86_64-std) KTYPE=32 CDVER=5.11.1 IUPVER=3.22 IMVER=3.12 # CHICKEN_VERSION=4.12.0 # CHICKEN_BASEVER=4.12.0 ;; Ubuntu-16.04-i686-std) KTYPE=32 CDVER=5.11.1 IUPVER=3.22 IMVER=3.12 # CHICKEN_VERSION=4.12.0 # CHICKEN_BASEVER=4.12.0 ;; SUSE_LINUX_11-x86_64-std) KTYPE=26g4 CDVER=5.11.1 IUPVER=3.22 IMVER=3.12 ;; CentOS_5.11-x86_64-std) KTYPE=24g3 CDVER=5.4.1 IUPVER=3.5 IMVER=3.6.3 ;; esac echo SYSTEM_TYPE=$SYSTEM_TYPE echo KTYPE=$KTYPE echo CDVER=$CDVER echo IUPVER=$IUPVER echo IMVER=$IMVER echo CHICKEN_VERSION=$CHICKEN_VERSION echo CHICKEN_BASEVER=$CHICKEN_BASEVER # NOTES: # # Centos with security setup may need to do commands such as following as root: # # NB// fix the paths first # # for a in /localdisk/chicken/4.8.0/lib/*.so;do chcon -t textrel_shlib_t $a; done echo ADDITIONAL_LIBPATH=$ADDITIONAL_LIBPATH echo echo To use previous IUP libraries set USEOLDIUP to yes echo USEOLDIUP=$USEOLDIUP echo echo Hit ^C now to do that # A nice way to run this script: # # script -c 'PREFIX=/tmp/delme ./installall.sh ' installall.log # logpro installall.logpro installall.html < installall.log # firefox installall.html sleep 5 if [[ $proxy == "" ]]; then echo 'Please set the environment variable "proxy" to host.com:port (e.g. foo.com:1234) to use a proxy' echo PROX="" else export http_proxy=http://$proxy export https_proxy=http://$proxy export PROX="-proxy $proxy" fi if [[ $KTYPE == "" ]]; then echo 'Using KTYPE=26' export KTYPE=26g4 else echo Using KTYPE=$KTYPE fi # Put all the downloaded tar files in tgz mkdir -p tgz # http://code.call-cc.org/releases/4.8.0/chicken-4.8.0.5.tar.gz chicken_targz=chicken-${CHICKEN_VERSION}.tar.gz if ! [[ -e tgz/$chicken_targz ]]; then wget http://code.call-cc.org/releases/${CHICKEN_BASEVER}/${chicken_targz} mv $chicken_targz tgz fi BUILDHOME=$PWD DEPLOYTARG=$BUILDHOME/deploy if [[ $PREFIX == "" ]]; then PREFIX=$PWD/inst fi export PATH=$PREFIX/bin:$PATH export LIBPATH=$PREFIX/lib:$PREFIX/lib64:$ADDITIONAL_LIBPATH export LD_LIBRARY_PATH=$LIBPATH export CHICKEN_INSTALL=$PREFIX/bin/chicken-install mkdir -p $PREFIX echo "export PATH=$PREFIX/bin:\$PATH" > $PREFIX/setup-chicken4x.sh echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:\$LD_LIBRARY_PATH" >> $PREFIX/setup-chicken4x.sh echo "export CHICKEN_DOC_PAGER=cat" >> $PREFIX/setup-chicken4x.sh echo "setenv PATH $PREFIX/bin:\$PATH" > $PREFIX/setup-chicken4x.csh echo "setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:\$LD_LIBRARY_PATH" >> $PREFIX/setup-chicken4x.csh echo "setenv CHICKEN_DOC_PAGER cat" >> $PREFIX/setup-chicken4x.csh echo PATH=$PATH |
︙ | ︙ | |||
156 157 158 159 160 161 162 | # Some eggs are quoted since they are reserved to Bash # for f in matchable readline apropos base64 regex-literals format "regex-case" "test" coops trace csv dot-locking posix-utils posix-extras directory-utils hostinfo tcp rpc csv-xml fmt json md5; do # $CHICKEN_INSTALL $PROX -keep-installed matchable readline apropos base64 regex-literals format "regex-case" "test" coops trace csv dot-locking posix-utils posix-extras directory-utils hostinfo tcp rpc csv-xml fmt json md5 awful http-client spiffy uri-common intarweb http-client spiffy-request-vars md5 message-digest http-client spiffy-directory-listing for egg in matchable readline apropos base64 regex-literals format "regex-case" "test" \ coops trace csv dot-locking posix-utils posix-extras directory-utils hostinfo \ tcp rpc csv-xml fmt json md5 awful http-client:0.7.1 spiffy uri-common intarweb http-client \ spiffy-request-vars s md5 message-digest spiffy-directory-listing ssax sxml-serializer \ | | | > > | 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 | # Some eggs are quoted since they are reserved to Bash # for f in matchable readline apropos base64 regex-literals format "regex-case" "test" coops trace csv dot-locking posix-utils posix-extras directory-utils hostinfo tcp rpc csv-xml fmt json md5; do # $CHICKEN_INSTALL $PROX -keep-installed matchable readline apropos base64 regex-literals format "regex-case" "test" coops trace csv dot-locking posix-utils posix-extras directory-utils hostinfo tcp rpc csv-xml fmt json md5 awful http-client spiffy uri-common intarweb http-client spiffy-request-vars md5 message-digest http-client spiffy-directory-listing for egg in matchable readline apropos base64 regex-literals format "regex-case" "test" \ coops trace csv dot-locking posix-utils posix-extras directory-utils hostinfo \ tcp rpc csv-xml fmt json md5 awful http-client:0.7.1 spiffy uri-common intarweb http-client \ spiffy-request-vars s md5 message-digest spiffy-directory-listing ssax sxml-serializer \ sxml-modifications z3 call-with-environment-variables \ pathname-expand typed-records \ logpro \ simple-exceptions numbers crypt parley srfi-42 \ alist-lib ansi-escape-sequences args basic-sequences bindings chicken-doc chicken-doc-cmd \ cock condition-utils debug define-record-and-printer easyffi easyffi-base \ expand-full ezxdisp filepath foof-loop ini-file irc lalr lazy-seq \ locale locale-builtin locale-categories locale-components locale-current locale-posix \ locale-timezone loops low-level-macros procedural-macros refdb rfc3339 scsh-process \ sexp-diff sha1 shell slice srfi-101 srfi-19 srfi-19-core srfi-19-date srfi-19-io \ srfi-19-period srfi-19-support srfi-19-time srfi-19-timezone srfi-29 srfi-37 srfi-78 syslog \ |
︙ | ︙ | |||
258 259 260 261 262 263 264 | cd ../margs $PREFIX/bin/chicken-install cd ../pkts $PREFIX/bin/chicken-install fi cd $BUILDHOME | | | 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 | cd ../margs $PREFIX/bin/chicken-install cd ../pkts $PREFIX/bin/chicken-install fi cd $BUILDHOME if [[ ! -e $PREFIX/bin/stmlrun ]] ; then #fossil clone http://www.kiatoa.com/fossils/stml stml.fossil wget -c -O stml.tar.gz 'http://www.kiatoa.com/fossils/stml/tarball?name=stml&uuid=trunk' tar -xzf stml.tar.gz cd stml #fossil open ../stml.fossil cp install.cfg.template install.cfg echo "TARGDIR=$PREFIX/bin" > install.cfg |
︙ | ︙ |