Overview
Comment: | Fixed blocking bug (I think). Minor additions to new user manual |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | blocking-on-FAIL-fix |
Files: | files | file ages | folders |
SHA1: |
68f98cd6bb6088983fd88a386b6de12a |
User & Date: | mrwellan on 2012-05-03 13:41:38 |
Other Links: | branch diff | manifest | tags |
Context
2012-05-03
| ||
13:52 | Merging blocking fix to trunk check-in: 41ef747b91 user: mrwellan tags: trunk | |
13:41 | Fixed blocking bug (I think). Minor additions to new user manual Closed-Leaf check-in: 68f98cd6bb user: mrwellan tags: blocking-on-FAIL-fix | |
01:07 | Added code needed to fix blocking problem but problem not quite solved check-in: 6d817c8583 user: matt tags: blocking-on-FAIL-fix | |
Changes
Modified docs/manual/getting_started.txt from [aef24f3018] to [cdb37e57c9].
1 |
| | | | | | | | | > > > > > | < | < < | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | Getting Started =============== [partintro] .Getting started with Megatest -- How to install Megatest and set it up for running your regressions and continuous integration process. -- Installation ------------ Dependencies ~~~~~~~~~~~~ Chicken scheme and a number of "eggs" are required for building Megatest. See the script installall.sch in the utils directory of the distribution for a mostly automated way to install everything needed for building Megatest on Linux. footnote:[An example footnote.] indexterm:[Example index entry] And now for something completely different: ((monkeys)), lions and tigers (Bengal and Siberian) using the alternative syntax index entries. (((Big cats,Lions))) (((Big cats,Tigers,Bengal Tiger))) (((Big cats,Tigers,Siberian Tiger))) |
︙ | ︙ |
Modified docs/manual/megatest_manual.html from [6d2ce20778] to [6c30402d62].
︙ | ︙ | |||
753 754 755 756 757 758 759 | <div class="sect1"> <h2 id="_thanks">Thanks</h2> <div class="sectionbody"> <div class="paragraph"><p>To all the great people from the various places I’ve worked over the years, thank you. I’ve benefitted from your patience and insight and I hope that some of what I’ve learned from you has been incorporated into Megatest.</p></div> | | | | | > > > > > > > > | < > | > > > > > > > > | < < < < > > > > > > > | < | | | | | > > > > | < < | < < < | 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 | <div class="sect1"> <h2 id="_thanks">Thanks</h2> <div class="sectionbody"> <div class="paragraph"><p>To all the great people from the various places I’ve worked over the years, thank you. I’ve benefitted from your patience and insight and I hope that some of what I’ve learned from you has been incorporated into Megatest.</p></div> <div class="paragraph"><p>Also, my thanks to all the wonderful open source creators. These truly free projects, from Linux, xemacs, chicken scheme, and fossil to asciidoc and bash are the philosophical and direct underpinnings of the Megatest project.</p></div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_preface">Preface</h2> <div class="sectionbody"> <div class="paragraph"><p>This book is organised as three sub-books; getting started, writing tests and reference.</p></div> <div class="sect2"> <h3 id="_why_megatest">Why Megatest?</h3> <div class="paragraph"><p>The Megatest project was started for two reasons, the first was an immediate and pressing need for a generalized tool to manage a suite of regression tests and the second was the fact that I had written or maintained several such tools at different companies over the years and it seemed a good thing to have a single open source tool, flexible enough to meet the needs of any team doing continuous integrating and or running a complex suite of tests for release qualification.</p></div> </div> <div class="sect2"> <h3 id="_megatest_design_philosophy">Megatest Design Philosophy</h3> <div class="paragraph"><p>Megatest is intended to provide the minimum needed resources to make writing a suite of tests and implementing continuous build for software, design engineering or process control (via owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome.</p></div> </div> <div class="sect2"> <h3 id="_megatest_architecture">Megatest Architecture</h3> <div class="paragraph"><p>All data to specify the tests and configure the system is stored in plain text files. All system state is stored in an sqlite3 database. Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database.</p></div> </div> </div> </div> <h1 id="_getting_started">Getting Started</h1> <div class="openblock"> <div class="title">Getting started with Megatest</div> <div class="content"> <div class="paragraph"><p>How to install Megatest and set it up for running your regressions and continuous integration process.</p></div> </div></div> <div class="sect1"> <h2 id="_installation">Installation</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_dependencies">Dependencies</h3> <div class="paragraph"><p>Chicken scheme and a number of "eggs" are required for building Megatest. See the script installall.sch in the utils directory of the distribution for a mostly automated way to install everything needed for building Megatest on Linux.</p></div> <div class="paragraph"><p><span class="footnote"><br />[An example footnote.]<br /></span></p></div> <div class="paragraph"><p>And now for something completely different: monkeys, lions and tigers (Bengal and Siberian) using the alternative syntax index entries. Note that multi-entry terms generate separate index entries.</p></div> |
︙ | ︙ | |||
846 847 848 849 850 851 852 853 854 855 856 857 858 859 | </table> </div> <div class="exampleblock"> <div class="title">Example 1. An example example</div> <div class="content"> <div class="paragraph"><p>Lorum ipum…</p></div> </div></div> <div class="sect2"> <h3 id="X1">Sub-section with Anchor</h3> <div class="paragraph"><p>Sub-section at level 2.</p></div> <div class="sect3"> <h4 id="_chapter_sub_section">Chapter Sub-section</h4> <div class="paragraph"><p>Sub-section at level 3.</p></div> <div class="sect4"> | > | 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 | </table> </div> <div class="exampleblock"> <div class="title">Example 1. An example example</div> <div class="content"> <div class="paragraph"><p>Lorum ipum…</p></div> </div></div> </div> <div class="sect2"> <h3 id="X1">Sub-section with Anchor</h3> <div class="paragraph"><p>Sub-section at level 2.</p></div> <div class="sect3"> <h4 id="_chapter_sub_section">Chapter Sub-section</h4> <div class="paragraph"><p>Sub-section at level 3.</p></div> <div class="sect4"> |
︙ | ︙ | |||
869 870 871 872 873 874 875 | </div> <div class="sect1"> <h2 id="_the_second_chapter">The Second Chapter</h2> <div class="sectionbody"> <div class="paragraph"><p>An example link to anchor at start of the <a href="#X1">first sub-section</a>.</p></div> <div class="paragraph"><p>An example link to a bibliography entry <a href="#taoup">[taoup]</a>.</p></div> </div> | < < < < < < < < | | 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 | </div> <div class="sect1"> <h2 id="_the_second_chapter">The Second Chapter</h2> <div class="sectionbody"> <div class="paragraph"><p>An example link to anchor at start of the <a href="#X1">first sub-section</a>.</p></div> <div class="paragraph"><p>An example link to a bibliography entry <a href="#taoup">[taoup]</a>.</p></div> </div> </div> <h1 id="_reference">Reference</h1> <div class="sect1"> <h2 id="_the_first_chapter_of_the_second_part">The First Chapter of the Second Part</h2> <div class="sectionbody"> <div class="paragraph"><p>Chapters grouped into book parts are at level 1 and can contain sub-sections.</p></div> </div> </div> <div class="sect1"> <h2 id="_the_testconfig_file">The testconfig File</h2> |
︙ | ︙ | |||
992 993 994 995 996 997 998 | </div> </div> </div> <div id="footnotes"><hr /></div> <div id="footer"> <div id="footer-text"> Version 1.0<br /> | | | 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 | </div> </div> </div> <div id="footnotes"><hr /></div> <div id="footer"> <div id="footer-text"> Version 1.0<br /> Last updated 2012-05-03 07:53:32 MST </div> </div> </body> </html> |
Modified docs/manual/megatest_manual.txt from [a336d0d8c0] to [de94f0104e].
︙ | ︙ | |||
17 18 19 20 21 22 23 | ------ To all the great people from the various places I've worked over the years, thank you. I've benefitted from your patience and insight and I hope that some of what I've learned from you has been incorporated into Megatest. | | | | > | > > > > > > > > > | > | | > > > > > > > > | < > > > > > > > > > | < | 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ------ To all the great people from the various places I've worked over the years, thank you. I've benefitted from your patience and insight and I hope that some of what I've learned from you has been incorporated into Megatest. Also, my thanks to all the wonderful open source creators. These truly free projects, from Linux, xemacs, chicken scheme, and fossil to asciidoc and bash are the philosophical and direct underpinnings of the Megatest project. [preface] Preface ======= This book is organised as three sub-books; getting started, writing tests and reference. Why Megatest? ~~~~~~~~~~~~~ The Megatest project was started for two reasons, the first was an immediate and pressing need for a generalized tool to manage a suite of regression tests and the second was the fact that I had written or maintained several such tools at different companies over the years and it seemed a good thing to have a single open source tool, flexible enough to meet the needs of any team doing continuous integrating and or running a complex suite of tests for release qualification. Megatest Design Philosophy ~~~~~~~~~~~~~~~~~~~~~~~~~~ Megatest is intended to provide the minimum needed resources to make writing a suite of tests and implementing continuous build for software, design engineering or process control (via owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome. Megatest Architecture ~~~~~~~~~~~~~~~~~~~~~ All data to specify the tests and configure the system is stored in plain text files. All system state is stored in an sqlite3 database. Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master sqlite3 database. include::getting_started.txt[] include::writing_tests.txt[] include::reference.txt[] [appendix] Example Appendix ================ One or more optional appendixes go here at section level zero. |
︙ | ︙ |
Modified runs.scm from [b407faba37] to [6f1beb94af].
︙ | ︙ | |||
278 279 280 281 282 283 284 | (let* ((test-record (hash-table-ref test-records hed)) (tconfig (tests:testqueue-get-testconfig test-record)) (waitons (tests:testqueue-get-waitons test-record)) (priority (tests:testqueue-get-priority test-record)) (itemdat (tests:testqueue-get-itemdat test-record)) ;; itemdat can be a string, list or #f (items (tests:testqueue-get-items test-record)) (item-path (item-list->path itemdat)) | | > > > > > > > | | | > < < < < < | | 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 | (let* ((test-record (hash-table-ref test-records hed)) (tconfig (tests:testqueue-get-testconfig test-record)) (waitons (tests:testqueue-get-waitons test-record)) (priority (tests:testqueue-get-priority test-record)) (itemdat (tests:testqueue-get-itemdat test-record)) ;; itemdat can be a string, list or #f (items (tests:testqueue-get-items test-record)) (item-path (item-list->path itemdat)) (newtal (append tal (list hed))) (calc-fails (lambda (prereqs-not-met) (filter (lambda (test) (debug:print 9 "test: " test) (and (vector? test) ;; not (string? test)) (not (member (db:test-get-status test) '("PASS" "WARN" "CHECK" "WAIVED"))))) prereqs-not-met)))) (debug:print 6 "itemdat: " itemdat "\n items: " items "\n item-path: " item-path) (cond ((not items) ;; when false the test is ok to be handed off to launch (but not before) (let* ((have-resources (runs:can-run-more-tests db test-record)) ;; look at the test jobgroup and tot jobs running (prereqs-not-met (db:get-prereqs-not-met db run-id waitons item-path)) (fails (calc-fails prereqs-not-met))) (debug:print 8 "INFO: have-resources: " have-resources " prereqs-not-met: " prereqs-not-met " fails: " fails) ;; Don't know at this time if the test have been launched at some time in the past ;; i.e. is this a re-launch? (cond ((and have-resources (null? prereqs-not-met)) ;; no loop here, just drop though and use the loop at the bottom (if (patt-list-match item-path item-patts) (run:test db run-id runname keyvallst test-record flags #f) (debug:print 1 "INFO: Skipping " (tests:testqueue-get-testname test-record) " " item-path " as it doesn't match " item-patts)) ;; else the run is stuck, temporarily or permanently ;; but should check if it is due to lack of resources vs. prerequisites ) ((not have-resources) ;; simply try again after waiting a second (thread-sleep! 1.0) (debug:print 1 "INFO: no resources to run new tests, waiting ...") ;; could have done hed tal here but doing car/cdr of newtal to rotate tests (loop (car newtal)(cdr newtal))) (else ;; must be we have unmet prerequisites (debug:print 4 "FAILS: " fails) ;; If one or more of the prereqs-not-met are FAIL then we can issue ;; a message and drop hed from the items to be processed. (if (null? fails) (begin ;; couldn't run, take a breather (debug:print 4 "INFO: Shouldn't really get here, race condition? Unable to launch more tests at this moment, killing time ...") (thread-sleep! 0.1) ;; long sleep here - no resources, may as well be patient ;; we made new tal by sticking hed at the back of the list (loop (car newtal)(cdr newtal))) ;; the waiton is FAIL so no point in trying to run hed ever again (if (not (null? tal)) (begin (debug:print 1 "WARN: Dropping test " (db:test-get-test-name hed) "/" (db:test-get-item-path hed) " from the launch list as it has prerequistes that are FAIL") (loop (car tal)(cdr tal))))))))) ;; case where an items came in as a list been processed ((and (list? items) ;; thus we know our items are already calculated (not itemdat)) ;; and not yet expanded into the list of things to be done (if (and (>= *verbosity* 1) (> (length items) 0) (> (length (car items)) 0)) |
︙ | ︙ | |||
359 360 361 362 363 364 365 | items) (if (not (null? tal)) (loop (car tal)(cdr tal)))) ;; if items is a proc then need to run items:get-items-from-config, get the list and loop ;; - but only do that if resources exist to kick off the job ((or (procedure? items)(eq? items 'have-procedure)) | | | > > > > > | | | | | | | | | | | | | | < < | | | | < < < < < < < | < < | < < < < < > | | | | 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 | items) (if (not (null? tal)) (loop (car tal)(cdr tal)))) ;; if items is a proc then need to run items:get-items-from-config, get the list and loop ;; - but only do that if resources exist to kick off the job ((or (procedure? items)(eq? items 'have-procedure)) (let* ((can-run-more (runs:can-run-more-tests db test-record)) (prereqs-not-met (db:get-prereqs-not-met db run-id waitons item-path)) (fails (calc-fails prereqs-not-met))) (debug:print 8 "INFO: can-run-more: " can-run-more " prereqs-not-met:\n " (intersperse prereqs-not-met "\n") " fails:\n " (intersperse fails "\n")) (cond ((and can-run-more (null? prereqs-not-met)) (let ((test-name (tests:testqueue-get-testname test-record))) (setenv "MT_TEST_NAME" test-name) ;; (setenv "MT_RUNNAME" runname) (set-megatest-env-vars db run-id) ;; these may be needed by the launching process (let ((items-list (items:get-items-from-config tconfig))) (if (list? items-list) (begin (tests:testqueue-set-items! test-record items-list) (loop hed tal)) (begin (debug:print 0 "ERROR: The proc from reading the setup did not yield a list - please report this") (exit 1)))))) ((and can-run-more (null? fails)) (debug:print 4 "INFO: Can't run more right now, killing a little time ...") (thread-sleep! 3.0) (loop (car newtal)(cdr newtal))) ;; an issue with prereqs not yet met? (else (debug:print 1 "INFO: test " hed " has failed prerequisite(s); " (string-intersperse (map db:test-get-testname fails) ", ") ", removing it from to-do list") (if (not (null? tal)) (loop (car tal)(cdr tal))))))) ;; this case should not happen, added to help catch any bugs ((and (list? items) itemdat) (debug:print 0 "ERROR: Should not have a list of items in a test and the itemspath set - please report this") (exit 1)))) ;; we get here on "drop through" - loop for next test in queue (if (null? tal) |
︙ | ︙ | |||
487 488 489 490 491 492 493 | ((equal? (test:get-state testdat) "NOT_STARTED")(set! runflag #t)) ;; not -rerun and PASS, WARN or CHECK, do no run ((and (or (not rerun) keepgoing) ;; Require to force re-run for COMPLETED or *anything* + PASS,WARN or CHECK (or (member (test:get-status testdat) '("PASS" "WARN" "CHECK")) (member (test:get-state testdat) '("COMPLETED")))) | | | 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 | ((equal? (test:get-state testdat) "NOT_STARTED")(set! runflag #t)) ;; not -rerun and PASS, WARN or CHECK, do no run ((and (or (not rerun) keepgoing) ;; Require to force re-run for COMPLETED or *anything* + PASS,WARN or CHECK (or (member (test:get-status testdat) '("PASS" "WARN" "CHECK")) (member (test:get-state testdat) '("COMPLETED")))) (debug:print 2 "INFO: running test " test-name "/" item-path " suppressed as it is " (test:get-state testdat) " and " (test:get-status testdat)) (set! runflag #f)) ;; -rerun and status is one of the specifed, run it ((and rerun (let* ((rerunlst (string-split rerun ",")) (must-rerun (member (test:get-status testdat) rerunlst))) (debug:print 3 "INFO: -rerun list: " rerun ", test-status: " (test:get-status testdat)", must-rerun: " must-rerun) must-rerun)) |
︙ | ︙ |