Megatest

Check-in [f160cf8d52]
Login
☰Home Timeline Files Branches Tags Tickets Wiki Login
Overview
Comment:Added area key
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | v1.6584-ck5
Files: files | file ages | folders
SHA1: f160cf8d521ae399bf0d84ecb8faca9d483c1eda
User & Date: matt on 2021-04-25 05:34:18
Other Links: branch diff | manifest | tags
Context
2021-04-25
22:29
Trying to start main.db server check-in: ef485de0ef user: matt tags: v1.6584-ck5
05:34
Added area key check-in: f160cf8d52 user: matt tags: v1.6584-ck5
05:14
Removed references to homehost check-in: 1d98834276 user: matt tags: v1.6584-ck5
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Modified commonmod.scm from [1116143dd3] to [91b24e1910].

︙︙
1048
1049
1050
1051
1052
1053
1054








1055
1056
1057







1058
1059

1060
1061
1062
1063
1064
1065
1066

					  tsname
					  (string-translate *toppath* "/" "."))
				    ))))
		(set! *db-cache-path* dbpath)
		dbpath))
	  #f)))









(define (common:get-area-path-signature)
  (message-digest-string (md5-primitive) *toppath*))








(define (common:get-signature str)
  (message-digest-string (md5-primitive) str))


;;======================================================================
;; E X I T   H A N D L I N G
;;======================================================================

(define (common:human-time)
  (time->string (seconds->local-time (current-seconds)) "%Y-%m-%d %H:%M:%S"))








>
>
>
>
>
>
>
>



>
>
>
>
>
>
>
|
<
>








1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073

1074
1075
1076
1077
1078
1079
1080
1081

					  tsname
					  (string-translate *toppath* "/" "."))
				    ))))
		(set! *db-cache-path* dbpath)
		dbpath))
	  #f)))

(define (common:get-signature str)
  (message-digest-string (md5-primitive) str))

;;======================================================================
;; TOPPATH/AREA
;;======================================================================

;; deprecate this one
(define (common:get-area-path-signature)
  (message-digest-string (md5-primitive) *toppath*))

(define (common:calc-area-key area-path #!optional (full #f))
  (let* ((top-dir-name (pathname-directory area-path))
	 (area-name    (pathname-strip-directory area-path))
	 (top-dir-key  (common:get-signature top-dir-name))
	 (short-key    (substring top-dir-key 0 6)))
    (conc (if full top-dir-key short-key) "-" area-name)))

(define (common:get-area-key)

  (common:calc-area-key *toppath*))

;;======================================================================
;; E X I T   H A N D L I N G
;;======================================================================

(define (common:human-time)
  (time->string (seconds->local-time (current-seconds)) "%Y-%m-%d %H:%M:%S"))
︙︙

Modified megatest.scm from [58763bc3e0] to [e84f4557e9].

︙︙
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534


;; Disabled help items
;;  -rollup                 : (currently disabled) fill run (set by :runname)  with latest test(s)
;;                            from prior runs with same keys
;;  -daemonize              : fork into background and disconnect from stdin/out
     
(define help (conc "
     Megatest, documentation at http://www.kiatoa.com/fossils/megatest
       version " megatest-version "
       license GPL, Copyright Matt Welland 2006-2017
      
     Usage: megatest [options]
       -h                      : this help
       -manual                 : show the Megatest user manual
       -version                : print megatest version (currently " megatest-version ")
     
     Launching and managing runs
       -run                    : run all tests or as specified by -testpatt
       -remove-runs            : remove the data for a run, requires -runname and -testpatt
                                 Optionally use :state and :status, use -keep-records to remove only
                                 the run data. Use -kill-wait to override the 10 second
                                 per test wait after kill delay (e.g. -kill-wait 0). 
       -kill-runs              : kill existing run(s) (all incomplete tests killed)
       -kill-rerun             : kill an existing run (all incomplete tests killed and run is rerun)
       -set-state-status X,Y   : set state to X and status to Y, requires controls per -remove-runs
       -rerun FAIL,WARN...     : force re-run for tests with specificed status(s)
       -rerun-clean            : set all tests not COMPLETED+PASS,WARN,WAIVED to NOT_STARTED,n/a
                                 and then run the specified testpatt with -preclean
       -rerun-all              : set all tests to NOT_STARTED,n/a and run with -preclean
       -lock                   : lock run specified by target and runname
       -unlock                 : unlock run specified by target and runname
       -set-run-status status  : sets status for run to status, requires -target and -runname
       -get-run-status         : gets status for run specified by target and runname
       -run-wait               : wait on run specified by target and runname
       -preclean               : remove the existing test directory before running the test
       -clean-cache            : remove the cached megatest.config and runconfigs.config files
       -no-cache               : do not use the cached config files. 
       -one-pass               : launch as many tests as you can but do not wait for more to be ready
       -remove-keep N          : remove all but N most recent runs per target; use '-actions, -age, -precmd'
       -age <age>              : 120d,3h,20m to apply only to runs older than the 
                                      specified age. NB// M=month, m=minute
       -actions <action>[,...] : actions to take; print,remove-runs,archive,kill-runs
       -precmd                 : insert a wrapper command in front of the commands run
     
     Selectors (e.g. use for -runtests, -remove-runs, -set-state-status, -list-runs etc.)
       -target key1/key2/...   : run for key1, key2, etc.
       -reqtarg key1/key2/...  : run for key1, key2, etc. but key1/key2 must be in runconfigs
       -testpatt patt1/patt2,patt3/...  : % is wildcard
       -runname                : required, name for this particular test run
       -state                  : Applies to runs, tests or steps depending on context
       -status                 : Applies to runs, tests or steps depending on context
       -modepatt key           : load testpatt from <key> in runconfigs instead of default TESTPATT if -testpatt and -tagexpr are not specified
       -tagexpr tag1,tag2%,..  : select tests with tags matching expression
       
     
     Test helpers (for use inside tests)
       -step stepname
       -test-status            : set the state and status of a test (use :state and :status)
       -setlog logfname        : set the path/filename to the final log relative to the test
                                 directory. may be used with -test-status
       -set-toplog logfname    : set the overall log for a suite of sub-tests
       -summarize-items        : for an itemized test create a summary html 
       -m comment              : insert a comment for this test
     
     Test data capture
       -set-values             : update or set values in the testdata table
       :category               : set the category field (optional)
       :variable               : set the variable name (optional)
       :value                  : value measured (required)
       :expected               : value expected (required)
       :tol                    : |value-expect| <= tol (required, can be <, >, >=, <= or number)
       :units                  : name of the units for value, expected_value etc. (optional)
       -load-test-data         : read test specific data for storage in the test_data table
                                 from standard in. Each line is comma delimited with four
                                 fields category,variable,value,comment
     
     Queries
       -list-runs patt         : list runs matching pattern \"patt\", % is the wildcard
       -show-keys              : show the keys used in this megatest setup
       -test-files targpatt    : get the most recent test path/file matching targpatt e.g. %/% or '*.log'
                                 returns list sorted by age ascending, see examples below
       -test-paths             : get the test paths matching target, runname, item and test
                                 patterns.
       -list-disks             : list the disks available for storing runs
       -list-targets           : list the targets in runconfigs.config
       -list-db-targets        : list the target combinations used in the db
       -show-config            : dump the internal representation of the megatest.config file
       -show-runconfig         : dump the internal representation of the runconfigs.config file
       -dumpmode MODE          : dump in MODE format instead of sexpr, MODE=json,ini,sexp etc. (add -debug 0,9 to see which file contributes each line)
       -show-cmdinfo           : dump the command info for a test (run in test environment)
       -section sectionName
       -var varName            : for config and runconfig lookup value for sectionName varName
       -since N                : get list of runs changed since time N (Unix seconds)
       -fields fieldspec       : fields to include in json dump; runs:id,runame+tests:testname+steps
       -sort fieldname         : in -list-runs sort tests by this field
       -testdata-csv [categorypatt/]varpatt  : dump testdata for given category
     
     Misc 
       -start-dir path         : switch to this directory before running megatest
       -contour cname          : add a level of hierarcy to the linktree and run paths
       -area-tag tagname       : add a tag to an area while syncing to pgdb
       -run-tag tagname        : add a tag to a run while syncing to pgdb
       -rebuild-db             : bring the database schema up to date
       -cleanup-db             : remove any orphan records, vacuum the db
       -import-megatest.db     : push data from megatest.db to cache db files in /tmp/$USER
       -sync-to-megatest.db    : pull data from cache files in /tmp/$USER to megatest.db
       -sync-to dest           : sync to new postgresql central style database
       -update-meta            : update the tests metadata for all tests
       -setvars VAR1=val1,VAR2=val2 : Add environment variables to a run NB// these are
                                      overwritten by values set in config files.
       -server -|hostname      : start the server (reduces contention on megatest.db), use
                                 - to automatically figure out hostname
       -adjutant C,M           : start the server/adjutant with allocated cores C and Mem M (Gig), 
                                 use 0,0 to auto use full machine
       -transport http|rpc     : use http or rpc for transport (default is http) 
       -log logfile            : send stdout and stderr to logfile
       -list-servers           : list the servers 
       -kill-servers           : kill all servers
       -repl                   : start a repl (useful for extending megatest)
       -load file.scm          : load and run file.scm
       -mark-incompletes       : find and mark incomplete tests
       -ping run-id|host:port  : ping server, exit with 0 if found
       -debug N|N,M,O...       : enable debug 0-N or N and M and O ...
       -debug-noprop N|M,M,O...: enable debug but do not propagate to subprocesses via MT_DEBUG
       -config fname           : override the megatest.config file with fname
       -append-config fname    : append fname to the megatest.config file
     
     Utilities
       -env2file fname         : write the environment to fname.csh and fname.sh
       -envcap a               : save current variables labeled as context 'a' in file envdat.db
       -envdelta a-b           : output enviroment delta from context a to context b to -o fname
                                 set the output mode with -dumpmode csh, bash or ini
                                 note: ini format will use calls to use curr and minimize path
       -refdb2dat refdb        : convert refdb to sexp or to format specified by s-dumpmode
                                 formats: perl, ruby, sqlite3, csv (for csv the -o param
                                 will substitute %s for the sheet name in generating 
                                 multiple sheets)
       -o                      : output file for refdb2dat (defaults to stdout)
       -archive cmd            : archive runs specified by selectors to one of disks specified
                                 in the [archive-disks] section.
                                 cmd: keep-html, restore, save, save-remove, get, replicate-db (use 
                                 -dest to set destination), -include path1,path2... to get or save specific files
       -generate-html          : create a simple html dashboard for browsing your runs
       -generate-html-structure  : create a top level html veiw to list targets/runs and a Run view within each run directory.  
       -list-run-time          : list time requered to complete runs. It supports following switches
                                 -run-patt <patt> -target-patt <patt> -dumpmode <csv,json,plain-text>
       -list-test-time	  : list time requered to complete each test in a run. It following following arguments
                                 -runname <patt> -target <patt> -dumpmode <csv,json,plain-text>
       -syscheck               : do some very basic checks; write access and space in tmp, home, runs, links and 
                                 is $DISPLAY valid 
       -list-waivers           : dump waivers for specified target, runname, testpatt to stdout
     
     Diff report
       -diff-rep               : generate diff report (must include -src-target, -src-runname, -target, -runname
                                                       and either -diff-email or -diff-html)
       -src-target <target>
       -src-runname <target>
       -diff-email <emails>    : comma separated list of email addresses to send diff report
       -diff-html  <rep.html>  : path to html file to generate
     
     Spreadsheet generation
       -extract-ods fname.ods  : extract an open document spreadsheet from the database
       -pathmod path           : insert path, i.e. path/runame/itempath/logfile.html
                                 will clear the field if no rundir/testname/itempath/logfile
                                 if it contains forward slashes the path will be converted
                                 to windows style
     Getting started
       -create-megatest-area   : create a skeleton megatest area. You will be prompted for paths
       -create-test testname   : create a skeleton megatest test. You will be prompted for info
     
     Examples
     
     # Get test path, use '.' to get a single path or a specific path/file pattern
     megatest -test-files 'logs/*.log' -target ubuntu/n%/no% -runname w49% -testpatt test_mt%
     
     Called as " (string-intersperse (argv) " ") "
     Version " megatest-version ", built from " megatest-fossil-hash ))
     
(define (main)
  (make-and-init-bigdata)

  
  ;; set up the functions in http transport
  (hash-table-set! *http-functions* 'api:process-request          api:process-request)








|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|








351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534


;; Disabled help items
;;  -rollup                 : (currently disabled) fill run (set by :runname)  with latest test(s)
;;                            from prior runs with same keys
;;  -daemonize              : fork into background and disconnect from stdin/out
     
(define help (conc "
Megatest, documentation at http://www.kiatoa.com/fossils/megatest
  version " megatest-version "
  license GPL, Copyright Matt Welland 2006-2017
 
Usage: megatest [options]
  -h                      : this help
  -manual                 : show the Megatest user manual
  -version                : print megatest version (currently " megatest-version ")

Launching and managing runs
  -run                    : run all tests or as specified by -testpatt
  -remove-runs            : remove the data for a run, requires -runname and -testpatt
                            Optionally use :state and :status, use -keep-records to remove only
                            the run data. Use -kill-wait to override the 10 second
                            per test wait after kill delay (e.g. -kill-wait 0). 
  -kill-runs              : kill existing run(s) (all incomplete tests killed)
  -kill-rerun             : kill an existing run (all incomplete tests killed and run is rerun)
  -set-state-status X,Y   : set state to X and status to Y, requires controls per -remove-runs
  -rerun FAIL,WARN...     : force re-run for tests with specificed status(s)
  -rerun-clean            : set all tests not COMPLETED+PASS,WARN,WAIVED to NOT_STARTED,n/a
                            and then run the specified testpatt with -preclean
  -rerun-all              : set all tests to NOT_STARTED,n/a and run with -preclean
  -lock                   : lock run specified by target and runname
  -unlock                 : unlock run specified by target and runname
  -set-run-status status  : sets status for run to status, requires -target and -runname
  -get-run-status         : gets status for run specified by target and runname
  -run-wait               : wait on run specified by target and runname
  -preclean               : remove the existing test directory before running the test
  -clean-cache            : remove the cached megatest.config and runconfigs.config files
  -no-cache               : do not use the cached config files. 
  -one-pass               : launch as many tests as you can but do not wait for more to be ready
  -remove-keep N          : remove all but N most recent runs per target; use '-actions, -age, -precmd'
  -age <age>              : 120d,3h,20m to apply only to runs older than the 
                                 specified age. NB// M=month, m=minute
  -actions <action>[,...] : actions to take; print,remove-runs,archive,kill-runs
  -precmd                 : insert a wrapper command in front of the commands run

Selectors (e.g. use for -runtests, -remove-runs, -set-state-status, -list-runs etc.)
  -target key1/key2/...   : run for key1, key2, etc.
  -reqtarg key1/key2/...  : run for key1, key2, etc. but key1/key2 must be in runconfigs
  -testpatt patt1/patt2,patt3/...  : % is wildcard
  -runname                : required, name for this particular test run
  -state                  : Applies to runs, tests or steps depending on context
  -status                 : Applies to runs, tests or steps depending on context
  -modepatt key           : load testpatt from <key> in runconfigs instead of default TESTPATT if -testpatt and -tagexpr are not specified
  -tagexpr tag1,tag2%,..  : select tests with tags matching expression
  

Test helpers (for use inside tests)
  -step stepname
  -test-status            : set the state and status of a test (use :state and :status)
  -setlog logfname        : set the path/filename to the final log relative to the test
                            directory. may be used with -test-status
  -set-toplog logfname    : set the overall log for a suite of sub-tests
  -summarize-items        : for an itemized test create a summary html 
  -m comment              : insert a comment for this test

Test data capture
  -set-values             : update or set values in the testdata table
  :category               : set the category field (optional)
  :variable               : set the variable name (optional)
  :value                  : value measured (required)
  :expected               : value expected (required)
  :tol                    : |value-expect| <= tol (required, can be <, >, >=, <= or number)
  :units                  : name of the units for value, expected_value etc. (optional)
  -load-test-data         : read test specific data for storage in the test_data table
                            from standard in. Each line is comma delimited with four
                            fields category,variable,value,comment

Queries
  -list-runs patt         : list runs matching pattern \"patt\", % is the wildcard
  -show-keys              : show the keys used in this megatest setup
  -test-files targpatt    : get the most recent test path/file matching targpatt e.g. %/% or '*.log'
                            returns list sorted by age ascending, see examples below
  -test-paths             : get the test paths matching target, runname, item and test
                            patterns.
  -list-disks             : list the disks available for storing runs
  -list-targets           : list the targets in runconfigs.config
  -list-db-targets        : list the target combinations used in the db
  -show-config            : dump the internal representation of the megatest.config file
  -show-runconfig         : dump the internal representation of the runconfigs.config file
  -dumpmode MODE          : dump in MODE format instead of sexpr, MODE=json,ini,sexp etc. (add -debug 0,9 to see which file contributes each line)
  -show-cmdinfo           : dump the command info for a test (run in test environment)
  -section sectionName
  -var varName            : for config and runconfig lookup value for sectionName varName
  -since N                : get list of runs changed since time N (Unix seconds)
  -fields fieldspec       : fields to include in json dump; runs:id,runame+tests:testname+steps
  -sort fieldname         : in -list-runs sort tests by this field
  -testdata-csv [categorypatt/]varpatt  : dump testdata for given category

Misc 
  -start-dir path         : switch to this directory before running megatest
  -contour cname          : add a level of hierarcy to the linktree and run paths
  -area-tag tagname       : add a tag to an area while syncing to pgdb
  -run-tag tagname        : add a tag to a run while syncing to pgdb
  -rebuild-db             : bring the database schema up to date
  -cleanup-db             : remove any orphan records, vacuum the db
  -import-megatest.db     : push data from megatest.db to cache db files in /tmp/$USER
  -sync-to-megatest.db    : pull data from cache files in /tmp/$USER to megatest.db
  -sync-to dest           : sync to new postgresql central style database
  -update-meta            : update the tests metadata for all tests
  -setvars VAR1=val1,VAR2=val2 : Add environment variables to a run NB// these are
                                 overwritten by values set in config files.
  -server -|hostname      : start the server (reduces contention on megatest.db), use
                            - to automatically figure out hostname
  -adjutant C,M           : start the server/adjutant with allocated cores C and Mem M (Gig), 
                            use 0,0 to auto use full machine
  -transport http|rpc     : use http or rpc for transport (default is http) 
  -log logfile            : send stdout and stderr to logfile
  -list-servers           : list the servers 
  -kill-servers           : kill all servers
  -repl                   : start a repl (useful for extending megatest)
  -load file.scm          : load and run file.scm
  -mark-incompletes       : find and mark incomplete tests
  -ping run-id|host:port  : ping server, exit with 0 if found
  -debug N|N,M,O...       : enable debug 0-N or N and M and O ...
  -debug-noprop N|M,M,O...: enable debug but do not propagate to subprocesses via MT_DEBUG
  -config fname           : override the megatest.config file with fname
  -append-config fname    : append fname to the megatest.config file

Utilities
  -env2file fname         : write the environment to fname.csh and fname.sh
  -envcap a               : save current variables labeled as context 'a' in file envdat.db
  -envdelta a-b           : output enviroment delta from context a to context b to -o fname
                            set the output mode with -dumpmode csh, bash or ini
                            note: ini format will use calls to use curr and minimize path
  -refdb2dat refdb        : convert refdb to sexp or to format specified by s-dumpmode
                            formats: perl, ruby, sqlite3, csv (for csv the -o param
                            will substitute %s for the sheet name in generating 
                            multiple sheets)
  -o                      : output file for refdb2dat (defaults to stdout)
  -archive cmd            : archive runs specified by selectors to one of disks specified
                            in the [archive-disks] section.
                            cmd: keep-html, restore, save, save-remove, get, replicate-db (use 
                            -dest to set destination), -include path1,path2... to get or save specific files
  -generate-html          : create a simple html dashboard for browsing your runs
  -generate-html-structure  : create a top level html veiw to list targets/runs and a Run view within each run directory.  
  -list-run-time          : list time requered to complete runs. It supports following switches
                            -run-patt <patt> -target-patt <patt> -dumpmode <csv,json,plain-text>
  -list-test-time	  : list time requered to complete each test in a run. It following following arguments
                            -runname <patt> -target <patt> -dumpmode <csv,json,plain-text>
  -syscheck               : do some very basic checks; write access and space in tmp, home, runs, links and 
                            is $DISPLAY valid 
  -list-waivers           : dump waivers for specified target, runname, testpatt to stdout

Diff report
  -diff-rep               : generate diff report (must include -src-target, -src-runname, -target, -runname
                                                  and either -diff-email or -diff-html)
  -src-target <target>
  -src-runname <target>
  -diff-email <emails>    : comma separated list of email addresses to send diff report
  -diff-html  <rep.html>  : path to html file to generate

Spreadsheet generation
  -extract-ods fname.ods  : extract an open document spreadsheet from the database
  -pathmod path           : insert path, i.e. path/runame/itempath/logfile.html
                            will clear the field if no rundir/testname/itempath/logfile
                            if it contains forward slashes the path will be converted
                            to windows style
Getting started
  -create-megatest-area   : create a skeleton megatest area. You will be prompted for paths
  -create-test testname   : create a skeleton megatest test. You will be prompted for info

Examples

# Get test path, use '.' to get a single path or a specific path/file pattern
megatest -test-files 'logs/*.log' -target ubuntu/n%/no% -runname w49% -testpatt test_mt%

Called as " (string-intersperse (argv) " ") "
Version " megatest-version ", built from " megatest-fossil-hash ))
     
(define (main)
  (make-and-init-bigdata)

  
  ;; set up the functions in http transport
  (hash-table-set! *http-functions* 'api:process-request          api:process-request)
︙︙

This page was generated in about 0.12s by Fossil 2.24 [452f8e1ffc] 2024-03-26 15:36:00