Megatest

Check-in [e9a41f7443]
Login
☰Home Timeline Files Branches Tags Tickets Wiki Login
Overview
Comment:Partial re-work of test status rollup for itemized tests. Initial implementation of per test summary generation
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | v1.60
Files: files | file ages | folders
SHA1: e9a41f74435703755d11ac0a93ff9274aee84810
User & Date: mrwellan on 2015-06-29 17:44:28
Other Links: branch diff | manifest | tags
Context
2015-06-29
23:15
Fixed rollup check-in: acc15ed887 user: matt tags: v1.60
17:44
Partial re-work of test status rollup for itemized tests. Initial implementation of per test summary generation check-in: e9a41f7443 user: mrwellan tags: v1.60
00:18
Display basics for areas check-in: 422ec08c45 user: matt tags: v1.60
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Modified api.scm from [194ab70fc6] to [ef0a5aac98].

︙
130
131
132
133
134
135
136
137

138
139
140
141
142
143
144

130
131
132
133
134
135
136

137
138
139
140
141
142
143
144








-
+








	    ;; TESTS
	    ((test-set-state-status-by-id)     (apply db:test-set-state-status-by-id dbstruct params))
	    ((delete-test-records)             (apply db:delete-test-records dbstruct params))
	    ((delete-old-deleted-test-records) (apply db:delete-old-deleted-test-records dbstruct params))
	    ((test-set-status-state)           (apply db:test-set-status-state dbstruct params))
	    ((test-set-top-process-pid)        (apply db:test-set-top-process-pid dbstruct params))
	    ((roll-up-pass-fail-counts)        (apply db:roll-up-pass-fail-counts dbstruct params))
	    ((update-fail-pass-counts)         (apply db:general-call dbstruct 'update-pass-fail-counts params))
	    ((update-pass-fail-counts)         (apply db:general-call dbstruct 'update-pass-fail-counts params))
	    ((test-set-archive-block-id)       (apply db:test-set-archive-block-id dbstruct params))

	    ;; RUNS
	    ((register-run)                 (apply db:register-run dbstruct params))
	    ((set-tests-state-status)       (apply db:set-tests-state-status dbstruct params))
	    ((delete-run)                   (apply db:delete-run dbstruct params))
	    ((lock/unlock-run)              (apply db:lock/unlock-run dbstruct params))
︙

Modified db.scm from [2e35c37b21] to [7acd23bc23].

︙
2829
2830
2831
2832
2833
2834
2835


2836
2837
2838
2839








2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853

2854
2855
2856
2857
2858
2859
2860

2829
2830
2831
2832
2833
2834
2835
2836
2837
2838



2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856




2857
2858
2859
2860
2861
2862
2863
2864








+
+

-
-
-
+
+
+
+
+
+
+
+










-
-
-
-
+








  (let ((dbdat  (db:get-db dbstruct run-id)))
    (if (member state '("LAUNCHED" "REMOTEHOSTSTART"))
	(db:general-call dbdat 'set-test-start-time (list test-id)))
    (if msg
	(db:general-call dbdat 'state-status-msg (list state status msg test-id))
	(db:general-call dbdat 'state-status     (list state status test-id)))))

;; call with state = #f to roll up with out accounting for state/status of this item
;;
(define (db:roll-up-pass-fail-counts dbstruct run-id test-name item-path state status)
  (if ;; (and
      (not (equal? item-path ""))
   ;; (not (member status '("PASS" "WARN" "FAIL" "WAIVED" "RUNNING" "CHECK" "SKIP" "LAUNCHED")))
  (if (or (not state)
	  (not (equal? item-path "")))
      ;; just do a rollup
      (begin
	(db:general-call dbdat 'update-pass-fail-counts (list test-name test-name test-name))
	(db:top-test-set-per-pf-counts db run-id test-name)
	#f)
      ;; (not (member status '("PASS" "WARN" "FAIL" "WAIVED" "RUNNING" "CHECK" "SKIP" "LAUNCHED")))
      (let ((dbdat (db:get-db dbstruct run-id)))
	(db:general-call dbdat 'update-pass-fail-counts (list test-name test-name test-name))
	;; NOTE: No else clause needed for this case
	(case (string->symbol status)
	 ((RUNNING)  (db:general-call dbdat 'top-test-set-running (list test-name)))
	 ((LAUNCHED) (db:general-call dbdat 'top-test-set (list "LAUNCHED" test-name)))
	 ((ABORT INCOMPLETE) (db:general-call dbdat 'top-test-set (list status test-name))))
	(let ((db (db:dbdat-get-db dbdat)))
	  (db:top-test-set-per-pf-counts db run-id test-name))
	#f)
      ;; if the test is not COMPLETED then this routine should not have been called
      (begin
	(debug:print 0 "ERROR: db:test-set-state-status called with state " state " and status " status)
	#f)))
      ))

(define (db:test-get-logfile-info dbstruct run-id test-name)
  (db:with-db
   dbstruct
   run-id
   #f
   (lambda (db)
︙
2928
2929
2930
2931
2932
2933
2934

2935
2936
2937
2938
2939
2940
2941

2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946








+








                                   WHEN (SELECT count(id) FROM tests 
                                                WHERE testname=?
                                                     AND item_path != '' 
                                                     AND status NOT IN ('TEN_STRIKES','BLOCKED')
                                                     AND state in ('RUNNING','NOT_STARTED','LAUNCHED','REMOTEHOSTSTART')) > 0 THEN 'RUNNING'
                                   ELSE 'COMPLETED' END,
                            status=CASE 
                                  WHEN fail_count > 0 THEN 'FAIL' 
                                  WHEN (SELECT count(id) FROM tests
                                         WHERE run_id=? AND testname=?
                                              AND item_path != ''
                                              AND state IN ('NOT_STARTED','BLOCKED','INCOMPLETE')) > 0 THEN 'FAIL'
                                  WHEN (SELECT count(id) FROM tests
                                         WHERE testname=?
                                              AND item_path != ''
︙
2949
2950
2951
2952
2953
2954
2955





2956
2957
2958
2959
2960
2961
2962

2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972








+
+
+
+
+








                                  WHEN (SELECT count(id) FROM tests
                                         WHERE testname=?
                                              AND item_path != ''
                                              AND state NOT IN ('DELETED')
                                              AND status IN ('STUCK/INCOMPLETE', 'INCOMPLETE')) > 0 THEN 'INCOMPLETE'
                                  WHEN (SELECT count(id) FROM tests
                                         WHERE testname=?
                                              AND item_path != ''
                                              AND state NOT IN ('DELETED')
                                              AND status = 'FAIL') > 0 THEN 'FAIL'
                                  WHEN (SELECT count(id) FROM tests
                                         WHERE testname=?
                                              AND item_path != ''
                                              AND state NOT IN ('DELETED')
                                              AND status = 'CHECK') > 0 THEN 'CHECK'
                                  WHEN (SELECT count(id) FROM tests
                                         WHERE testname=?
                                              AND item_path != ''
                                              AND state NOT IN ('DELETED')
︙
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986

2982
2983
2984
2985
2986
2987
2988

2989
2990
2991
2992
2993
2994
2995








-








                                              AND state NOT IN ('DELETED')
                                              AND status = 'WAIVED') > 0 THEN 'WAIVED'
                                  WHEN (SELECT count(id) FROM tests
                                         WHERE testname=? 
                                              AND item_path != ''
                                              AND state NOT IN ('DELETED')
                                              AND status NOT IN ('PASS','FAIL','WARN','WAIVED')) > 0 THEN 'ABORT'
                                  WHEN fail_count > 0 THEN 'FAIL' 
                                  WHEN pass_count > 0 AND fail_count=0 THEN 'PASS' 
                                  ELSE 'UNKNOWN' END
                       WHERE testname=? AND item_path='';") ;; DONE

	;; STEPS
	'(delete-test-step-records "UPDATE test_steps SET status='DELETED' WHERE test_id=?;")
	'(delete-test-data-records "UPDATE test_data  SET status='DELETED' WHERE test_id=?;") ;; using status since no state field
︙

Modified megatest-version.scm from [7d7023c394] to [a98e35a9bb]. 

1
2
3
4
5
6

7

1
2
3
4
5

6
7






-
+


;; Always use two or four digit decimal
;; 1.01, 1.02...1.10,1.11,1,1101 ... 1.99,2.00..

(declare (unit megatest-version))

(define megatest-version 1.6017)
(define megatest-version 1.6018)

Modified rmt.scm from [84b644dacc] to [9384bcca50].

︙
525
526
527
528
529
530
531
532

533
534
535
536
537
538
539

525
526
527
528
529
530
531

532
533
534
535
536
537
538
539








-
+









;; state and status are extra hints not usually used in the calculation
;;
(define (rmt:roll-up-pass-fail-counts run-id test-name item-path state status)
  (rmt:send-receive 'roll-up-pass-fail-counts run-id (list run-id test-name item-path state status)))

(define (rmt:update-pass-fail-counts run-id test-name)
  (rmt:general-call 'update-fail-pass-counts run-id (list run-id test-name run-id test-name run-id test-name)))
  (rmt:general-call 'update-pass-fail-counts run-id (list run-id test-name run-id test-name run-id test-name)))

;;======================================================================
;;  R U N S
;;======================================================================

(define (rmt:get-run-info run-id)
  (rmt:send-receive 'get-run-info run-id (list run-id)))
︙

Modified tests.scm from [018cc1b606] to [e113e6dd48].

︙
327
328
329
330
331
332
333
334

335




336

337
338
339
340
341
342
343

327
328
329
330
331
332
333

334
335
336
337
338
339

340
341
342
343
344
345
346
347








-
+

+
+
+
+
-
+








    (if (or (equal? logf "logs/final.log")
	    (equal? logf outputfilename)
	    force)
	(let ((my-start-time (current-seconds))
	      (lockf         (conc outputfilename ".lock")))
	  (let loop ((have-lock  (common:simple-file-lock lockf)))
	    (if have-lock
		(begin
		(let ((script (configf:lookup *configdat* "testrollup" test-name)))
		  (print "Obtained lock for " outputfilename)
		  ;; (rmt:top-test-set-per-pf-counts run-id test-name)
		  (rmt:roll-up-pass-fail-counts run-id test-name "" #f #f)
		  (if script
		      (system (conc script " > " outputfilename " & "))
		  (tests:generate-html-summary-for-iterated-test run-id test-id test-name outputfilename)
		      (tests:generate-html-summary-for-iterated-test run-id test-id test-name outputfilename))
		  (common:simple-file-release-lock lockf)
		  (change-directory orig-dir)
		  ;; NB// tests:test-set-toplog! is remote internal...
		  (tests:test-set-toplog! run-id test-name outputfilename))
		;; didn't get the lock, check to see if current update started later than this 
		;; update, if so we can exit without doing any work
		(if (> my-start-time (file-modification-time lockf))
︙

Modified tests/fullrun/megatest.config from [b0dbb892fd] to [9182ed7cc3].

︙
238
239
240
241
242
243
244




245
246
247
248
249
250
251

238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255








+
+
+
+









# launcher bsub -q priority -o $MT_TEST_RUN_DIR/openlava.log 

# launcher #{shell if which bsub > /dev/null;then echo bsub -q priority -o openlava.log;else echo sleeprunner;fi}
launcher nbfake

[configf:settings trim-trailing-spaces yes]

# Override the rollup for specific tests
[testrollup]
runfirst ls

[test]
# VAL1 has trailing spaces
VAL1 Foo    
VAL2 ==>#{get test VAL1}Bar<== no spaces between Foo and Bar to pass

ltest #{scheme (case (string->symbol (conc (getenv "datapath"))) \
︙

Modified tests/fullrun/tests/priority_1/testconfig from [737fc2258c] to [5fb118212f].

︙
8
9
10
11
12
13
14




8
9
10
11
12
13
14
15
16
17








+
+
+

jobgroup sqlite3
author matt
owner  bob
description This test checks that a multi-lineitem test with mix of pass and non-fail rolls up a PASS

tags first,single
reviewed 09/10/2011, by Matt

[triggers]
COMPLETED/ echo $MT_TEST_NAME > $MT_RUN_AREA_HOME/foo

This page was generated in about 0.123s by Fossil 2.24 [452f8e1ffc] 2024-03-26 15:36:00