# Copyright 2006-2017, Matthew Welland.
#
# This file is part of Megatest.
#
# Megatest is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Megatest is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Megatest. If not, see <http://www.gnu.org/licenses/>.
(server:writable-watchdog-bruteforce dbstruct)
(server:writable-watchdog-deltasync dbstruct)
=====================================================================
NOTES from looking at branch v1.62-rpc
=====================================================================
*last-db-access* or *db-last-access* ==> which is it to be?
seen in singletest: ERROR: Unrecognised arguments: :first_err This is the first error
======================================================================
New way of launching needed to accomodate different target hosttypes
for items
======================================================================
[flavors]
general ssh #{getbesthost general}
nbgeneral nbjob run JOBCOMMAND -log $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME.$MT_TESTNAME-$MT_ITEM_PATH.lgo
[hosts]
general cubian xena
[launchers]
envsetup general
xor/%/n 4C16G
% nbgeneral
[jobtools]
launcher internal
======================================================================
Try writing to in-memory db and every 2-5 seconds syncing to megatest.db
======================================================================
First, how much time will it take to write back the changes:
1. Get the run table
(define (get-all db)(let ((res '()))(for-each-row (lambda (a . b)(set! res (cons (apply vector a b) res))) db "SELECT * FROM tests;") res))
(define tdata (let ((start (current-milliseconds))(res (get-all *db*)))(print (- (current-milliseconds) start))res))
Result ranges from 34ms to 89ms but mostly around 40ms for 623 records on moosefs
Projecting to 15000 records:
Slow 2 seconds to read all
Median 1 second to read all
This seems like it would work with an update period of 2-5 seconds
TODO
----
1. open-db opens in-memory db and megatest.db, put handles in *memdb* and *db*, *memdb* is < run-id dbh >
2. Server is part of runtests
a. server start cycle - adapt to per run-id
i. states; starting, started, stopping, stopped
b. turn off write coalesing
3. Calls to -runtests, -remove-runs etc.
a. Might talk to running server if run specific
b. Can talk to megatest.db but not a generally good idea
c. Can start a runserver
4. Dashboard is fine except for writes?
======================================================================
Routines to convert for runs.scm
cdb:remote-run db:register-run
cdb:delete-tests-in-state *runremote*
cdb:get-test-info-by-id *runremote*
cdb:remote-run db:delete-old-deleted-test-records
cdb:remote-run db:delete-run
cdb:remote-run db:delete-test-records
cdb:remote-run db:delete-tests-for-run
cdb:remote-run db:find-and-mark-incomplete
cdb:remote-run db:get-count-tests-running
cdb:remote-run db:get-count-tests-running-in-jobgroup
cdb:remote-run db:get-keys
cdb:remote-run db:get-run-info
cdb:remote-run db:get-run-key-val
cdb:remote-run db:get-run-name-from-id
cdb:remote-run db:get-steps-for-test
cdb:remote-run db:get-test-id-cached
cdb:remote-run db:get-tests-for-runs-mindata
cdb:remote-run db:lock/unlock-run
cdb:remote-run db:set-sync
cdb:remote-run db:set-tests-state-status
cdb:remote-run db:set-var
cdb:remote-run db:testmeta-add-record
cdb:remote-run db:testmeta-get-record
cdb:remote-run db:testmeta-update-field
cdb:remote-run db:update-run-event_time
cdb:remote-run instead
cdb:remote-run server:start
cdb:remote-run test:get-matching-previous-test-run-records
cdb:tests-register-test *runremote*
(define (runs:shrink-can-run-more-tests-count) ;; the db is a dummy var so we can use cdb:remote-run
======================================================================
[87cbe68f31]
[be405e8e2e]
# FROM andyjpg on #chicken
(let ((original-exit (exit-handler)))
(exit-handler (lambda (#!optional (exit-code 0))
(printf "Preparing to exit...\n" exit-code)
(for-each (lambda (pid)
(printf "Sending signal/term to ~A\n" pid)
(process-signal pid signal/term)) (children))
(original-exit exit-code))))
1. All run control access to db is direct.
2. All test machines must have megatest available
3. Tests may or may not have file system access to the originating
run area. rsync is used to pull the test area to the home host
if and only if the originating area can not be seen via file
system. NO LONGER TRUE. Rsync is used but file system must be visible.
4. All db access is done via the home host. NOT IMPLEMENTED YET.
REMOTE ACCESS DB LOADS
INFO: (0) Max cached queries was 10
INFO: (0) Number of cached writes 27043
INFO: (0) Average cached write time 15.0634544983915 ms
INFO: (0) Number non-cached queries 71928
INFO: (0) Average non-cached time 5.15547491936381 ms
INFO: (0) Server shutdown complete. Exiting
fdktestqa on Apr 29, 2013: 1812 tests
INFO: (0) Max cached queries was 10
INFO: (0) Number of cached writes 41335
INFO: (0) Average cached write time 206.081553163179 ms
INFO: (0) Number non-cached queries 74289
INFO: (0) Average non-cached time 1055.09826488444 ms
INFO: (0) Server shutdown complete. Exiting
Start: 0 at Sun Apr 28 22:18:25 MST 2013
Max: 52 at Sun Apr 28 23:06:59 MST 2013
End: 6 at Sun Apr 28 23:47:51 MST 2013
========================================================================