1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
|
// This file is part of Megatest.
//
// Megatest is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Megatest is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Megatest. If not, see <http://www.gnu.org/licenses/>.
// Copyright 2006-2020, Matthew Welland.
matt@kiatoa.com with requests or bug reports. Requests from inside
Intel generally take priority.
Dashboard and runs
. Multi-area dashboard view
Tests Support
. Add variable $MT_RUNPATH = $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME
. Improve [script], especially indent handling
Scalability
. Overflow database methodology - combine the best of the v1.63
multi-db approach and the current db-in-tmp approach (currently
slowness can be seen when number of tests in a db goes over 50-100k,
with the overflow db it will be able to handle 1000's of runs with
50-100k tests per run). High priority - goal is to complete this by
20Q3.
. Switch to using simple runs query everywhere
. Add end_time to runs and add a rollup call that sets state, status and end_time
. Aggressive megatest.config and runconfig.config caching.
.. Cache the configs in $MT_RUNPATH
.. Following invocations of –run, -rerun* will calculate the new config but only overwrite the cached file IF changed
. If the cached file changes ALL existing tests go from COMPLETED -> STALE, I’m not sure what to do about RUNNING tests
. !VARS in runconfigs are not exported to the environment. They are accessed via rget as if the ! was not there.
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
|
<
<
<
<
<
<
<
<
<
<
<
<
>
>
|
<
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
|
TODO / Road Map
---------------
Note: This road-map is a wish list and not a formal plan. Items are in
rough priority but are subject to change. Development is driven by
user requests, developer "itch" and bug reports. Please contact
matt@kiatoa.com with requests or bug reports.
Dashboard and runs
. Multi-area dashboard view
Tests Support
. Add variable $MT_RUNPATH = $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME
. Improve [script], especially indent handling
Mtutils/CI
. Enable mtutil calls from dashboard (for remote control)
. Logs browser (esp. for surfacing mtutil related activities)
. Embed ftfplan for distributed automation, completed activities trigger QA runs which trigger deployment etc.
. Jenkins junit XML support [DONE]
. Add output flushing in teamcity support
Build system
. ./configure => ubuntu, sles11, sles12, rh7 [WIP]
Code refactoring/quality/performance
. Switch to scsh-process pipeline management for job execution/control
. Use call-with-environment-variables where possible.
. Switch to using simple runs query everywhere
. Add end_time to runs and add a rollup call that sets state, status and end_time
Migration to inmem db and or overflow db
. Re-work the dbstruct data structure?
.. [ run-id.db inmemdb last-mod last-read last-sync inuse ]
Some ideas for Megatest 2.0
. Generate cache files of megatest.config and runconfig.config and use them consistently. No magic.
.. Cache the configs in $MT_RUNPATH
.. Following invocations of –run, -rerun* will calculate the new config but only overwrite the cached file IF changed
. If the cached file changes ALL existing tests go from COMPLETED -> STALE, I’m not sure what to do about RUNNING tests
. Per test copy commands (example is incomplete).
----------------
[testcopy]
%/iind% unison SRC DEST
% cp –r SRC DEST
----------------
|
157
158
159
160
161
162
163
|
. Added option to compile IUP (needed for VMs)
. Server side run launching
. Wizards for creating tests, regression areas (current ones are text only and limited).
. Fully functional built in web service (currently you can browse runs but it is very simplistic).
. Gui panels for editing megatest.config and runconfigs.config
. Fully isolated tests (no use of NFS to see regression area files)
. Windows version
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
|
. Added option to compile IUP (needed for VMs)
. Server side run launching
. Wizards for creating tests, regression areas (current ones are text only and limited).
. Fully functional built in web service (currently you can browse runs but it is very simplistic).
. Gui panels for editing megatest.config and runconfigs.config
. Fully isolated tests (no use of NFS to see regression area files)
. Windows version
WW14
. Streamline compilation - DONE, all non-official egg modules are now bundled.
WW15
. syscheck; touch file in home, tmp, runs, links and start xterm
. pull in ftfplan (not integrated, just code pulled in)
. fill newview matrix with data, filter pipeline gui elements
. improve [script], especially indent handling
WW16
. split db into megatest.db (runs etc.) db/<something>.db
. release basic newview implementation
. add section "archive" to testconfig with save yes to save the test to archive at end of test run.
. add -getdata and -savedata; writes a path to archive using provided target, runname, test and item_patt and path
WW18
. release split db implementation
. mtutil calls from dashboard (for remote control)
. logs browser (esp. for surfacing mtutil related activities)
WW19
.
WW20
. Switch to using simple runs query everywhere
. Add end_time to runs and add a rollup call that sets state, status and end_time
Future
. Switch to scsh-process pipeline management for job execution/control
. Use call-with-environment-variables more.
Migration to inmem db plus per run db
-------------------------------------
. Re-work the dbstruct data structure?
.. Move main.db to global?
.. [ run-id.db inmemdb last-mod last-read last-sync inuse ]
. Open main.db directly in calls to -runtests etc. No need to talk remote?
|