The Megatest Users Manual
=========================
Matt Welland <matt@kiatoa.com>
v1.5, June 2020
:doctype: book
[preface]
Preface
-------
This book is organised as three sub-books; getting started, writing tests and reference.
.License
----------------------------
Copyright 2006-2020, Matthew Welland.
This document is part of Megatest.
Megatest is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
Megatest is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with Megatest. If not, see <http://www.gnu.org/licenses/>.
----------------------------
Why Megatest?
-------------
Megatest was created to provide a generalized tool for managing suites
of regression tests and to provide a multi-host, distributed
alternative to "make". The EDA world is littered with proprietory,
company-specific tools for this purpose and by going open source and
keeping the tool flexible the hope is that Megatest could be useful to
any team at any company for continuous integration and almost any
other general automation tasks.
Megatest Design Philosophy
--------------------------
Megatest is a distributed system intended to provide the minimum needed
resources to make writing a suite of tests and tasks for implementing
continuous build for software, design engineering or process control (via
owlfs for example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or FAIL
of a test or task. In most cases megatest is best used in conjunction with
logpro or a similar tool to parse, analyze and decide on the test outcome.
* Self-checking - make it as easy as possible to write self-checking
tests (as opposed to using deltas, i.e. tests that compare with a
previous measurement to deterine PASS/FAIL).
* Traceable - environment variables, host OS and other possibly
influential variables are captured and kept recorded.
* Immutable - once a test is run it cannot be easily overwritten or
modified accidentally.
* Repeatable - test results can be recreated in the future using all
the original variables.
* Relocatable - the testsuite or automation area can be checked out
and the tests run anywhere in the disk hierarchy.
* Encapsulated - the tests run in self-contained directories and all
inputs and outputs to the process can be found in the run areas.
* Deployable - a testsuite is self-contained and can be bundled with
a software project and easily used by others with little to no
setup burden.
Megatest Architecture
---------------------
Data separation
~~~~~~~~~~~~~~~
All data to specify the tests and configure the system is stored in
plain text config files. All system state is stored in an sqlite3
database.
Distributed Compute
~~~~~~~~~~~~~~~~~~~
Tests are launched using the launching system available for the
distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database. Megatest has been used with the Intel Netbatch and
lsf (also known as openlava) batch systems and it should be
straightforward to use it with other similar systems.
// :leveloffset: 0
include::overview.txt[]
include::plan.txt[]
include::installation.txt[]
include::getting_started.txt[]
include::study_plan.txt[]
include::writing_tests.txt[]
include::debugging.txt[]
include::howto.txt[]
include::reference.txt[]
include::testplan.txt[]
Megatest Internals
------------------
["graphviz", "server.png"]
----------------------------------------------------------------------
include::server.dot[]
----------------------------------------------------------------------
// include::plan.txt[]
// to allow the getting_started.txt to be a stand-alone document use level
// shifting, note that the preceding blank line is needed.
// :leveloffset: 2
// [appendix]
// Example Appendix
// ================
// One or more optional appendixes go here at section level zero.
//
// Appendix Sub-section
// ~~~~~~~~~~~~~~~~~~~
// NOTE: Preface and appendix subsections start out of sequence at level
// 2 (level 1 is skipped). This only applies to multi-part book
// documents.
//
//
//
// [bibliography]
// Example Bibliography
// ====================
// The bibliography list is a style of AsciiDoc bulleted list.
//
// [bibliography]
// - [[[taoup]]] Eric Steven Raymond. 'The Art of Unix
// Programming'. Addison-Wesley. ISBN 0-13-142901-9.
// - [[[walsh-muellner]]] Norman Walsh & Leonard Muellner.
// 'DocBook - The Definitive Guide'. O'Reilly & Associates. 1999.
// ISBN 1-56592-580-7.
//
//
// [glossary]
// Example Glossary
// ================
// Glossaries are optional. Glossaries entries are an example of a style
// of AsciiDoc labeled lists.
//
// [glossary]
// A glossary term::
// The corresponding (indented) definition.
//
// A second glossary term::
// The corresponding (indented) definition.
//
//
// [colophon]
// Example Colophon
// ================
// Text at the end of a book describing facts about its production.
[index]
Index
-----
////////////////////////////////////////////////////////////////
The index is normally left completely empty, it's contents are
generated automatically by the DocBook toolchain.
////////////////////////////////////////////////////////////////