Megatest: Check-in [851bcc0c6b]

Overview

Comment:	Merged changes from trunk
Downloads:	Tarball \| ZIP archive \| SQL archive
Timelines:	family \| ancestors \| descendants \| both \| v2.01-try-1
Files:	files \| file ages \| folders
SHA1:	851bcc0c6bb239b6eee6dd067020d606e6b7c01e
User & Date:	mrwellan on 2018-11-27 15:43:00
Other Links:	branch diff \| manifest \| tags

Context

2018-12-06
09:52		Initial version of tquery check-in: 06b3f69cfc user: jmoon18 tags: v2.01-tquery, v2.01-try-1
2018-11-28
14:44		Corrected couple mis-ported items for the mt* stuff Leaf check-in: c6761db385 user: mrwellan tags: v2.01-try-1
2018-11-27
15:43		Merged changes from trunk check-in: 851bcc0c6b user: mrwellan tags: v2.01-try-1
14:11		More manual cleanup and improvements. check-in: c5e11fa496 user: mrwellan tags: trunk
2018-11-15
23:06		Switched to using mtconfigf egg instead of configf.scm check-in: 61cc0c58dd user: matt tags: v2.01-try-1

Changes

Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Added docs/manual/megatest-stand-alone-area.png version [b4666b70f8].

cannot compute difference between binary files

Added docs/manual/megatest-system-architecture.png version [d1d2f4a2b8].

cannot compute difference between binary files

Added docs/manual/megatest-system-architecture.svg version [1f9c743227].

Modified docs/manual/megatest_manual.html from [08360c42a9] to [4f03cc70cf].

Modified docs/manual/megatest_manual.txt from [bb97544027] to [2589d78a96].

Added docs/manual/stand-alone-megatest-area.svg version [a33455507d].

cannot compute difference between binary files

︙
833 834 835 836 837 838 839 ~~840 841~~ 842 843 844 845 ~~846 847~~ 848 849 850 851 852 853 854 855 856 857 ~~858~~ 859 860 861 862 ~~863~~ 864 865 866 867 ~~868 869~~ 870 871 872 873 ~~874~~ 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 ~~895 896 897~~ 898 899 900 901 902 903 904	833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968	- - + + + - - + + - + + - + + - - + + - + + + - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +	owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test or task. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome.</p></div> <div class="ulist"><ul> <li> <p> ~~Self-checking -~~Rep~~eatable stri~~ve for directed or~~ self-checking ~~test~~ as opposed to delta ~~based~~ tests.~~ Self-checking - make it as easy as possible to write self-checking tests (as opposed to using deltas, i.e. tests that compare with a previous measurement to deterine PASS/FAIL). </p> </li> <li> <p> ~~Traceable - environment variables, host OS and other possibly ~~influential~~ variables are captured and kept recorded.~~ Traceable - environment variables, host OS and other possibly influential variables are captured and kept recorded. </p> </li> <li> <p> Immutable - once a test is run it cannot be easily overwritten or modified accidentally. </p> </li> <li> <p> ~~Repeatable - test results can be recreated in the future using all ~~the original variables.~~~~ Repeatable - test results can be recreated in the future using all the original variables. </p> </li> <li> <p> ~~Relocatable - the testsuite or automation area can be checked out ~~and the tests run anywhere in the disk hierarchy.~~~~ Relocatable - the testsuite or automation area can be checked out and the tests run anywhere in the disk hierarchy. </p> </li> <li> <p> ~~Encapsulated - the tests run in self-contained directories and all ~~inputs~~ and outputs to the process can be found in the run areas.~~ Encapsulated - the tests run in self-contained directories and all inputs and outputs to the process can be found in the run areas. </p> </li> <li> <p> ~~Deployable - a testsuite is self-contained and can be bundled with ~~a software project.~~~~ Deployable - a testsuite is self-contained and can be bundled with a software project and easily used by others with little to no setup burden. </p> </li> </ul></div> </div> </div> <div class="sect1"> <h2 id="_megatest_architecture">Megatest Architecture</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_data_separation">Data separation</h3> <div class="paragraph"><p>All data to specify the tests and configure the system is stored in plain text config files. All system state is stored in an sqlite3 database.</p></div> </div> <div class="sect2"> <h3 id="_distributed_compute">Distributed Compute</h3> <div class="paragraph"><p>Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to call home to your master ~~sqlite3 database. Megatest has been used with the Intel Netbatch ~~system, the~~ lsf batch system and it should be ~~possible to use it with~~ other similar systems.</p></div>~~ sqlite3 database. Megatest has been used with the Intel Netbatch and lsf (also known as openlava) batch systems and it should be straightforward to use it with other similar systems.</p></div> </div> </div> </div> <div class="sect1"> <h2 id="_overview">Overview</h2> <div class="sectionbody"> <div class="sect2"> <h3 id="_stand_alone_megatest_area">Stand-alone Megatest Area</h3> <div class="paragraph"><p>A single, stand-alone, Megatest based testsuite or "area" is sufficient for most validation, automation and build problems.</p></div> <div class="imageblock"> <div class="content"> <img src="megatest-stand-alone-area.png" alt="Static"> </div> </div> <div class="paragraph"><p>Megatest is designed as a distributed or decoupled system. This means you can run the areas stand-alone with no additional infrastructure. I.e. there are no databases, web servers or other centralized resources needed. However as your needs grow you can integrate multiple areas into a bigger system.</p></div> <div class="sect3"> <h4 id="_component_descriptions">Component Descriptions</h4> <div class="olist arabic"><ol class="arabic"> <li> <p> Multi-area dashboard and xterm. A gui (the dashboard) is usually the best option for controlling and launching runs but all operations can also be done from the commandline. Note: The not yet released multi-area dashboard replaces the old dashboard for browsing and controlling runs but for managing a single area the old dashboard works very well. </p> </li> <li> <p> Area/testsuite. This is your testsuite or automation definition and consists of the information in megatest.config, runconfigs.config and your testconfigs along with any custom scripting that can’t be done with the native Megatest features. </p> </li> <li> <p> If your testsuite or build automation is too large to run on a single instance you can distribute your jobs into a compute server pool. The only current requirements are password-less ssh access and a network filesystem. </p> </li> </ol></div> </div> </div> <div class="sect2"> <h3 id="_full_system_architecture">Full System Architecture</h3> <div class="imageblock"> <div class="content"> <img src="megatest-system-architecture.png" alt="Static"> </div> </div> </div> </div> </div> <div class="sect1"> <h2 id="_installation">Installation</h2> <div class="sectionbody"> <div class="sect2">
︙
2967 2968 2969 2970 2971 2972 2973 ~~2974~~ 2975 2976 2977 2978	3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042	- +	</div> </div> </div> <div id="footnotes"><hr></div> <div id="footer"> <div id="footer-text"> Version 1.0<br> ~~Last updated 2018-10-24 15:~~59:59~~ PDT~~ Last updated 2018-11-27 13:07:36 PST </div> </div> </body> </html>