SoTest High-Level Setup¶

The overview page already gave a short glance on the overall architecture of SoTest deployments. This section looks at SoTest deployments from the administrator perspective.

All in all, the following services are required to run SoTest:

• SoTest Web UI
• SoTest Controller app
• Database, e.g. PostgreSQL
• DHCP server, e.g. dnsmasq
• TFTP server, e.g. nginx
• optional HTTP server, e.g. nginx for speeding up large image downloads
• optional iSCSI server for serving OS images

The admin pages are structured as follows:

• this page explains the difference between Single Host and Multiple Host SoTest Instances and gives an overview over the SoTest Controller.
• Deployment with NixOS module describes how to configure the entire SoTest setup using NixOS modules
• Deployment without NixOS describes how to deploy SoTest manually without NixOS
• Configuration files describes the options in the SoTest configuration files
• Network and Boot Configuration shows how to set up the DHCP and TFTP server
• Generalized Power Interface explains SoTest configuration options for the GPI
• Step-by-Step for creating a NixOS Test VM walks you through creating a simple test VM for trying out and getting started with SoTest

Single Host SoTest Instances¶

It is possible to run SoTest on a single host or to employ multiple SoTest Controllers that reside on different host servers.

A full SoTest instance can be hosted on one host:

Both the Web UI and Controller services run on the same machine and communicate with the same PostgreSQL®¹ database instance.

Multi Controller SoTest Instances¶

With test hardware in multiple racks that are located at different sites, it is possible to have multiple Controllers on different hosts.

In such cases, all Controllers still connect to the same database but can be managed by different teams.

Controller Setup¶

SoTest Controller instances are the actual work horses of SoTest deployments: Being configured for certain test hardware, they recurringly poll the SoTest database for new boot items that they can execute on the hardware. After test execution, they post back the results.

Typically, dedicated servers are used to drive bigger racks of test hardware. However, it is also possible to convert a developer laptop that is wired with test hardware to become a SoTest Controller.

A SoTest Controller setup usually looks like this:

The SoTest Controller host runs the following services:

• SoTest Controller: The SoTest Controller service recurringly queries the SoTest database for open boot items to test and drives the test hardware to execute those.
• DHCP Server: A DHCP service needs to be configured in order to give all wired test hardware IP addresses when they are powered up. The SoTest Controller is configured to know which hardware has which MAC address and which IP address the hardware obtains.
• TFTP Server: The TFTP Service serves the binaries that shall be booted on the hardware after being powered up. This hardware differs for each specific test. The SoTest Controller puts the right binaries for the right test/hardware into the TFTP root folder.
• HTTP Server: As the HTTP protocol is faster than TFTP for serving larger binaries, an HTTP server can be optionally configured to serve boot binaries.

In order to drive a commodity PC/Server as test hardware, the SoTest Controller needs to be able to drive it using three different interfaces:

• Power Interface: To power the test hardware on or off before/after a test, SoTest needs remote access to the machine. This is usually done via AMT, IPMI, network-switchable power sockets (Antrax), or the Generalized Power Interface.
• Boot Files Interface: After the test hardware has been powered up, it obtains test binaries via network using the PXE protocol.
• Log Interface: Test output which follows the SoTest protocol is collected via this interface. The SoTest Controller interprets success/failure indicators out of its content. Serial interfaces can be used for this, or XHCI debug cables.

For more information on the specific interfaces that are supported and can be configured per machine, please refer to the machine configuration section of the documentation.

In the example diagram, the hardware platforms are configured as follows:

• HW1:
  • Boot files and PXE Boot: The Controller power cycles the test HW via IPMI. Both PXE boot and IPMI work via the same network cable.
  • Serial: The Controller connects to the legacy COM port of the test HW using a USB-serial adaptor.
• HW2:
  • Boot files, power control, as well as serial output are served using Intel AMT over network.
• HW3:
  • Boot files are served over network.
  • Power: This machine provides no network-controllable power interface, so its power button is directly actuated using optocouplers in the Generalized Power Interface, which is controlled via network.
  • Serial: This device uses an XHCI debug interface, which is transposed to a plain network socket. The SoTest Controller connects to it via network.