Deployment with NixOS Module¶
This page describes how to configure SoTest services via the NixOS module. Deployment of SoTest using the NixOS module is the easy and best-supported way to deploy SoTest.
When using NixOS, all system configuration options such as installed services
and their configuration, created user accounts, or network options are placed
in a configuration.nix
file. This means that the entire system configuration
can be stored transparently, versioned and even rolled back. Additional services
can be integrated using NixOS modules. The modules do not only control
configuration parameters but also service configuration via systemd. SoTest
provides such a module in the file sotest.nix
.
Administrators who choose this method of SoTest deployment can largely ignore the per-service documentation, as this is all already handled for them via the NixOS module.
SoTest module configuration¶
The first step to using SoTest on a native NixOS host is importing the
sotest.nix
module.
imports = [
"<path to sotest.nix>"
];
Then the service can be configured as described in the following template by
specifying the options for sotest-db
, sotest-webui
and sotest-controller
.
An exhaustive list of all SoTest NixOS module options along with descriptions is
presented at the end of this document page.
services = {
# The database configuration used by the sotest-webui and the
# sotest-controller. The database itself needs to be configured somewhere
# else.
sotest-db = {
host = "<hostname of database>";
port = 1234;
user = "<username for database>";
password = "<password for user>";
database = "<database name>";
};
# The SoTest Web UI
sotest-webui = {
enable = true;
openFirewall = true;
port = 1235;
autoMigration = true;
# Parameters for test data upload and its garbage collection
storageDir = "/path/to/storage/";
storageUrl = "http://host/storage/url";
storageGCSizeStopMB = 1000;
storageGCSizeTriggerMB = 2000;
# You may also configure user logins (see sotest-webui.logins)
# or enable prediction (sotest-webui.prediction) here
};
# The SoTest Controller
sotest-controller = {
enable = true;
binaryTmpDir = "/tmp/sotest-binaries";
tftpFolder = "/var/lib/sotest_controller/tftp_root";
machines = [
# ...
];
};
};
The configuration of the used machine
s is parsed by the SoTest Controller at
system startup and not handled by the NixOS module. You can find the
documentation for the respective fields
here. When using the SoTest NixOS
module, the configuration names and their usage remains the same as when using
YAML. However, you need to adhere to the nix language syntax and you can
therefore benefit from language features such as recursive sets for
accessing previously defined variables (mac_address
or tftpFolder
) that you
would have to type multiple times when using YAML.
A machine configuration might look something like this:
machines = [
{
id = "machine_id";
name = "machine name";
tags = [ "tag1" "tag2" ];
power_interface = "none";
mac = "02-03-04-05-06-07";
config_vars = [ "name = value" "name2 = value2" ];
serial_interfaces = [
{
type = "custom";
cmd = "qemu-system-x86_64 [...]";
}
];
}
{
id = "machine_id_2";
name = "machine name 2";
tags = [ "laptop" ];
mac = "02-03-04-05-06-08";
power_interface = "ipmi";
ipmi_ip = "192.168.1.123";
ipmi_user = "ADMIN";
ipmi_pass = "ADMINPW";
serial_interfaces = [
{
type = "native";
serial_file = "/dev/tty.lenovoserial";
}
];
}
];
For full example config files, please refer to the Step-by-Step VM guide and to the SoTest Deployment.
Database configuration¶
Setting up the database is very straightforward in NixOS. While the internal SoTest production environment uses PostgreSQL, using a different database for your setup is also possible.
The NixOS configuration allows you to read the values (such as port, username,
or database) you configured for the SoTest services and re-use them for the
database configuration. This way you can, for example, specify the database
port using port = config.services.sotest-db.port;
instead of declaring it in
two places.
An example that uses PostgreSQL can be found in the Step-by-Step VM guide.
Network Configuration¶
Network configuration on a new SoTest instance can be delayed until everything
else works. For general steps and explanations, first check out Network Configuration.
With the SoTest NixOS module, it’s possible to integrate the dnsmasq
configuration shown there and then add some NixOS-specific options to your
configuration.nix
.
The dnsmasq.service
is enabled and configured using the following lines in the
NixOS configuration:
services.dnsmasq = {
enable = true;
resolveLocalQueries = false;
extraConfig = '';
};
extraConfig
is where the general configuration (e.g. dhcp-host
assignments)
goes.
The port configuration can be done using the networking
module. The following
example assumes that the test machines are connected to eth1
, the webserver
runs on port 80
, and the database on port 5432
. UDP ports 67
and 69
are
used for DHCP and TFTP respectively.
networking = {
firewall = {
allowedTCPPorts = [ 80 ];
allowedUDPPorts = [ ];
};
interfaces.eth1 = {
useDHCP = false;
ipv4.addresses = [ { address = "192.168.1.100"; prefixLength = 24; } ];
};
firewall.interfaces.eth1 = {
allowedUDPPorts = [ 67 69 ];
allowedTCPPorts = [ 80 5432 ];
};
};
The SoTest NixOS module creates the boot configuration files when it’s installed
and then automatically places them in the tftpFolder
when the Controller
starts up. The TFTP server is then responsible for serving the boot
configuration files to the test machines. Because NixOS uses a restrictive
service configuration with systemd by default, it may be necessary to disable
the use of a PrivateTmp
in the service config:
systemd.services.dnsmasq = {
# Required so that dnsmasq can access the tftpFolder
serviceConfig.PrivateTmp = lib.mkForce false;
};
Prediction configuration¶
The failure probability prediction can be enabled to optimize test execution.
The default approach is to enable the services.sotest-model-generation
module
and the services.sotest-webui.prediction
field in the Web UI module.
The sotest-model-generation
service evaluates past Test Runs and stores
failure probability information for each of the Execution Items in a file
named model.pkl
. The services.sotest-webui.prediction
then uses the given
command at services.sotest-webui.predictionCommand
and the data from
model.pkl
to predict which upcoming Test Execution Items are most likely to
fail. SoTest’s scheduling takes this into account and schedules these first.
A simple configuration might look like this:
# The SoTest Web UI
sotest-webui = {
enable = true;
openFirewall = true;
port = 5000;
autoMigration = true;
# Set to true to enable failure probability prediction. Either enable
# sotest-model-generation or provide your own `predictionCommand`.
prediction = true;
storageDir = "/path/to/storage/";
storageUrl = "http://host/storage/url";
};
# Enable to regularly generate a prediction model from the database.
sotest-model-generation = {
enable = true;
nJobs = 1;
timer = {
enable = true;
interval = "Sat 22:00"; # execute service each Saturday at 10pm
};
};
Enabling the timer
option will lead to regular updates to the model file by
executing the service in the given interval
. The format is described in
systemd.time(7).
The nJobs
option defines the number of CPUs used to run the model
generation.
If desired, the predictionCommand
can be changed. Please have a look at
machine learning configuration
for the necessary format that the predictionCommand
script needs to output.
For more details about each option please have a look at the last chapter of this page: SoTest NixOS Module Options.
SoTest NixOS Module Options¶
This section describes all existing SoTest NixOS module options.
/nix/store/0fh882s4849006i97lzi3jqv31wn51c3-options.md