Tutorial¶
The purpose of this tutorial is to give you a full walkthrough from test package preparation to Test Run creation until test result polling.
Test Run Preparation: Binary Packaging¶
Let us create a very simple Test Run that consists of one boot item that shall be booted on all hardware platforms of our SoTest cluster.
We begin with a Linux kernel and a companion initial RAM disk that can be booted
together.
The RAM disk contains the dmidecode
app and will print many machine details
along with SoTest protocol output.
This kernel and initial RAM disk act as the build output of some example project.
Download them in a ZIP bundle here: tutorial-binaries.zip If interested, see how you can build them yourself at the end of this page.
The ZIP contains the two files:
bzImage
(the kernel image)initrd
(the initial RAM disk)
Test Run Creation: Project Config¶
Now we have all files together and can prepare a description for SoTest on
how to boot these files as a boot item.
Save the following text as project_config.yaml
:
boot_items:
- name: "Linux + initial RAM disk test"
boot_source:
bios:
exec: bzImage console=@{linux_terminal}
load:
- initrd
As described in the project configuration doc page, the
project configuration consists at least of a list of boot items under the field
name boot_items
.
Our boot item has a name, an exec line and a load line.
The exec line says that we are going to boot the bzimage-5.4.8
Linux kernel
file, and supply it with the command line console=@{linux_terminal}
.
The expression @{linux_terminal}
is a platform specific variable
.
The administrator of a SoTest cluster can provide all configured
hardware platforms with their own definitions of key-value pairs.
In this case, the variable @{linux_terminal}
is substituted by ttyS0
on one
machine, and ttyS4
on another machine, depending on which serial device is
hardwired in the setup for test output.
The load
line says that we are also loading the initial RAM disk into memory
so that the Linux kernel can discover and mount it at boot time.
Test Run Creation¶
Both our binary package and project config need to be sent to the SoTest server.
As SoTest does not store binaries itself but only where they are stored, we need to upload them somewhere (if not done by our CI automatically).
Now we can do a SoTest REST API call in order to create a new Test Run:
$ curl --netrc \
-F 'boot_files=@downloaded-tutorial-binaries.zip' \
-F 'url=https://docs.sotest.io/tutorial' \
-F 'name=Tutorial Test Run' \
-F 'config=@project_config.yaml' \
-X POST \
https://opensource.sotest.io/test_runs
45
We can note down the 45
as this is the ID of our Test Run.
Test Run Results¶
If the hardware platform resources are currently free, we can observe how they pick up our jobs:
While the testing takes a few minutes (which depends on how fast the hardware boots and how long our actual test programs take), we can use the REST API to poll for the Test Run progress, which is useful for CIs and generally scripts:
$ curl https://opensource.sotest.io/test_runs/45/status
"unfinished"
$ curl https://opensource.sotest.io/test_runs/45/status
"unfinished"
$ curl https://opensource.sotest.io/test_runs/45/status
"success"
In the Test Run results page, we can select individual machines and look at their output, which looks like this:
[ 3.781442] Run /init as init process
[ 4.448661] tsc: Refined TSC clocksource calibration: 2100.014 MHz
[ 4.454848] clocksource: tsc: mask: 0xffffffffffffffff
max_cycles: 0x1e453ddc726, max_idle_ns: 440795276965 ns
[ 4.464890] clocksource: Switched to clocksource tsc
Hello! Running Linux (none) 5.4.8 #1-NixOS SMP Sat Jan 4 18:19:19 UTC 2020
x86_64 GNU/Linux
SoTest protocol output
SOTEST VERSION 1 BEGIN 1
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.
75 structures occupying 2919 bytes.
Table at 0x000EB550.
Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
Vendor: American Megatrends Inc.
Version: 2.0a
Release Date: 02/01/2013
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 11264 kB
Characteristics:
PCI is supported
BIOS is upgradeable
# lots of output omitted....
SOTEST SUCCESS
SOTEST END
The lines prefixed with SOTEST ...
are the ones that the SoTest interpreter
uses to distinguish successful from failing tests.
Appendix: Build linux kernel and initial RAM disk yourself¶
In order to provide all source code and information needed to reproduce the binaries, we provide nix expressions that can be built locally on Linux at the following URL:
https://github.com/cyberus-technology/cbspkgs-public
You will need to install nix
before executing the build command, which you can
download at https://nixos.org/nix/.
With nix installed, you can run:
$ nix-build \
https://github.com/cyberus-technology/cbspkgs-public/tarball/73ecf3928e2fb06b3732d60aa86a38dda3ca3481 \
-A initrds.initrds.dmidecode \
-A sotest-kernels.linux_latest
# ... lots of output omitted
/nix/store/7b8cm04m1hadiy9qvrpp4w0iy79b59ys-initrd
/nix/store/50hkk8796wx5616dw906wsww7s89x3fp-linux-5.4.8
The files will then be located in the paths that this command emitted on
stdout
.
You can zip them excluding the nix store paths as follows:
zip --junk-paths sotest-binaries.zip /nix/store/<hash>-initrd/initrd /nix/store/<hash>-linux-<version>/bzImage