Tutorial¶

The purpose of this tutorial is to give you a full walkthrough from test package preparation to Test Run creation until test result polling.

Test Run Preparation: Binary Packaging¶

Let us create a very simple Test Run that consists of one boot item that shall be booted on all hardware platforms of our SoTest cluster.

We begin with a Linux kernel and a companion initial RAM disk that can be booted together. The RAM disk contains the dmidecode app and will print many machine details along with SoTest protocol output.

This kernel and initial RAM disk act as the build output of some example project.

Download them in a ZIP bundle here: tutorial-binaries.zip If interested, see how you can build them yourself at the end of this page.

The ZIP contains the two files:

bzImage (the kernel image)
initrd (the initial RAM disk)

Test Run Creation: Project Config¶

Now we have all files together and can prepare a description for SoTest on how to boot these files as a boot item. Save the following text as project_config.yaml:

boot_items:
  - name: "Linux + initial RAM disk test"
    boot_source:
      bios:
        exec: bzImage console=@{linux_terminal}
        load:
          - initrd

As described in the project configuration doc page, the project configuration consists at least of a list of boot items under the field name boot_items.

Our boot item has a name, an exec line and a load line. The exec line says that we are going to boot the bzimage-5.4.8 Linux kernel file, and supply it with the command line console=@{linux_terminal}.

The expression @{linux_terminal} is a platform specific variable . The administrator of a SoTest cluster can provide all configured hardware platforms with their own definitions of key-value pairs. In this case, the variable @{linux_terminal} is substituted by ttyS0 on one machine, and ttyS4 on another machine, depending on which serial device is hardwired in the setup for test output.

The load line says that we are also loading the initial RAM disk into memory so that the Linux kernel can discover and mount it at boot time.

Test Run Creation¶

Both our binary package and project config need to be sent to the SoTest server.

As SoTest does not store binaries itself but only where they are stored, we need to upload them somewhere (if not done by our CI automatically).

Now we can do a SoTest REST API call in order to create a new Test Run:

$ curl --netrc \
     -F 'boot_files=@downloaded-tutorial-binaries.zip' \
     -F 'url=https://docs.sotest.io/tutorial' \
     -F 'name=Tutorial Test Run' \
     -F 'config=@project_config.yaml' \
     -X POST \
     https://opensource.sotest.io/test_runs
45

We can note down the 45 as this is the ID of our Test Run.

Test Run Results¶

If the hardware platform resources are currently free, we can observe how they pick up our jobs:

Animated image of the Test Run progress bar

While the testing takes a few minutes (which depends on how fast the hardware boots and how long our actual test programs take), we can use the REST API to poll for the Test Run progress, which is useful for CIs and generally scripts:

$ curl https://opensource.sotest.io/test_runs/45/status
"unfinished"
$ curl https://opensource.sotest.io/test_runs/45/status
"unfinished"
$ curl https://opensource.sotest.io/test_runs/45/status
"success"

In the Test Run results page, we can select individual machines and look at their output, which looks like this:

[    3.781442] Run /init as init process
[    4.448661] tsc: Refined TSC clocksource calibration: 2100.014 MHz
[    4.454848] clocksource: tsc: mask: 0xffffffffffffffff
max_cycles: 0x1e453ddc726, max_idle_ns: 440795276965 ns
[    4.464890] clocksource: Switched to clocksource tsc
Hello! Running Linux (none) 5.4.8 #1-NixOS SMP Sat Jan 4 18:19:19 UTC 2020
x86_64 GNU/Linux

SoTest protocol output
SOTEST VERSION 1 BEGIN 1

# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.
75 structures occupying 2919 bytes.
Table at 0x000EB550.

Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
    Vendor: American Megatrends Inc.
    Version: 2.0a
    Release Date: 02/01/2013
    Address: 0xF0000
    Runtime Size: 64 kB
    ROM Size: 11264 kB
    Characteristics:
        PCI is supported
        BIOS is upgradeable
# lots of output omitted....

SOTEST SUCCESS
SOTEST END

The lines prefixed with SOTEST ... are the ones that the SoTest interpreter uses to distinguish successful from failing tests.

Appendix: Build linux kernel and initial RAM disk yourself¶

In order to provide all source code and information needed to reproduce the binaries, we provide nix expressions that can be built locally on Linux at the following URL:

https://github.com/cyberus-technology/cbspkgs-public

You will need to install nix before executing the build command, which you can download at https://nixos.org/nix/.

With nix installed, you can run:

$ nix-build \
  https://github.com/cyberus-technology/cbspkgs-public/tarball/73ecf3928e2fb06b3732d60aa86a38dda3ca3481 \
  -A initrds.initrds.dmidecode \
  -A sotest-kernels.linux_latest

# ... lots of output omitted
/nix/store/7b8cm04m1hadiy9qvrpp4w0iy79b59ys-initrd
/nix/store/50hkk8796wx5616dw906wsww7s89x3fp-linux-5.4.8

The files will then be located in the paths that this command emitted on stdout.

You can zip them excluding the nix store paths as follows:

zip --junk-paths sotest-binaries.zip /nix/store/<hash>-initrd/initrd /nix/store/<hash>-linux-<version>/bzImage