diff options
Diffstat (limited to 'nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml')
| -rw-r--r-- | nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml | 421 |
1 files changed, 421 insertions, 0 deletions
diff --git a/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml b/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml new file mode 100644 index 00000000000..6be2d0a4d23 --- /dev/null +++ b/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml @@ -0,0 +1,421 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-writing-nixos-tests"> + <title>Writing Tests</title> + + <para> + A NixOS test is a Nix expression that has the following structure: +<programlisting> +import ./make-test.nix { + + # Either the configuration of a single machine: + machine = + { config, pkgs, ... }: + { <replaceable>configuration…</replaceable> + }; + + # Or a set of machines: + nodes = + { <replaceable>machine1</replaceable> = + { config, pkgs, ... }: { <replaceable>…</replaceable> }; + <replaceable>machine2</replaceable> = + { config, pkgs, ... }: { <replaceable>…</replaceable> }; + … + }; + + testScript = + '' + <replaceable>Perl code…</replaceable> + ''; +} +</programlisting> + The attribute <literal>testScript</literal> is a bit of Perl code that + executes the test (described below). During the test, it will start one or + more virtual machines, the configuration of which is described by the + attribute <literal>machine</literal> (if you need only one machine in your + test) or by the attribute <literal>nodes</literal> (if you need multiple + machines). For instance, + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename> + only needs a single machine to test whether users can log in on the virtual + console, whether device ownership is correctly maintained when switching + between consoles, and so on. On the other hand, + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs.nix">nfs.nix</filename>, + which tests NFS client and server functionality in the Linux kernel + (including whether locks are maintained across server crashes), requires + three machines: a server and two clients. + </para> + + <para> + There are a few special NixOS configuration options for test VMs: +<!-- FIXME: would be nice to generate this automatically. --> + <variablelist> + <varlistentry> + <term> + <option>virtualisation.memorySize</option> + </term> + <listitem> + <para> + The memory of the VM in megabytes. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>virtualisation.vlans</option> + </term> + <listitem> + <para> + The virtual networks to which the VM is connected. See + <filename + xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename> + for an example. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>virtualisation.writableStore</option> + </term> + <listitem> + <para> + By default, the Nix store in the VM is not writable. If you enable this + option, a writable union file system is mounted on top of the Nix store + to make it appear writable. This is necessary for tests that run Nix + operations that modify the store. + </para> + </listitem> + </varlistentry> + </variablelist> + For more options, see the module + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>. + </para> + + <para> + The test script is a sequence of Perl statements that perform various + actions, such as starting VMs, executing commands in the VMs, and so on. Each + virtual machine is represented as an object stored in the variable + <literal>$<replaceable>name</replaceable></literal>, where + <replaceable>name</replaceable> is the identifier of the machine (which is + just <literal>machine</literal> if you didn’t specify multiple machines + using the <literal>nodes</literal> attribute). For instance, the following + starts the machine, waits until it has finished booting, then executes a + command and checks that the output is more-or-less correct: +<programlisting> +$machine->start; +$machine->waitForUnit("default.target"); +$machine->succeed("uname") =~ /Linux/ or die; +</programlisting> + The first line is actually unnecessary; machines are implicitly started when + you first execute an action on them (such as <literal>waitForUnit</literal> + or <literal>succeed</literal>). If you have multiple machines, you can speed + up the test by starting them in parallel: +<programlisting> +startAll; +</programlisting> + </para> + + <para> + The following methods are available on machine objects: + <variablelist> + <varlistentry> + <term> + <methodname>start</methodname> + </term> + <listitem> + <para> + Start the virtual machine. This method is asynchronous — it does not + wait for the machine to finish booting. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>shutdown</methodname> + </term> + <listitem> + <para> + Shut down the machine, waiting for the VM to exit. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>crash</methodname> + </term> + <listitem> + <para> + Simulate a sudden power failure, by telling the VM to exit immediately. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>block</methodname> + </term> + <listitem> + <para> + Simulate unplugging the Ethernet cable that connects the machine to the + other machines. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>unblock</methodname> + </term> + <listitem> + <para> + Undo the effect of <methodname>block</methodname>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>screenshot</methodname> + </term> + <listitem> + <para> + Take a picture of the display of the virtual machine, in PNG format. The + screenshot is linked from the HTML log. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>getScreenText</methodname> + </term> + <listitem> + <para> + Return a textual representation of what is currently visible on the + machine's screen using optical character recognition. + </para> + <note> + <para> + This requires passing <option>enableOCR</option> to the test attribute + set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>sendMonitorCommand</methodname> + </term> + <listitem> + <para> + Send a command to the QEMU monitor. This is rarely used, but allows doing + stuff such as attaching virtual USB disks to a running machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>sendKeys</methodname> + </term> + <listitem> + <para> + Simulate pressing keys on the virtual keyboard, e.g., + <literal>sendKeys("ctrl-alt-delete")</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>sendChars</methodname> + </term> + <listitem> + <para> + Simulate typing a sequence of characters on the virtual keyboard, e.g., + <literal>sendKeys("foobar\n")</literal> will type the string + <literal>foobar</literal> followed by the Enter key. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>execute</methodname> + </term> + <listitem> + <para> + Execute a shell command, returning a list + <literal>(<replaceable>status</replaceable>, + <replaceable>stdout</replaceable>)</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>succeed</methodname> + </term> + <listitem> + <para> + Execute a shell command, raising an exception if the exit status is not + zero, otherwise returning the standard output. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>fail</methodname> + </term> + <listitem> + <para> + Like <methodname>succeed</methodname>, but raising an exception if the + command returns a zero status. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitUntilSucceeds</methodname> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it succeeds. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitUntilFails</methodname> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it fails. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForUnit</methodname> + </term> + <listitem> + <para> + Wait until the specified systemd unit has reached the “active” state. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForFile</methodname> + </term> + <listitem> + <para> + Wait until the specified file exists. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForOpenPort</methodname> + </term> + <listitem> + <para> + Wait until a process is listening on the given TCP port (on + <literal>localhost</literal>, at least). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForClosedPort</methodname> + </term> + <listitem> + <para> + Wait until nobody is listening on the given TCP port. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForX</methodname> + </term> + <listitem> + <para> + Wait until the X11 server is accepting connections. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForText</methodname> + </term> + <listitem> + <para> + Wait until the supplied regular expressions matches the textual contents + of the screen by using optical character recognition (see + <methodname>getScreenText</methodname>). + </para> + <note> + <para> + This requires passing <option>enableOCR</option> to the test attribute + set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>waitForWindow</methodname> + </term> + <listitem> + <para> + Wait until an X11 window has appeared whose name matches the given + regular expression, e.g., <literal>waitForWindow(qr/Terminal/)</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>copyFileFromHost</methodname> + </term> + <listitem> + <para> + Copies a file from host to machine, e.g., + <literal>copyFileFromHost("myfile", "/etc/my/important/file")</literal>. + </para> + <para> + The first argument is the file on the host. The file needs to be + accessible while building the nix derivation. The second argument is the + location of the file on the machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>systemctl</methodname> + </term> + <listitem> + <para> + Runs <literal>systemctl</literal> commands with optional support for + <literal>systemctl --user</literal> + </para> + <para> +<programlisting> +$machine->systemctl("list-jobs --no-pager"); // runs `systemctl list-jobs --no-pager` +$machine->systemctl("list-jobs --no-pager", "any-user"); // spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` +</programlisting> + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + + <para> + To test user units declared by <literal>systemd.user.services</literal> the + optional <literal>$user</literal> argument can be used: +<programlisting> +$machine->start; +$machine->waitForX; +$machine->waitForUnit("xautolock.service", "x-session-user"); +</programlisting> + This applies to <literal>systemctl</literal>, <literal>getUnitInfo</literal>, + <literal>waitForUnit</literal>, <literal>startJob</literal> and + <literal>stopJob</literal>. + </para> +</section> |