diff options
Diffstat (limited to 'infra/libkookie/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml')
| -rw-r--r-- | infra/libkookie/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml | 453 |
1 files changed, 453 insertions, 0 deletions
diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml new file mode 100644 index 000000000000..cab4c067e0d3 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/development/writing-nixos-tests.xml @@ -0,0 +1,453 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-writing-nixos-tests"> + <title>Writing Tests</title> + + <para> + A NixOS test is a Nix expression that has the following structure: +<programlisting> +import ./make-test-python.nix { + + # Either the configuration of a single machine: + machine = + { config, pkgs, ... }: + { <replaceable>configuration…</replaceable> + }; + + # Or a set of machines: + nodes = + { <replaceable>machine1</replaceable> = + { config, pkgs, ... }: { <replaceable>…</replaceable> }; + <replaceable>machine2</replaceable> = + { config, pkgs, ... }: { <replaceable>…</replaceable> }; + … + }; + + testScript = + '' + <replaceable>Python code…</replaceable> + ''; +} +</programlisting> + The attribute <literal>testScript</literal> is a bit of Python code that + executes the test (described below). During the test, it will start one or + more virtual machines, the configuration of which is described by the + attribute <literal>machine</literal> (if you need only one machine in your + test) or by the attribute <literal>nodes</literal> (if you need multiple + machines). For instance, + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename> + only needs a single machine to test whether users can log in on the virtual + console, whether device ownership is correctly maintained when switching + between consoles, and so on. On the other hand, + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs/simple.nix">nfs/simple.nix</filename>, + which tests NFS client and server functionality in the Linux kernel + (including whether locks are maintained across server crashes), requires + three machines: a server and two clients. + </para> + + <para> + There are a few special NixOS configuration options for test VMs: +<!-- FIXME: would be nice to generate this automatically. --> + <variablelist> + <varlistentry> + <term> + <option>virtualisation.memorySize</option> + </term> + <listitem> + <para> + The memory of the VM in megabytes. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>virtualisation.vlans</option> + </term> + <listitem> + <para> + The virtual networks to which the VM is connected. See + <filename + xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename> + for an example. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <option>virtualisation.writableStore</option> + </term> + <listitem> + <para> + By default, the Nix store in the VM is not writable. If you enable this + option, a writable union file system is mounted on top of the Nix store + to make it appear writable. This is necessary for tests that run Nix + operations that modify the store. + </para> + </listitem> + </varlistentry> + </variablelist> + For more options, see the module + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>. + </para> + + <para> + The test script is a sequence of Python statements that perform various + actions, such as starting VMs, executing commands in the VMs, and so on. Each + virtual machine is represented as an object stored in the variable + <literal><replaceable>name</replaceable></literal> if this is also the + identifier of the machine in the declarative config. + If you didn't specify multiple machines using the <literal>nodes</literal> + attribute, it is just <literal>machine</literal>. + The following example starts the machine, waits until it has finished booting, + then executes a command and checks that the output is more-or-less correct: +<programlisting> +machine.start() +machine.wait_for_unit("default.target") +if not "Linux" in machine.succeed("uname"): + raise Exception("Wrong OS") +</programlisting> + The first line is actually unnecessary; machines are implicitly started when + you first execute an action on them (such as <literal>wait_for_unit</literal> + or <literal>succeed</literal>). If you have multiple machines, you can speed + up the test by starting them in parallel: +<programlisting> +start_all() +</programlisting> + </para> + + <para> + The following methods are available on machine objects: + <variablelist> + <varlistentry> + <term> + <methodname>start</methodname> + </term> + <listitem> + <para> + Start the virtual machine. This method is asynchronous — it does not + wait for the machine to finish booting. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>shutdown</methodname> + </term> + <listitem> + <para> + Shut down the machine, waiting for the VM to exit. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>crash</methodname> + </term> + <listitem> + <para> + Simulate a sudden power failure, by telling the VM to exit immediately. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>block</methodname> + </term> + <listitem> + <para> + Simulate unplugging the Ethernet cable that connects the machine to the + other machines. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>unblock</methodname> + </term> + <listitem> + <para> + Undo the effect of <methodname>block</methodname>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>screenshot</methodname> + </term> + <listitem> + <para> + Take a picture of the display of the virtual machine, in PNG format. The + screenshot is linked from the HTML log. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>get_screen_text</methodname> + </term> + <listitem> + <para> + Return a textual representation of what is currently visible on the + machine's screen using optical character recognition. + </para> + <note> + <para> + This requires passing <option>enableOCR</option> to the test attribute + set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>send_monitor_command</methodname> + </term> + <listitem> + <para> + Send a command to the QEMU monitor. This is rarely used, but allows doing + stuff such as attaching virtual USB disks to a running machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>send_key</methodname> + </term> + <listitem> + <para> + Simulate pressing keys on the virtual keyboard, e.g., + <literal>send_key("ctrl-alt-delete")</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>send_chars</methodname> + </term> + <listitem> + <para> + Simulate typing a sequence of characters on the virtual keyboard, e.g., + <literal>send_chars("foobar\n")</literal> will type the string + <literal>foobar</literal> followed by the Enter key. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>execute</methodname> + </term> + <listitem> + <para> + Execute a shell command, returning a list + <literal>(<replaceable>status</replaceable>, + <replaceable>stdout</replaceable>)</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>succeed</methodname> + </term> + <listitem> + <para> + Execute a shell command, raising an exception if the exit status is not + zero, otherwise returning the standard output. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>fail</methodname> + </term> + <listitem> + <para> + Like <methodname>succeed</methodname>, but raising an exception if the + command returns a zero status. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_until_succeeds</methodname> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it succeeds. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_until_fails</methodname> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it fails. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_unit</methodname> + </term> + <listitem> + <para> + Wait until the specified systemd unit has reached the “active” state. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_file</methodname> + </term> + <listitem> + <para> + Wait until the specified file exists. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_open_port</methodname> + </term> + <listitem> + <para> + Wait until a process is listening on the given TCP port (on + <literal>localhost</literal>, at least). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_closed_port</methodname> + </term> + <listitem> + <para> + Wait until nobody is listening on the given TCP port. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_x</methodname> + </term> + <listitem> + <para> + Wait until the X11 server is accepting connections. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_text</methodname> + </term> + <listitem> + <para> + Wait until the supplied regular expressions matches the textual contents + of the screen by using optical character recognition (see + <methodname>get_screen_text</methodname>). + </para> + <note> + <para> + This requires passing <option>enableOCR</option> to the test attribute + set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_console_text</methodname> + </term> + <listitem> + <para> + Wait until the supplied regular expressions match a line of the serial + console output. This method is useful when OCR is not possibile or + accurate enough. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>wait_for_window</methodname> + </term> + <listitem> + <para> + Wait until an X11 window has appeared whose name matches the given + regular expression, e.g., <literal>wait_for_window("Terminal")</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>copy_from_host</methodname> + </term> + <listitem> + <para> + Copies a file from host to machine, e.g., + <literal>copy_from_host("myfile", "/etc/my/important/file")</literal>. + </para> + <para> + The first argument is the file on the host. The file needs to be + accessible while building the nix derivation. The second argument is the + location of the file on the machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <methodname>systemctl</methodname> + </term> + <listitem> + <para> + Runs <literal>systemctl</literal> commands with optional support for + <literal>systemctl --user</literal> + </para> + <para> +<programlisting> +machine.systemctl("list-jobs --no-pager") # runs `systemctl list-jobs --no-pager` +machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` +</programlisting> + </para> + </listitem> + </varlistentry> + </variablelist> + </para> + + <para> + To test user units declared by <literal>systemd.user.services</literal> the + optional <literal>user</literal> argument can be used: +<programlisting> +machine.start() +machine.wait_for_x() +machine.wait_for_unit("xautolock.service", "x-session-user") +</programlisting> + This applies to <literal>systemctl</literal>, <literal>get_unit_info</literal>, + <literal>wait_for_unit</literal>, <literal>start_job</literal> and + <literal>stop_job</literal>. + </para> + + <para> + For faster dev cycles it's also possible to disable the code-linters (this shouldn't + be commited though): +<programlisting> +import ./make-test-python.nix { + skipLint = true; + machine = + { config, pkgs, ... }: + { <replaceable>configuration…</replaceable> + }; + + testScript = + '' + <replaceable>Python code…</replaceable> + ''; +} +</programlisting> + </para> +</section> |