diff options
Diffstat (limited to 'infra/libkookie/nixpkgs/nixos')
364 files changed, 7857 insertions, 2357 deletions
diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/README b/infra/libkookie/nixpkgs/nixos/doc/manual/README index 587f6275197a..120c127d7af2 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/README +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/README @@ -1,12 +1,3 @@ -To build the manual, you need Nix installed on your system (no need -for NixOS). To install Nix, follow the instructions at +Moved to: ./contributing-to-this-manual.xml. Link: - https://nixos.org/nix/download.html - -When you have Nix on your system, in the root directory of the project -(i.e., `nixpkgs`), run: - - nix-build nixos/release.nix -A manual.x86_64-linux - -When this command successfully finishes, it will tell you where the -manual got generated. +https://nixos.org/manual/nixos/unstable/#chap-contributing diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/administration/service-mgmt.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/administration/service-mgmt.xml index 1c5d48a5bcf0..863b0d47f6c7 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/administration/service-mgmt.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/administration/service-mgmt.xml @@ -6,7 +6,7 @@ <title>Service Management</title> <para> In NixOS, all system services are started and monitored using the systemd - program. Systemd is the “init” process of the system (i.e. PID 1), the + program. systemd is the “init” process of the system (i.e. PID 1), the parent of all other processes. It manages a set of so-called “units”, which can be things like system services (programs), but also mount points, swap files, devices, targets (groups of units) and more. Units can have @@ -16,10 +16,17 @@ dependencies of this unit cause all system services to be started, file systems to be mounted, swap files to be activated, and so on. </para> - <para> - The command <command>systemctl</command> is the main way to interact with - <command>systemd</command>. Without any arguments, it shows the status of - active units: + <section xml:id="sect-nixos-systemd-general"> + <title>Interacting with a running systemd</title> + <para> + The command <command>systemctl</command> is the main way to interact with + <command>systemd</command>. The following paragraphs demonstrate ways to + interact with any OS running systemd as init system. NixOS is of no + exception. The <link xlink:href="#sect-nixos-systemd-nixos">next section + </link> explains NixOS specific things worth knowing. + </para> + <para> + Without any arguments, <literal>systmctl</literal> the status of active units: <screen> <prompt>$ </prompt>systemctl -.mount loaded active mounted / @@ -28,10 +35,10 @@ sshd.service loaded active running SSH Daemon graphical.target loaded active active Graphical Interface <replaceable>...</replaceable> </screen> - </para> - <para> - You can ask for detailed status information about a unit, for instance, the - PostgreSQL database service: + </para> + <para> + You can ask for detailed status information about a unit, for instance, the + PostgreSQL database service: <screen> <prompt>$ </prompt>systemctl status postgresql.service postgresql.service - PostgreSQL Server @@ -62,11 +69,72 @@ Jan 07 15:55:57 hagbard systemd[1]: Started PostgreSQL Server. <prompt># </prompt>systemctl start postgresql.service <prompt># </prompt>systemctl restart postgresql.service </screen> - These operations are synchronous: they wait until the service has finished - starting or stopping (or has failed). Starting a unit will cause the - dependencies of that unit to be started as well (if necessary). - </para> -<!-- - cgroups: each service and user session is a cgroup + These operations are synchronous: they wait until the service has finished + starting or stopping (or has failed). Starting a unit will cause the + dependencies of that unit to be started as well (if necessary). + </para> + <!-- TODO: document cgroups, draft: + each service and user session is a cgroup -- cgroup resource management --> + - cgroup resource management --> + </section> + <section xml:id="sect-nixos-systemd-nixos"> + <title>systemd in NixOS</title> + <para> + Packages in Nixpkgs sometimes provide systemd units with them, usually in + e.g <literal>#pkg-out#/lib/systemd/</literal>. Putting such a package in + <literal>environment.systemPackages</literal> doesn't make the service + available to users or the system. + </para> + <para> + In order to enable a systemd <emphasis>system</emphasis> service with + provided upstream package, use (e.g): +<programlisting> +<xref linkend="opt-systemd.packages"/> = [ pkgs.packagekit ]; +</programlisting> + </para> + <para> + Usually NixOS modules written by the community do the above, plus take care of + other details. If a module was written for a service you are interested in, + you'd probably need only to use + <literal>services.#name#.enable = true;</literal>. These services are defined + in Nixpkgs' + <link xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/modules"> + <literal>nixos/modules/</literal> directory </link>. In case the service is + simple enough, the above method should work, and start the service on boot. + </para> + <para> + <emphasis>User</emphasis> systemd services on the other hand, should be + treated differently. Given a package that has a systemd unit file at + <literal>#pkg-out#/lib/systemd/user/</literal>, using + <xref linkend="opt-systemd.packages"/> will make you able to start the service via + <literal>systemctl --user start</literal>, but it won't start automatically on login. + <!-- TODO: Document why systemd.packages doesn't work for user services or fix this. + https://github.com/NixOS/nixpkgs/blob/2cd6594a8710a801038af2b72348658f732ce84a/nixos/modules/system/boot/systemd-lib.nix#L177-L198 + + This has been talked over at https://discourse.nixos.org/t/how-to-enable-upstream-systemd-user-services-declaratively/7649/5 + --> + However, You can imperatively enable it by adding the package's attribute to + <link linkend="opt-environment.systemPackages"> + <literal>systemd.packages</literal></link> and then do this (e.g): +<screen> +<prompt>$ </prompt>mkdir -p ~/.config/systemd/user/default.target.wants +<prompt>$ </prompt>ln -s /run/current-system/sw/lib/systemd/user/syncthing.service ~/.config/systemd/user/default.target.wants/ +<prompt>$ </prompt>systemctl --user daemon-reload +<prompt>$ </prompt>systemctl --user enable syncthing.service +</screen> + If you are interested in a timer file, use <literal>timers.target.wants</literal> + instead of <literal>default.target.wants</literal> in the 1st and 2nd command. + </para> + <para> + Using <literal>systemctl --user enable syncthing.service</literal> instead of + the above, will work, but it'll use the absolute path of + <literal>syncthing.service</literal> for the symlink, and this path is in + <literal>/nix/store/.../lib/systemd/user/</literal>. Hence + <link xlink:href="#sec-nix-gc">garbage collection</link> will remove that file + and you will wind up with a broken symlink in your systemd configuration, which + in turn will not make the service / timer start on login. + </para> + </section> </chapter> + diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/configuration.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/configuration.xml index 6eb8f50bacac..6949189b8883 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/configuration.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/configuration.xml @@ -18,10 +18,12 @@ <xi:include href="user-mgmt.xml" /> <xi:include href="file-systems.xml" /> <xi:include href="x-windows.xml" /> + <xi:include href="wayland.xml" /> <xi:include href="gpu-accel.xml" /> <xi:include href="xfce.xml" /> <xi:include href="networking.xml" /> <xi:include href="linux-kernel.xml" /> + <xi:include href="subversion.xml" /> <xi:include href="../generated/modules.xml" xpointer="xpointer(//section[@id='modules']/*)" /> <xi:include href="profiles.xml" /> <xi:include href="kubernetes.xml" /> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/gpu-accel.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/gpu-accel.xml index 95ee13f4796e..3695a5ff445a 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/gpu-accel.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/gpu-accel.xml @@ -65,16 +65,16 @@ Platform Vendor Advanced Micro Devices, Inc.</screen> <title>AMD</title> <para> - Modern AMD <link - xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics - Core Next</link> (GCN) GPUs are supported through the - <package>rocm-opencl-icd</package> package. Adding this package to - <xref linkend="opt-hardware.opengl.extraPackages"/> enables OpenCL - support: - - <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ - rocm-opencl-icd -];</programlisting> + Modern AMD <link + xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics + Core Next</link> (GCN) GPUs are supported through the + <package>rocm-opencl-icd</package> package. Adding this package to + <xref linkend="opt-hardware.opengl.extraPackages"/> enables OpenCL + support: + + <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ + rocm-opencl-icd + ];</programlisting> </para> </section> @@ -100,9 +100,9 @@ Platform Vendor Advanced Micro Devices, Inc.</screen> support. For example, for Gen8 and later GPUs, the following configuration can be used: - <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ - intel-compute-runtime -];</programlisting> + <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ + intel-compute-runtime + ];</programlisting> </para> </section> @@ -173,31 +173,31 @@ GPU1: <title>AMD</title> <para> - Modern AMD <link - xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics - Core Next</link> (GCN) GPUs are supported through either radv, which is - part of <package>mesa</package>, or the <package>amdvlk</package> package. - Adding the <package>amdvlk</package> package to - <xref linkend="opt-hardware.opengl.extraPackages"/> makes both drivers - available for applications and lets them choose. A specific driver can - be forced as follows: - - <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ - pkgs.<package>amdvlk</package> -]; - -# To enable Vulkan support for 32-bit applications, also add: -<xref linkend="opt-hardware.opengl.extraPackages32"/> = [ - pkgs.driversi686Linux.<package>amdvlk</package> -]; - -# For amdvlk -<xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = - "/run/opengl-driver/share/vulkan/icd.d/amd_icd64.json"; -# For radv -<xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = - "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json"; -</programlisting> + Modern AMD <link + xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics + Core Next</link> (GCN) GPUs are supported through either radv, which is + part of <package>mesa</package>, or the <package>amdvlk</package> package. + Adding the <package>amdvlk</package> package to + <xref linkend="opt-hardware.opengl.extraPackages"/> makes both drivers + available for applications and lets them choose. A specific driver can + be forced as follows: + + <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ + pkgs.<package>amdvlk</package> + ]; + + # To enable Vulkan support for 32-bit applications, also add: + <xref linkend="opt-hardware.opengl.extraPackages32"/> = [ + pkgs.driversi686Linux.<package>amdvlk</package> + ]; + + # For amdvlk + <xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = + "/run/opengl-driver/share/vulkan/icd.d/amd_icd64.json"; + # For radv + <xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = + "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json"; + </programlisting> </para> </section> </section> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/subversion.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/subversion.xml new file mode 100644 index 000000000000..940d63cc4e6d --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/subversion.xml @@ -0,0 +1,140 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="module-services-subversion"> + <title>Subversion</title> + + <para> + <link xlink:href="https://subversion.apache.org/">Subversion</link> + is a centralized version-control system. It can use a <link + xlink:href="http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.serverconfig.choosing">variety + of protocols</link> for communication between client and server. + </para> + <section xml:id="module-services-subversion-apache-httpd"> + <title>Subversion inside Apache HTTP</title> + + <para> + This section focuses on configuring a web-based server on top of + the Apache HTTP server, which uses + <link xlink:href="http://www.webdav.org/">WebDAV</link>/<link + xlink:href="http://www.webdav.org/deltav/WWW10/deltav-intro.htm">DeltaV</link> + for communication. + </para> + + <para>For more information on the general setup, please refer to + the <link + xlink:href="http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.serverconfig.httpd">the + appropriate section of the Subversion book</link>. + </para> + + <para>To configure, include in + <literal>/etc/nixos/configuration.nix</literal> code to activate + Apache HTTP, setting <xref linkend="opt-services.httpd.adminAddr" /> + appropriately: + </para> + + <para> +<programlisting> + services.httpd.enable = true; + services.httpd.adminAddr = ...; + networking.firewall.allowedTCPPorts = [ 80 443 ]; +</programlisting> + </para> + + <para>For a simple Subversion server with basic authentication, + configure the Subversion module for Apache as follows, setting + <literal>hostName</literal> and <literal>documentRoot</literal> + appropriately, and <literal>SVNParentPath</literal> to the parent + directory of the repositories, + <literal>AuthzSVNAccessFile</literal> to the location of the + <code>.authz</code> file describing access permission, and + <literal>AuthUserFile</literal> to the password file. + </para> + <para> +<programlisting> +services.httpd.extraModules = [ + # note that order is *super* important here + { name = "dav_svn"; path = "${pkgs.apacheHttpdPackages.subversion}/modules/mod_dav_svn.so"; } + { name = "authz_svn"; path = "${pkgs.apacheHttpdPackages.subversion}/modules/mod_authz_svn.so"; } + ]; + services.httpd.virtualHosts = { + "svn" = { + hostName = HOSTNAME; + documentRoot = DOCUMENTROOT; + locations."/svn".extraConfig = '' + DAV svn + SVNParentPath REPO_PARENT + AuthzSVNAccessFile ACCESS_FILE + AuthName "SVN Repositories" + AuthType Basic + AuthUserFile PASSWORD_FILE + Require valid-user + ''; + } +</programlisting> + </para> + + <para> + The key <code>"svn"</code> is just a symbolic name identifying the + virtual host. The <code>"/svn"</code> in + <code>locations."/svn".extraConfig</code> is the path underneath + which the repositories will be served. + </para> + + <para><link + xlink:href="https://wiki.archlinux.org/index.php/Subversion">This + page</link> explains how to set up the Subversion configuration + itself. This boils down to the following: + </para> + <para> + Underneath <literal>REPO_PARENT</literal> repositories can be set up + as follows: + </para> + <para> +<screen> +<prompt>$ </prompt> svn create REPO_NAME +</screen> + </para> + <para>Repository files need to be accessible by + <literal>wwwrun</literal>: + </para> + <para> +<screen> +<prompt>$ </prompt> chown -R wwwrun:wwwrun REPO_PARENT +</screen> + </para> + <para> + The password file <literal>PASSWORD_FILE</literal> can be created as follows: + </para> + <para> +<screen> +<prompt>$ </prompt> htpasswd -cs PASSWORD_FILE USER_NAME +</screen> + </para> + <para> + Additional users can be set up similarly, omitting the + <code>c</code> flag: + </para> + <para> +<screen> +<prompt>$ </prompt> htpasswd -s PASSWORD_FILE USER_NAME +</screen> + </para> + <para> + The file describing access permissions + <literal>ACCESS_FILE</literal> will look something like + the following: + </para> + <para> +<programlisting> +[/] +* = r + +[REPO_NAME:/] +USER_NAME = rw +</programlisting> + </para> + <para>The Subversion repositories will be accessible as <code>http://HOSTNAME/svn/REPO_NAME</code>.</para> + </section> +</chapter> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/user-mgmt.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/user-mgmt.xml index cbec83814c9a..e83e7b75ef54 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/user-mgmt.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/user-mgmt.xml @@ -38,7 +38,7 @@ assigned by setting the user's <link linkend="opt-users.users._name_.hashedPassword">hashedPassword</link> option. A hashed password can be generated using <command>mkpasswd -m - sha-512</command> after installing the <literal>mkpasswd</literal> package. + sha-512</command>. </para> <para> A user ID (uid) is assigned automatically. You can also specify a uid diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/wayland.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/wayland.xml new file mode 100644 index 000000000000..231c219cf081 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/configuration/wayland.xml @@ -0,0 +1,23 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-wayland"> + <title>Wayland</title> + + <para> + While X11 (see <xref linkend="sec-x11"/>) is still the primary display + technology on NixOS, Wayland support is steadily improving. + Where X11 separates the X Server and the window manager, on Wayland those + are combined: a Wayland Compositor is like an X11 window manager, but also + embeds the Wayland 'Server' functionality. This means it is sufficient to + install a Wayland Compositor such as <package>sway</package> without + separately enabling a Wayland server: +<programlisting> +<xref linkend="opt-programs.sway.enable"/> = true; +</programlisting> + This installs the <package>sway</package> compositor along with some + essential utilities. Now you can start <package>sway</package> from the TTY + console. + </para> +</chapter> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/contributing-to-this-manual.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/contributing-to-this-manual.xml new file mode 100644 index 000000000000..935dd66bc141 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/contributing-to-this-manual.xml @@ -0,0 +1,22 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="chap-contributing"> + <title>Contributing to this documentation</title> + <para> + The DocBook sources of NixOS' manual are in the <filename +xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/doc/manual"> +nixos/doc/manual</filename> subdirectory of the <link +xlink:href="https://github.com/NixOS/nixpkgs">Nixpkgs</link> repository. + </para> + <para> + You can quickly check your edits with the following: + </para> +<screen> +<prompt>$ </prompt>cd /path/to/nixpkgs +<prompt>$ </prompt>nix-build nixos/release.nix -A manual.x86_64-linux +</screen> + <para> + If the build succeeds, the manual will be in + <filename>./result/share/doc/nixos/index.html</filename>. + </para> +</chapter> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/default.nix b/infra/libkookie/nixpkgs/nixos/doc/manual/default.nix index 6ca75f869f45..af7a2e08220c 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/default.nix +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/default.nix @@ -63,6 +63,7 @@ let "--stringparam html.script './highlightjs/highlight.pack.js ./highlightjs/loader.js'" "--param xref.with.number.and.title 1" "--param toc.section.depth 0" + "--param generate.consistent.ids 1" "--stringparam admon.style ''" "--stringparam callout.graphics.extension .svg" "--stringparam current.docid manual" diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/development/running-nixos-tests-interactively.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/development/running-nixos-tests-interactively.xml index a11a9382764d..a6044d5f89e8 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/development/running-nixos-tests-interactively.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/development/running-nixos-tests-interactively.xml @@ -9,7 +9,7 @@ The test itself can be run interactively. This is particularly useful when developing or debugging a test: <screen> -<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driver +<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driverInteractive <prompt>$ </prompt>./result/bin/nixos-test-driver starting VDE switch for network 1 <prompt>></prompt> @@ -30,7 +30,7 @@ starting VDE switch for network 1 <para> To just start and experiment with the VMs, run: <screen> -<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driver +<prompt>$ </prompt>nix-build nixos/tests/login.nix -A driverInteractive <prompt>$ </prompt>./result/bin/nixos-run-vms </screen> The script <command>nixos-run-vms</command> starts the virtual machines diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing-from-other-distro.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing-from-other-distro.xml index f10a7d658879..43f69b923d14 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing-from-other-distro.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing-from-other-distro.xml @@ -161,6 +161,13 @@ nixpkgs https://nixos.org/channels/nixpkgs-unstable</screen> existing systems without the help of a rescue USB drive or similar. </para> </warning> + <note> + <para> + On some distributions there are separate PATHS for programs intended only for root. + In order for the installation to succeed, you might have to use <literal>PATH="$PATH:/usr/sbin:/sbin"</literal> + in the following command. + </para> + </note> <screen><prompt>$ </prompt>sudo PATH="$PATH" NIX_PATH="$NIX_PATH" `which nixos-install` --root /mnt</screen> <para> Again, please refer to the <literal>nixos-install</literal> step in diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing.xml index 6df1d8303483..bedeb7ccfa8e 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/installation/installing.xml @@ -480,13 +480,8 @@ Retype new UNIX password: ***</screen> <prompt>$ </prompt>passwd eelco</screen> </para> <para> - You may also want to install some software. For instance, -<screen> -<prompt>$ </prompt>nix-env -qaP \*</screen> - shows what packages are available, and -<screen> -<prompt>$ </prompt>nix-env -f '<nixpkgs>' -iA w3m</screen> - installs the <literal>w3m</literal> browser. + You may also want to install some software. This will be covered + in <xref linkend="sec-package-management" />. </para> </listitem> </orderedlist> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/installation/upgrading.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/installation/upgrading.xml index 08780051d5f6..15ba5db9a37b 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/installation/upgrading.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/installation/upgrading.xml @@ -14,7 +14,7 @@ <para> <emphasis>Stable channels</emphasis>, such as <literal - xlink:href="https://nixos.org/channels/nixos-20.03">nixos-20.03</literal>. + xlink:href="https://nixos.org/channels/nixos-20.09">nixos-20.09</literal>. These only get conservative bug fixes and package upgrades. For instance, a channel update may cause the Linux kernel on your system to be upgraded from 4.19.34 to 4.19.38 (a minor bug fix), but not from @@ -38,7 +38,7 @@ <para> <emphasis>Small channels</emphasis>, such as <literal - xlink:href="https://nixos.org/channels/nixos-20.03-small">nixos-20.03-small</literal> + xlink:href="https://nixos.org/channels/nixos-20.09-small">nixos-20.09-small</literal> or <literal xlink:href="https://nixos.org/channels/nixos-unstable-small">nixos-unstable-small</literal>. @@ -63,8 +63,8 @@ <para> When you first install NixOS, you’re automatically subscribed to the NixOS channel that corresponds to your installation source. For instance, if you - installed from a 20.03 ISO, you will be subscribed to the - <literal>nixos-20.03</literal> channel. To see which NixOS channel you’re + installed from a 20.09 ISO, you will be subscribed to the + <literal>nixos-20.09</literal> channel. To see which NixOS channel you’re subscribed to, run the following as root: <screen> <prompt># </prompt>nix-channel --list | grep nixos @@ -75,13 +75,13 @@ nixos https://nixos.org/channels/nixos-unstable <prompt># </prompt>nix-channel --add https://nixos.org/channels/<replaceable>channel-name</replaceable> nixos </screen> (Be sure to include the <literal>nixos</literal> parameter at the end.) For - instance, to use the NixOS 20.03 stable channel: + instance, to use the NixOS 20.09 stable channel: <screen> -<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.03 nixos +<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09 nixos </screen> If you have a server, you may want to use the “small” channel instead: <screen> -<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.03-small nixos +<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09-small nixos </screen> And if you want to live on the bleeding edge: <screen> @@ -132,7 +132,7 @@ nixos https://nixos.org/channels/nixos-unstable kernel, initrd or kernel modules. You can also specify a channel explicitly, e.g. <programlisting> -<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.03; +<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.09; </programlisting> </para> </section> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/man-nixos-install.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/man-nixos-install.xml index b205e2309687..91542d37cbd5 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/man-nixos-install.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/man-nixos-install.xml @@ -50,6 +50,12 @@ </arg> <arg> + <group choice='req'> + <arg choice='plain'><option>--impure</option></arg> + </group> + </arg> + + <arg> <arg choice='plain'> <option>--channel</option> </arg> @@ -102,6 +108,12 @@ <arg> <arg choice='plain'> + <option>--keep-going</option> + </arg> + </arg> + + <arg> + <arg choice='plain'> <option>--help</option> </arg> </arg> @@ -297,6 +309,17 @@ </varlistentry> <varlistentry> <term> + <option>--keep-going</option> + </term> + <listitem> + <para> + Causes Nix to continue building derivations as far as possible + in the face of failed builds. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> <option>--help</option> </term> <listitem> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/manual.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/manual.xml index 18a67a2dd941..db9e7313831d 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/manual.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/manual.xml @@ -19,5 +19,6 @@ <xi:include href="./generated/options-db.xml" xpointer="configuration-variable-list" /> </appendix> + <xi:include href="contributing-to-this-manual.xml" /> <xi:include href="release-notes/release-notes.xml" /> </book> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2009.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2009.xml index d9ff51ae3df0..afb09d7c5d26 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2009.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2009.xml @@ -3,8 +3,11 @@ xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0" xml:id="sec-release-20.09"> - <title>Release 20.09 (“Nightingale”, 2020.09/??)</title> + <title>Release 20.09 (“Nightingale”, 2020.10/27)</title> + <para> + Support is planned until the end of April 2021, handing over to 21.03. + </para> <section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" @@ -13,88 +16,611 @@ <title>Highlights</title> <para> - In addition to numerous new and upgraded packages, this release has the + In addition to 7349 new, 14442 updated, and 8181 removed packages, this release has the following highlights: </para> <itemizedlist> <listitem> <para> - Support is planned until the end of April 2021, handing over to 21.03. + Core version changes: </para> + <itemizedlist> + <listitem> + <para> + gcc: 9.2.0 -> 9.3.0 + </para> + </listitem> + <listitem> + <para> + glibc: 2.30 -> 2.31 + </para> + </listitem> + <listitem> + <para> + linux: still defaults to 5.4.x, all supported kernels available + </para> + </listitem> + <listitem> + <para> + mesa: 19.3.5 -> 20.1.7 + </para> + </listitem> + </itemizedlist> </listitem> <listitem> - <para>GNOME desktop environment was upgraded to 3.36, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.36/">release notes</link>.</para> - </listitem> - <listitem> - <para> - The Cinnamon desktop environment (v4.6) has been added. <varname>services.xserver.desktopManager.cinnamon.enable = true;</varname> to try it out! - Remember that, with any new feature it's possible you could run into issues, so please send all support requests to <link xlink:href="https://github.com/NixOS/nixpkgs/issues">github.com/NixOS/nixpkgs</link> to notify the maintainers. - </para> + <para> + Desktop Environments: + </para> + <itemizedlist> + <listitem> + <para> + plasma5: 5.17.5 -> 5.18.5 + </para> + </listitem> + <listitem> + <para> + kdeApplications: 19.12.3 -> 20.08.1 + </para> + </listitem> + <listitem> + <para> + gnome3: 3.34 -> 3.36, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.36/">release notes</link> + </para> + </listitem> + <listitem> + <para> + cinnamon: added at 4.6 + </para> + </listitem> + <listitem> + <para> + NixOS now distributes an official <link xlink:href="https://nixos.org/download.html#nixos-iso">GNOME ISO</link> + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - Quickly configure a complete, private, self-hosted video - conferencing solution with the new Jitsi Meet module. + Programming Languages and Frameworks: </para> + <itemizedlist> + + <listitem> + <para> + Agda ecosystem was heavily reworked (see more details below) + </para> + </listitem> + <listitem> + <para> + PHP now defaults to PHP 7.4, updated from 7.3 + </para> + </listitem> + <listitem> + <para> + PHP 7.2 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 20.09 release + </para> + </listitem> + <listitem> + <para> + Python 3 now defaults to Python 3.8 instead of 3.7 + </para> + </listitem> + <listitem> + <para> + Python 3.5 reached its upstream EOL at the end of September 2020: it + has been removed from the list of available packages + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - <package>maxx</package> package removed along with <varname>services.xserver.desktopManager.maxx</varname> module. - Please migrate to <package>cdesktopenv</package> and <varname>services.xserver.desktopManager.cde</varname> module. + Databases and Service Monitoring: </para> + <itemizedlist> + <listitem> + <para> + MariaDB has been updated to 10.4, MariaDB Galera to 26.4. Please read the related upgrade instructions under <link linkend="sec-release-20.09-incompatibilities">backwards incompatibilities</link> before upgrading. + </para> + </listitem> + <listitem> + <para> + Zabbix now defaults to 5.0, updated from 4.4. Please read related sections under <link linkend="sec-release-20.09-incompatibilities">backwards compatibilities</link> before upgrading. + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - We now distribute a GNOME ISO. + Major module changes: </para> + <itemizedlist> + <listitem> + <para> + Quickly configure a complete, private, self-hosted video + conferencing solution with the new Jitsi Meet module. + </para> + </listitem> + <listitem> + <para> + Two new options, <link linkend="opt-services.openssh.authorizedKeysCommand">authorizedKeysCommand</link> + and <link linkend="opt-services.openssh.authorizedKeysCommandUser">authorizedKeysCommandUser</link>, have + been added to the <literal>openssh</literal> module. If you have <literal>AuthorizedKeysCommand</literal> + in your <link linkend="opt-services.openssh.extraConfig">services.openssh.extraConfig</link> you should + make use of these new options instead. + </para> + </listitem> + <listitem> + <para> + There is a new module for Podman (<varname>virtualisation.podman</varname>), a drop-in replacement for the Docker command line. + </para> + </listitem> + <listitem> + <para> + The new <varname>virtualisation.containers</varname> module manages configuration shared by the CRI-O and Podman modules. + </para> + </listitem> + <listitem> + <para> + Declarative Docker containers are renamed from <varname>docker-containers</varname> to <varname>virtualisation.oci-containers.containers</varname>. + This is to make it possible to use <literal>podman</literal> instead of <literal>docker</literal>. + </para> + </listitem> + <listitem> + <para> + The new option <link linkend="opt-documentation.man.generateCaches">documentation.man.generateCaches</link> + has been added to automatically generate the <literal>man-db</literal> caches, which are needed by utilities + like <command>whatis</command> and <command>apropos</command>. The caches are generated during the build of + the NixOS configuration: since this can be expensive when a large number of packages are installed, the + feature is disabled by default. + </para> + </listitem> + <listitem> + <para> + <varname>services.postfix.sslCACert</varname> was replaced by <varname>services.postfix.tlsTrustedAuthorities</varname> which now defaults to system certificate authorities. + </para> + </listitem> + <listitem> + <para> + The various documented workarounds to use steam have been converted to a module. <varname>programs.steam.enable</varname> enables steam, controller support and the workarounds. + </para> + </listitem> + <listitem> + <para> + Support for built-in LCDs in various pieces of Logitech hardware (keyboards and USB speakers). <varname>hardware.logitech.lcd.enable</varname> enables support for all hardware supported by the <link xlink:href="https://sourceforge.net/projects/g15daemon/">g15daemon project</link>. + </para> + </listitem> + <listitem> + <para> + The GRUB module gained support for basic password protection, which + allows to restrict non-default entries in the boot menu to one or more + users. The users and passwords are defined via the option + <option>boot.loader.grub.users</option>. + Note: Password support is only available in GRUB version 2. + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - PHP now defaults to PHP 7.4, updated from 7.3. + NixOS module changes: </para> + <itemizedlist> + <listitem> + <para> + The NixOS module system now supports freeform modules as a mix between <literal>types.attrsOf</literal> and <literal>types.submodule</literal>. These allow you to explicitly declare a subset of options while still permitting definitions without an associated option. See <xref linkend='sec-freeform-modules'/> for how to use them. + </para> + </listitem> + <listitem> + <para> + Following its deprecation in 20.03, the Perl NixOS test driver has been removed. + All remaining tests have been ported to the Python test framework. + Code outside nixpkgs using <filename>make-test.nix</filename> or + <filename>testing.nix</filename> needs to be ported to + <filename>make-test-python.nix</filename> and + <filename>testing-python.nix</filename> respectively. + </para> + </listitem> + <listitem> + <para> + Subordinate GID and UID mappings are now set up automatically for all normal users. + This will make container tools like Podman work as non-root users out of the box. + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - PHP 7.2 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 20.09 release. + Starting with this release, the hydra-build-result + <literal>nixos-<replaceable>YY.MM</replaceable></literal> + branches no longer exist in the <link + xlink:href="https://github.com/nixos/nixpkgs-channels">deprecated + nixpkgs-channels repository</link>. These branches are now in + <link xlink:href="https://github.com/nixos/nixpkgs">the main nixpkgs + repository</link>. </para> </listitem> + </itemizedlist> + </section> + + <section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-release-20.09-new-services"> + <title>New Services</title> + + <para> + In addition to 1119 new, 118 updated, and 476 removed options; 61 new modules were added since the last release: + </para> + + <itemizedlist> <listitem> <para> - Python 3 now defaults to Python 3.8 instead of 3.7. + Hardware: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-hardware.system76.firmware-daemon.enable" /> adds easy support of system76 firmware + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.uinput.enable" /> loads uinput kernel module + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.video.hidpi.enable" /> enable good defaults for HiDPI displays + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.wooting.enable" /> support for Wooting keyboards + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.xpadneo.enable" /> xpadneo driver for Xbox One wireless controllers + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - Python 3.5 has reached its upstream EOL at the end of September 2020: it - has been removed from the list of available packages. + Programs: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-programs.hamster.enable" /> enable hamster time tracking + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-programs.steam.enable" /> adds easy enablement of steam and related system configuration + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - Two new options, <link linkend="opt-services.openssh.authorizedKeysCommand">authorizedKeysCommand</link> - and <link linkend="opt-services.openssh.authorizedKeysCommandUser">authorizedKeysCommandUser</link>, have - been added to the <literal>openssh</literal> module. If you have <literal>AuthorizedKeysCommand</literal> - in your <link linkend="opt-services.openssh.extraConfig">services.openssh.extraConfig</link> you should - make use of these new options instead. + Security: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-security.doas.enable" /> alternative to sudo, allows non-root users to execute commands as root + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-security.tpm2.enable" /> add Trusted Platform Module 2 support + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - There is a new module for Podman(<varname>virtualisation.podman</varname>), a drop-in replacement for the Docker command line. + System: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-boot.initrd.network.openvpn.enable" /> start an OpenVPN client during initrd boot + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - The new <varname>virtualisation.containers</varname> module manages configuration shared by the CRI-O and Podman modules. + Virtualization: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-boot.enableContainers" /> use nixos-containers + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-virtualisation.oci-containers.containers" /> run OCI (Docker) containers + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-virtualisation.podman.enable" /> daemonless container engine + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - Declarative Docker containers are renamed from <varname>docker-containers</varname> to <varname>virtualisation.oci-containers.containers</varname>. - This is to make it possible to use <literal>podman</literal> instead of <literal>docker</literal>. + Services: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-services.ankisyncd.enable" /> Anki sync server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.bazarr.enable" /> Subtitle manager for Sonarr and Radarr + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.biboumi.enable" /> Biboumi XMPP gateway to IRC + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.blockbook-frontend" /> Blockbook-frontend, a service for the Trezor wallet + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.cage.enable" /> Wayland cage service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.convos.enable" /> IRC daemon, which can be accessed throught the browser + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.engelsystem.enable" /> Tool for coordinating volunteers and shifts on large events + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.espanso.enable" /> text-expander written in rust + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.foldingathome.enable" /> Folding@home client + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.gerrit.enable" /> Web-based team code collaboration tool + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.go-neb.enable" /> Matrix bot + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.hardware.xow.enable" /> xow as a systemd service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.hercules-ci-agent.enable" /> Hercules CI build agent + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jicofo.enable" /> Jitsi Conference Focus, component of Jitsi Meet + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jirafeau.enable" /> A web file repository + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jitsi-meet.enable" /> Secure, simple and scalable video conferences + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jitsi-videobridge.enable" /> Jitsi Videobridge, a WebRTC compatible router + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jupyterhub.enable" /> Jupyterhub development server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.k3s.enable" /> Lightweight Kubernetes distribution + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.magic-wormhole-mailbox-server.enable" /> Magic Wormhole Mailbox Server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.malcontent.enable" /> Parental Control support + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.matrix-appservice-discord.enable" /> Matrix and Discord bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.mautrix-telegram.enable" /> Matrix-Telegram puppeting/relaybot bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.mirakurun.enable" /> Japanese DTV Tuner Server Service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.molly-brown.enable" /> Molly-Brown Gemini server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.mullvad-vpn.enable" /> Mullvad VPN daemon + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.ncdns.enable" /> Namecoin to DNS bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.nextdns.enable" /> NextDNS to DoH Proxy service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.nix-store-gcs-proxy" /> Google storage bucket to be used as a nix store + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.onedrive.enable" /> OneDrive sync service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.pinnwand.enable" /> Pastebin-like service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.pixiecore.enable" /> Manage network booting of machines + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.privacyidea.enable" /> Privacy authentication server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.quorum.enable" /> Quorum blockchain daemon + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.robustirc-bridge.enable" /> RobustIRC bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.rss-bridge.enable" /> Generate RSS and Atom feeds + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.rtorrent.enable" /> rTorrent service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.smartdns.enable" /> SmartDNS DNS server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.sogo.enable" /> SOGo groupware + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.teeworlds.enable" /> Teeworlds game server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.torque.mom.enable" /> torque computing node + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.torque.server.enable" /> torque server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.tuptime.enable" /> A total uptime service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.urserver.enable" /> X11 remote server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.wasabibackend.enable" /> Wasabi backend service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.yubikey-agent.enable" /> Yubikey agent + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.zigbee2mqtt.enable" /> Zigbee to MQTT bridge + </para> + </listitem> + </itemizedlist> </listitem> + + </itemizedlist> + + </section> + + <section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-release-20.09-incompatibilities"> + <title>Backward Incompatibilities</title> + + <para> + When upgrading from a previous release, please be aware of the following + incompatible changes: + </para> + + <itemizedlist> <listitem> <para> MariaDB has been updated to 10.4, MariaDB Galera to 26.4. @@ -144,36 +670,7 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION; from the default of <literal>mysql</literal> to a different user please change <literal>'mysql'@'localhost'</literal> to the corresponding user instead. </para> </listitem> - <listitem> - <para> - The new option <link linkend="opt-documentation.man.generateCaches">documentation.man.generateCaches</link> - has been added to automatically generate the <literal>man-db</literal> caches, which are needed by utilities - like <command>whatis</command> and <command>apropos</command>. The caches are generated during the build of - the NixOS configuration: since this can be expensive when a large number of packages are installed, the - feature is disabled by default. - </para> - </listitem> - <listitem> - <para> - <varname>services.postfix.sslCACert</varname> was replaced by <varname>services.postfix.tlsTrustedAuthorities</varname> which now defaults to system certificate authorities. - </para> - </listitem> - <listitem> - <para> - Subordinate GID and UID mappings are now set up automatically for all normal users. - This will make container tools like Podman work as non-root users out of the box. - </para> - </listitem> - <listitem> - <para> - The various documented workarounds to use steam have been converted to a module. <varname>programs.steam.enable</varname> enables steam, controller support and the workarounds. - </para> - </listitem> - <listitem> - <para> - Support for built-in LCDs in various pieces of Logitech hardware (keyboards and USB speakers). <varname>hardware.logitech.lcd.enable</varname> enables support for all hardware supported by the g15daemon project. - </para> - </listitem> + <listitem> <para> Zabbix now defaults to 5.0, updated from 4.4. Please carefully read through @@ -208,72 +705,13 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION; </programlisting> </para> </listitem> - <listitem> - <para> - The NixOS module system now supports freeform modules as a mix between <literal>types.attrsOf</literal> and <literal>types.submodule</literal>. These allow you to explicitly declare a subset of options while still permitting definitions without an associated option. See <xref linkend='sec-freeform-modules'/> for how to use them. - </para> - </listitem> - <listitem> - <para> - The GRUB module gained support for basic password protection, which - allows to restrict non-default entries in the boot menu to one or more - users. The users and passwords are defined via the option - <option>boot.loader.grub.users</option>. - Note: Password support is only avaiable in GRUB version 2. - </para> - </listitem> - <listitem> - <para> - Following its deprecation in 20.03, the Perl NixOS test driver has been removed. - All remaining tests have been ported to the Python test framework. - Code outside nixpkgs using <filename>make-test.nix</filename> or - <filename>testing.nix</filename> needs to be ported to - <filename>make-test-python.nix</filename> and - <filename>testing-python.nix</filename> respectively. - </para> - </listitem> - </itemizedlist> - </section> - <section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-release-20.09-new-services"> - <title>New Services</title> - - <para> - The following new services were added since the last release: - </para> - - <itemizedlist> <listitem> <para> - There is a new <xref linkend="opt-security.doas.enable"/> module that provides <command>doas</command>, a lighter alternative to <command>sudo</command> with many of the same features. - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://hercules-ci.com">Hercules CI</link> Agent is a specialized build agent for projects built with Nix. See the <link xlink:href="https://nixos.org/nixos/options.html#services.hercules-ci-agent">options</link> and <link xlink:href="https://docs.hercules-ci.com/hercules-ci/getting-started/#deploy-agent">setup</link>. + <package>maxx</package> package removed along with <varname>services.xserver.desktopManager.maxx</varname> module. + Please migrate to <package>cdesktopenv</package> and <varname>services.xserver.desktopManager.cde</varname> module. </para> </listitem> - </itemizedlist> - - </section> - - <section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-release-20.09-incompatibilities"> - <title>Backward Incompatibilities</title> - - <para> - When upgrading from a previous release, please be aware of the following - incompatible changes: - </para> - - <itemizedlist> <listitem> <para> The <link linkend="opt-services.matrix-synapse.enable">matrix-synapse</link> module no longer includes optional dependencies by default, they have to be added through the <link linkend="opt-services.matrix-synapse.plugins">plugins</link> option. @@ -300,7 +738,7 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION; It can still be enabled by providing <literal>phantomJsSupport = true</literal> to the package instantiation: <programlisting>{ services.grafana.package = pkgs.grafana.overrideAttrs (oldAttrs: rec { - phantomJsSupport = false; + phantomJsSupport = true; }); }</programlisting> </para> @@ -451,12 +889,23 @@ php.override { <listitem> <para> Nginx web server now starting with additional sandbox/hardening options. By default, write access - to <literal>services.nginx.stateDir</literal> is allowed. To allow writing to other folders, + to <literal>/var/log/nginx</literal> and <literal>/var/cache/nginx</literal> is allowed. To allow writing to other folders, use <literal>systemd.services.nginx.serviceConfig.ReadWritePaths</literal> <programlisting> systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/www" ]; </programlisting> </para> + <para> + Nginx is also started with the systemd option <literal>ProtectHome = mkDefault true;</literal> + which forbids it to read anything from <literal>/home</literal>, <literal>/root</literal> + and <literal>/run/user</literal> (see + <link xlink:href="https://www.freedesktop.org/software/systemd/man/systemd.exec.html#ProtectHome=">ProtectHome docs</link> + for details). + If you require serving files from home directories, you may choose to set e.g. +<programlisting> +systemd.services.nginx.serviceConfig.ProtectHome = "read-only"; +</programlisting> + </para> </listitem> <listitem> <para> @@ -643,6 +1092,13 @@ systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/www" ]; In the <literal>resilio</literal> module, <xref linkend="opt-services.resilio.httpListenAddr"/> has been changed to listen to <literal>[::1]</literal> instead of <literal>0.0.0.0</literal>. </para> </listitem> + <listitem> + <para> + <literal>sslh</literal> has been updated to version + <literal>1.21</literal>. The <literal>ssl</literal> probe must be + renamed to <literal>tls</literal> in <xref linkend="opt-services.sslh.appendConfig"/>. + </para> + </listitem> <listitem> <para> Users of <link xlink:href="http://openafs.org">OpenAFS 1.6</link> must @@ -1102,6 +1558,8 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; <package>nextcloud18</package> before upgrading to <package>nextcloud19</package> since Nextcloud doesn't support upgrades across multiple major versions. </para> + </listitem> + <listitem> <para> The <literal>nixos-run-vms</literal> script now deletes the previous run machines states on test startup. You can use the @@ -1145,30 +1603,30 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; <para> Agda has been heavily reworked. <itemizedlist> - <listitem> - <para> - <literal>agda.mkDerivation</literal> has been heavily changed and - is now located at <package>agdaPackages.mkDerivation</package>. - </para> - </listitem> - <listitem> - <para> - New top-level packages <package>agda</package> and - <literal>agda.withPackages</literal> have been added, the second - of which sets up agda with access to chosen libraries. - </para> - </listitem> - <listitem> - <para> - All agda libraries now live under - <literal>agdaPackages</literal>. - </para> - </listitem> - <listitem> - <para> - Many broken libraries have been removed. - </para> - </listitem> + <listitem> + <para> + <literal>agda.mkDerivation</literal> has been heavily changed and + is now located at <package>agdaPackages.mkDerivation</package>. + </para> + </listitem> + <listitem> + <para> + New top-level packages <package>agda</package> and + <literal>agda.withPackages</literal> have been added, the second + of which sets up agda with access to chosen libraries. + </para> + </listitem> + <listitem> + <para> + All agda libraries now live under + <literal>agdaPackages</literal>. + </para> + </listitem> + <listitem> + <para> + Many broken libraries have been removed. + </para> + </listitem> </itemizedlist> See the <link xlink:href="https://nixos.org/nixpkgs/manual/#agda">new @@ -1181,7 +1639,7 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; nixpkgs. It was a work in progress to package the <link xlink:href="https://www.deepin.org/en/dde/">Deepin Desktop Environment (DDE)</link>, including libraries, tools and applications, and it was still - missing a service to lauch the desktop environment. It has shown + missing a service to launch the desktop environment. It has shown to no longer be a feasible goal due to reasons discussed in <link xlink:href="https://github.com/NixOS/nixpkgs/issues/94870">issue #94870</link>. The package <literal>netease-cloud-music</literal> has also been @@ -1226,4 +1684,131 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; </listitem> </itemizedlist> </section> + + <section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-release-20.09-contributions"> + <title>Contributions</title> + <para> + I, Jonathan Ringer, would like to thank the following individuals for their work on nixpkgs. This release could not be done without the hard work of the NixOS community. There were 31282 contributions across 1313 contributors. + </para> + <orderedlist> + <para> + Top contributors to NixOS/Nixpkgs from the 20.03 release to the 20.09 release: + </para> + <listitem> + <para> + 2288 Mario Rodas + </para> + </listitem> + <listitem> + <para> + 1837 Frederik Rietdijk + </para> + </listitem> + <listitem> + <para> + 946 Jörg Thalheim + </para> + </listitem> + <listitem> + <para> + 925 Maximilian Bosch + </para> + </listitem> + <listitem> + <para> + 687 Jonathan Ringer + </para> + </listitem> + <listitem> + <para> + 651 Jan Tojnar + </para> + </listitem> + <listitem> + <para> + 622 Daniël de Kok + </para> + </listitem> + <listitem> + <para> + 605 WORLDofPEACE + </para> + </listitem> + <listitem> + <para> + 597 Florian Klink + </para> + </listitem> + <listitem> + <para> + 528 José Romildo Malaquias + </para> + </listitem> + </orderedlist> + + <orderedlist> + <para> + Top contributors to stabilizing this release (Zero Hydra Failures period): + </para> + <listitem> + <para> + 281 volth + </para> + </listitem> + <listitem> + <para> + 101 Robert Scott + </para> + </listitem> + <listitem> + <para> + 86 Tim Steinbach + </para> + </listitem> + <listitem> + <para> + 76 WORLDofPEACE + </para> + </listitem> + <listitem> + <para> + 49 Maximilian Bosch + </para> + </listitem> + <listitem> + <para> + 42 Thomas Tuegel + </para> + </listitem> + <listitem> + <para> + 37 Doron Behar + </para> + </listitem> + <listitem> + <para> + 36 Vladimír Čunát + </para> + </listitem> + <listitem> + <para> + 27 Jonathan Ringer + </para> + </listitem> + <listitem> + <para> + 27 Maciej Krüger + </para> + </listitem> + </orderedlist> + + <para> + I, Jonathan Ringer, would also like to personally thank @WORLDofPEACE for their help in mentoring me on the release process. Special thanks also goes to Thomas Tuegel for helping immensely with stabilizing Qt, KDE, and Plasma5; I would also like to thank Robert Scott for his numerous fixes and pull request reviews. + </para> + + </section> </section> diff --git a/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2103.xml b/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2103.xml index fb1d50434b80..ffb00aa0362e 100644 --- a/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2103.xml +++ b/infra/libkookie/nixpkgs/nixos/doc/manual/release-notes/rl-2103.xml @@ -23,6 +23,22 @@ Support is planned until the end of October 2021, handing over to 21.09. </para> </listitem> + <listitem> + <para>GNOME desktop environment was upgraded to 3.38, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.38/">release notes</link>.</para> + </listitem> + <listitem> + <para> + <link xlink:href="https://www.gnuradio.org/">GNURadio</link> 3.8 was + <link xlink:href="https://github.com/NixOS/nixpkgs/issues/82263">finnally</link> + packaged, along with a rewrite to the Nix expressions, allowing users to + override the features upstream supports selecting to compile or not to. + Additionally, the attribute <code>gnuradio</code> and <code>gnuradio3_7</code> + now point to an externally wrapped by default derivations, that allow you to + also add `extraPythonPackages` to the Python interpreter used by GNURadio. + Missing environmental variables needed for operational GUI were also added + (<link xlink:href="https://github.com/NixOS/nixpkgs/issues/75478">#7547</link>). + </para> + </listitem> </itemizedlist> </section> @@ -39,7 +55,24 @@ <itemizedlist> <listitem> - <para /> + <para> + <link xlink:href="https://www.keycloak.org/">Keycloak</link>, + an open source identity and access management server with + support for <link + xlink:href="https://openid.net/connect/">OpenID Connect</link>, + <link xlink:href="https://oauth.net/2/">OAUTH 2.0</link> and + <link xlink:href="https://en.wikipedia.org/wiki/SAML_2.0">SAML + 2.0</link>. + </para> + <para> + See the <link linkend="module-services-keycloak">Keycloak + section of the NixOS manual</link> for more information. + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.samba-wsdd.enable" /> Web Services Dynamic Discovery host daemon + </para> </listitem> </itemizedlist> @@ -75,6 +108,25 @@ </para> </listitem> <listitem> + <para> + <literal>rubyMinimal</literal> was removed due to being unused and + unusable. The default ruby interpreter includes JIT support, which makes + it reference it's compiler. Since JIT support is probably needed by some + Gems, it was decided to enable this feature with all cc references by + default, and allow to build a Ruby derivation without references to cc, + by setting <literal>jitSupport = false;</literal> in an overlay. See + <link xlink:href="https://github.com/NixOS/nixpkgs/pull/90151">#90151</link> + for more info. + </para> + </listitem> + <listitem> + <para> + Setting <option>services.openssh.authorizedKeysFiles</option> now also affects which keys <option>security.pam.enableSSHAgentAuth</option> will use. + + WARNING: If you are using these options in combination do make sure that any key paths you use are present in <option>services.openssh.authorizedKeysFiles</option>! + </para> + </listitem> + <listitem> <para> The option <option>fonts.enableFontDir</option> has been renamed to <xref linkend="opt-fonts.fontDir.enable"/>. The path of font directory @@ -93,6 +145,126 @@ <literal>kicad/default.nix</literal>. </para> </listitem> + <listitem> + <para> + The socket for the <literal>pdns-recursor</literal> module was moved from <literal>/var/lib/pdns-recursor</literal> + to <literal>/run/pdns-recursor</literal> to match upstream. + </para> + </listitem> + <listitem> + <para> + Paperwork was updated to version 2. The on-disk format slightly changed, + and it is not possible to downgrade from Paperwork 2 back to Paperwork + 1.3. Back your documents up before upgrading. See <link xlink:href="https://forum.openpaper.work/t/paperwork-2-0/112/5">this thread</link> for more details. + </para> + </listitem> + <listitem> + <para> + PowerDNS has been updated from <literal>4.2.x</literal> to <literal>4.3.x</literal>. Please + be sure to review the <link xlink:href="https://doc.powerdns.com/authoritative/upgrading.html#x-to-4-3-0">Upgrade Notes</link> + provided by upstream before upgrading. Worth specifically noting is that the service now runs + entirely as a dedicated <literal>pdns</literal> user, instead of starting as <literal>root</literal> + and dropping privileges, as well as the default <literal>socket-dir</literal> location changing from + <literal>/var/lib/powerdns</literal> to <literal>/run/pdns</literal>. + </para> + </listitem> + <listitem> + <para> + <package>btc1</package> has been abandoned upstream, and removed. + </para> + </listitem> + <listitem> + <para> + <package>cpp_ethereum</package> (aleth) has been abandoned upstream, and removed. + </para> + </listitem> + <listitem> + <para> + <package>riak-cs</package> package removed along with <varname>services.riak-cs</varname> module. + </para> + </listitem> + <listitem> + <para> + <package>stanchion</package> package removed along with <varname>services.stanchion</varname> module. + </para> + </listitem> + <listitem> + <para> + <package>mutt</package> has been updated to a new major version (2.x), which comes with + some backward incompatible changes that are described in the + <link xlink:href="http://www.mutt.org/relnotes/2.0/">release notes for Mutt 2.0</link>. + </para> + </listitem> + <listitem> + <para> + <literal>vim</literal> switched to Python 3, dropping all Python 2 support. + </para> + </listitem> + <listitem> + <para> + <link linkend="opt-boot.zfs.forceImportAll">boot.zfs.forceImportAll</link> + previously did nothing, but has been fixed. However its default has been + changed to <literal>false</literal> to preserve the existing default + behaviour. If you have this explicitly set to <literal>true</literal>, + please note that your non-root pools will now be forcibly imported. + </para> + </listitem> + <listitem> + <para> + <package>openafs</package> now points to <package>openafs_1_8</package>, + which is the new stable release. OpenAFS 1.6 was removed. + </para> + </listitem> + <listitem> + <para> + The <literal>openldap</literal> module now has support for OLC-style + configuration, users of the <literal>configDir</literal> option may wish + to migrate. If you continue to use <literal>configDir</literal>, ensure that + <literal>olcPidFile</literal> is set to <literal>/run/slapd/slapd.pid</literal>. + </para> + <para> + As a result, <literal>extraConfig</literal> and <literal>extraDatabaseConfig</literal> + are removed. To help with migration, you can convert your <literal>slapd.conf</literal> + file to OLC configuration with the following script (find the location of this + configuration file by running <literal>systemctl status openldap</literal>, it is the + <literal>-f</literal> option. + </para> + <programlisting> + TMPDIR=$(mktemp -d) + slaptest -f /path/to/slapd.conf $TMPDIR + slapcat -F $TMPDIR -n0 -H 'ldap:///???(!(objectClass=olcSchemaConfig))' + </programlisting> + <para> + This will dump your current configuration in LDIF format, which should be + straightforward to convert into Nix settings. This does not show your schema + configuration, as this is unnecessarily verbose for users of the default schemas + and <literal>slaptest</literal> is buggy with schemas directly in the config file. + </para> + </listitem> + <listitem> + <para> + Amazon EC2 and OpenStack Compute (nova) images now re-fetch instance meta data and user data from the instance + metadata service (IMDS) on each boot. For example: stopping an EC2 instance, changing its user data, and + restarting the instance will now cause it to fetch and apply the new user data. + </para> + <warning> + <para> + Specifically, <literal>/etc/ec2-metadata</literal> is re-populated on each boot. Some NixOS scripts that read + from this directory are guarded to only run if the files they want to manipulate do not already exist, and so + will not re-apply their changes if the IMDS response changes. Examples: <literal>root</literal>'s SSH key is + only added if <literal>/root/.ssh/authorized_keys</literal> does not exist, and SSH host keys are only set from + user data if they do not exist in <literal>/etc/ssh</literal>. + </para> + </warning> + </listitem> + <listitem> + <para> + The <literal>rspamd</literal> services is now sandboxed. It is run as + a dynamic user instead of root, so secrets and other files may have to + be moved or their permissions may have to be fixed. The sockets are now + located in <literal>/run/rspamd</literal> instead of <literal>/run</literal>. + </para> + </listitem> </itemizedlist> </section> @@ -116,6 +288,88 @@ to <package>nextcloud20</package>. </para> </listitem> + <listitem> + <para> + The setting <xref linkend="opt-services.redis.bind" /> defaults to <literal>127.0.0.1</literal> now, making Redis listen on the loopback interface only, and not all public network interfaces. + </para> + </listitem> + <listitem> + <para> + NixOS now emits a deprecation warning if systemd's <literal>StartLimitInterval</literal> setting is used in a <literal>serviceConfig</literal> section instead of in a <literal>unitConfig</literal>; that setting is deprecated and now undocumented for the service section by systemd upstream, but still effective and somewhat buggy there, which can be confusing. See <link xlink:href="https://github.com/NixOS/nixpkgs/issues/45785">#45785</link> for details. + </para> + <para> + All services should use <xref linkend="opt-systemd.services._name_.startLimitIntervalSec" /> or <literal>StartLimitIntervalSec</literal> in <xref linkend="opt-systemd.services._name_.unitConfig" /> instead. + </para> + </listitem> + <listitem> + <para> + The Unbound DNS resolver service (<literal>services.unbound</literal>) has been refactored to allow reloading, control sockets and to fix startup ordering issues. + </para> + + <para> + It is now possible to enable a local UNIX control socket for unbound by setting the <xref linkend="opt-services.unbound.localControlSocketPath" /> + option. + </para> + + <para> + Previously we just applied a very minimal set of restrictions and + trusted unbound to properly drop root privs and capabilities. + </para> + + <para> + As of this we are (for the most part) just using the upstream + example unit file for unbound. The main difference is that we start + unbound as <literal>unbound</literal> user with the required capabilities instead of + letting unbound do the chroot & uid/gid changes. + </para> + + <para> + The upstream unit configuration this is based on is a lot stricter with + all kinds of permissions then our previous variant. It also came with + the default of having the <literal>Type</literal> set to <literal>notify</literal>, therefore we are now also + using the <literal>unbound-with-systemd</literal> package here. Unbound will start up, + read the configuration files and start listening on the configured ports + before systemd will declare the unit <literal>active (running)</literal>. + This will likely help with startup order and the occasional race condition during system + activation where the DNS service is started but not yet ready to answer + queries. Services depending on <literal>nss-lookup.target</literal> or <literal>unbound.service</literal> + are now be able to use unbound when those targets have been reached. + </para> + + <para> + Aditionally to the much stricter runtime environmet the + <literal>/dev/urandom</literal> mount lines we previously had in the code (that would + randomly failed during the stop-phase) have been removed as systemd will take care of those for us. + </para> + + <para> + The <literal>preStart</literal> script is now only required if we enabled the trust + anchor updates (which are still enabled by default). + </para> + + <para> + Another benefit of the refactoring is that we can now issue reloads via + either <literal>pkill -HUP unbound</literal> and <literal>systemctl reload unbound</literal> to reload the + running configuration without taking the daemon offline. A prerequisite + of this was that unbound configuration is available on a well known path + on the file system. We are using the path <literal>/etc/unbound/unbound.conf</literal> as that is the + default in the CLI tooling which in turn enables us to use + <literal>unbound-control</literal> without passing a custom configuration location. + </para> + </listitem> + <listitem> + <para> + NixOS now defaults to the unified cgroup hierarchy (cgroupsv2). + See the <link xlink:href="https://www.redhat.com/sysadmin/fedora-31-control-group-v2">Fedora Article for 31</link> + for details on why this is desirable, and how it impacts containers. + </para> + <para> + If you want to run containers with a runtime that does not yet support cgroupsv2, + you can switch back to the old behaviour by setting + <xref linkend="opt-systemd.enableUnifiedCgroupHierarchy"/> = <literal>false</literal>; + and rebooting. + </para> + </listitem> </itemizedlist> </section> </section> diff --git a/infra/libkookie/nixpkgs/nixos/doc/varlistentry-fixer.rb b/infra/libkookie/nixpkgs/nixos/doc/varlistentry-fixer.rb index 6c7cc1e6439b..02168016b554 100755 --- a/infra/libkookie/nixpkgs/nixos/doc/varlistentry-fixer.rb +++ b/infra/libkookie/nixpkgs/nixos/doc/varlistentry-fixer.rb @@ -15,8 +15,8 @@ require "rexml/document" include REXML if ARGV.length < 1 then - $stderr.puts "Needs a filename." - exit 1 + $stderr.puts "Needs a filename." + exit 1 end filename = ARGV.shift @@ -51,17 +51,17 @@ $touched = false # Generates: --optionnamevalue # ^^ ^^ doc.elements.each("//varlistentry/term") do |term| - ["varname", "function", "option", "replaceable"].each do |prev_name| - term.elements.each(prev_name) do |el| - if el.next_element and - el.next_element.name == "replaceable" and - el.next_sibling_node.class == Element - then - $touched = true - term.insert_after(el, Text.new(" ")) - end - end - end + ["varname", "function", "option", "replaceable"].each do |prev_name| + term.elements.each(prev_name) do |el| + if el.next_element and + el.next_element.name == "replaceable" and + el.next_sibling_node.class == Element + then + $touched = true + term.insert_after(el, Text.new(" ")) + end + end + end end @@ -75,17 +75,17 @@ end # Generates: -Ipath # ^^ doc.elements.each("//cmdsynopsis/arg") do |term| - ["option", "replaceable"].each do |prev_name| - term.elements.each(prev_name) do |el| - if el.next_element and - el.next_element.name == "replaceable" and - el.next_sibling_node.class == Element - then - $touched = true - term.insert_after(el, Text.new(" ")) - end - end - end + ["option", "replaceable"].each do |prev_name| + term.elements.each(prev_name) do |el| + if el.next_element and + el.next_element.name == "replaceable" and + el.next_sibling_node.class == Element + then + $touched = true + term.insert_after(el, Text.new(" ")) + end + end + end end # <cmdsynopsis> @@ -104,21 +104,21 @@ end # Generates: [{--profile-name | -p }name] # ^^^^ doc.elements.each("//cmdsynopsis/arg") do |term| - ["group"].each do |prev_name| - term.elements.each(prev_name) do |el| - if el.next_element and - el.next_element.name == "replaceable" and - el.next_sibling_node.class == Element - then - $touched = true - term.insert_after(el, Text.new(" ")) - end - end - end + ["group"].each do |prev_name| + term.elements.each(prev_name) do |el| + if el.next_element and + el.next_element.name == "replaceable" and + el.next_sibling_node.class == Element + then + $touched = true + term.insert_after(el, Text.new(" ")) + end + end + end end if $touched then - doc.context[:attribute_quote] = :quote - doc.write(output: File.open(filename, "w")) + doc.context[:attribute_quote] = :quote + doc.write(output: File.open(filename, "w")) end diff --git a/infra/libkookie/nixpkgs/nixos/lib/build-vms.nix b/infra/libkookie/nixpkgs/nixos/lib/build-vms.nix index b1575fc13bbd..ebbb0296bef6 100644 --- a/infra/libkookie/nixpkgs/nixos/lib/build-vms.nix +++ b/infra/libkookie/nixpkgs/nixos/lib/build-vms.nix @@ -18,9 +18,6 @@ rec { inherit pkgs; - qemu = pkgs.qemu_test; - - # Build a virtual network from an attribute set `{ machine1 = # config1; ... machineN = configN; }', where `machineX' is the # hostname and `configX' is a NixOS system configuration. Each @@ -39,7 +36,6 @@ rec { [ ../modules/virtualisation/qemu-vm.nix ../modules/testing/test-instrumentation.nix # !!! should only get added for automated test runs { key = "no-manual"; documentation.nixos.enable = false; } - { key = "qemu"; system.build.qemu = qemu; } { key = "nodes"; _module.args.nodes = nodes; } ] ++ optional minimal ../modules/testing/minimal-kernel.nix; }; diff --git a/infra/libkookie/nixpkgs/nixos/lib/make-disk-image.nix b/infra/libkookie/nixpkgs/nixos/lib/make-disk-image.nix index 8aa606a56af8..0ad0cf1fef5a 100644 --- a/infra/libkookie/nixpkgs/nixos/lib/make-disk-image.nix +++ b/infra/libkookie/nixpkgs/nixos/lib/make-disk-image.nix @@ -28,6 +28,9 @@ # partition of reasonable size is created in addition to the root partition. # For "legacy", the msdos partition table is used and a single large root # partition is created. + # For "legacy+gpt", the GPT partition table is used, a 1MiB no-fs partition for + # use by the bootloader is created, and a single large root partition is + # created. # For "hybrid", the GPT partition table is used and a mandatory ESP # partition of reasonable size is created in addition to the root partition. # Also a legacy MBR will be present. @@ -54,7 +57,7 @@ format ? "raw" }: -assert partitionTableType == "legacy" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none"; +assert partitionTableType == "legacy" || partitionTableType == "legacy+gpt" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none"; # We use -E offset=X below, which is only supported by e2fsprogs assert partitionTableType != "none" -> fsType == "ext4"; @@ -75,6 +78,7 @@ let format' = format; in let rootPartition = { # switch-case legacy = "1"; + "legacy+gpt" = "2"; efi = "2"; hybrid = "3"; }.${partitionTableType}; @@ -85,6 +89,16 @@ let format' = format; in let mklabel msdos \ mkpart primary ext4 1MiB -1 ''; + "legacy+gpt" = '' + parted --script $diskImage -- \ + mklabel gpt \ + mkpart no-fs 1MB 2MB \ + set 1 bios_grub on \ + align-check optimal 1 \ + mkpart primary ext4 2MB -1 \ + align-check optimal 2 \ + print + ''; efi = '' parted --script $diskImage -- \ mklabel gpt \ @@ -120,7 +134,7 @@ let format' = format; in let binPath = with pkgs; makeBinPath ( [ rsync - utillinux + util-linux parted e2fsprogs lkl @@ -225,7 +239,7 @@ let format' = format; in let in pkgs.vmTools.runInLinuxVM ( pkgs.runCommand name { preVM = prepareImage; - buildInputs = with pkgs; [ utillinux e2fsprogs dosfstools ]; + buildInputs = with pkgs; [ util-linux e2fsprogs dosfstools ]; postVM = '' ${if format == "raw" then '' mv $diskImage $out/${filename} diff --git a/infra/libkookie/nixpkgs/nixos/lib/make-iso9660-image.nix b/infra/libkookie/nixpkgs/nixos/lib/make-iso9660-image.nix index 6a0e0e7c635a..549530965f6e 100644 --- a/infra/libkookie/nixpkgs/nixos/lib/make-iso9660-image.nix +++ b/infra/libkookie/nixpkgs/nixos/lib/make-iso9660-image.nix @@ -48,7 +48,7 @@ assert usbBootable -> isohybridMbrImage != ""; stdenv.mkDerivation { name = isoName; builder = ./make-iso9660-image.sh; - buildInputs = [ xorriso syslinux zstd libossp_uuid ]; + nativeBuildInputs = [ xorriso syslinux zstd libossp_uuid ]; inherit isoName bootable bootImage compressImage volumeID efiBootImage efiBootable isohybridMbrImage usbBootable; diff --git a/infra/libkookie/nixpkgs/nixos/lib/make-system-tarball.nix b/infra/libkookie/nixpkgs/nixos/lib/make-system-tarball.nix index dee91a6ce3f4..dab168f4a481 100644 --- a/infra/libkookie/nixpkgs/nixos/lib/make-system-tarball.nix +++ b/infra/libkookie/nixpkgs/nixos/lib/make-system-tarball.nix @@ -37,7 +37,7 @@ in stdenv.mkDerivation { name = "tarball"; builder = ./make-system-tarball.sh; - buildInputs = extraInputs; + nativeBuildInputs = extraInputs; inherit fileName extraArgs extraCommands compressCommand; diff --git a/infra/libkookie/nixpkgs/nixos/lib/test-driver/test-driver.py b/infra/libkookie/nixpkgs/nixos/lib/test-driver/test-driver.py index 156392ad1e32..96b75a49928f 100644 --- a/infra/libkookie/nixpkgs/nixos/lib/test-driver/test-driver.py +++ b/infra/libkookie/nixpkgs/nixos/lib/test-driver/test-driver.py @@ -110,7 +110,6 @@ def create_vlan(vlan_nr: str) -> Tuple[str, str, "subprocess.Popen[bytes]", Any] pty_master, pty_slave = pty.openpty() vde_process = subprocess.Popen( ["vde_switch", "-s", vde_socket, "--dirmode", "0700"], - bufsize=1, stdin=pty_slave, stdout=subprocess.PIPE, stderr=subprocess.PIPE, @@ -635,8 +634,7 @@ class Machine: shutil.copy(intermediate, abs_target) def dump_tty_contents(self, tty: str) -> None: - """Debugging: Dump the contents of the TTY<n> - """ + """Debugging: Dump the contents of the TTY<n>""" self.execute("fold -w 80 /dev/vcs{} | systemd-cat".format(tty)) def get_screen_text(self) -> str: @@ -748,7 +746,6 @@ class Machine: self.process = subprocess.Popen( self.script, - bufsize=1, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, @@ -862,8 +859,7 @@ class Machine: self.send_monitor_command("set_link virtio-net-pci.1 off") def unblock(self) -> None: - """Make the machine reachable. - """ + """Make the machine reachable.""" self.send_monitor_command("set_link virtio-net-pci.1 on") diff --git a/infra/libkookie/nixpkgs/nixos/lib/testing-python.nix b/infra/libkookie/nixpkgs/nixos/lib/testing-python.nix index 498f97336c02..13abfb9a111d 100644 --- a/infra/libkookie/nixpkgs/nixos/lib/testing-python.nix +++ b/infra/libkookie/nixpkgs/nixos/lib/testing-python.nix @@ -3,13 +3,13 @@ # Use a minimal kernel? , minimal ? false # Ignored -, config ? {} +, config ? { } # !!! See comment about args in lib/modules.nix -, specialArgs ? {} +, specialArgs ? { } # Modules to add to each VM -, extraConfigurations ? [] }: +, extraConfigurations ? [ ] +}: -with import ./build-vms.nix { inherit system pkgs minimal specialArgs extraConfigurations; }; with pkgs; rec { @@ -17,40 +17,41 @@ rec { inherit pkgs; - testDriver = let - testDriverScript = ./test-driver/test-driver.py; - in stdenv.mkDerivation { - name = "nixos-test-driver"; - - nativeBuildInputs = [ makeWrapper ]; - buildInputs = [ (python3.withPackages (p: [ p.ptpython ])) ]; - checkInputs = with python3Packages; [ pylint black mypy ]; - - dontUnpack = true; + mkTestDriver = + let + testDriverScript = ./test-driver/test-driver.py; + in + qemu_pkg: stdenv.mkDerivation { + name = "nixos-test-driver"; - preferLocalBuild = true; + nativeBuildInputs = [ makeWrapper ]; + buildInputs = [ (python3.withPackages (p: [ p.ptpython ])) ]; + checkInputs = with python3Packages; [ pylint black mypy ]; - doCheck = true; - checkPhase = '' - mypy --disallow-untyped-defs \ - --no-implicit-optional \ - --ignore-missing-imports ${testDriverScript} - pylint --errors-only ${testDriverScript} - black --check --diff ${testDriverScript} - ''; + dontUnpack = true; - installPhase = - '' - mkdir -p $out/bin - cp ${testDriverScript} $out/bin/nixos-test-driver - chmod u+x $out/bin/nixos-test-driver - # TODO: copy user script part into this file (append) + preferLocalBuild = true; - wrapProgram $out/bin/nixos-test-driver \ - --prefix PATH : "${lib.makeBinPath [ qemu_test vde2 netpbm coreutils ]}" \ + doCheck = true; + checkPhase = '' + mypy --disallow-untyped-defs \ + --no-implicit-optional \ + --ignore-missing-imports ${testDriverScript} + pylint --errors-only ${testDriverScript} + black --check --diff ${testDriverScript} ''; - }; + installPhase = + '' + mkdir -p $out/bin + cp ${testDriverScript} $out/bin/nixos-test-driver + chmod u+x $out/bin/nixos-test-driver + # TODO: copy user script part into this file (append) + + wrapProgram $out/bin/nixos-test-driver \ + --prefix PATH : "${lib.makeBinPath [ qemu_pkg vde2 netpbm coreutils ]}" \ + ''; + }; # Run an automated test suite in the given virtual network. # `driver' is the script that runs the network. @@ -73,11 +74,10 @@ rec { { testScript , enableOCR ? false , name ? "unnamed" - # Skip linting (mainly intended for faster dev cycles) + # Skip linting (mainly intended for faster dev cycles) , skipLint ? false , ... } @ t: - let # A standard store path to the vm monitor is built like this: # /tmp/nix-build-vm-test-run-$name.drv-0/vm-state-machine/monitor @@ -86,25 +86,7 @@ rec { maxTestNameLen = 50; testNameLen = builtins.stringLength name; - testDriverName = with builtins; - if testNameLen > maxTestNameLen then - abort ("The name of the test '${name}' must not be longer than ${toString maxTestNameLen} " + - "it's currently ${toString testNameLen} characters long.") - else - "nixos-test-driver-${name}"; - nodes = buildVirtualNetwork ( - t.nodes or (if t ? machine then { machine = t.machine; } else { })); - - testScript' = - # Call the test script with the computed nodes. - if lib.isFunction testScript - then testScript { inherit nodes; } - else testScript; - - vlans = map (m: m.config.virtualisation.vlans) (lib.attrValues nodes); - - vms = map (m: m.config.system.build.vm) (lib.attrValues nodes); ocrProg = tesseract4.override { enableLanguages = [ "eng" ]; }; @@ -113,72 +95,124 @@ rec { # Generate convenience wrappers for running the test driver # interactively with the specified network, and for starting the # VMs from the command line. - driver = let warn = if skipLint then lib.warn "Linting is disabled!" else lib.id; in warn (runCommand testDriverName - { buildInputs = [ makeWrapper]; - testScript = testScript'; - preferLocalBuild = true; - testName = name; - } - '' - mkdir -p $out/bin - - echo -n "$testScript" > $out/test-script - ${lib.optionalString (!skipLint) '' - ${python3Packages.black}/bin/black --check --diff $out/test-script - ''} - - ln -s ${testDriver}/bin/nixos-test-driver $out/bin/ - vms=($(for i in ${toString vms}; do echo $i/bin/run-*-vm; done)) - wrapProgram $out/bin/nixos-test-driver \ - --add-flags "''${vms[*]}" \ - ${lib.optionalString enableOCR - "--prefix PATH : '${ocrProg}/bin:${imagemagick_tiff}/bin'"} \ - --run "export testScript=\"\$(${coreutils}/bin/cat $out/test-script)\"" \ - --set VLANS '${toString vlans}' - ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-run-vms - wrapProgram $out/bin/nixos-run-vms \ - --add-flags "''${vms[*]}" \ - ${lib.optionalString enableOCR "--prefix PATH : '${ocrProg}/bin'"} \ - --set tests 'start_all(); join_all();' \ - --set VLANS '${toString vlans}' \ - ${lib.optionalString (builtins.length vms == 1) "--set USE_SERIAL 1"} - ''); # " + mkDriver = qemu_pkg: + let + build-vms = import ./build-vms.nix { + inherit system pkgs minimal specialArgs; + extraConfigurations = extraConfigurations ++ (pkgs.lib.optional (qemu_pkg != null) + { + virtualisation.qemu.package = qemu_pkg; + } + ); + }; + + # FIXME: get this pkg from the module system + testDriver = mkTestDriver (if qemu_pkg == null then pkgs.qemu_test else qemu_pkg); + + nodes = build-vms.buildVirtualNetwork ( + t.nodes or (if t ? machine then { machine = t.machine; } else { }) + ); + vlans = map (m: m.config.virtualisation.vlans) (lib.attrValues nodes); + vms = map (m: m.config.system.build.vm) (lib.attrValues nodes); + + testScript' = + # Call the test script with the computed nodes. + if lib.isFunction testScript + then testScript { inherit nodes; } + else testScript; + + testDriverName = with builtins; + if testNameLen > maxTestNameLen then + abort + ("The name of the test '${name}' must not be longer than ${toString maxTestNameLen} " + + "it's currently ${toString testNameLen} characters long.") + else + "nixos-test-driver-${name}"; + + warn = if skipLint then lib.warn "Linting is disabled!" else lib.id; + in + warn (runCommand testDriverName + { + buildInputs = [ makeWrapper ]; + testScript = testScript'; + preferLocalBuild = true; + testName = name; + passthru = { + inherit nodes; + }; + } + '' + mkdir -p $out/bin + + echo -n "$testScript" > $out/test-script + ${lib.optionalString (!skipLint) '' + ${python3Packages.black}/bin/black --check --diff $out/test-script + ''} + + ln -s ${testDriver}/bin/nixos-test-driver $out/bin/ + vms=($(for i in ${toString vms}; do echo $i/bin/run-*-vm; done)) + wrapProgram $out/bin/nixos-test-driver \ + --add-flags "''${vms[*]}" \ + ${lib.optionalString enableOCR + "--prefix PATH : '${ocrProg}/bin:${imagemagick_tiff}/bin'"} \ + --run "export testScript=\"\$(${coreutils}/bin/cat $out/test-script)\"" \ + --set VLANS '${toString vlans}' + ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-run-vms + wrapProgram $out/bin/nixos-run-vms \ + --add-flags "''${vms[*]}" \ + ${lib.optionalString enableOCR "--prefix PATH : '${ocrProg}/bin'"} \ + --set tests 'start_all(); join_all();' \ + --set VLANS '${toString vlans}' \ + ${lib.optionalString (builtins.length vms == 1) "--set USE_SERIAL 1"} + ''); # " passMeta = drv: drv // lib.optionalAttrs (t ? meta) { - meta = (drv.meta or {}) // t.meta; + meta = (drv.meta or { }) // t.meta; }; + driver = mkDriver null; + driverInteractive = mkDriver pkgs.qemu; + test = passMeta (runTests driver); - nodeNames = builtins.attrNames nodes; + nodeNames = builtins.attrNames driver.nodes; invalidNodeNames = lib.filter - (node: builtins.match "^[A-z_]([A-z0-9_]+)?$" node == null) nodeNames; + (node: builtins.match "^[A-z_]([A-z0-9_]+)?$" node == null) + nodeNames; in - if lib.length invalidNodeNames > 0 then - throw '' - Cannot create machines out of (${lib.concatStringsSep ", " invalidNodeNames})! - All machines are referenced as python variables in the testing framework which will break the - script when special characters are used. + if lib.length invalidNodeNames > 0 then + throw '' + Cannot create machines out of (${lib.concatStringsSep ", " invalidNodeNames})! + All machines are referenced as python variables in the testing framework which will break the + script when special characters are used. - Please stick to alphanumeric chars and underscores as separation. - '' - else - test // { - inherit nodes driver test; - }; + Please stick to alphanumeric chars and underscores as separation. + '' + else + test // { + inherit test driver driverInteractive; + inherit (driver) nodes; + }; runInMachine = { drv , machine , preBuild ? "" , postBuild ? "" + , qemu ? pkgs.qemu_test , ... # ??? }: let - vm = buildVM { } - [ machine - { key = "run-in-machine"; + build-vms = import ./build-vms.nix { + inherit system pkgs minimal specialArgs extraConfigurations; + }; + + vm = build-vms.buildVM { } + [ + machine + { + key = "run-in-machine"; networking.hostName = "client"; nix.readOnlyStore = false; virtualisation.writableStore = false; @@ -221,20 +255,20 @@ rec { unset xchg export tests='${testScript}' - ${testDriver}/bin/nixos-test-driver ${vm.config.system.build.vm}/bin/run-*-vm + ${mkTestDriver qemu}/bin/nixos-test-driver --keep-vm-state ${vm.config.system.build.vm}/bin/run-*-vm ''; # */ in - lib.overrideDerivation drv (attrs: { - requiredSystemFeatures = [ "kvm" ]; - builder = "${bash}/bin/sh"; - args = ["-e" vmRunCommand]; - origArgs = attrs.args; - origBuilder = attrs.builder; - }); + lib.overrideDerivation drv (attrs: { + requiredSystemFeatures = [ "kvm" ]; + builder = "${bash}/bin/sh"; + args = [ "-e" vmRunCommand ]; + origArgs = attrs.args; + origBuilder = attrs.builder; + }); - runInMachineWithX = { require ? [], ... } @ args: + runInMachineWithX = { require ? [ ], ... } @ args: let client = { ... }: @@ -250,13 +284,13 @@ rec { services.xserver.windowManager.icewm.enable = true; }; in - runInMachine ({ - machine = client; - preBuild = - '' - client.wait_for_x() - ''; - } // args); + runInMachine ({ + machine = client; + preBuild = + '' + client.wait_for_x() + ''; + } // args); simpleTest = as: (makeTest as).test; diff --git a/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/amazon-image.nix b/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/amazon-image.nix index b09f4ca47a3f..0ecf07669a11 100644 --- a/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/amazon-image.nix +++ b/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/amazon-image.nix @@ -57,7 +57,7 @@ in { inherit (cfg) contents format name; pkgs = import ../../../.. { inherit (pkgs) system; }; # ensure we use the regular qemu-kvm package partitionTableType = if config.ec2.efi then "efi" - else if config.ec2.hvm then "legacy" + else if config.ec2.hvm then "legacy+gpt" else "none"; diskSize = cfg.sizeMB; fsType = "ext4"; diff --git a/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/create-amis.sh b/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/create-amis.sh index 89e24f2ccfd9..ec2eb5366790 100755 --- a/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/create-amis.sh +++ b/infra/libkookie/nixpkgs/nixos/maintainers/scripts/ec2/create-amis.sh @@ -1,13 +1,15 @@ #!/usr/bin/env nix-shell #!nix-shell -p awscli -p jq -p qemu -i bash +# shellcheck shell=bash # Uploads and registers NixOS images built from the # <nixos/release.nix> amazonImage attribute. Images are uploaded and # registered via a home region, and then copied to other regions. -# The home region requires an s3 bucket, and a "vmimport" IAM role -# with access to the S3 bucket. Configuration of the vmimport role is -# documented in +# The home region requires an s3 bucket, and an IAM role named "vmimport" +# (by default) with access to the S3 bucket. The name can be +# configured with the "service_role_name" variable. Configuration of the +# vmimport role is documented in # https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html # set -x @@ -17,6 +19,7 @@ set -euo pipefail state_dir=$HOME/amis/ec2-images home_region=eu-west-1 bucket=nixos-amis +service_role_name=vmimport regions=(eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1 us-east-1 us-east-2 us-west-1 us-west-2 @@ -64,7 +67,7 @@ image_logical_bytes=$(read_image_info .logical_bytes) # Derived attributes -image_logical_gigabytes=$((($image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB +image_logical_gigabytes=$(((image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB case "$image_system" in aarch64-linux) @@ -100,7 +103,7 @@ write_state() { local type=$2 local val=$3 - mkdir -p $state_dir + mkdir -p "$state_dir" echo "$val" > "$state_dir/$state_key.$type" } @@ -110,8 +113,8 @@ wait_for_import() { local state snapshot_id log "Waiting for import task $task_id to be completed" while true; do - read state progress snapshot_id < <( - aws ec2 describe-import-snapshot-tasks --region $region --import-task-ids "$task_id" | \ + read -r state progress snapshot_id < <( + aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" | \ jq -r '.ImportSnapshotTasks[].SnapshotTaskDetail | "\(.Status) \(.Progress) \(.SnapshotId)"' ) log " ... state=$state progress=$progress snapshot_id=$snapshot_id" @@ -125,6 +128,8 @@ wait_for_import() { ;; *) log "Unexpected snapshot import state: '${state}'" + log "Full response: " + aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" >&2 exit 1 ;; esac @@ -138,8 +143,8 @@ wait_for_image() { log "Waiting for image $ami_id to be available" while true; do - read state < <( - aws ec2 describe-images --image-ids "$ami_id" --region $region | \ + read -r state < <( + aws ec2 describe-images --image-ids "$ami_id" --region "$region" | \ jq -r ".Images[].State" ) log " ... state=$state" @@ -163,7 +168,7 @@ make_image_public() { local region=$1 local ami_id=$2 - wait_for_image $region "$ami_id" + wait_for_image "$region" "$ami_id" log "Making image $ami_id public" @@ -177,27 +182,30 @@ upload_image() { local aws_path=${image_file#/} local state_key="$region.$image_label.$image_system" - local task_id=$(read_state "$state_key" task_id) - local snapshot_id=$(read_state "$state_key" snapshot_id) - local ami_id=$(read_state "$state_key" ami_id) + local task_id + task_id=$(read_state "$state_key" task_id) + local snapshot_id + snapshot_id=$(read_state "$state_key" snapshot_id) + local ami_id + ami_id=$(read_state "$state_key" ami_id) if [ -z "$task_id" ]; then log "Checking for image on S3" if ! aws s3 ls --region "$region" "s3://${bucket}/${aws_path}" >&2; then log "Image missing from aws, uploading" - aws s3 cp --region $region "$image_file" "s3://${bucket}/${aws_path}" >&2 + aws s3 cp --region "$region" "$image_file" "s3://${bucket}/${aws_path}" >&2 fi log "Importing image from S3 path s3://$bucket/$aws_path" - task_id=$(aws ec2 import-snapshot --disk-container "{ + task_id=$(aws ec2 import-snapshot --role-name "$service_role_name" --disk-container "{ \"Description\": \"nixos-image-${image_label}-${image_system}\", \"Format\": \"vhd\", \"UserBucket\": { \"S3Bucket\": \"$bucket\", \"S3Key\": \"$aws_path\" } - }" --region $region | jq -r '.ImportTaskId') + }" --region "$region" | jq -r '.ImportTaskId') write_state "$state_key" task_id "$task_id" fi @@ -221,16 +229,16 @@ upload_image() { --virtualization-type hvm ) - block_device_mappings+=(DeviceName=/dev/sdb,VirtualName=ephemeral0) - block_device_mappings+=(DeviceName=/dev/sdc,VirtualName=ephemeral1) - block_device_mappings+=(DeviceName=/dev/sdd,VirtualName=ephemeral2) - block_device_mappings+=(DeviceName=/dev/sde,VirtualName=ephemeral3) + block_device_mappings+=("DeviceName=/dev/sdb,VirtualName=ephemeral0") + block_device_mappings+=("DeviceName=/dev/sdc,VirtualName=ephemeral1") + block_device_mappings+=("DeviceName=/dev/sdd,VirtualName=ephemeral2") + block_device_mappings+=("DeviceName=/dev/sde,VirtualName=ephemeral3") ami_id=$( aws ec2 register-image \ --name "$image_name" \ --description "$image_description" \ - --region $region \ + --region "$region" \ --architecture $amazon_arch \ --block-device-mappings "${block_device_mappings[@]}" \ "${extra_flags[@]}" \ @@ -240,7 +248,7 @@ upload_image() { write_state "$state_key" ami_id "$ami_id" fi - make_image_public $region "$ami_id" + make_image_public "$region" "$ami_id" echo "$ami_id" } @@ -268,7 +276,7 @@ copy_to_region() { write_state "$state_key" ami_id "$ami_id" fi - make_image_public $region "$ami_id" + make_image_public "$region" "$ami_id" echo "$ami_id" } diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/fonts/fontdir.nix b/infra/libkookie/nixpkgs/nixos/modules/config/fonts/fontdir.nix index 264d73ebafa5..c4bd3a077d33 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/fonts/fontdir.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/fonts/fontdir.nix @@ -8,7 +8,7 @@ let x11Fonts = pkgs.runCommand "X11-fonts" { preferLocalBuild = true; } '' mkdir -p "$out/share/X11/fonts" - font_regexp='.*\.\(ttf\|otf\|pcf\|pfa\|pfb\|bdf\)\(\.gz\)?' + font_regexp='.*\.\(ttf\|ttc\|otf\|pcf\|pfa\|pfb\|bdf\)\(\.gz\)?' find ${toString config.fonts.fonts} -regex "$font_regexp" \ -exec ln -sf -t "$out/share/X11/fonts" '{}' \; cd "$out/share/X11/fonts" diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/malloc.nix b/infra/libkookie/nixpkgs/nixos/modules/config/malloc.nix index 31a659ee83fe..a3eb55d8a42e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/malloc.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/malloc.nix @@ -23,7 +23,7 @@ let }; scudo = { - libPath = "${pkgs.llvmPackages.compiler-rt}/lib/linux/libclang_rt.scudo-x86_64.so"; + libPath = "${pkgs.llvmPackages_latest.compiler-rt}/lib/linux/libclang_rt.scudo-x86_64.so"; description = '' A user-mode allocator based on LLVM Sanitizer’s CombinedAllocator, which aims at providing additional mitigations against heap based diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/no-x-libs.nix b/infra/libkookie/nixpkgs/nixos/modules/config/no-x-libs.nix index 941ab78f8632..c3120c2bf30d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/no-x-libs.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/no-x-libs.nix @@ -30,11 +30,12 @@ with lib; cairo = super.cairo.override { x11Support = false; }; dbus = super.dbus.override { x11Support = false; }; networkmanager-fortisslvpn = super.networkmanager-fortisslvpn.override { withGnome = false; }; + networkmanager-iodine = super.networkmanager-iodine.override { withGnome = false; }; networkmanager-l2tp = super.networkmanager-l2tp.override { withGnome = false; }; networkmanager-openconnect = super.networkmanager-openconnect.override { withGnome = false; }; networkmanager-openvpn = super.networkmanager-openvpn.override { withGnome = false; }; + networkmanager-sstp = super.networkmanager-vpnc.override { withGnome = false; }; networkmanager-vpnc = super.networkmanager-vpnc.override { withGnome = false; }; - networkmanager-iodine = super.networkmanager-iodine.override { withGnome = false; }; gobject-introspection = super.gobject-introspection.override { x11Support = false; }; qemu = super.qemu.override { gtkSupport = false; spiceSupport = false; sdlSupport = false; }; })); diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/pulseaudio.nix b/infra/libkookie/nixpkgs/nixos/modules/config/pulseaudio.nix index 408d0a9c33f2..a77524d75d8d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/pulseaudio.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/pulseaudio.nix @@ -36,6 +36,8 @@ let ${addModuleIf cfg.zeroconf.discovery.enable "module-zeroconf-discover"} ${addModuleIf cfg.tcp.enable (concatStringsSep " " ([ "module-native-protocol-tcp" ] ++ allAnon ++ ipAnon))} + ${addModuleIf config.services.jack.jackd.enable "module-jack-sink"} + ${addModuleIf config.services.jack.jackd.enable "module-jack-source"} ${cfg.extraConfig} ''; }; @@ -144,7 +146,9 @@ in { package = mkOption { type = types.package; - default = pkgs.pulseaudio; + default = if config.services.jack.jackd.enable + then pkgs.pulseaudioFull + else pkgs.pulseaudio; defaultText = "pkgs.pulseaudio"; example = literalExample "pkgs.pulseaudioFull"; description = '' @@ -259,7 +263,7 @@ in { (drv: drv.override { pulseaudio = overriddenPackage; }) cfg.extraModules; modulePaths = builtins.map - (drv: "${drv}/lib/pulse-${overriddenPackage.version}/modules") + (drv: "${drv}/${overriddenPackage.pulseDir}/modules") # User-provided extra modules take precedence (overriddenModules ++ [ overriddenPackage ]); in lib.concatStringsSep ":" modulePaths; @@ -284,6 +288,8 @@ in { RestartSec = "500ms"; PassEnvironment = "DISPLAY"; }; + } // optionalAttrs config.services.jack.jackd.enable { + environment.JACK_PROMISCUOUS_SERVER = "jackaudio"; }; sockets.pulseaudio = { wantedBy = [ "sockets.target" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/swap.nix b/infra/libkookie/nixpkgs/nixos/modules/config/swap.nix index adb4e2294213..4bb66e9b5144 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/swap.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/swap.nix @@ -187,7 +187,7 @@ in before = [ "${realDevice'}.swap" ]; # If swap is encrypted, depending on rngd resolves a possible entropy starvation during boot after = mkIf (config.security.rngd.enable && sw.randomEncryption.enable) [ "rngd.service" ]; - path = [ pkgs.utillinux ] ++ optional sw.randomEncryption.enable pkgs.cryptsetup; + path = [ pkgs.util-linux ] ++ optional sw.randomEncryption.enable pkgs.cryptsetup; script = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/system-path.nix b/infra/libkookie/nixpkgs/nixos/modules/config/system-path.nix index c46937f80081..27d1cef849bc 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/system-path.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/system-path.nix @@ -33,10 +33,11 @@ let pkgs.ncurses pkgs.netcat config.programs.ssh.package + pkgs.mkpasswd pkgs.procps pkgs.su pkgs.time - pkgs.utillinux + pkgs.util-linux pkgs.which pkgs.zstd ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/update-users-groups.pl b/infra/libkookie/nixpkgs/nixos/modules/config/update-users-groups.pl index e220aa610908..fd3affae899c 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/update-users-groups.pl +++ b/infra/libkookie/nixpkgs/nixos/modules/config/update-users-groups.pl @@ -16,8 +16,7 @@ my $gidMap = -e $gidMapFile ? decode_json(read_file($gidMapFile)) : {}; sub updateFile { my ($path, $contents, $perms) = @_; - write_file("$path.tmp", { binmode => ':utf8', perms => $perms // 0644 }, $contents); - rename("$path.tmp", $path) or die; + write_file($path, { atomic => 1, binmode => ':utf8', perms => $perms // 0644 }, $contents) or die; } @@ -98,7 +97,7 @@ sub parseGroup { return ($f[0], { name => $f[0], password => $f[1], gid => $gid, members => $f[3] }); } -my %groupsCur = -f "/etc/group" ? map { parseGroup } read_file("/etc/group") : (); +my %groupsCur = -f "/etc/group" ? map { parseGroup } read_file("/etc/group", { binmode => ":utf8" }) : (); # Read the current /etc/passwd. sub parseUser { @@ -109,20 +108,19 @@ sub parseUser { return ($f[0], { name => $f[0], fakePassword => $f[1], uid => $uid, gid => $f[3], description => $f[4], home => $f[5], shell => $f[6] }); } - -my %usersCur = -f "/etc/passwd" ? map { parseUser } read_file("/etc/passwd") : (); +my %usersCur = -f "/etc/passwd" ? map { parseUser } read_file("/etc/passwd", { binmode => ":utf8" }) : (); # Read the groups that were created declaratively (i.e. not by groups) # in the past. These must be removed if they are no longer in the # current spec. my $declGroupsFile = "/var/lib/nixos/declarative-groups"; my %declGroups; -$declGroups{$_} = 1 foreach split / /, -e $declGroupsFile ? read_file($declGroupsFile) : ""; +$declGroups{$_} = 1 foreach split / /, -e $declGroupsFile ? read_file($declGroupsFile, { binmode => ":utf8" }) : ""; # Idem for the users. my $declUsersFile = "/var/lib/nixos/declarative-users"; my %declUsers; -$declUsers{$_} = 1 foreach split / /, -e $declUsersFile ? read_file($declUsersFile) : ""; +$declUsers{$_} = 1 foreach split / /, -e $declUsersFile ? read_file($declUsersFile, { binmode => ":utf8" }) : ""; # Generate a new /etc/group containing the declared groups. @@ -175,7 +173,7 @@ foreach my $name (keys %groupsCur) { # Rewrite /etc/group. FIXME: acquire lock. my @lines = map { join(":", $_->{name}, $_->{password}, $_->{gid}, $_->{members}) . "\n" } (sort { $a->{gid} <=> $b->{gid} } values(%groupsOut)); -updateFile($gidMapFile, encode_json($gidMap)); +updateFile($gidMapFile, to_json($gidMap)); updateFile("/etc/group", \@lines); system("nscd --invalidate group"); @@ -251,7 +249,7 @@ foreach my $name (keys %usersCur) { # Rewrite /etc/passwd. FIXME: acquire lock. @lines = map { join(":", $_->{name}, $_->{fakePassword}, $_->{uid}, $_->{gid}, $_->{description}, $_->{home}, $_->{shell}) . "\n" } (sort { $a->{uid} <=> $b->{uid} } (values %usersOut)); -updateFile($uidMapFile, encode_json($uidMap)); +updateFile($uidMapFile, to_json($uidMap)); updateFile("/etc/passwd", \@lines); system("nscd --invalidate passwd"); @@ -260,7 +258,7 @@ system("nscd --invalidate passwd"); my @shadowNew; my %shadowSeen; -foreach my $line (-f "/etc/shadow" ? read_file("/etc/shadow") : ()) { +foreach my $line (-f "/etc/shadow" ? read_file("/etc/shadow", { binmode => ":utf8" }) : ()) { chomp $line; my ($name, $hashedPassword, @rest) = split(':', $line, -9); my $u = $usersOut{$name};; diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/users-groups.nix b/infra/libkookie/nixpkgs/nixos/modules/config/users-groups.nix index 1bb1317a8e85..72285fe631da 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/users-groups.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/users-groups.nix @@ -35,8 +35,7 @@ let ''; hashedPasswordDescription = '' - To generate a hashed password install the <literal>mkpasswd</literal> - package and run <literal>mkpasswd -m sha-512</literal>. + To generate a hashed password run <literal>mkpasswd -m sha-512</literal>. If set to an empty string (<literal>""</literal>), this user will be able to log in without being asked for a password (but not via remote @@ -139,6 +138,20 @@ let ''; }; + pamMount = mkOption { + type = with types; attrsOf str; + default = {}; + description = '' + Attributes for user's entry in + <filename>pam_mount.conf.xml</filename>. + Useful attributes might include <code>path</code>, + <code>options</code>, <code>fstype</code>, and <code>server</code>. + See <link + xlink:href="http://pam-mount.sourceforge.net/pam_mount.conf.5.html" /> + for more information. + ''; + }; + shell = mkOption { type = types.either types.shellPackage types.path; default = pkgs.shadow; diff --git a/infra/libkookie/nixpkgs/nixos/modules/config/zram.nix b/infra/libkookie/nixpkgs/nixos/modules/config/zram.nix index 5e9870bf6b1c..1f513b7e4dae 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/config/zram.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/config/zram.nix @@ -80,6 +80,15 @@ in ''; }; + memoryMax = mkOption { + default = null; + type = with types; nullOr int; + description = '' + Maximum total amount of memory (in bytes) that can be used by the zram + swap devices. + ''; + }; + priority = mkOption { default = 5; type = types.int; @@ -146,11 +155,16 @@ in # Calculate memory to use for zram mem=$(${pkgs.gawk}/bin/awk '/MemTotal: / { - print int($2*${toString cfg.memoryPercent}/100.0/${toString devicesCount}*1024) + value=int($2*${toString cfg.memoryPercent}/100.0/${toString devicesCount}*1024); + ${lib.optionalString (cfg.memoryMax != null) '' + memory_max=int(${toString cfg.memoryMax}/${toString devicesCount}); + if (value > memory_max) { value = memory_max } + ''} + print value }' /proc/meminfo) - ${pkgs.utillinux}/sbin/zramctl --size $mem --algorithm ${cfg.algorithm} /dev/${dev} - ${pkgs.utillinux}/sbin/mkswap /dev/${dev} + ${pkgs.util-linux}/sbin/zramctl --size $mem --algorithm ${cfg.algorithm} /dev/${dev} + ${pkgs.util-linux}/sbin/mkswap /dev/${dev} ''; restartIfChanged = false; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/hardware/acpilight.nix b/infra/libkookie/nixpkgs/nixos/modules/hardware/acpilight.nix index 34e8a2220965..2de448a265c7 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/hardware/acpilight.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/hardware/acpilight.nix @@ -19,6 +19,7 @@ in }; config = mkIf cfg.enable { + environment.systemPackages = with pkgs; [ acpilight ]; services.udev.packages = with pkgs; [ acpilight ]; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/hardware/rtl-sdr.nix b/infra/libkookie/nixpkgs/nixos/modules/hardware/rtl-sdr.nix new file mode 100644 index 000000000000..77c8cb59a3d5 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/hardware/rtl-sdr.nix @@ -0,0 +1,20 @@ +{ config, lib, pkgs, ... }: + +let + cfg = config.hardware.rtl-sdr; + +in { + options.hardware.rtl-sdr = { + enable = lib.mkEnableOption '' + Enables rtl-sdr udev rules and ensures 'plugdev' group exists. + This is a prerequisite to using devices supported by rtl-sdr without + being root, since rtl-sdr USB descriptors will be owned by plugdev + through udev. + ''; + }; + + config = lib.mkIf cfg.enable { + services.udev.packages = [ pkgs.rtl-sdr ]; + users.groups.plugdev = {}; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/hardware/video/nvidia.nix b/infra/libkookie/nixpkgs/nixos/modules/hardware/video/nvidia.nix index 2acb891f1a9a..d1cf7d05c1b8 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/hardware/video/nvidia.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/hardware/video/nvidia.nix @@ -235,7 +235,7 @@ in hardware.opengl.extraPackages32 = optional offloadCfg.enable nvidia_libs32; environment.systemPackages = [ nvidia_x11.bin nvidia_x11.settings ] - ++ filter (p: p != null) [ nvidia_x11.persistenced ]; + ++ optionals nvidiaPersistencedEnabled [ nvidia_x11.persistenced ]; systemd.packages = optional cfg.powerManagement.enable nvidia_x11.out; diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix b/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix index 8c98691116dc..803bae4212ef 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix @@ -9,7 +9,14 @@ with lib; isoImage.edition = "gnome"; - services.xserver.desktopManager.gnome3.enable = true; + services.xserver.desktopManager.gnome3 = { + # Add firefox to favorite-apps + favoriteAppsOverride = '' + [org.gnome.shell] + favorite-apps=[ 'firefox.desktop', 'org.gnome.Geary.desktop', 'org.gnome.Calendar.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop' ] + ''; + enable = true; + }; services.xserver.displayManager = { gdm = { diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/sd-image.nix b/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/sd-image.nix index 231c7bf0a6c2..c8c5a46dfb87 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/sd-image.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/sd-image.nix @@ -147,10 +147,10 @@ in sdImage.storePaths = [ config.system.build.toplevel ]; system.build.sdImage = pkgs.callPackage ({ stdenv, dosfstools, e2fsprogs, - mtools, libfaketime, utillinux, zstd }: stdenv.mkDerivation { + mtools, libfaketime, util-linux, zstd }: stdenv.mkDerivation { name = config.sdImage.imageName; - nativeBuildInputs = [ dosfstools e2fsprogs mtools libfaketime utillinux zstd ]; + nativeBuildInputs = [ dosfstools e2fsprogs mtools libfaketime util-linux zstd ]; inherit (config.sdImage) compressImage; @@ -221,11 +221,12 @@ in set -euo pipefail set -x # Figure out device names for the boot device and root filesystem. - rootPart=$(${pkgs.utillinux}/bin/findmnt -n -o SOURCE /) + rootPart=$(${pkgs.util-linux}/bin/findmnt -n -o SOURCE /) bootDevice=$(lsblk -npo PKNAME $rootPart) + partNum=$(lsblk -npo MAJ:MIN $rootPart | awk -F: '{print $2}') # Resize the root partition and the filesystem to fit the disk - echo ",+," | sfdisk -N2 --no-reread $bootDevice + echo ",+," | sfdisk -N$partNum --no-reread $bootDevice ${pkgs.parted}/bin/partprobe ${pkgs.e2fsprogs}/bin/resize2fs $rootPart diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/system-tarball-sheevaplug.nix b/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/system-tarball-sheevaplug.nix index 8408f56f94f9..0e67ae7de698 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/system-tarball-sheevaplug.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/cd-dvd/system-tarball-sheevaplug.nix @@ -96,7 +96,7 @@ in boot.initrd.extraUtilsCommands = '' - copy_bin_and_libs ${pkgs.utillinux}/sbin/hwclock + copy_bin_and_libs ${pkgs.util-linux}/sbin/hwclock ''; boot.initrd.postDeviceCommands = diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nix-fallback-paths.nix b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nix-fallback-paths.nix index a15a2dbadb8c..699fb555615b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nix-fallback-paths.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nix-fallback-paths.nix @@ -1,6 +1,6 @@ { - x86_64-linux = "/nix/store/4vz8sh9ngx34ivi0bw5hlycxdhvy5hvz-nix-2.3.7"; - i686-linux = "/nix/store/dzxkg9lpp60bjmzvagns42vqlz3yq5kx-nix-2.3.7"; - aarch64-linux = "/nix/store/cfvf8nl8mwyw817by5y8zd3s8pnf5m9f-nix-2.3.7"; - x86_64-darwin = "/nix/store/5ira7xgs92inqz1x8l0n1wci4r79hnd0-nix-2.3.7"; + x86_64-linux = "/nix/store/fwak7l5jjl0py4wldsqjbv7p7rdzql0b-nix-2.3.9"; + i686-linux = "/nix/store/jlqrx9zw3vkwcczndaar5ban1j8g519z-nix-2.3.9"; + aarch64-linux = "/nix/store/kzvpzlm12185hw27l5znrprgvcja54d0-nix-2.3.9"; + x86_64-darwin = "/nix/store/kanh3awpf370pxfnjfvkh2m343wr3hj0-nix-2.3.9"; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix index 0c9f8522cc12..e49ceba24245 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix @@ -15,4 +15,4 @@ with import ../../../../lib/testing-python.nix { pkgs = import ../../../../.. { inherit system config; }; }; -(makeTest { inherit nodes; testScript = ""; }).driver +(makeTest { inherit nodes; testScript = ""; }).driverInteractive diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-generate-config.pl b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-generate-config.pl index 54b84521bdff..6e3ddb875e1b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-generate-config.pl +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-generate-config.pl @@ -183,6 +183,11 @@ sub pciCheck { push @imports, "(modulesPath + \"/hardware/network/broadcom-43xx.nix\")"; } + # In case this is a virtio scsi device, we need to explicitly make this available. + if ($vendor eq "0x1af4" && $device eq "0x1004") { + push @initrdAvailableKernelModules, "virtio_scsi"; + } + # Can't rely on $module here, since the module may not be loaded # due to missing firmware. Ideally we would check modules.pcimap # here. @@ -625,6 +630,10 @@ EOF my $networkingDhcpConfig = generateNetworkingDhcpConfig(); + (my $desktopConfiguration = <<EOF)=~s/^/ /gm; +@desktopConfiguration@ +EOF + write_file($fn, <<EOF); @configuration@ EOF diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-install.sh b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-install.sh index 90555f145803..9d49d4055e43 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-install.sh +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/nixos-install.sh @@ -64,7 +64,7 @@ while [ "$#" -gt 0 ]; do --no-bootloader) noBootLoader=1 ;; - --show-trace) + --show-trace|--impure|--keep-going) extraBuildFlags+=("$i") ;; --help) diff --git a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/tools.nix b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/tools.nix index 666a1eff0e62..0582812f92d2 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/installer/tools/tools.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/installer/tools/tools.nix @@ -45,7 +45,7 @@ let src = ./nixos-generate-config.pl; path = lib.optionals (lib.elem "btrfs" config.boot.supportedFilesystems) [ pkgs.btrfs-progs ]; perl = "${pkgs.perl}/bin/perl -I${pkgs.perlPackages.FileSlurp}/${pkgs.perl.libPrefix}"; - inherit (config.system.nixos-generate-config) configuration; + inherit (config.system.nixos-generate-config) configuration desktopConfiguration; }; nixos-option = @@ -78,24 +78,42 @@ in { - options.system.nixos-generate-config.configuration = mkOption { - internal = true; - type = types.str; - description = '' - The NixOS module that <literal>nixos-generate-config</literal> - saves to <literal>/etc/nixos/configuration.nix</literal>. - - This is an internal option. No backward compatibility is guaranteed. - Use at your own risk! + options.system.nixos-generate-config = { + configuration = mkOption { + internal = true; + type = types.str; + description = '' + The NixOS module that <literal>nixos-generate-config</literal> + saves to <literal>/etc/nixos/configuration.nix</literal>. + + This is an internal option. No backward compatibility is guaranteed. + Use at your own risk! + + Note that this string gets spliced into a Perl script. The perl + variable <literal>$bootLoaderConfig</literal> can be used to + splice in the boot loader configuration. + ''; + }; - Note that this string gets spliced into a Perl script. The perl - variable <literal>$bootLoaderConfig</literal> can be used to - splice in the boot loader configuration. - ''; + desktopConfiguration = mkOption { + internal = true; + type = types.str; + default = ""; + description = '' + Text to preseed the desktop configuration that <literal>nixos-generate-config</literal> + saves to <literal>/etc/nixos/configuration.nix</literal>. + + This is an internal option. No backward compatibility is guaranteed. + Use at your own risk! + + Note that this string gets spliced into a Perl script. The perl + variable <literal>$bootLoaderConfig</literal> can be used to + splice in the boot loader configuration. + ''; + }; }; config = { - system.nixos-generate-config.configuration = mkDefault '' # Edit this configuration file to define what should be installed on # your system. Help is available in the configuration.nix(5) man page @@ -113,6 +131,9 @@ in # networking.hostName = "nixos"; # Define your hostname. # networking.wireless.enable = true; # Enables wireless support via wpa_supplicant. + # Set your time zone. + # time.timeZone = "Europe/Amsterdam"; + $networkingDhcpConfig # Configure network proxy if necessary # networking.proxy.default = "http://user:password\@proxy:port/"; @@ -125,13 +146,32 @@ in # keyMap = "us"; # }; - # Set your time zone. - # time.timeZone = "Europe/Amsterdam"; + $desktopConfiguration + # Configure keymap in X11 + # services.xserver.layout = "us"; + # services.xserver.xkbOptions = "eurosign:e"; + + # Enable CUPS to print documents. + # services.printing.enable = true; + + # Enable sound. + # sound.enable = true; + # hardware.pulseaudio.enable = true; + + # Enable touchpad support (enabled default in most desktopManager). + # services.xserver.libinput.enable = true; + + # Define a user account. Don't forget to set a password with ‘passwd’. + # users.users.jane = { + # isNormalUser = true; + # extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user. + # }; # List packages installed in system profile. To search, run: # \$ nix search wget # environment.systemPackages = with pkgs; [ # wget vim + # firefox # ]; # Some programs need SUID wrappers, can be configured further or are @@ -140,7 +180,6 @@ in # programs.gnupg.agent = { # enable = true; # enableSSHSupport = true; - # pinentryFlavor = "gnome3"; # }; # List services that you want to enable: @@ -154,31 +193,6 @@ in # Or disable the firewall altogether. # networking.firewall.enable = false; - # Enable CUPS to print documents. - # services.printing.enable = true; - - # Enable sound. - # sound.enable = true; - # hardware.pulseaudio.enable = true; - - # Enable the X11 windowing system. - # services.xserver.enable = true; - # services.xserver.layout = "us"; - # services.xserver.xkbOptions = "eurosign:e"; - - # Enable touchpad support. - # services.xserver.libinput.enable = true; - - # Enable the KDE Desktop Environment. - # services.xserver.displayManager.sddm.enable = true; - # services.xserver.desktopManager.plasma5.enable = true; - - # Define a user account. Don't forget to set a password with ‘passwd’. - # users.users.jane = { - # isNormalUser = true; - # extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user. - # }; - # This value determines the NixOS release from which the default # settings for stateful data, like file locations and database versions # on your system were taken. It‘s perfectly fine and recommended to leave diff --git a/infra/libkookie/nixpkgs/nixos/modules/misc/documentation.nix b/infra/libkookie/nixpkgs/nixos/modules/misc/documentation.nix index 71a40b4f4d6e..fe0263f158f4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/misc/documentation.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/misc/documentation.nix @@ -40,9 +40,9 @@ let in scrubbedEval.options; }; - helpScript = pkgs.writeScriptBin "nixos-help" - '' - #! ${pkgs.runtimeShell} -e + + nixos-help = let + helpScript = pkgs.writeShellScriptBin "nixos-help" '' # Finds first executable browser in a colon-separated list. # (see how xdg-open defines BROWSER) browser="$( @@ -59,14 +59,22 @@ let exec "$browser" ${manual.manualHTMLIndex} ''; - desktopItem = pkgs.makeDesktopItem { - name = "nixos-manual"; - desktopName = "NixOS Manual"; - genericName = "View NixOS documentation in a web browser"; - icon = "nix-snowflake"; - exec = "${helpScript}/bin/nixos-help"; - categories = "System"; - }; + desktopItem = pkgs.makeDesktopItem { + name = "nixos-manual"; + desktopName = "NixOS Manual"; + genericName = "View NixOS documentation in a web browser"; + icon = "nix-snowflake"; + exec = "nixos-help"; + categories = "System"; + }; + + in pkgs.symlinkJoin { + name = "nixos-help"; + paths = [ + helpScript + desktopItem + ]; + }; in @@ -209,7 +217,7 @@ in manualCache = pkgs.runCommandLocal "man-cache" { } '' echo "MANDB_MAP ${manualPages}/share/man $out" > man.conf - ${pkgs.man-db}/bin/mandb -C man.conf -psc + ${pkgs.man-db}/bin/mandb -C man.conf -psc >/dev/null 2>&1 ''; in '' @@ -250,8 +258,8 @@ in environment.systemPackages = [] ++ optional cfg.man.enable manual.manpages - ++ optionals cfg.doc.enable ([ manual.manualHTML helpScript ] - ++ optionals config.services.xserver.enable [ desktopItem pkgs.nixos-icons ]); + ++ optionals cfg.doc.enable ([ manual.manualHTML nixos-help ] + ++ optionals config.services.xserver.enable [ pkgs.nixos-icons ]); services.mingetty.helpLine = mkIf cfg.doc.enable ( "\nRun 'nixos-help' for the NixOS manual." diff --git a/infra/libkookie/nixpkgs/nixos/modules/misc/ids.nix b/infra/libkookie/nixpkgs/nixos/modules/misc/ids.nix index 81d5d04fa5ea..cf0198d7b93d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/misc/ids.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/misc/ids.nix @@ -135,7 +135,7 @@ in #keys = 96; # unused #haproxy = 97; # dynamically allocated as of 2020-03-11 mongodb = 98; - openldap = 99; + #openldap = 99; # dynamically allocated as of PR#94610 #users = 100; # unused cgminer = 101; munin = 102; @@ -290,14 +290,14 @@ in hound = 259; leaps = 260; ipfs = 261; - stanchion = 262; - riak-cs = 263; + # stanchion = 262; # unused, removed 2020-10-14 + # riak-cs = 263; # unused, removed 2020-10-14 infinoted = 264; sickbeard = 265; headphones = 266; couchpotato = 267; gogs = 268; - pdns-recursor = 269; + #pdns-recursor = 269; # dynamically allocated as of 2020-20-18 #kresd = 270; # switched to "knot-resolver" with dynamic ID rpc = 271; geoip = 272; @@ -451,7 +451,7 @@ in keys = 96; #haproxy = 97; # dynamically allocated as of 2020-03-11 #mongodb = 98; # unused - openldap = 99; + #openldap = 99; # dynamically allocated as of PR#94610 munin = 102; #logcheck = 103; # unused #nix-ssh = 104; # unused @@ -468,7 +468,7 @@ in #minecraft = 114; # unused vault = 115; #ripped = 116; # unused - #murmur = 117; # unused + murmur = 117; foundationdb = 118; newrelic = 119; starbound = 120; @@ -593,8 +593,8 @@ in hound = 259; leaps = 260; ipfs = 261; - stanchion = 262; - riak-cs = 263; + # stanchion = 262; # unused, removed 2020-10-14 + # riak-cs = 263; # unused, removed 2020-10-14 infinoted = 264; sickbeard = 265; headphones = 266; diff --git a/infra/libkookie/nixpkgs/nixos/modules/module-list.nix b/infra/libkookie/nixpkgs/nixos/modules/module-list.nix index d2f8dcbea629..1c277da586a5 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/module-list.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/module-list.nix @@ -59,6 +59,7 @@ ./hardware/pcmcia.nix ./hardware/printers.nix ./hardware/raid/hpsa.nix + ./hardware/rtl-sdr.nix ./hardware/steam-hardware.nix ./hardware/system-76.nix ./hardware/tuxedo-keyboard.nix @@ -146,6 +147,7 @@ ./programs/npm.nix ./programs/oblogout.nix ./programs/plotinus.nix + ./programs/proxychains.nix ./programs/qt5ct.nix ./programs/screen.nix ./programs/sedutil.nix @@ -172,6 +174,7 @@ ./programs/wavemon.nix ./programs/waybar.nix ./programs/wireshark.nix + ./programs/wshowkeys.nix ./programs/x2goserver.nix ./programs/xfs_quota.nix ./programs/xonsh.nix @@ -294,8 +297,6 @@ ./services/databases/postgresql.nix ./services/databases/redis.nix ./services/databases/riak.nix - ./services/databases/riak-cs.nix - ./services/databases/stanchion.nix ./services/databases/victoriametrics.nix ./services/databases/virtuoso.nix ./services/desktops/accountsservice.nix @@ -392,16 +393,17 @@ ./services/logging/logcheck.nix ./services/logging/logrotate.nix ./services/logging/logstash.nix + ./services/logging/promtail.nix ./services/logging/rsyslogd.nix ./services/logging/syslog-ng.nix ./services/logging/syslogd.nix + ./services/logging/vector.nix ./services/mail/clamsmtp.nix ./services/mail/davmail.nix ./services/mail/dkimproxy-out.nix ./services/mail/dovecot.nix ./services/mail/dspam.nix ./services/mail/exim.nix - ./services/mail/freepops.nix ./services/mail/mail.nix ./services/mail/mailcatcher.nix ./services/mail/mailhog.nix @@ -480,6 +482,7 @@ ./services/misc/mediatomb.nix ./services/misc/metabase.nix ./services/misc/mwlib.nix + ./services/misc/n8n.nix ./services/misc/nix-daemon.nix ./services/misc/nix-gc.nix ./services/misc/nix-optimise.nix @@ -542,6 +545,7 @@ ./services/monitoring/kapacitor.nix ./services/monitoring/loki.nix ./services/monitoring/longview.nix + ./services/monitoring/mackerel-agent.nix ./services/monitoring/monit.nix ./services/monitoring/munin.nix ./services/monitoring/nagios.nix @@ -582,6 +586,7 @@ ./services/network-filesystems/orangefs/client.nix ./services/network-filesystems/rsyncd.nix ./services/network-filesystems/samba.nix + ./services/network-filesystems/samba-wsdd.nix ./services/network-filesystems/tahoe.nix ./services/network-filesystems/diod.nix ./services/network-filesystems/u9fs.nix @@ -681,6 +686,7 @@ ./services/networking/murmur.nix ./services/networking/mxisd.nix ./services/networking/namecoind.nix + ./services/networking/nar-serve.nix ./services/networking/nat.nix ./services/networking/ndppd.nix ./services/networking/networkmanager.nix @@ -864,6 +870,7 @@ ./services/web-apps/ihatemoney ./services/web-apps/jirafeau.nix ./services/web-apps/jitsi-meet.nix + ./services/web-apps/keycloak.nix ./services/web-apps/limesurvey.nix ./services/web-apps/mattermost.nix ./services/web-apps/mediawiki.nix diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/bandwhich.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/bandwhich.nix index 5413044f4614..1cffb5fa2765 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/programs/bandwhich.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/bandwhich.nix @@ -4,7 +4,7 @@ with lib; let cfg = config.programs.bandwhich; in { - meta.maintainers = with maintainers; [ filalex77 ]; + meta.maintainers = with maintainers; [ Br1ght0ne ]; options = { programs.bandwhich = { diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/firejail.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/firejail.nix index 484f9eb44406..ad4ef1a39459 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/programs/firejail.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/firejail.nix @@ -11,10 +11,20 @@ let } '' mkdir -p $out/bin - ${lib.concatStringsSep "\n" (lib.mapAttrsToList (command: binary: '' + ${lib.concatStringsSep "\n" (lib.mapAttrsToList (command: value: + let + opts = if builtins.isAttrs value + then value + else { executable = value; profile = null; extraArgs = []; }; + args = lib.escapeShellArgs ( + (optional (opts.profile != null) "--profile=${toString opts.profile}") + ++ opts.extraArgs + ); + in + '' cat <<_EOF >$out/bin/${command} #! ${pkgs.runtimeShell} -e - exec /run/wrappers/bin/firejail ${binary} "\$@" + exec /run/wrappers/bin/firejail ${args} -- ${toString opts.executable} "\$@" _EOF chmod 0755 $out/bin/${command} '') cfg.wrappedBinaries)} @@ -25,12 +35,38 @@ in { enable = mkEnableOption "firejail"; wrappedBinaries = mkOption { - type = types.attrsOf types.path; + type = types.attrsOf (types.either types.path (types.submodule { + options = { + executable = mkOption { + type = types.path; + description = "Executable to run sandboxed"; + example = literalExample "''${lib.getBin pkgs.firefox}/bin/firefox"; + }; + profile = mkOption { + type = types.nullOr types.path; + default = null; + description = "Profile to use"; + example = literalExample "''${pkgs.firejail}/etc/firejail/firefox.profile"; + }; + extraArgs = mkOption { + type = types.listOf types.str; + default = []; + description = "Extra arguments to pass to firejail"; + example = [ "--private=~/.firejail_home" ]; + }; + }; + })); default = {}; example = literalExample '' { - firefox = "''${lib.getBin pkgs.firefox}/bin/firefox"; - mpv = "''${lib.getBin pkgs.mpv}/bin/mpv"; + firefox = { + executable = "''${lib.getBin pkgs.firefox}/bin/firefox"; + profile = "''${pkgs.firejail}/etc/firejail/firefox.profile"; + }; + mpv = { + executable = "''${lib.getBin pkgs.mpv}/bin/mpv"; + profile = "''${pkgs.firejail}/etc/firejail/mpv.profile"; + }; } ''; description = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/fish.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/fish.nix index 39b92edf2ac2..50d1077dd410 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/programs/fish.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/fish.nix @@ -103,7 +103,7 @@ in programs.fish.shellAliases = mapAttrs (name: mkDefault) cfge.shellAliases; # Required for man completions - documentation.man.generateCaches = true; + documentation.man.generateCaches = lib.mkDefault true; environment.etc."fish/foreign-env/shellInit".text = cfge.shellInit; environment.etc."fish/foreign-env/loginShellInit".text = cfge.loginShellInit; diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/proxychains.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/proxychains.nix new file mode 100644 index 000000000000..7743f79c1c0a --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/proxychains.nix @@ -0,0 +1,165 @@ +{ config, lib, pkgs, ... }: +with lib; +let + + cfg = config.programs.proxychains; + + configFile = '' + ${cfg.chain.type}_chain + ${optionalString (cfg.chain.type == "random") + "chain_len = ${builtins.toString cfg.chain.length}"} + ${optionalString cfg.proxyDNS "proxy_dns"} + ${optionalString cfg.quietMode "quiet_mode"} + remote_dns_subnet ${builtins.toString cfg.remoteDNSSubnet} + tcp_read_time_out ${builtins.toString cfg.tcpReadTimeOut} + tcp_connect_time_out ${builtins.toString cfg.tcpConnectTimeOut} + localnet ${cfg.localnet} + [ProxyList] + ${builtins.concatStringsSep "\n" + (lib.mapAttrsToList (k: v: "${v.type} ${v.host} ${builtins.toString v.port}") + (lib.filterAttrs (k: v: v.enable) cfg.proxies))} + ''; + + proxyOptions = { + options = { + enable = mkEnableOption "this proxy"; + + type = mkOption { + type = types.enum [ "http" "socks4" "socks5" ]; + description = "Proxy type."; + }; + + host = mkOption { + type = types.str; + description = "Proxy host or IP address."; + }; + + port = mkOption { + type = types.port; + description = "Proxy port"; + }; + }; + }; + +in { + + ###### interface + + options = { + + programs.proxychains = { + + enable = mkEnableOption "installing proxychains configuration"; + + chain = { + type = mkOption { + type = types.enum [ "dynamic" "strict" "random" ]; + default = "strict"; + description = '' + <literal>dynamic</literal> - Each connection will be done via chained proxies + all proxies chained in the order as they appear in the list + at least one proxy must be online to play in chain + (dead proxies are skipped) + otherwise <literal>EINTR</literal> is returned to the app. + + <literal>strict</literal> - Each connection will be done via chained proxies + all proxies chained in the order as they appear in the list + all proxies must be online to play in chain + otherwise <literal>EINTR</literal> is returned to the app. + + <literal>random</literal> - Each connection will be done via random proxy + (or proxy chain, see <option>programs.proxychains.chain.length</option>) from the list. + ''; + }; + length = mkOption { + type = types.nullOr types.int; + default = null; + description = '' + Chain length for random chain. + ''; + }; + }; + + proxyDNS = mkOption { + type = types.bool; + default = true; + description = "Proxy DNS requests - no leak for DNS data."; + }; + + quietMode = mkEnableOption "Quiet mode (no output from the library)."; + + remoteDNSSubnet = mkOption { + type = types.enum [ 10 127 224 ]; + default = 224; + description = '' + Set the class A subnet number to use for the internal remote DNS mapping, uses the reserved 224.x.x.x range by default. + ''; + }; + + tcpReadTimeOut = mkOption { + type = types.int; + default = 15000; + description = "Connection read time-out in milliseconds."; + }; + + tcpConnectTimeOut = mkOption { + type = types.int; + default = 8000; + description = "Connection time-out in milliseconds."; + }; + + localnet = mkOption { + type = types.str; + default = "127.0.0.0/255.0.0.0"; + description = "By default enable localnet for loopback address ranges."; + }; + + proxies = mkOption { + type = types.attrsOf (types.submodule proxyOptions); + description = '' + Proxies to be used by proxychains. + ''; + + example = literalExample '' + { myproxy = + { type = "socks4"; + host = "127.0.0.1"; + port = 1337; + }; + } + ''; + }; + + }; + + }; + + ###### implementation + + meta.maintainers = with maintainers; [ sorki ]; + + config = mkIf cfg.enable { + + assertions = singleton { + assertion = cfg.chain.type != "random" && cfg.chain.length == null; + message = '' + Option `programs.proxychains.chain.length` + only makes sense with `programs.proxychains.chain.type` = "random". + ''; + }; + + programs.proxychains.proxies = mkIf config.services.tor.client.enable + { + torproxy = mkDefault { + enable = true; + type = "socks4"; + host = "127.0.0.1"; + port = 9050; + }; + }; + + environment.etc."proxychains.conf".text = configFile; + environment.systemPackages = [ pkgs.proxychains ]; + }; + +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/ssmtp.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/ssmtp.nix index 15d2750c193f..1f49ddc91bb3 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/programs/ssmtp.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/ssmtp.nix @@ -1,6 +1,6 @@ # Configuration for `ssmtp', a trivial mail transfer agent that can # replace sendmail/postfix on simple systems. It delivers email -# directly to an SMTP server defined in its configuration file, wihout +# directly to an SMTP server defined in its configuration file, without # queueing mail locally. { config, lib, pkgs, ... }: @@ -142,6 +142,13 @@ in config = mkIf cfg.enable { + assertions = [ + { + assertion = cfg.useSTARTTLS -> cfg.useTLS; + message = "services.ssmtp.useSTARTTLS has no effect without services.ssmtp.useTLS"; + } + ]; + services.ssmtp.settings = mkMerge [ ({ MailHub = cfg.hostName; diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/wshowkeys.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/wshowkeys.nix new file mode 100644 index 000000000000..09b008af1d5d --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/wshowkeys.nix @@ -0,0 +1,22 @@ +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.programs.wshowkeys; +in { + meta.maintainers = with maintainers; [ primeos ]; + + options = { + programs.wshowkeys = { + enable = mkEnableOption '' + wshowkeys (displays keypresses on screen on supported Wayland + compositors). It requires root permissions to read input events, but + these permissions are dropped after startup''; + }; + }; + + config = mkIf cfg.enable { + security.wrappers.wshowkeys.source = "${pkgs.wshowkeys}/bin/wshowkeys"; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/programs/x2goserver.nix b/infra/libkookie/nixpkgs/nixos/modules/programs/x2goserver.nix index 7d74231e956b..05707a56542f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/programs/x2goserver.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/programs/x2goserver.nix @@ -110,7 +110,7 @@ in { "L+ /usr/local/bin/chmod - - - - ${coreutils}/bin/chmod" "L+ /usr/local/bin/cp - - - - ${coreutils}/bin/cp" "L+ /usr/local/bin/sed - - - - ${gnused}/bin/sed" - "L+ /usr/local/bin/setsid - - - - ${utillinux}/bin/setsid" + "L+ /usr/local/bin/setsid - - - - ${util-linux}/bin/setsid" "L+ /usr/local/bin/xrandr - - - - ${xorg.xrandr}/bin/xrandr" "L+ /usr/local/bin/xmodmap - - - - ${xorg.xmodmap}/bin/xmodmap" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/security/acme.nix b/infra/libkookie/nixpkgs/nixos/modules/security/acme.nix index 5732620f2908..47f6bead7c3e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/security/acme.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/security/acme.nix @@ -63,7 +63,7 @@ let script = with builtins; concatStringsSep "\n" (mapAttrsToList (cert: data: '' for fixpath in /var/lib/acme/${escapeShellArg cert} /var/lib/acme/.lego/${escapeShellArg cert}; do if [ -d "$fixpath" ]; then - chmod -R 750 "$fixpath" + chmod -R u=rwX,g=rX,o= "$fixpath" chown -R acme:${data.group} "$fixpath" fi done @@ -271,7 +271,7 @@ let mv domainhash.txt certificates/ chmod 640 certificates/* - chmod -R 700 accounts/* + chmod -R u=rwX,g=,o= accounts/* # Group might change between runs, re-apply it chown 'acme:${data.group}' certificates/* diff --git a/infra/libkookie/nixpkgs/nixos/modules/security/doas.nix b/infra/libkookie/nixpkgs/nixos/modules/security/doas.nix index b81f2d0c2d52..27f6870aaf37 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/security/doas.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/security/doas.nix @@ -12,6 +12,7 @@ let mkOpts = rule: concatStringsSep " " [ (optionalString rule.noPass "nopass") + (optionalString rule.noLog "nolog") (optionalString rule.persist "persist") (optionalString rule.keepEnv "keepenv") "setenv { SSH_AUTH_SOCK ${concatStringsSep " " rule.setEnv} }" @@ -118,6 +119,16 @@ in ''; }; + noLog = mkOption { + type = with types; bool; + default = false; + description = '' + If <code>true</code>, successful executions will not be logged + to + <citerefentry><refentrytitle>syslogd</refentrytitle><manvolnum>8</manvolnum></citerefentry>. + ''; + }; + persist = mkOption { type = with types; bool; default = false; diff --git a/infra/libkookie/nixpkgs/nixos/modules/security/hidepid.nix b/infra/libkookie/nixpkgs/nixos/modules/security/hidepid.nix index 55a48ea3c9c6..4953f517e93b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/security/hidepid.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/security/hidepid.nix @@ -23,5 +23,9 @@ with lib; boot.specialFileSystems."/proc".options = [ "hidepid=2" "gid=${toString config.ids.gids.proc}" ]; systemd.services.systemd-logind.serviceConfig.SupplementaryGroups = [ "proc" ]; + + # Disable cgroupsv2, which doesn't work with hidepid. + # https://github.com/NixOS/nixpkgs/pull/104094#issuecomment-729996203 + systemd.enableUnifiedCgroupHierarchy = false; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/security/pam.nix b/infra/libkookie/nixpkgs/nixos/modules/security/pam.nix index a20d0a243a8e..a428103eaa96 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/security/pam.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/security/pam.nix @@ -318,6 +318,42 @@ let ''; }; + gnupg = { + enable = mkOption { + type = types.bool; + default = false; + description = '' + If enabled, pam_gnupg will attempt to automatically unlock the + user's GPG keys with the login password via + <command>gpg-agent</command>. The keygrips of all keys to be + unlocked should be written to <filename>~/.pam-gnupg</filename>, + and can be queried with <command>gpg -K --with-keygrip</command>. + Presetting passphrases must be enabled by adding + <literal>allow-preset-passphrase</literal> in + <filename>~/.gnupg/gpg-agent.conf</filename>. + ''; + }; + + noAutostart = mkOption { + type = types.bool; + default = false; + description = '' + Don't start <command>gpg-agent</command> if it is not running. + Useful in conjunction with starting <command>gpg-agent</command> as + a systemd user service. + ''; + }; + + storeOnly = mkOption { + type = types.bool; + default = false; + description = '' + Don't send the password immediately after login, but store for PAM + <literal>session</literal>. + ''; + }; + }; + text = mkOption { type = types.nullOr types.lines; description = "Contents of the PAM service file."; @@ -360,7 +396,7 @@ let ${optionalString cfg.logFailures "auth required pam_tally.so"} ${optionalString (config.security.pam.enableSSHAgentAuth && cfg.sshAgentAuth) - "auth sufficient ${pkgs.pam_ssh_agent_auth}/libexec/pam_ssh_agent_auth.so file=~/.ssh/authorized_keys:~/.ssh/authorized_keys2:/etc/ssh/authorized_keys.d/%u"} + "auth sufficient ${pkgs.pam_ssh_agent_auth}/libexec/pam_ssh_agent_auth.so file=${lib.concatStringsSep ":" config.services.openssh.authorizedKeysFiles}"} ${optionalString cfg.fprintAuth "auth sufficient ${pkgs.fprintd}/lib/security/pam_fprintd.so"} ${let p11 = config.security.pam.p11; in optionalString cfg.p11Auth @@ -386,6 +422,7 @@ let || cfg.enableKwallet || cfg.enableGnomeKeyring || cfg.googleAuthenticator.enable + || cfg.gnupg.enable || cfg.duoSecurity.enable)) '' auth required pam_unix.so ${optionalString cfg.allowNullPassword "nullok"} ${optionalString cfg.nodelay "nodelay"} likeauth ${optionalString config.security.pam.enableEcryptfs @@ -397,6 +434,10 @@ let " kwalletd=${pkgs.kdeFrameworks.kwallet.bin}/bin/kwalletd5")} ${optionalString cfg.enableGnomeKeyring "auth optional ${pkgs.gnome3.gnome-keyring}/lib/security/pam_gnome_keyring.so"} + ${optionalString cfg.gnupg.enable + "auth optional ${pkgs.pam_gnupg}/lib/security/pam_gnupg.so" + + optionalString cfg.gnupg.storeOnly " store-only" + } ${optionalString cfg.googleAuthenticator.enable "auth required ${pkgs.googleAuthenticator}/lib/security/pam_google_authenticator.so no_increment_hotp"} ${optionalString cfg.duoSecurity.enable @@ -472,6 +513,10 @@ let " kwalletd=${pkgs.kdeFrameworks.kwallet.bin}/bin/kwalletd5")} ${optionalString (cfg.enableGnomeKeyring) "session optional ${pkgs.gnome3.gnome-keyring}/lib/security/pam_gnome_keyring.so auto_start"} + ${optionalString cfg.gnupg.enable + "session optional ${pkgs.pam_gnupg}/lib/security/pam_gnupg.so" + + optionalString cfg.gnupg.noAutostart " no-autostart" + } ${optionalString (config.virtualisation.lxc.lxcfs.enable) "session optional ${pkgs.lxc}/lib/security/pam_cgfs.so -c all"} ''); diff --git a/infra/libkookie/nixpkgs/nixos/modules/security/pam_mount.nix b/infra/libkookie/nixpkgs/nixos/modules/security/pam_mount.nix index 77e22a96b553..9a0143c155c5 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/security/pam_mount.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/security/pam_mount.nix @@ -39,8 +39,16 @@ in environment.etc."security/pam_mount.conf.xml" = { source = let - extraUserVolumes = filterAttrs (n: u: u.cryptHomeLuks != null) config.users.users; - userVolumeEntry = user: "<volume user=\"${user.name}\" path=\"${user.cryptHomeLuks}\" mountpoint=\"${user.home}\" />\n"; + extraUserVolumes = filterAttrs (n: u: u.cryptHomeLuks != null || u.pamMount != {}) config.users.users; + mkAttr = k: v: ''${k}="${v}"''; + userVolumeEntry = user: let + attrs = { + user = user.name; + path = user.cryptHomeLuks; + mountpoint = user.home; + } // user.pamMount; + in + "<volume ${concatStringsSep " " (mapAttrsToList mkAttr attrs)} />\n"; in pkgs.writeText "pam_mount.conf.xml" '' <?xml version="1.0" encoding="utf-8" ?> @@ -52,7 +60,7 @@ in <!-- if activated, requires ofl from hxtools to be present --> <logout wait="0" hup="no" term="no" kill="no" /> <!-- set PATH variable for pam_mount module --> - <path>${pkgs.utillinux}/bin</path> + <path>${pkgs.util-linux}/bin</path> <!-- create mount point if not present --> <mkmountpoint enable="1" remove="true" /> diff --git a/infra/libkookie/nixpkgs/nixos/modules/security/wrappers/default.nix b/infra/libkookie/nixpkgs/nixos/modules/security/wrappers/default.nix index 52de21bca9bf..de6213714ac3 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/security/wrappers/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/security/wrappers/default.nix @@ -163,8 +163,8 @@ in # These are mount related wrappers that require the +s permission. fusermount.source = "${pkgs.fuse}/bin/fusermount"; fusermount3.source = "${pkgs.fuse3}/bin/fusermount3"; - mount.source = "${lib.getBin pkgs.utillinux}/bin/mount"; - umount.source = "${lib.getBin pkgs.utillinux}/bin/umount"; + mount.source = "${lib.getBin pkgs.util-linux}/bin/mount"; + umount.source = "${lib.getBin pkgs.util-linux}/bin/umount"; }; boot.specialFileSystems.${parentWrapperDir} = { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/master.nix b/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/master.nix index c6b1b0cc0bd8..a3069c81c19a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/master.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/master.nix @@ -45,7 +45,7 @@ in wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; path = with pkgs; [ - utillinux # for dmesg + util-linux # for dmesg ]; serviceConfig = { ExecStart = "${pkgs.salt}/bin/salt-master"; @@ -59,5 +59,5 @@ in }; }; - meta.maintainers = with lib.maintainers; [ aneeshusa ]; + meta.maintainers = with lib.maintainers; [ Flakebi ]; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/minion.nix b/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/minion.nix index c8fa9461a209..ac124c570d8d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/minion.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/admin/salt/minion.nix @@ -50,7 +50,7 @@ in wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; path = with pkgs; [ - utillinux + util-linux ]; serviceConfig = { ExecStart = "${pkgs.salt}/bin/salt-minion"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/audio/jack.nix b/infra/libkookie/nixpkgs/nixos/modules/services/audio/jack.nix index ceff366d0bbb..bee97dbfc6b3 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/audio/jack.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/audio/jack.nix @@ -246,6 +246,9 @@ in { description = "JACK Audio Connection Kit"; serviceConfig = { User = "jackaudio"; + SupplementaryGroups = lib.optional + (config.hardware.pulseaudio.enable + && !config.hardware.pulseaudio.systemWide) "users"; ExecStart = "${cfg.jackd.package}/bin/jackd ${lib.escapeShellArgs cfg.jackd.extraOptions}"; LimitRTPRIO = 99; LimitMEMLOCK = "infinity"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/backup/syncoid.nix b/infra/libkookie/nixpkgs/nixos/modules/services/backup/syncoid.nix index fff119c2cf00..e72e3fa59cf9 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/backup/syncoid.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/backup/syncoid.nix @@ -4,6 +4,15 @@ with lib; let cfg = config.services.syncoid; + + # Extract pool names of local datasets (ones that don't contain "@") that + # have the specified type (either "source" or "target") + getPools = type: unique (map (d: head (builtins.match "([^/]+).*" d)) ( + # Filter local datasets + filter (d: !hasInfix "@" d) + # Get datasets of the specified type + (catAttrs type (attrValues cfg.commands)) + )); in { # Interface @@ -26,14 +35,25 @@ in { user = mkOption { type = types.str; - default = "root"; + default = "syncoid"; example = "backup"; description = '' - The user for the service. Sudo or ZFS privilege delegation must be - configured to use a user other than root. + The user for the service. ZFS privilege delegation will be + automatically configured for any local pools used by syncoid if this + option is set to a user other than root. The user will be given the + "hold" and "send" privileges on any pool that has datasets being sent + and the "create", "mount", "receive", and "rollback" privileges on + any pool that has datasets being received. ''; }; + group = mkOption { + type = types.str; + default = "syncoid"; + example = "backup"; + description = "The group for the service."; + }; + sshKey = mkOption { type = types.nullOr types.path; # Prevent key from being copied to store @@ -150,6 +170,18 @@ in { # Implementation config = mkIf cfg.enable { + users = { + users = mkIf (cfg.user == "syncoid") { + syncoid = { + group = cfg.group; + isSystemUser = true; + }; + }; + groups = mkIf (cfg.group == "syncoid") { + syncoid = {}; + }; + }; + systemd.services.syncoid = { description = "Syncoid ZFS synchronization service"; script = concatMapStringsSep "\n" (c: lib.escapeShellArgs @@ -160,10 +192,22 @@ in { ++ c.extraArgs ++ [ "--sendoptions" c.sendOptions "--recvoptions" c.recvOptions + "--no-privilege-elevation" c.source c.target ])) (attrValues cfg.commands); after = [ "zfs.target" ]; - serviceConfig.User = cfg.user; + serviceConfig = { + ExecStartPre = (map (pool: lib.escapeShellArgs [ + "+/run/booted-system/sw/bin/zfs" "allow" + cfg.user "hold,send" pool + ]) (getPools "source")) ++ + (map (pool: lib.escapeShellArgs [ + "+/run/booted-system/sw/bin/zfs" "allow" + cfg.user "create,mount,receive,rollback" pool + ]) (getPools "target")); + User = cfg.user; + Group = cfg.group; + }; startAt = cfg.interval; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/backup/tarsnap.nix b/infra/libkookie/nixpkgs/nixos/modules/services/backup/tarsnap.nix index 6d99a1efb613..e1200731c2ca 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/backup/tarsnap.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/backup/tarsnap.nix @@ -308,7 +308,7 @@ in requires = [ "network-online.target" ]; after = [ "network-online.target" ]; - path = with pkgs; [ iputils tarsnap utillinux ]; + path = with pkgs; [ iputils tarsnap util-linux ]; # In order for the persistent tarsnap timer to work reliably, we have to # make sure that the tarsnap server is reachable after systemd starts up @@ -355,7 +355,7 @@ in description = "Tarsnap restore '${name}'"; requires = [ "network-online.target" ]; - path = with pkgs; [ iputils tarsnap utillinux ]; + path = with pkgs; [ iputils tarsnap util-linux ]; script = let tarsnap = ''tarsnap --configfile "/etc/tarsnap/${name}.conf"''; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/cluster/k3s/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/cluster/k3s/default.nix index 2e8bf20a68fc..f0317fdbd160 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/cluster/k3s/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/cluster/k3s/default.nix @@ -76,6 +76,10 @@ in enable = mkDefault true; }; + # TODO: disable this once k3s supports cgroupsv2, either by docker + # supporting it, or their bundled containerd + systemd.enableUnifiedCgroupHierarchy = false; + systemd.services.k3s = { description = "k3s service"; after = mkIf cfg.docker [ "docker.service" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/cluster/kubernetes/kubelet.nix b/infra/libkookie/nixpkgs/nixos/modules/services/cluster/kubernetes/kubelet.nix index c3d67552cc8c..2b6e45ba1b90 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/cluster/kubernetes/kubelet.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/cluster/kubernetes/kubelet.nix @@ -241,7 +241,7 @@ in description = "Kubernetes Kubelet Service"; wantedBy = [ "kubernetes.target" ]; after = [ "network.target" "docker.service" "kube-apiserver.service" ]; - path = with pkgs; [ gitMinimal openssh docker utillinux iproute ethtool thin-provisioning-tools iptables socat ] ++ top.path; + path = with pkgs; [ gitMinimal openssh docker util-linux iproute ethtool thin-provisioning-tools iptables socat ] ++ top.path; preStart = '' ${concatMapStrings (img: '' echo "Seeding docker image: ${img}" diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/mom.nix b/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/mom.nix index 0c5f43cf3e6a..6747bd4b0d5a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/mom.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/mom.nix @@ -32,7 +32,7 @@ in environment.systemPackages = [ pkgs.torque ]; systemd.services.torque-mom-init = { - path = with pkgs; [ torque utillinux procps inetutils ]; + path = with pkgs; [ torque util-linux procps inetutils ]; script = '' pbs_mkdirs -v aux diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/server.nix b/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/server.nix index 21c5a4f46724..8d923fc04d46 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/server.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/computing/torque/server.nix @@ -21,7 +21,7 @@ in environment.systemPackages = [ pkgs.torque ]; systemd.services.torque-server-init = { - path = with pkgs; [ torque utillinux procps inetutils ]; + path = with pkgs; [ torque util-linux procps inetutils ]; script = '' tmpsetup=$(mktemp -t torque-XXXX) diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/gitlab-runner.nix b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/gitlab-runner.nix index 431555309cc9..c358a5db77c2 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/gitlab-runner.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/gitlab-runner.nix @@ -541,7 +541,7 @@ in jq moreutils remarshal - utillinux + util-linux cfg.package ] ++ cfg.extraPackages; reloadIfChanged = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix index d2e7e8e18f94..79d1ce580545 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix @@ -25,19 +25,18 @@ in ]; config = mkIf cfg.enable { - systemd.services.hercules-ci-agent = { wantedBy = [ "multi-user.target" ]; after = [ "network-online.target" ]; wants = [ "network-online.target" ]; path = [ config.nix.package ]; + startLimitBurst = 30 * 1000000; # practically infinite serviceConfig = { User = "hercules-ci-agent"; ExecStart = command; ExecStartPre = testCommand; Restart = "on-failure"; RestartSec = 120; - StartLimitBurst = 30 * 1000000; # practically infinite }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hydra/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hydra/default.nix index 502a5898a5de..252ca17006da 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hydra/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/hydra/default.nix @@ -37,8 +37,6 @@ let haveLocalDB = cfg.dbi == localDB; - inherit (config.system) stateVersion; - hydra-package = let makeWrapperArgs = concatStringsSep " " (mapAttrsToList (key: value: "--set \"${key}\" \"${value}\"") hydraEnv); @@ -96,7 +94,8 @@ in package = mkOption { type = types.package; - defaultText = "pkgs.hydra"; + default = pkgs.hydra-unstable; + defaultText = "pkgs.hydra-unstable"; description = "The Hydra package."; }; @@ -225,34 +224,6 @@ in config = mkIf cfg.enable { - warnings = optional (cfg.package.migration or false) '' - You're currently deploying an older version of Hydra which is needed to - make some required database changes[1]. As soon as this is done, it's recommended - to run `hydra-backfill-ids` and set `services.hydra.package` to `pkgs.hydra-unstable` - after that. - - [1] https://github.com/NixOS/hydra/pull/711 - ''; - - services.hydra.package = with pkgs; - mkDefault ( - if pkgs ? hydra - then throw '' - The Hydra package doesn't exist anymore in `nixpkgs`! It probably exists - due to an overlay. To upgrade Hydra, you need to take two steps as some - bigger changes in the database schema were implemented recently[1]. You first - need to deploy `pkgs.hydra-migration`, run `hydra-backfill-ids` on the server - and then deploy `pkgs.hydra-unstable`. - - If you want to use `pkgs.hydra` from your overlay, please set `services.hydra.package` - explicitly to `pkgs.hydra` and make sure you know what you're doing. - - [1] https://github.com/NixOS/hydra/pull/711 - '' - else if versionOlder stateVersion "20.03" then hydra-migration - else hydra-unstable - ); - users.groups.hydra = { gid = config.ids.gids.hydra; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/jenkins/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/jenkins/default.nix index 1477c471f8ab..cdc3b4b5c58f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/jenkins/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/continuous-integration/jenkins/default.nix @@ -86,8 +86,8 @@ in { }; packages = mkOption { - default = [ pkgs.stdenv pkgs.git pkgs.jdk config.programs.ssh.package pkgs.nix ]; - defaultText = "[ pkgs.stdenv pkgs.git pkgs.jdk config.programs.ssh.package pkgs.nix ]"; + default = [ pkgs.stdenv pkgs.git pkgs.jdk11 config.programs.ssh.package pkgs.nix ]; + defaultText = "[ pkgs.stdenv pkgs.git pkgs.jdk11 config.programs.ssh.package pkgs.nix ]"; type = types.listOf types.package; description = '' Packages to add to PATH for the jenkins process. @@ -207,7 +207,7 @@ in { # For reference: https://wiki.jenkins.io/display/JENKINS/JenkinsLinuxStartupScript script = '' - ${pkgs.jdk}/bin/java ${concatStringsSep " " cfg.extraJavaOptions} -jar ${cfg.package}/webapps/jenkins.war --httpListenAddress=${cfg.listenAddress} \ + ${pkgs.jdk11}/bin/java ${concatStringsSep " " cfg.extraJavaOptions} -jar ${cfg.package}/webapps/jenkins.war --httpListenAddress=${cfg.listenAddress} \ --httpPort=${toString cfg.port} \ --prefix=${cfg.prefix} \ -Djava.awt.headless=true \ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/foundationdb.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/foundationdb.nix index 18727acc7c75..e22127403e91 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/foundationdb.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/databases/foundationdb.nix @@ -233,7 +233,7 @@ in type = types.str; default = "Check.Valid=1,Check.Unexpired=1"; description = '' - "Peer verification string". This may be used to adjust which TLS + "Peer verification string". This may be used to adjust which TLS client certificates a server will accept, as a form of user authorization; for example, it may only accept TLS clients who offer a certificate abiding by some locality or organization name. diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/openldap.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/openldap.nix index 7472538b887e..94a5c573768b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/openldap.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/databases/openldap.nix @@ -1,43 +1,121 @@ { config, lib, pkgs, ... }: with lib; - let - cfg = config.services.openldap; + legacyOptions = [ "rootpwFile" "suffix" "dataDir" "rootdn" "rootpw" ]; openldap = cfg.package; - - dataFile = pkgs.writeText "ldap-contents.ldif" cfg.declarativeContents; - configFile = pkgs.writeText "slapd.conf" ((optionalString cfg.defaultSchemas '' - include ${openldap.out}/etc/schema/core.schema - include ${openldap.out}/etc/schema/cosine.schema - include ${openldap.out}/etc/schema/inetorgperson.schema - include ${openldap.out}/etc/schema/nis.schema - '') + '' - ${cfg.extraConfig} - database ${cfg.database} - suffix ${cfg.suffix} - rootdn ${cfg.rootdn} - ${if (cfg.rootpw != null) then '' - rootpw ${cfg.rootpw} - '' else '' - include ${cfg.rootpwFile} - ''} - directory ${cfg.dataDir} - ${cfg.extraDatabaseConfig} - ''); - configOpts = if cfg.configDir == null then "-f ${configFile}" - else "-F ${cfg.configDir}"; -in - -{ - - ###### interface - + configDir = if cfg.configDir != null then cfg.configDir else "/etc/openldap/slapd.d"; + + ldapValueType = let + # Can't do types.either with multiple non-overlapping submodules, so define our own + singleLdapValueType = lib.mkOptionType rec { + name = "LDAP"; + description = "LDAP value"; + check = x: lib.isString x || (lib.isAttrs x && (x ? path || x ? base64)); + merge = lib.mergeEqualOption; + }; + # We don't coerce to lists of single values, as some values must be unique + in types.either singleLdapValueType (types.listOf singleLdapValueType); + + ldapAttrsType = + let + options = { + attrs = mkOption { + type = types.attrsOf ldapValueType; + default = {}; + description = "Attributes of the parent entry."; + }; + children = mkOption { + # Hide the child attributes, to avoid infinite recursion in e.g. documentation + # Actual Nix evaluation is lazy, so this is not an issue there + type = let + hiddenOptions = lib.mapAttrs (name: attr: attr // { visible = false; }) options; + in types.attrsOf (types.submodule { options = hiddenOptions; }); + default = {}; + description = "Child entries of the current entry, with recursively the same structure."; + example = lib.literalExample '' + { + "cn=schema" = { + # The attribute used in the DN must be defined + attrs = { cn = "schema"; }; + children = { + # This entry's DN is expanded to "cn=foo,cn=schema" + "cn=foo" = { ... }; + }; + # These includes are inserted after "cn=schema", but before "cn=foo,cn=schema" + includes = [ ... ]; + }; + } + ''; + }; + includes = mkOption { + type = types.listOf types.path; + default = []; + description = '' + LDIF files to include after the parent's attributes but before its children. + ''; + }; + }; + in types.submodule { inherit options; }; + + valueToLdif = attr: values: let + listValues = if lib.isList values then values else lib.singleton values; + in map (value: + if lib.isAttrs value then + if lib.hasAttr "path" value + then "${attr}:< file://${value.path}" + else "${attr}:: ${value.base64}" + else "${attr}: ${lib.replaceStrings [ "\n" ] [ "\n " ] value}" + ) listValues; + + attrsToLdif = dn: { attrs, children, includes, ... }: ['' + dn: ${dn} + ${lib.concatStringsSep "\n" (lib.flatten (lib.mapAttrsToList valueToLdif attrs))} + ''] ++ (map (path: "include: file://${path}\n") includes) ++ ( + lib.flatten (lib.mapAttrsToList (name: value: attrsToLdif "${name},${dn}" value) children) + ); +in { + imports = let + deprecationNote = "This option is removed due to the deprecation of `slapd.conf` upstream. Please migrate to `services.openldap.settings`, see the release notes for advice with this process."; + mkDatabaseOption = old: new: + lib.mkChangedOptionModule [ "services" "openldap" old ] [ "services" "openldap" "settings" "children" ] + (config: let + database = lib.getAttrFromPath [ "services" "openldap" "database" ] config; + value = lib.getAttrFromPath [ "services" "openldap" old ] config; + in lib.setAttrByPath ([ "olcDatabase={1}${database}" "attrs" ] ++ new) value); + in [ + (lib.mkRemovedOptionModule [ "services" "openldap" "extraConfig" ] deprecationNote) + (lib.mkRemovedOptionModule [ "services" "openldap" "extraDatabaseConfig" ] deprecationNote) + + (lib.mkChangedOptionModule [ "services" "openldap" "logLevel" ] [ "services" "openldap" "settings" "attrs" "olcLogLevel" ] + (config: lib.splitString " " (lib.getAttrFromPath [ "services" "openldap" "logLevel" ] config))) + (lib.mkChangedOptionModule [ "services" "openldap" "defaultSchemas" ] [ "services" "openldap" "settings" "children" "cn=schema" "includes"] + (config: lib.optionals (lib.getAttrFromPath [ "services" "openldap" "defaultSchemas" ] config) ( + map (schema: "${openldap}/etc/schema/${schema}.ldif") [ "core" "cosine" "inetorgperson" "nis" ]))) + + (lib.mkChangedOptionModule [ "services" "openldap" "database" ] [ "services" "openldap" "settings" "children" ] + (config: let + database = lib.getAttrFromPath [ "services" "openldap" "database" ] config; + in { + "olcDatabase={1}${database}".attrs = { + # objectClass is case-insensitive, so don't need to capitalize ${database} + objectClass = [ "olcdatabaseconfig" "olc${database}config" ]; + olcDatabase = "{1}${database}"; + olcDbDirectory = lib.mkDefault "/var/db/openldap"; + }; + "cn=schema".includes = lib.mkDefault ( + map (schema: "${openldap}/etc/schema/${schema}.ldif") [ "core" "cosine" "inetorgperson" "nis" ] + ); + })) + (mkDatabaseOption "rootpwFile" [ "olcRootPW" "path" ]) + (mkDatabaseOption "suffix" [ "olcSuffix" ]) + (mkDatabaseOption "dataDir" [ "olcDbDirectory" ]) + (mkDatabaseOption "rootdn" [ "olcRootDN" ]) + (mkDatabaseOption "rootpw" [ "olcRootPW" ]) + ]; options = { - services.openldap = { - enable = mkOption { type = types.bool; default = false; @@ -77,224 +155,170 @@ in example = [ "ldaps:///" ]; }; - dataDir = mkOption { - type = types.path; - default = "/var/db/openldap"; - description = "The database directory."; - }; - - defaultSchemas = mkOption { - type = types.bool; - default = true; - description = '' - Include the default schemas core, cosine, inetorgperson and nis. - This setting will be ignored if configDir is set. - ''; - }; - - database = mkOption { - type = types.str; - default = "mdb"; - description = '' - Database type to use for the LDAP. - This setting will be ignored if configDir is set. - ''; - }; - - suffix = mkOption { - type = types.str; - example = "dc=example,dc=org"; - description = '' - Specify the DN suffix of queries that will be passed to this backend - database. - This setting will be ignored if configDir is set. - ''; - }; - - rootdn = mkOption { - type = types.str; - example = "cn=admin,dc=example,dc=org"; - description = '' - Specify the distinguished name that is not subject to access control - or administrative limit restrictions for operations on this database. - This setting will be ignored if configDir is set. - ''; - }; - - rootpw = mkOption { - type = types.nullOr types.str; - default = null; - description = '' - Password for the root user. - This setting will be ignored if configDir is set. - Using this option will store the root password in plain text in the - world-readable nix store. To avoid this the <literal>rootpwFile</literal> can be used. + settings = mkOption { + type = ldapAttrsType; + description = "Configuration for OpenLDAP, in OLC format"; + example = lib.literalExample '' + { + attrs.olcLogLevel = [ "stats" ]; + children = { + "cn=schema".includes = [ + "\${pkgs.openldap}/etc/schema/core.ldif" + "\${pkgs.openldap}/etc/schema/cosine.ldif" + "\${pkgs.openldap}/etc/schema/inetorgperson.ldif" + ]; + "olcDatabase={-1}frontend" = { + attrs = { + objectClass = "olcDatabaseConfig"; + olcDatabase = "{-1}frontend"; + olcAccess = [ "{0}to * by dn.exact=uidNumber=0+gidNumber=0,cn=peercred,cn=external,cn=auth manage stop by * none stop" ]; + }; + }; + "olcDatabase={0}config" = { + attrs = { + objectClass = "olcDatabaseConfig"; + olcDatabase = "{0}config"; + olcAccess = [ "{0}to * by * none break" ]; + }; + }; + "olcDatabase={1}mdb" = { + attrs = { + objectClass = [ "olcDatabaseConfig" "olcMdbConfig" ]; + olcDatabase = "{1}mdb"; + olcDbDirectory = "/var/db/ldap"; + olcDbIndex = [ + "objectClass eq" + "cn pres,eq" + "uid pres,eq" + "sn pres,eq,subany" + ]; + olcSuffix = "dc=example,dc=com"; + olcAccess = [ "{0}to * by * read break" ]; + }; + }; + }; + }; ''; }; - rootpwFile = mkOption { - type = types.nullOr types.str; - default = null; - description = '' - Password file for the root user. - The file should contain the string <literal>rootpw</literal> followed by the password. - e.g.: <literal>rootpw mysecurepassword</literal> - ''; - }; - - logLevel = mkOption { - type = types.str; - default = "0"; - example = "acl trace"; - description = "The log level selector of slapd."; - }; - + # This option overrides settings configDir = mkOption { type = types.nullOr types.path; default = null; - description = "Use this optional config directory instead of using slapd.conf"; + description = '' + Use this config directory instead of generating one from the + <literal>settings</literal> option. Overrides all NixOS settings. If + you use this option,ensure `olcPidFile` is set to `/run/slapd/slapd.conf`. + ''; example = "/var/db/slapd.d"; }; - extraConfig = mkOption { - type = types.lines; - default = ""; - description = " - slapd.conf configuration - "; - example = literalExample '' - ''' - include ${openldap.out}/etc/schema/core.schema - include ${openldap.out}/etc/schema/cosine.schema - include ${openldap.out}/etc/schema/inetorgperson.schema - include ${openldap.out}/etc/schema/nis.schema - - database bdb - suffix dc=example,dc=org - rootdn cn=admin,dc=example,dc=org - # NOTE: change after first start - rootpw secret - directory /var/db/openldap - ''' - ''; - }; - declarativeContents = mkOption { - type = with types; nullOr lines; - default = null; + type = with types; attrsOf lines; + default = {}; description = '' - Declarative contents for the LDAP database, in LDIF format. + Declarative contents for the LDAP database, in LDIF format by suffix. - Note a few facts when using it. First, the database - <emphasis>must</emphasis> be stored in the directory defined by - <code>dataDir</code>. Second, all <code>dataDir</code> will be erased - when starting the LDAP server. Third, modifications to the database - are not prevented, they are just dropped on the next reboot of the - server. Finally, performance-wise the database and indexes are rebuilt - on each server startup, so this will slow down server startup, + All data will be erased when starting the LDAP server. Modifications + to the database are not prevented, they are just dropped on the next + reboot of the server. Performance-wise the database and indexes are + rebuilt on each server startup, so this will slow down server startup, especially with large databases. ''; - example = '' - dn: dc=example,dc=org - objectClass: domain - dc: example - - dn: ou=users,dc=example,dc=org - objectClass = organizationalUnit - ou: users - - # ... + example = lib.literalExample '' + { + "dc=example,dc=org" = ''' + dn= dn: dc=example,dc=org + objectClass: domain + dc: example + + dn: ou=users,dc=example,dc=org + objectClass = organizationalUnit + ou: users + + # ... + '''; + } ''; }; - - extraDatabaseConfig = mkOption { - type = types.lines; - default = ""; - description = '' - slapd.conf configuration after the database option. - This setting will be ignored if configDir is set. - ''; - example = '' - # Indices to maintain for this directory - # unique id so equality match only - index uid eq - # allows general searching on commonname, givenname and email - index cn,gn,mail eq,sub - # allows multiple variants on surname searching - index sn eq,sub - # sub above includes subintial,subany,subfinal - # optimise department searches - index ou eq - # if searches will include objectClass uncomment following - # index objectClass eq - # shows use of default index parameter - index default eq,sub - # indices missing - uses default eq,sub - index telephonenumber - - # other database parameters - # read more in slapd.conf reference section - cachesize 10000 - checkpoint 128 15 - ''; - }; - }; - - }; - - meta = { - maintainers = [ lib.maintainers.mic92 ]; }; - - ###### implementation + meta.maintainers = with lib.maintainters; [ mic92 kwohlfahrt ]; config = mkIf cfg.enable { - assertions = [ - { - assertion = cfg.configDir != null || cfg.rootpwFile != null || cfg.rootpw != null; - message = "services.openldap: Unless configDir is set, either rootpw or rootpwFile must be set"; - } - ]; - + assertions = map (opt: { + assertion = ((getAttr opt cfg) != "_mkMergedOptionModule") -> (cfg.database != "_mkMergedOptionModule"); + message = "Legacy OpenLDAP option `services.openldap.${opt}` requires `services.openldap.database` (use value \"mdb\" if unsure)"; + }) legacyOptions; environment.systemPackages = [ openldap ]; + # Literal attributes must always be set + services.openldap.settings = { + attrs = { + objectClass = "olcGlobal"; + cn = "config"; + olcPidFile = "/run/slapd/slapd.pid"; + }; + children."cn=schema".attrs = { + cn = "schema"; + objectClass = "olcSchemaConfig"; + }; + }; + systemd.services.openldap = { description = "LDAP server"; wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; - preStart = '' + preStart = let + settingsFile = pkgs.writeText "config.ldif" (lib.concatStringsSep "\n" (attrsToLdif "cn=config" cfg.settings)); + + dbSettings = lib.filterAttrs (name: value: lib.hasPrefix "olcDatabase=" name) cfg.settings.children; + dataDirs = lib.mapAttrs' (name: value: lib.nameValuePair value.attrs.olcSuffix value.attrs.olcDbDirectory) + (lib.filterAttrs (_: value: value.attrs ? olcDbDirectory) dbSettings); + dataFiles = lib.mapAttrs (dn: contents: pkgs.writeText "${dn}.ldif" contents) cfg.declarativeContents; + mkLoadScript = dn: let + dataDir = lib.escapeShellArg (getAttr dn dataDirs); + in '' + rm -rf ${dataDir}/* + ${openldap}/bin/slapadd -F ${lib.escapeShellArg configDir} -b ${dn} -l ${getAttr dn dataFiles} + chown -R "${cfg.user}:${cfg.group}" ${dataDir} + ''; + in '' mkdir -p /run/slapd chown -R "${cfg.user}:${cfg.group}" /run/slapd - ${optionalString (cfg.declarativeContents != null) '' - rm -Rf "${cfg.dataDir}" - ''} - mkdir -p "${cfg.dataDir}" - ${optionalString (cfg.declarativeContents != null) '' - ${openldap.out}/bin/slapadd ${configOpts} -l ${dataFile} - ''} - chown -R "${cfg.user}:${cfg.group}" "${cfg.dataDir}" - ${openldap}/bin/slaptest ${configOpts} + mkdir -p ${lib.escapeShellArg configDir} ${lib.escapeShellArgs (lib.attrValues dataDirs)} + chown "${cfg.user}:${cfg.group}" ${lib.escapeShellArg configDir} ${lib.escapeShellArgs (lib.attrValues dataDirs)} + + ${lib.optionalString (cfg.configDir == null) ('' + rm -Rf ${configDir}/* + ${openldap}/bin/slapadd -F ${configDir} -bcn=config -l ${settingsFile} + '')} + chown -R "${cfg.user}:${cfg.group}" ${lib.escapeShellArg configDir} + + ${lib.concatStrings (map mkLoadScript (lib.attrNames cfg.declarativeContents))} + ${openldap}/bin/slaptest -u -F ${lib.escapeShellArg configDir} ''; - serviceConfig.ExecStart = - "${openldap.out}/libexec/slapd -d '${cfg.logLevel}' " + - "-u '${cfg.user}' -g '${cfg.group}' " + - "-h '${concatStringsSep " " cfg.urlList}' " + - "${configOpts}"; + serviceConfig = { + ExecStart = lib.escapeShellArgs ([ + "${openldap}/libexec/slapd" "-u" cfg.user "-g" cfg.group "-F" configDir + "-h" (lib.concatStringsSep " " cfg.urlList) + ]); + Type = "forking"; + PIDFile = cfg.settings.attrs.olcPidFile; + }; }; - users.users.openldap = - { name = cfg.user; + users.users = lib.optionalAttrs (cfg.user == "openldap") { + openldap = { group = cfg.group; - uid = config.ids.uids.openldap; - }; - - users.groups.openldap = - { name = cfg.group; - gid = config.ids.gids.openldap; + isSystemUser = true; }; + }; + users.groups = lib.optionalAttrs (cfg.group == "openldap") { + openldap = {}; + }; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/postgresql.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/postgresql.nix index 5056d50153f6..f582b0592774 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/postgresql.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/databases/postgresql.nix @@ -69,11 +69,16 @@ in type = types.lines; default = ""; description = '' - Defines how users authenticate themselves to the server. By - default, "trust" access to local users will always be granted - along with any other custom options. If you do not want this, - set this option using "lib.mkForce" to override this - behaviour. + Defines how users authenticate themselves to the server. See the + <link xlink:href="https://www.postgresql.org/docs/current/auth-pg-hba-conf.html"> + PostgreSQL documentation for pg_hba.conf</link> + for details on the expected format of this option. By default, + peer based authentication will be used for users connecting + via the Unix socket, and md5 password authentication will be + used for users connecting via TCP. Any added rules will be + inserted above the default rules. If you'd like to replace the + default rules entirely, you can use <function>lib.mkForce</function> in your + module. ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/redis.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/redis.nix index f1777854e141..6b8853ae390b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/redis.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/databases/redis.nix @@ -87,9 +87,12 @@ in bind = mkOption { type = with types; nullOr str; - default = null; # All interfaces - description = "The IP interface to bind to."; - example = "127.0.0.1"; + default = "127.0.0.1"; + description = '' + The IP interface to bind to. + <literal>null</literal> means "all interfaces". + ''; + example = "192.0.2.1"; }; unixSocket = mkOption { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak-cs.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak-cs.nix deleted file mode 100644 index fa6ac8863318..000000000000 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak-cs.nix +++ /dev/null @@ -1,202 +0,0 @@ -{ config, lib, pkgs, ... }: - -with lib; - -let - - cfg = config.services.riak-cs; - -in - -{ - - ###### interface - - options = { - - services.riak-cs = { - - enable = mkEnableOption "riak-cs"; - - package = mkOption { - type = types.package; - default = pkgs.riak-cs; - defaultText = "pkgs.riak-cs"; - example = literalExample "pkgs.riak-cs"; - description = '' - Riak package to use. - ''; - }; - - nodeName = mkOption { - type = types.str; - default = "riak-cs@127.0.0.1"; - description = '' - Name of the Erlang node. - ''; - }; - - anonymousUserCreation = mkOption { - type = types.bool; - default = false; - description = '' - Anonymous user creation. - ''; - }; - - riakHost = mkOption { - type = types.str; - default = "127.0.0.1:8087"; - description = '' - Name of riak hosting service. - ''; - }; - - listener = mkOption { - type = types.str; - default = "127.0.0.1:8080"; - description = '' - Name of Riak CS listening service. - ''; - }; - - stanchionHost = mkOption { - type = types.str; - default = "127.0.0.1:8085"; - description = '' - Name of stanchion hosting service. - ''; - }; - - stanchionSsl = mkOption { - type = types.bool; - default = true; - description = '' - Tell stanchion to use SSL. - ''; - }; - - distributedCookie = mkOption { - type = types.str; - default = "riak"; - description = '' - Cookie for distributed node communication. All nodes in the - same cluster should use the same cookie or they will not be able to - communicate. - ''; - }; - - dataDir = mkOption { - type = types.path; - default = "/var/db/riak-cs"; - description = '' - Data directory for Riak CS. - ''; - }; - - logDir = mkOption { - type = types.path; - default = "/var/log/riak-cs"; - description = '' - Log directory for Riak CS. - ''; - }; - - extraConfig = mkOption { - type = types.lines; - default = ""; - description = '' - Additional text to be appended to <filename>riak-cs.conf</filename>. - ''; - }; - - extraAdvancedConfig = mkOption { - type = types.lines; - default = ""; - description = '' - Additional text to be appended to <filename>advanced.config</filename>. - ''; - }; - }; - - }; - - ###### implementation - - config = mkIf cfg.enable { - - environment.systemPackages = [ cfg.package ]; - environment.etc."riak-cs/riak-cs.conf".text = '' - nodename = ${cfg.nodeName} - distributed_cookie = ${cfg.distributedCookie} - - platform_log_dir = ${cfg.logDir} - - riak_host = ${cfg.riakHost} - listener = ${cfg.listener} - stanchion_host = ${cfg.stanchionHost} - - anonymous_user_creation = ${if cfg.anonymousUserCreation then "on" else "off"} - - ${cfg.extraConfig} - ''; - - environment.etc."riak-cs/advanced.config".text = '' - ${cfg.extraAdvancedConfig} - ''; - - users.users.riak-cs = { - name = "riak-cs"; - uid = config.ids.uids.riak-cs; - group = "riak"; - description = "Riak CS server user"; - }; - - systemd.services.riak-cs = { - description = "Riak CS Server"; - - wantedBy = [ "multi-user.target" ]; - after = [ "network.target" ]; - - path = [ - pkgs.utillinux # for `logger` - pkgs.bash - ]; - - environment.HOME = "${cfg.dataDir}"; - environment.RIAK_CS_DATA_DIR = "${cfg.dataDir}"; - environment.RIAK_CS_LOG_DIR = "${cfg.logDir}"; - environment.RIAK_CS_ETC_DIR = "/etc/riak"; - - preStart = '' - if ! test -e ${cfg.logDir}; then - mkdir -m 0755 -p ${cfg.logDir} - chown -R riak-cs ${cfg.logDir} - fi - - if ! test -e ${cfg.dataDir}; then - mkdir -m 0700 -p ${cfg.dataDir} - chown -R riak-cs ${cfg.dataDir} - fi - ''; - - serviceConfig = { - ExecStart = "${cfg.package}/bin/riak-cs console"; - ExecStop = "${cfg.package}/bin/riak-cs stop"; - StandardInput = "tty"; - User = "riak-cs"; - Group = "riak-cs"; - PermissionsStartOnly = true; - # Give Riak a decent amount of time to clean up. - TimeoutStopSec = 120; - LimitNOFILE = 65536; - }; - - unitConfig.RequiresMountsFor = [ - "${cfg.dataDir}" - "${cfg.logDir}" - "/etc/riak" - ]; - }; - }; -} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak.nix index 885215209bdf..657eeea87bf4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/databases/riak.nix @@ -118,7 +118,7 @@ in after = [ "network.target" ]; path = [ - pkgs.utillinux # for `logger` + pkgs.util-linux # for `logger` pkgs.bash ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/stanchion.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/stanchion.nix deleted file mode 100644 index 97e55bc70c47..000000000000 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/stanchion.nix +++ /dev/null @@ -1,194 +0,0 @@ -{ config, lib, pkgs, ... }: - -with lib; - -let - - cfg = config.services.stanchion; - -in - -{ - - ###### interface - - options = { - - services.stanchion = { - - enable = mkEnableOption "stanchion"; - - package = mkOption { - type = types.package; - default = pkgs.stanchion; - defaultText = "pkgs.stanchion"; - example = literalExample "pkgs.stanchion"; - description = '' - Stanchion package to use. - ''; - }; - - nodeName = mkOption { - type = types.str; - default = "stanchion@127.0.0.1"; - description = '' - Name of the Erlang node. - ''; - }; - - adminKey = mkOption { - type = types.str; - default = ""; - description = '' - Name of admin user. - ''; - }; - - adminSecret = mkOption { - type = types.str; - default = ""; - description = '' - Name of admin secret - ''; - }; - - riakHost = mkOption { - type = types.str; - default = "127.0.0.1:8087"; - description = '' - Name of riak hosting service. - ''; - }; - - listener = mkOption { - type = types.str; - default = "127.0.0.1:8085"; - description = '' - Name of Riak CS listening service. - ''; - }; - - stanchionHost = mkOption { - type = types.str; - default = "127.0.0.1:8085"; - description = '' - Name of stanchion hosting service. - ''; - }; - - distributedCookie = mkOption { - type = types.str; - default = "riak"; - description = '' - Cookie for distributed node communication. All nodes in the - same cluster should use the same cookie or they will not be able to - communicate. - ''; - }; - - dataDir = mkOption { - type = types.path; - default = "/var/db/stanchion"; - description = '' - Data directory for Stanchion. - ''; - }; - - logDir = mkOption { - type = types.path; - default = "/var/log/stanchion"; - description = '' - Log directory for Stanchion. - ''; - }; - - extraConfig = mkOption { - type = types.lines; - default = ""; - description = '' - Additional text to be appended to <filename>stanchion.conf</filename>. - ''; - }; - }; - }; - - ###### implementation - - config = mkIf cfg.enable { - - environment.systemPackages = [ cfg.package ]; - - environment.etc."stanchion/advanced.config".text = '' - [{stanchion, []}]. - ''; - - environment.etc."stanchion/stanchion.conf".text = '' - listener = ${cfg.listener} - - riak_host = ${cfg.riakHost} - - ${optionalString (cfg.adminKey == "") "#"} admin.key=${optionalString (cfg.adminKey != "") cfg.adminKey} - ${optionalString (cfg.adminSecret == "") "#"} admin.secret=${optionalString (cfg.adminSecret != "") cfg.adminSecret} - - platform_bin_dir = ${pkgs.stanchion}/bin - platform_data_dir = ${cfg.dataDir} - platform_etc_dir = /etc/stanchion - platform_lib_dir = ${pkgs.stanchion}/lib - platform_log_dir = ${cfg.logDir} - - nodename = ${cfg.nodeName} - - distributed_cookie = ${cfg.distributedCookie} - - ${cfg.extraConfig} - ''; - - users.users.stanchion = { - name = "stanchion"; - uid = config.ids.uids.stanchion; - group = "stanchion"; - description = "Stanchion server user"; - }; - - users.groups.stanchion.gid = config.ids.gids.stanchion; - - systemd.tmpfiles.rules = [ - "d '${cfg.logDir}' - stanchion stanchion --" - "d '${cfg.dataDir}' 0700 stanchion stanchion --" - ]; - - systemd.services.stanchion = { - description = "Stanchion Server"; - - wantedBy = [ "multi-user.target" ]; - after = [ "network.target" ]; - - path = [ - pkgs.utillinux # for `logger` - pkgs.bash - ]; - - environment.HOME = "${cfg.dataDir}"; - environment.STANCHION_DATA_DIR = "${cfg.dataDir}"; - environment.STANCHION_LOG_DIR = "${cfg.logDir}"; - environment.STANCHION_ETC_DIR = "/etc/stanchion"; - - serviceConfig = { - ExecStart = "${cfg.package}/bin/stanchion console"; - ExecStop = "${cfg.package}/bin/stanchion stop"; - StandardInput = "tty"; - User = "stanchion"; - Group = "stanchion"; - # Give Stanchion a decent amount of time to clean up. - TimeoutStopSec = 120; - LimitNOFILE = 65536; - }; - - unitConfig.RequiresMountsFor = [ - "${cfg.dataDir}" - "${cfg.logDir}" - "/etc/stanchion" - ]; - }; - }; -} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/databases/victoriametrics.nix b/infra/libkookie/nixpkgs/nixos/modules/services/databases/victoriametrics.nix index 0af5d2adf372..5b09115bb2fb 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/databases/victoriametrics.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/databases/victoriametrics.nix @@ -40,10 +40,10 @@ let cfg = config.services.victoriametrics; in systemd.services.victoriametrics = { description = "VictoriaMetrics time series database"; after = [ "network.target" ]; + startLimitBurst = 5; serviceConfig = { Restart = "on-failure"; RestartSec = 1; - StartLimitBurst = 5; StateDirectory = "victoriametrics"; DynamicUser = true; ExecStart = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/flatpak.nix b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/flatpak.nix index 7da92cc9f264..d0f6b66328a4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/flatpak.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/flatpak.nix @@ -15,6 +15,18 @@ in { options = { services.flatpak = { enable = mkEnableOption "flatpak"; + + guiPackages = mkOption { + internal = true; + type = types.listOf types.package; + default = []; + example = literalExample "[ pkgs.gnome3.gnome-software ]"; + description = '' + Packages that provide an interface for flatpak + (like gnome-software) that will be automatically available + to all users when flatpak is enabled. + ''; + }; }; }; @@ -28,7 +40,7 @@ in { } ]; - environment.systemPackages = [ pkgs.flatpak ]; + environment.systemPackages = [ pkgs.flatpak ] ++ cfg.guiPackages; services.dbus.packages = [ pkgs.flatpak ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/pipewire.nix b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/pipewire.nix index 5aee59cfdcce..0ef988d9e69f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/pipewire.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/pipewire.nix @@ -5,8 +5,18 @@ with lib; let cfg = config.services.pipewire; - packages = with pkgs; [ pipewire ]; + enable32BitAlsaPlugins = cfg.alsa.support32Bit + && pkgs.stdenv.isx86_64 + && pkgs.pkgsi686Linux.pipewire != null; + # The package doesn't output to $out/lib/pipewire directly so that the + # overlays can use the outputs to replace the originals in FHS environments. + # + # This doesn't work in general because of missing development information. + jack-libs = pkgs.runCommand "jack-libs" {} '' + mkdir -p "$out/lib" + ln -s "${cfg.package.jack}/lib" "$out/lib/pipewire" + ''; in { meta = { @@ -18,6 +28,16 @@ in { services.pipewire = { enable = mkEnableOption "pipewire service"; + package = mkOption { + type = types.package; + default = pkgs.pipewire; + defaultText = "pkgs.pipewire"; + example = literalExample "pkgs.pipewire"; + description = '' + The pipewire derivation to use. + ''; + }; + socketActivation = mkOption { default = true; type = types.bool; @@ -25,17 +45,139 @@ in { Automatically run pipewire when connections are made to the pipewire socket. ''; }; + + extraConfig = mkOption { + type = types.lines; + default = ""; + description = '' + Literal string to append to /etc/pipewire/pipewire.conf. + ''; + }; + + sessionManager = mkOption { + type = types.nullOr types.string; + default = null; + example = literalExample ''"''${pipewire}/bin/pipewire-media-session"''; + description = '' + Path to the pipewire session manager executable. + ''; + }; + + sessionManagerArguments = mkOption { + type = types.listOf types.string; + default = []; + example = literalExample ''[ "-p" "bluez5.msbc-support=true" ]''; + description = '' + Arguments passed to the pipewire session manager. + ''; + }; + + alsa = { + enable = mkEnableOption "ALSA support"; + support32Bit = mkEnableOption "32-bit ALSA support on 64-bit systems"; + }; + + jack = { + enable = mkEnableOption "JACK audio emulation"; + }; + + pulse = { + enable = mkEnableOption "PulseAudio server emulation"; + }; }; }; ###### implementation config = mkIf cfg.enable { - environment.systemPackages = packages; + assertions = [ + { + assertion = cfg.pulse.enable -> !config.hardware.pulseaudio.enable; + message = "PipeWire based PulseAudio server emulation replaces PulseAudio"; + } + { + assertion = cfg.jack.enable -> !config.services.jack.jackd.enable; + message = "PipeWire based JACK emulation doesn't use the JACK service"; + } + ]; + + services.pipewire.sessionManager = mkDefault "${cfg.package}/bin/pipewire-media-session"; + + environment.systemPackages = [ cfg.package ] + ++ lib.optional cfg.jack.enable jack-libs; - systemd.packages = packages; + systemd.packages = [ cfg.package ] + ++ lib.optional cfg.pulse.enable cfg.package.pulse; + # PipeWire depends on DBUS but doesn't list it. Without this booting + # into a terminal results in the service crashing with an error. systemd.user.sockets.pipewire.wantedBy = lib.mkIf cfg.socketActivation [ "sockets.target" ]; - }; + systemd.user.sockets.pipewire-pulse.wantedBy = lib.mkIf (cfg.socketActivation && cfg.pulse.enable) ["sockets.target"]; + systemd.user.services.pipewire.bindsTo = [ "dbus.service" ]; + services.udev.packages = [ cfg.package ]; + + # If any paths are updated here they must also be updated in the package test. + environment.etc."alsa/conf.d/49-pipewire-modules.conf" = mkIf cfg.alsa.enable { + text = '' + pcm_type.pipewire { + libs.native = ${cfg.package.lib}/lib/alsa-lib/libasound_module_pcm_pipewire.so ; + ${optionalString enable32BitAlsaPlugins + "libs.32Bit = ${pkgs.pkgsi686Linux.pipewire.lib}/lib/alsa-lib/libasound_module_pcm_pipewire.so ;"} + } + ctl_type.pipewire { + libs.native = ${cfg.package.lib}/lib/alsa-lib/libasound_module_ctl_pipewire.so ; + ${optionalString enable32BitAlsaPlugins + "libs.32Bit = ${pkgs.pkgsi686Linux.pipewire.lib}/lib/alsa-lib/libasound_module_ctl_pipewire.so ;"} + } + ''; + }; + environment.etc."alsa/conf.d/50-pipewire.conf" = mkIf cfg.alsa.enable { + source = "${cfg.package}/share/alsa/alsa.conf.d/50-pipewire.conf"; + }; + environment.etc."alsa/conf.d/99-pipewire-default.conf" = mkIf cfg.alsa.enable { + source = "${cfg.package}/share/alsa/alsa.conf.d/99-pipewire-default.conf"; + }; + environment.sessionVariables.LD_LIBRARY_PATH = + lib.optional cfg.jack.enable "/run/current-system/sw/lib/pipewire"; + + environment.etc."pipewire/pipewire.conf" = { + # Adapted from src/daemon/pipewire.conf.in + text = '' + set-prop link.max-buffers 16 # version < 3 clients can't handle more + + add-spa-lib audio.convert* audioconvert/libspa-audioconvert + add-spa-lib api.alsa.* alsa/libspa-alsa + add-spa-lib api.v4l2.* v4l2/libspa-v4l2 + add-spa-lib api.libcamera.* libcamera/libspa-libcamera + add-spa-lib api.bluez5.* bluez5/libspa-bluez5 + add-spa-lib api.vulkan.* vulkan/libspa-vulkan + add-spa-lib api.jack.* jack/libspa-jack + add-spa-lib support.* support/libspa-support + + load-module libpipewire-module-rtkit # rt.prio=20 rt.time.soft=200000 rt.time.hard=200000 + load-module libpipewire-module-protocol-native + load-module libpipewire-module-profiler + load-module libpipewire-module-metadata + load-module libpipewire-module-spa-device-factory + load-module libpipewire-module-spa-node-factory + load-module libpipewire-module-client-node + load-module libpipewire-module-client-device + load-module libpipewire-module-portal + load-module libpipewire-module-access + load-module libpipewire-module-adapter + load-module libpipewire-module-link-factory + load-module libpipewire-module-session-manager + + create-object spa-node-factory factory.name=support.node.driver node.name=Dummy priority.driver=8000 + exec ${cfg.sessionManager} ${lib.concatStringsSep " " cfg.sessionManagerArguments} + + ${cfg.extraConfig} + ''; + }; + + environment.etc."pipewire/media-session.d/with-alsa" = mkIf cfg.alsa.enable { text = ""; }; + environment.etc."pipewire/media-session.d/with-pulseaudio" = mkIf cfg.pulse.enable { text = ""; }; + environment.etc."pipewire/media-session.d/with-jack" = mkIf cfg.jack.enable { text = ""; }; + }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/profile-sync-daemon.nix b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/profile-sync-daemon.nix index a8ac22ac1276..6206295272fc 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/profile-sync-daemon.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/profile-sync-daemon.nix @@ -36,7 +36,7 @@ in { description = "Profile Sync daemon"; wants = [ "psd-resync.service" ]; wantedBy = [ "default.target" ]; - path = with pkgs; [ rsync kmod gawk nettools utillinux profile-sync-daemon ]; + path = with pkgs; [ rsync kmod gawk nettools util-linux profile-sync-daemon ]; unitConfig = { RequiresMountsFor = [ "/home/" ]; }; @@ -55,7 +55,7 @@ in { wants = [ "psd-resync.timer" ]; partOf = [ "psd.service" ]; wantedBy = [ "default.target" ]; - path = with pkgs; [ rsync kmod gawk nettools utillinux profile-sync-daemon ]; + path = with pkgs; [ rsync kmod gawk nettools util-linux profile-sync-daemon ]; serviceConfig = { Type = "oneshot"; ExecStart = "${pkgs.profile-sync-daemon}/bin/profile-sync-daemon resync"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/telepathy.nix b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/telepathy.nix index 34596bf78184..8c50d860e5bb 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/desktops/telepathy.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/desktops/telepathy.nix @@ -38,6 +38,11 @@ with lib; services.dbus.packages = [ pkgs.telepathy-mission-control ]; + # Enable runtime optional telepathy in gnome-shell + services.xserver.desktopManager.gnome3.sessionPath = with pkgs; [ + telepathy-glib + telepathy-logger + ]; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/development/hoogle.nix b/infra/libkookie/nixpkgs/nixos/modules/services/development/hoogle.nix index 1a98f005602a..cbf13f027de2 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/development/hoogle.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/development/hoogle.nix @@ -61,10 +61,8 @@ in { Restart = "always"; ExecStart = ''${hoogleEnv}/bin/hoogle server --local --port ${toString cfg.port} --home ${cfg.home}''; - User = "nobody"; - Group = "nogroup"; + DynamicUser = true; - PrivateTmp = true; ProtectHome = true; RuntimeDirectory = "hoogle"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/games/factorio.nix b/infra/libkookie/nixpkgs/nixos/modules/services/games/factorio.nix index 4b2e1a3c07f0..73099ae33634 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/games/factorio.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/games/factorio.nix @@ -49,8 +49,13 @@ in default = 34197; description = '' The port to which the service should bind. - - This option will also open up the UDP port in the firewall configuration. + ''; + }; + openFirewall = mkOption { + type = types.bool; + default = false; + description = '' + Whether to automatically open the specified UDP port in the firewall. ''; }; saveName = mkOption { @@ -237,6 +242,6 @@ in }; }; - networking.firewall.allowedUDPPorts = [ cfg.port ]; + networking.firewall.allowedUDPPorts = if cfg.openFirewall then [ cfg.port ] else []; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/bluetooth.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/bluetooth.nix index dfa39e7f6024..6f5a6d3bf288 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/bluetooth.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/bluetooth.nix @@ -15,6 +15,8 @@ in { hardware.bluetooth = { enable = mkEnableOption "support for Bluetooth"; + hsphfpd.enable = mkEnableOption "support for hsphfpd[-prototype] implementation"; + powerOnBoot = mkOption { type = types.bool; default = true; @@ -72,7 +74,8 @@ in { }; }; - environment.systemPackages = [ bluez-bluetooth ]; + environment.systemPackages = [ bluez-bluetooth ] + ++ optionals cfg.hsphfpd.enable [ pkgs.hsphfpd ]; environment.etc."bluetooth/main.conf"= { source = pkgs.writeText "main.conf" @@ -80,19 +83,42 @@ in { }; services.udev.packages = [ bluez-bluetooth ]; - services.dbus.packages = [ bluez-bluetooth ]; + services.dbus.packages = [ bluez-bluetooth ] + ++ optionals cfg.hsphfpd.enable [ pkgs.hsphfpd ]; systemd.packages = [ bluez-bluetooth ]; systemd.services = { bluetooth = { wantedBy = [ "bluetooth.target" ]; aliases = [ "dbus-org.bluez.service" ]; + # restarting can leave people without a mouse/keyboard + unitConfig.X-RestartIfChanged = false; }; - }; + } + // (optionalAttrs cfg.hsphfpd.enable { + hsphfpd = { + after = [ "bluetooth.service" ]; + requires = [ "bluetooth.service" ]; + wantedBy = [ "multi-user.target" ]; + + description = "A prototype implementation used for connecting HSP/HFP Bluetooth devices"; + serviceConfig.ExecStart = "${pkgs.hsphfpd}/bin/hsphfpd.pl"; + }; + }) + ; systemd.user.services = { obex.aliases = [ "dbus-org.bluez.obex.service" ]; - }; + } + // (optionalAttrs cfg.hsphfpd.enable { + telephony_client = { + wantedBy = [ "default.target"]; + + description = "telephony_client for hsphfpd"; + serviceConfig.ExecStart = "${pkgs.hsphfpd}/bin/telephony_client.pl"; + }; + }) + ; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/fwupd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/fwupd.nix index 222ac8e487eb..51eca19dca32 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/fwupd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/fwupd.nix @@ -11,8 +11,8 @@ let "fwupd/daemon.conf" = { source = pkgs.writeText "daemon.conf" '' [fwupd] - BlacklistDevices=${lib.concatStringsSep ";" cfg.blacklistDevices} - BlacklistPlugins=${lib.concatStringsSep ";" cfg.blacklistPlugins} + DisabledDevices=${lib.concatStringsSep ";" cfg.disabledDevices} + DisabledPlugins=${lib.concatStringsSep ";" cfg.disabledPlugins} ''; }; "fwupd/uefi.conf" = { @@ -59,21 +59,21 @@ in { ''; }; - blacklistDevices = mkOption { + disabledDevices = mkOption { type = types.listOf types.str; default = []; example = [ "2082b5e0-7a64-478a-b1b2-e3404fab6dad" ]; description = '' - Allow blacklisting specific devices by their GUID + Allow disabling specific devices by their GUID ''; }; - blacklistPlugins = mkOption { + disabledPlugins = mkOption { type = types.listOf types.str; default = []; example = [ "udev" ]; description = '' - Allow blacklisting specific plugins + Allow disabling specific plugins ''; }; @@ -105,11 +105,15 @@ in { }; }; + imports = [ + (mkRenamedOptionModule [ "services" "fwupd" "blacklistDevices"] [ "services" "fwupd" "disabledDevices" ]) + (mkRenamedOptionModule [ "services" "fwupd" "blacklistPlugins"] [ "services" "fwupd" "disabledPlugins" ]) + ]; ###### implementation config = mkIf cfg.enable { # Disable test related plug-ins implicitly so that users do not have to care about them. - services.fwupd.blacklistPlugins = cfg.package.defaultBlacklistedPlugins; + services.fwupd.disabledPlugins = cfg.package.defaultDisabledPlugins; environment.systemPackages = [ cfg.package ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/lcd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/lcd.nix index d78d742cd318..dc8595ea60cd 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/lcd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/lcd.nix @@ -151,14 +151,13 @@ in with lib; { description = "LCDproc - client"; after = [ "lcdd.service" ]; wantedBy = [ "lcd.target" ]; + # Allow restarting for eternity + startLimitIntervalSec = lib.mkIf cfg.client.restartForever 0; serviceConfig = serviceCfg // { ExecStart = "${pkg}/bin/lcdproc -f -c ${clientCfg}"; # If the server is being restarted at the same time, the client will # fail as it cannot connect, so space it out a bit. RestartSec = "5"; - # Allow restarting for eternity - StartLimitIntervalSec = lib.mkIf cfg.client.restartForever "0"; - StartLimitBurst = lib.mkIf cfg.client.restartForever "0"; }; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/sane.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/sane.nix index b344dfc20610..03070a8f9e7c 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/sane.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/sane.nix @@ -148,7 +148,7 @@ in # saned needs to distinguish between IPv4 and IPv6 to open matching data sockets. BindIPv6Only = "ipv6-only"; Accept = true; - MaxConnections = 1; + MaxConnections = 64; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/thermald.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/thermald.nix index b7be0e89d0c6..241490c5aae7 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/thermald.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/thermald.nix @@ -23,15 +23,6 @@ in { default = null; description = "the thermald manual configuration file."; }; - - adaptive = mkOption { - type = types.bool; - default = false; - description = '' - Whether to enable adaptive mode, only working on kernel versions greater than 5.8. - Thermald will detect this itself, safe to enable on kernel versions below 5.8. - ''; - }; }; }; @@ -48,8 +39,8 @@ in { --no-daemon \ ${optionalString cfg.debug "--loglevel=debug"} \ ${optionalString (cfg.configFile != null) "--config-file ${cfg.configFile}"} \ - ${optionalString cfg.adaptive "--adaptive"} \ --dbus-enable + --adaptive ''; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/tlp.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/tlp.nix index 4230f2edd279..eb53f565a67f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/tlp.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/tlp.nix @@ -39,7 +39,7 @@ in default = ""; description = '' Verbatim additional configuration variables for TLP. - DEPRECATED: use services.tlp.config instead. + DEPRECATED: use services.tlp.settings instead. ''; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/udev.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/udev.nix index 587b9b0234aa..a212adb7342d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/udev.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/udev.nix @@ -57,8 +57,8 @@ let substituteInPlace $i \ --replace \"/sbin/modprobe \"${pkgs.kmod}/bin/modprobe \ --replace \"/sbin/mdadm \"${pkgs.mdadm}/sbin/mdadm \ - --replace \"/sbin/blkid \"${pkgs.utillinux}/sbin/blkid \ - --replace \"/bin/mount \"${pkgs.utillinux}/bin/mount \ + --replace \"/sbin/blkid \"${pkgs.util-linux}/sbin/blkid \ + --replace \"/bin/mount \"${pkgs.util-linux}/bin/mount \ --replace /usr/bin/readlink ${pkgs.coreutils}/bin/readlink \ --replace /usr/bin/basename ${pkgs.coreutils}/bin/basename done @@ -280,7 +280,7 @@ in services.udev.packages = [ extraUdevRules extraHwdbFile ]; - services.udev.path = [ pkgs.coreutils pkgs.gnused pkgs.gnugrep pkgs.utillinux udev ]; + services.udev.path = [ pkgs.coreutils pkgs.gnused pkgs.gnugrep pkgs.util-linux udev ]; boot.kernelParams = mkIf (!config.networking.usePredictableInterfaceNames) [ "net.ifnames=0" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/undervolt.nix b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/undervolt.nix index 054ffa35050a..9c2f78a755dd 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/hardware/undervolt.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/hardware/undervolt.nix @@ -3,7 +3,12 @@ with lib; let cfg = config.services.undervolt; - cliArgs = lib.cli.toGNUCommandLineShell {} { + + mkPLimit = limit: window: + if (isNull limit && isNull window) then null + else assert asserts.assertMsg (!isNull limit && !isNull window) "Both power limit and window must be set"; + "${toString limit} ${toString window}"; + cliArgs = lib.cli.toGNUCommandLine {} { inherit (cfg) verbose temp @@ -21,6 +26,9 @@ let temp-bat = cfg.tempBat; temp-ac = cfg.tempAc; + + power-limit-long = mkPLimit cfg.p1.limit cfg.p1.window; + power-limit-short = mkPLimit cfg.p2.limit cfg.p2.window; }; in { @@ -104,6 +112,40 @@ in ''; }; + p1.limit = mkOption { + type = with types; nullOr int; + default = null; + description = '' + The P1 Power Limit in Watts. + Both limit and window must be set. + ''; + }; + p1.window = mkOption { + type = with types; nullOr (oneOf [ float int ]); + default = null; + description = '' + The P1 Time Window in seconds. + Both limit and window must be set. + ''; + }; + + p2.limit = mkOption { + type = with types; nullOr int; + default = null; + description = '' + The P2 Power Limit in Watts. + Both limit and window must be set. + ''; + }; + p2.window = mkOption { + type = with types; nullOr (oneOf [ float int ]); + default = null; + description = '' + The P2 Time Window in seconds. + Both limit and window must be set. + ''; + }; + useTimer = mkOption { type = types.bool; default = false; @@ -133,7 +175,7 @@ in serviceConfig = { Type = "oneshot"; Restart = "no"; - ExecStart = "${pkgs.undervolt}/bin/undervolt ${cliArgs}"; + ExecStart = "${pkgs.undervolt}/bin/undervolt ${toString cliArgs}"; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/logging/promtail.nix b/infra/libkookie/nixpkgs/nixos/modules/services/logging/promtail.nix new file mode 100644 index 000000000000..19b12daa4152 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/logging/promtail.nix @@ -0,0 +1,86 @@ +{ config, lib, pkgs, ... }: with lib; +let + cfg = config.services.promtail; + + prettyJSON = conf: pkgs.runCommandLocal "promtail-config.json" {} '' + echo '${builtins.toJSON conf}' | ${pkgs.buildPackages.jq}/bin/jq 'del(._module)' > $out + ''; + + allowSystemdJournal = cfg.configuration ? scrape_configs && lib.any (v: v ? journal) cfg.configuration.scrape_configs; +in { + options.services.promtail = with types; { + enable = mkEnableOption "the Promtail ingresser"; + + + configuration = mkOption { + type = (pkgs.formats.json {}).type; + description = '' + Specify the configuration for Promtail in Nix. + ''; + }; + + extraFlags = mkOption { + type = listOf str; + default = []; + example = [ "--server.http-listen-port=3101" ]; + description = '' + Specify a list of additional command line flags, + which get escaped and are then passed to Loki. + ''; + }; + }; + + config = mkIf cfg.enable { + services.promtail.configuration.positions.filename = mkDefault "/var/cache/promtail/positions.yaml"; + + systemd.services.promtail = { + description = "Promtail log ingress"; + wantedBy = [ "multi-user.target" ]; + stopIfChanged = false; + + serviceConfig = { + Restart = "on-failure"; + + ExecStart = "${pkgs.grafana-loki}/bin/promtail -config.file=${prettyJSON cfg.configuration} ${escapeShellArgs cfg.extraFlags}"; + + ProtectSystem = "strict"; + ProtectHome = true; + PrivateTmp = true; + PrivateDevices = true; + ProtectKernelTunables = true; + ProtectControlGroups = true; + RestrictSUIDSGID = true; + PrivateMounts = true; + CacheDirectory = "promtail"; + + User = "promtail"; + Group = "promtail"; + + CapabilityBoundingSet = ""; + NoNewPrivileges = true; + + ProtectKernelModules = true; + SystemCallArchitectures = "native"; + ProtectKernelLogs = true; + ProtectClock = true; + + LockPersonality = true; + ProtectHostname = true; + RestrictRealtime = true; + MemoryDenyWriteExecute = true; + PrivateUsers = true; + + SupplementaryGroups = lib.optional (allowSystemdJournal) "systemd-journal"; + } // (optionalAttrs (!pkgs.stdenv.isAarch64) { # FIXME: figure out why this breaks on aarch64 + SystemCallFilter = "@system-service"; + }); + }; + + users.groups.promtail = {}; + users.users.promtail = { + description = "Promtail service user"; + isSystemUser = true; + group = "promtail"; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/logging/vector.nix b/infra/libkookie/nixpkgs/nixos/modules/services/logging/vector.nix new file mode 100644 index 000000000000..a7c54ad75fde --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/logging/vector.nix @@ -0,0 +1,61 @@ +{ config, lib, pkgs, ... }: + +with lib; +let cfg = config.services.vector; + +in { + options.services.vector = { + enable = mkEnableOption "Vector"; + + journaldAccess = mkOption { + type = types.bool; + default = false; + description = '' + Enable Vector to access journald. + ''; + }; + + settings = mkOption { + type = (pkgs.formats.json { }).type; + default = { }; + description = '' + Specify the configuration for Vector in Nix. + ''; + }; + }; + + config = mkIf cfg.enable { + + users.groups.vector = { }; + users.users.vector = { + description = "Vector service user"; + group = "vector"; + isSystemUser = true; + }; + systemd.services.vector = { + description = "Vector event and log aggregator"; + wantedBy = [ "multi-user.target" ]; + after = [ "network-online.target" ]; + requires = [ "network-online.target" ]; + serviceConfig = let + format = pkgs.formats.toml { }; + conf = format.generate "vector.toml" cfg.settings; + validateConfig = file: + pkgs.runCommand "validate-vector-conf" { } '' + ${pkgs.vector}/bin/vector validate --no-topology --no-environment "${file}" + ln -s "${file}" "$out" + ''; + in { + ExecStart = "${pkgs.vector}/bin/vector --config ${validateConfig conf}"; + User = "vector"; + Group = "vector"; + Restart = "no"; + StateDirectory = "vector"; + ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; + AmbientCapabilities = "CAP_NET_BIND_SERVICE"; + # This group is required for accessing journald. + SupplementaryGroups = mkIf cfg.journaldAccess "systemd-journal"; + }; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/mail/dovecot.nix b/infra/libkookie/nixpkgs/nixos/modules/services/mail/dovecot.nix index f5c5f795dc1b..03e7e40e388e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/mail/dovecot.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/mail/dovecot.nix @@ -427,12 +427,12 @@ in wantedBy = [ "multi-user.target" ]; restartTriggers = [ cfg.configFile modulesDir ]; + startLimitIntervalSec = 60; # 1 min serviceConfig = { ExecStart = "${dovecotPkg}/sbin/dovecot -F"; ExecReload = "${dovecotPkg}/sbin/doveadm reload"; Restart = "on-failure"; RestartSec = "1s"; - StartLimitInterval = "1min"; RuntimeDirectory = [ "dovecot2" ]; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/mail/freepops.nix b/infra/libkookie/nixpkgs/nixos/modules/services/mail/freepops.nix deleted file mode 100644 index 5b729ca50a5e..000000000000 --- a/infra/libkookie/nixpkgs/nixos/modules/services/mail/freepops.nix +++ /dev/null @@ -1,89 +0,0 @@ -{ config, lib, pkgs, ... }: - -with lib; - -let - cfg = config.services.mail.freepopsd; -in - -{ - options = { - services.mail.freepopsd = { - enable = mkOption { - default = false; - type = with types; bool; - description = '' - Enables Freepops, a POP3 webmail wrapper. - ''; - }; - - port = mkOption { - default = 2000; - type = with types; uniq int; - description = '' - Port on which the pop server will listen. - ''; - }; - - threads = mkOption { - default = 5; - type = with types; uniq int; - description = '' - Max simultaneous connections. - ''; - }; - - bind = mkOption { - default = "0.0.0.0"; - type = types.str; - description = '' - Bind over an IPv4 address instead of any. - ''; - }; - - logFile = mkOption { - default = "/var/log/freepopsd"; - example = "syslog"; - type = types.str; - description = '' - Filename of the log file or syslog to rely on the logging daemon. - ''; - }; - - suid = { - user = mkOption { - default = "nobody"; - type = types.str; - description = '' - User name under which freepopsd will be after binding the port. - ''; - }; - - group = mkOption { - default = "nogroup"; - type = types.str; - description = '' - Group under which freepopsd will be after binding the port. - ''; - }; - }; - - }; - }; - - config = mkIf cfg.enable { - systemd.services.freepopsd = { - description = "Freepopsd (webmail over POP3)"; - after = [ "network.target" ]; - wantedBy = [ "multi-user.target" ]; - script = '' - ${pkgs.freepops}/bin/freepopsd \ - -p ${toString cfg.port} \ - -t ${toString cfg.threads} \ - -b ${cfg.bind} \ - -vv -l ${cfg.logFile} \ - -s ${cfg.suid.user}.${cfg.suid.group} - ''; - }; - }; -} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/mail/postfix.nix b/infra/libkookie/nixpkgs/nixos/modules/services/mail/postfix.nix index fd4d16cdc37b..319b3b638444 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/mail/postfix.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/mail/postfix.nix @@ -834,12 +834,6 @@ in }; services.postfix.masterConfig = { - smtp_inet = { - name = "smtp"; - type = "inet"; - private = false; - command = "smtpd"; - }; pickup = { private = false; wakeup = 60; @@ -921,6 +915,12 @@ in in concatLists (mapAttrsToList mkKeyVal cfg.submissionOptions); }; } // optionalAttrs cfg.enableSmtp { + smtp_inet = { + name = "smtp"; + type = "inet"; + private = false; + command = "smtpd"; + }; smtp = {}; relay = { command = "smtp"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/mail/roundcube.nix b/infra/libkookie/nixpkgs/nixos/modules/services/mail/roundcube.nix index a0bbab64985b..ee7aa7e22fb9 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/mail/roundcube.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/mail/roundcube.nix @@ -204,6 +204,11 @@ in }; systemd.services.phpfpm-roundcube.after = [ "roundcube-setup.service" ]; + # Restart on config changes. + systemd.services.phpfpm-roundcube.restartTriggers = [ + config.environment.etc."roundcube/config.inc.php".source + ]; + systemd.services.roundcube-setup = mkMerge [ (mkIf (cfg.database.host == "localhost") { requires = [ "postgresql.service" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/mail/rspamd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/mail/rspamd.nix index aacdbe2aeed2..2f9d28195bd8 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/mail/rspamd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/mail/rspamd.nix @@ -153,7 +153,7 @@ let ${concatStringsSep "\n" (mapAttrsToList (name: value: let includeName = if name == "rspamd_proxy" then "proxy" else name; - tryOverride = if value.extraConfig == "" then "true" else "false"; + tryOverride = boolToString (value.extraConfig == ""); in '' worker "${value.type}" { type = "${value.type}"; @@ -371,6 +371,9 @@ in }; services.postfix.config = mkIf cfg.postfix.enable cfg.postfix.config; + systemd.services.postfix.serviceConfig.SupplementaryGroups = + mkIf cfg.postfix.enable [ postfixCfg.group ]; + # Allow users to run 'rspamc' and 'rspamadm'. environment.systemPackages = [ pkgs.rspamd ]; @@ -394,21 +397,50 @@ in restartTriggers = [ rspamdDir ]; serviceConfig = { - ExecStart = "${pkgs.rspamd}/bin/rspamd ${optionalString cfg.debug "-d"} --user=${cfg.user} --group=${cfg.group} --pid=/run/rspamd.pid -c /etc/rspamd/rspamd.conf -f"; + ExecStart = "${pkgs.rspamd}/bin/rspamd ${optionalString cfg.debug "-d"} -c /etc/rspamd/rspamd.conf -f"; Restart = "always"; + + User = "${cfg.user}"; + Group = "${cfg.group}"; + SupplementaryGroups = mkIf cfg.postfix.enable [ postfixCfg.group ]; + RuntimeDirectory = "rspamd"; + RuntimeDirectoryMode = "0755"; + StateDirectory = "rspamd"; + StateDirectoryMode = "0700"; + + AmbientCapabilities = []; + CapabilityBoundingSet = []; + DevicePolicy = "closed"; + LockPersonality = true; + NoNewPrivileges = true; + PrivateDevices = true; + PrivateMounts = true; PrivateTmp = true; + # we need to chown socket to rspamd-milter + PrivateUsers = !cfg.postfix.enable; + ProtectClock = true; + ProtectControlGroups = true; + ProtectHome = true; + ProtectHostname = true; + ProtectKernelLogs = true; + ProtectKernelModules = true; + ProtectKernelTunables = true; + ProtectSystem = "strict"; + RemoveIPC = true; + RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ]; + RestrictNamespaces = true; + RestrictRealtime = true; + RestrictSUIDSGID = true; + SystemCallArchitectures = "native"; + SystemCallFilter = "@system-service"; + UMask = "0077"; }; - - preStart = '' - ${pkgs.coreutils}/bin/mkdir -p /var/lib/rspamd - ${pkgs.coreutils}/bin/chown ${cfg.user}:${cfg.group} /var/lib/rspamd - ''; }; }; imports = [ (mkRemovedOptionModule [ "services" "rspamd" "socketActivation" ] - "Socket activation never worked correctly and could at this time not be fixed and so was removed") + "Socket activation never worked correctly and could at this time not be fixed and so was removed") (mkRenamedOptionModule [ "services" "rspamd" "bindSocket" ] [ "services" "rspamd" "workers" "normal" "bindSockets" ]) (mkRenamedOptionModule [ "services" "rspamd" "bindUISocket" ] [ "services" "rspamd" "workers" "controller" "bindSockets" ]) (mkRemovedOptionModule [ "services" "rmilter" ] "Use services.rspamd.* instead to set up milter service") diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/autorandr.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/autorandr.nix index cf7fb5f78d3d..dfb418af6ede 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/autorandr.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/autorandr.nix @@ -37,9 +37,9 @@ in { description = "Autorandr execution hook"; after = [ "sleep.target" ]; + startLimitIntervalSec = 5; + startLimitBurst = 1; serviceConfig = { - StartLimitInterval = 5; - StartLimitBurst = 1; ExecStart = "${pkgs.autorandr}/bin/autorandr --batch --change --default ${cfg.defaultTarget}"; Type = "oneshot"; RemainAfterExit = false; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/cfdyndns.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/cfdyndns.nix index dcf416022734..15af1f50da1d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/cfdyndns.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/cfdyndns.nix @@ -6,6 +6,12 @@ let cfg = config.services.cfdyndns; in { + imports = [ + (mkRemovedOptionModule + [ "services" "cfdyndns" "apikey" ] + "Use services.cfdyndns.apikeyFile instead.") + ]; + options = { services.cfdyndns = { enable = mkEnableOption "Cloudflare Dynamic DNS Client"; @@ -17,10 +23,12 @@ in ''; }; - apikey = mkOption { - type = types.str; + apikeyFile = mkOption { + default = null; + type = types.nullOr types.str; description = '' - The API Key to use to authenticate to CloudFlare. + The path to a file containing the API Key + used to authenticate with CloudFlare. ''; }; @@ -45,13 +53,17 @@ in Type = "simple"; User = config.ids.uids.cfdyndns; Group = config.ids.gids.cfdyndns; - ExecStart = "/bin/sh -c '${pkgs.cfdyndns}/bin/cfdyndns'"; }; environment = { CLOUDFLARE_EMAIL="${cfg.email}"; - CLOUDFLARE_APIKEY="${cfg.apikey}"; CLOUDFLARE_RECORDS="${concatStringsSep "," cfg.records}"; }; + script = '' + ${optionalString (cfg.apikeyFile != null) '' + export CLOUDFLARE_APIKEY="$(cat ${escapeShellArg cfg.apikeyFile})" + ''} + ${pkgs.cfdyndns}/bin/cfdyndns + ''; }; users.users = { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/cgminer.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/cgminer.nix index 7635c2a0f4e9..fa9c8c54509e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/cgminer.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/cgminer.nix @@ -126,12 +126,12 @@ in GPU_USE_SYNC_OBJECTS = "1"; }; + startLimitIntervalSec = 60; # 1 min serviceConfig = { ExecStart = "${pkgs.cgminer}/bin/cgminer --syslog --text-only --config ${cgminerConfig}"; User = cfg.user; RestartSec = "30s"; Restart = "always"; - StartLimitInterval = "1m"; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/disnix.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/disnix.nix index 69386cdbb381..41483d80a2dd 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/disnix.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/disnix.nix @@ -34,6 +34,14 @@ in defaultText = "pkgs.disnix"; }; + enableProfilePath = mkEnableOption "exposing the Disnix profiles in the system's PATH"; + + profiles = mkOption { + type = types.listOf types.string; + default = [ "default" ]; + example = [ "default" ]; + description = "Names of the Disnix profiles to expose in the system's PATH"; + }; }; }; @@ -44,6 +52,7 @@ in dysnomia.enable = true; environment.systemPackages = [ pkgs.disnix ] ++ optional cfg.useWebServiceInterface pkgs.DisnixWebService; + environment.variables.PATH = lib.optionals cfg.enableProfilePath (map (profileName: "/nix/var/nix/profiles/disnix/${profileName}/bin" ) cfg.profiles); services.dbus.enable = true; services.dbus.packages = [ pkgs.disnix ]; @@ -68,7 +77,8 @@ in ++ optional config.services.postgresql.enable "postgresql.service" ++ optional config.services.tomcat.enable "tomcat.service" ++ optional config.services.svnserve.enable "svnserve.service" - ++ optional config.services.mongodb.enable "mongodb.service"; + ++ optional config.services.mongodb.enable "mongodb.service" + ++ optional config.services.influxdb.enable "influxdb.service"; restartIfChanged = false; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/dysnomia.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/dysnomia.nix index 4b52963500d1..eb94791fbbff 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/dysnomia.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/dysnomia.nix @@ -66,6 +66,19 @@ let ) (builtins.attrNames cfg.components)} ''; }; + + dysnomiaFlags = { + enableApacheWebApplication = config.services.httpd.enable; + enableAxis2WebService = config.services.tomcat.axis2.enable; + enableDockerContainer = config.virtualisation.docker.enable; + enableEjabberdDump = config.services.ejabberd.enable; + enableMySQLDatabase = config.services.mysql.enable; + enablePostgreSQLDatabase = config.services.postgresql.enable; + enableTomcatWebApplication = config.services.tomcat.enable; + enableMongoDatabase = config.services.mongodb.enable; + enableSubversionRepository = config.services.svnserve.enable; + enableInfluxDatabase = config.services.influxdb.enable; + }; in { options = { @@ -117,6 +130,12 @@ in description = "A list of paths containing additional modules that are added to the search folders"; default = []; }; + + enableLegacyModules = mkOption { + type = types.bool; + default = true; + description = "Whether to enable Dysnomia legacy process and wrapper modules"; + }; }; }; @@ -142,34 +161,48 @@ in environment.systemPackages = [ cfg.package ]; - dysnomia.package = pkgs.dysnomia.override (origArgs: { - enableApacheWebApplication = config.services.httpd.enable; - enableAxis2WebService = config.services.tomcat.axis2.enable; - enableEjabberdDump = config.services.ejabberd.enable; - enableMySQLDatabase = config.services.mysql.enable; - enablePostgreSQLDatabase = config.services.postgresql.enable; - enableSubversionRepository = config.services.svnserve.enable; - enableTomcatWebApplication = config.services.tomcat.enable; - enableMongoDatabase = config.services.mongodb.enable; - enableInfluxDatabase = config.services.influxdb.enable; + dysnomia.package = pkgs.dysnomia.override (origArgs: dysnomiaFlags // lib.optionalAttrs (cfg.enableLegacyModules) { + enableLegacy = builtins.trace '' + WARNING: Dysnomia has been configured to use the legacy 'process' and 'wrapper' + modules for compatibility reasons! If you rely on these modules, consider + migrating to better alternatives. + + More information: https://raw.githubusercontent.com/svanderburg/dysnomia/f65a9a84827bcc4024d6b16527098b33b02e4054/README-legacy.md + + If you have migrated already or don't rely on these Dysnomia modules, you can + disable legacy mode with the following NixOS configuration option: + + dysnomia.enableLegacyModules = false; + + In a future version of Dysnomia (and NixOS) the legacy option will go away! + '' true; }); dysnomia.properties = { hostname = config.networking.hostName; inherit (config.nixpkgs.localSystem) system; - supportedTypes = (import "${pkgs.stdenv.mkDerivation { - name = "supportedtypes"; - buildCommand = '' - ( echo -n "[ " - cd ${cfg.package}/libexec/dysnomia - for i in * - do - echo -n "\"$i\" " - done - echo -n " ]") > $out - ''; - }}"); + supportedTypes = [ + "echo" + "fileset" + "process" + "wrapper" + + # These are not base modules, but they are still enabled because they work with technology that are always enabled in NixOS + "systemd-unit" + "sysvinit-script" + "nixos-configuration" + ] + ++ optional (dysnomiaFlags.enableApacheWebApplication) "apache-webapplication" + ++ optional (dysnomiaFlags.enableAxis2WebService) "axis2-webservice" + ++ optional (dysnomiaFlags.enableDockerContainer) "docker-container" + ++ optional (dysnomiaFlags.enableEjabberdDump) "ejabberd-dump" + ++ optional (dysnomiaFlags.enableInfluxDatabase) "influx-database" + ++ optional (dysnomiaFlags.enableMySQLDatabase) "mysql-database" + ++ optional (dysnomiaFlags.enablePostgreSQLDatabase) "postgresql-database" + ++ optional (dysnomiaFlags.enableTomcatWebApplication) "tomcat-webapplication" + ++ optional (dysnomiaFlags.enableMongoDatabase) "mongo-database" + ++ optional (dysnomiaFlags.enableSubversionRepository) "subversion-repository"; }; dysnomia.containers = lib.recursiveUpdate ({ @@ -185,9 +218,9 @@ in }; } // lib.optionalAttrs (config.services.mysql.enable) { mysql-database = { mysqlPort = config.services.mysql.port; + mysqlSocket = "/run/mysqld/mysqld.sock"; } // lib.optionalAttrs cfg.enableAuthentication { mysqlUsername = "root"; - mysqlPassword = builtins.readFile (config.services.mysql.rootPassword); }; } // lib.optionalAttrs (config.services.postgresql.enable) { postgresql-database = { @@ -199,6 +232,13 @@ in tomcatPort = 8080; }; } // lib.optionalAttrs (config.services.mongodb.enable) { mongo-database = {}; } + // lib.optionalAttrs (config.services.influxdb.enable) { + influx-database = { + influxdbUsername = config.services.influxdb.user; + influxdbDataDir = "${config.services.influxdb.dataDir}/data"; + influxdbMetaDir = "${config.services.influxdb.dataDir}/meta"; + }; + } // lib.optionalAttrs (config.services.svnserve.enable) { subversion-repository = { svnBaseDir = config.services.svnserve.svnBaseDir; }; }) cfg.extraContainerProperties; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/fstrim.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/fstrim.nix index b8841a7fe74c..5258f5acb410 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/fstrim.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/fstrim.nix @@ -31,7 +31,7 @@ in { config = mkIf cfg.enable { - systemd.packages = [ pkgs.utillinux ]; + systemd.packages = [ pkgs.util-linux ]; systemd.timers.fstrim = { timerConfig = { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/gitlab.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/gitlab.nix index 122bc3000b41..3ee7a81dc375 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/gitlab.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/gitlab.nix @@ -43,9 +43,13 @@ let [gitlab-shell] dir = "${cfg.packages.gitlab-shell}" + + [gitlab] secret_file = "${cfg.statePath}/gitlab_shell_secret" - gitlab_url = "http+unix://${pathUrlQuote gitlabSocket}" - http_settings = { self_signed_cert = false } + url = "http+unix://${pathUrlQuote gitlabSocket}" + + [gitlab.http-settings] + self_signed_cert = false ${concatStringsSep "\n" (attrValues (mapAttrs (k: v: '' [[storage]] @@ -119,6 +123,7 @@ let receive_pack = true; }; workhorse.secret_file = "${cfg.statePath}/.gitlab_workhorse_secret"; + gitlab_kas.secret_file = "${cfg.statePath}/.gitlab_kas_secret"; git.bin_path = "git"; monitoring = { ip_whitelist = [ "127.0.0.0/8" "::1/128" ]; @@ -653,7 +658,7 @@ in { script = '' set -eu - PSQL="${pkgs.utillinux}/bin/runuser -u ${pgsql.superUser} -- psql --port=${toString pgsql.port}" + PSQL="${pkgs.util-linux}/bin/runuser -u ${pgsql.superUser} -- psql --port=${toString pgsql.port}" $PSQL -tAc "SELECT 1 FROM pg_database WHERE datname = '${cfg.databaseName}'" | grep -q 1 || $PSQL -tAc 'CREATE DATABASE "${cfg.databaseName}" OWNER "${cfg.databaseUsername}"' current_owner=$($PSQL -tAc "SELECT pg_catalog.pg_get_userbyid(datdba) FROM pg_catalog.pg_database WHERE datname = '${cfg.databaseName}'") @@ -668,6 +673,7 @@ in { rm "${config.services.postgresql.dataDir}/.reassigning_${cfg.databaseName}" fi $PSQL '${cfg.databaseName}' -tAc "CREATE EXTENSION IF NOT EXISTS pg_trgm" + $PSQL '${cfg.databaseName}' -tAc "CREATE EXTENSION IF NOT EXISTS btree_gist;" ''; serviceConfig = { @@ -750,7 +756,8 @@ in { }; systemd.services.gitaly = { - after = [ "network.target" ]; + after = [ "network.target" "gitlab.service" ]; + bindsTo = [ "gitlab.service" ]; wantedBy = [ "multi-user.target" ]; path = with pkgs; [ openssh @@ -839,7 +846,7 @@ in { }; systemd.services.gitlab = { - after = [ "gitlab-workhorse.service" "gitaly.service" "network.target" "gitlab-postgresql.service" "redis.service" ]; + after = [ "gitlab-workhorse.service" "network.target" "gitlab-postgresql.service" "redis.service" ]; requires = [ "gitlab-sidekiq.service" ]; wantedBy = [ "multi-user.target" ]; environment = gitlabEnv; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/gogs.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/gogs.nix index c5070aaa356a..d7233f10c7cb 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/gogs.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/gogs.nix @@ -25,7 +25,6 @@ let HTTP_ADDR = ${cfg.httpAddress} HTTP_PORT = ${toString cfg.httpPort} ROOT_URL = ${cfg.rootUrl} - STATIC_ROOT_PATH = ${cfg.staticRootPath} [session] COOKIE_NAME = session @@ -179,13 +178,6 @@ in ''; }; - staticRootPath = mkOption { - type = types.str; - default = "${pkgs.gogs.data}"; - example = "/var/lib/gogs/data"; - description = "Upper level of template and static files path."; - }; - extraConfig = mkOption { type = types.str; default = ""; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/home-assistant.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/home-assistant.nix index 0477254e7c18..1f2e13f37325 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/home-assistant.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/home-assistant.nix @@ -245,7 +245,11 @@ in { Group = "hass"; Restart = "on-failure"; ProtectSystem = "strict"; - ReadWritePaths = "${cfg.configDir}"; + ReadWritePaths = let + cfgPath = [ "config" "homeassistant" "allowlist_external_dirs" ]; + value = attrByPath cfgPath [] cfg; + allowPaths = if isList value then value else singleton value; + in [ "${cfg.configDir}" ] ++ allowPaths; KillSignal = "SIGINT"; PrivateTmp = true; RemoveIPC = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/jellyfin.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/jellyfin.nix index 0493dadea94e..6a47dc3628f4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/jellyfin.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/jellyfin.nix @@ -45,6 +45,46 @@ in CacheDirectory = "jellyfin"; ExecStart = "${cfg.package}/bin/jellyfin --datadir '/var/lib/${StateDirectory}' --cachedir '/var/cache/${CacheDirectory}'"; Restart = "on-failure"; + + # Security options: + + NoNewPrivileges = true; + + AmbientCapabilities = ""; + CapabilityBoundingSet = ""; + + # ProtectClock= adds DeviceAllow=char-rtc r + DeviceAllow = ""; + + LockPersonality = true; + + PrivateTmp = true; + PrivateDevices = true; + PrivateUsers = true; + + ProtectClock = true; + ProtectControlGroups = true; + ProtectHostname = true; + ProtectKernelLogs = true; + ProtectKernelModules = true; + ProtectKernelTunables = true; + + RemoveIPC = true; + + RestrictNamespaces = true; + # AF_NETLINK needed because Jellyfin monitors the network connection + RestrictAddressFamilies = [ "AF_NETLINK" "AF_INET" "AF_INET6" ]; + RestrictRealtime = true; + RestrictSUIDSGID = true; + + SystemCallArchitectures = "native"; + SystemCallErrorNumber = "EPERM"; + SystemCallFilter = [ + "@system-service" + + "~@chown" "~@cpu-emulation" "~@debug" "~@keyring" "~@memlock" "~@module" + "~@obsolete" "~@privileged" "~@setuid" + ]; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/matrix-synapse.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/matrix-synapse.nix index 7f42184735c8..3abb9b7d69c8 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/matrix-synapse.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/matrix-synapse.nix @@ -713,7 +713,7 @@ in { ${ concatMapStringsSep "\n " (x: "--config-path ${x} \\") ([ configFile ] ++ cfg.extraConfigFiles) } --keys-directory ${cfg.dataDir} ''; - ExecReload = "${pkgs.utillinux}/bin/kill -HUP $MAINPID"; + ExecReload = "${pkgs.util-linux}/bin/kill -HUP $MAINPID"; Restart = "on-failure"; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/mautrix-telegram.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/mautrix-telegram.nix index c5e8a5b85ec2..caeb4b04164f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/mautrix-telegram.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/mautrix-telegram.nix @@ -21,6 +21,7 @@ in { default = { appservice = rec { database = "sqlite:///${dataDir}/mautrix-telegram.db"; + database_opts = {}; hostname = "0.0.0.0"; port = 8080; address = "http://localhost:${toString port}"; @@ -29,6 +30,8 @@ in { bridge = { permissions."*" = "relaybot"; relaybot.whitelist = [ ]; + double_puppet_server_map = {}; + login_shared_secret_map = {}; }; logging = { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/n8n.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/n8n.nix new file mode 100644 index 000000000000..516d0f70ef0b --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/n8n.nix @@ -0,0 +1,78 @@ +{ config, pkgs, lib, ... }: + +with lib; + +let + cfg = config.services.n8n; + format = pkgs.formats.json {}; + configFile = format.generate "n8n.json" cfg.settings; +in +{ + options.services.n8n = { + + enable = mkEnableOption "n8n server"; + + openFirewall = mkOption { + type = types.bool; + default = false; + description = "Open ports in the firewall for the n8n web interface."; + }; + + settings = mkOption { + type = format.type; + default = {}; + description = '' + Configuration for n8n, see <link xlink:href="https://docs.n8n.io/reference/configuration.html"/> + for supported values. + ''; + }; + + }; + + config = mkIf cfg.enable { + services.n8n.settings = { + # We use this to open the firewall, so we need to know about the default at eval time + port = lib.mkDefault 5678; + }; + + systemd.services.n8n = { + description = "N8N service"; + after = [ "network.target" ]; + wantedBy = [ "multi-user.target" ]; + environment = { + # This folder must be writeable as the application is storing + # its data in it, so the StateDirectory is a good choice + N8N_USER_FOLDER = "/var/lib/n8n"; + N8N_CONFIG_FILES = "${configFile}"; + }; + serviceConfig = { + Type = "simple"; + ExecStart = "${pkgs.n8n}/bin/n8n"; + Restart = "on-failure"; + StateDirectory = "n8n"; + + # Basic Hardening + NoNewPrivileges = "yes"; + PrivateTmp = "yes"; + PrivateDevices = "yes"; + DevicePolicy = "closed"; + DynamicUser = "true"; + ProtectSystem = "strict"; + ProtectHome = "read-only"; + ProtectControlGroups = "yes"; + ProtectKernelModules = "yes"; + ProtectKernelTunables = "yes"; + RestrictAddressFamilies = "AF_UNIX AF_INET AF_INET6 AF_NETLINK"; + RestrictNamespaces = "yes"; + RestrictRealtime = "yes"; + RestrictSUIDSGID = "yes"; + MemoryDenyWriteExecute = "yes"; + LockPersonality = "yes"; + }; + }; + + networking.firewall = mkIf cfg.openFirewall { + allowedTCPPorts = [ cfg.settings.port ]; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/nix-daemon.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/nix-daemon.nix index 2680b1cc0d3b..0eeff31d6c4d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/nix-daemon.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/nix-daemon.nix @@ -45,7 +45,7 @@ let trusted-substituters = ${toString cfg.trustedBinaryCaches} trusted-public-keys = ${toString cfg.binaryCachePublicKeys} auto-optimise-store = ${boolToString cfg.autoOptimiseStore} - require-sigs = ${if cfg.requireSignedBinaryCaches then "true" else "false"} + require-sigs = ${boolToString cfg.requireSignedBinaryCaches} trusted-users = ${toString cfg.trustedUsers} allowed-users = ${toString cfg.allowedUsers} ${optionalString (!cfg.distributedBuilds) '' @@ -539,7 +539,7 @@ in systemd.sockets.nix-daemon.wantedBy = [ "sockets.target" ]; systemd.services.nix-daemon = - { path = [ nix pkgs.utillinux config.programs.ssh.package ] + { path = [ nix pkgs.util-linux config.programs.ssh.package ] ++ optionals cfg.distributedBuilds [ pkgs.gzip ]; environment = cfg.envVars diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/octoprint.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/octoprint.nix index e2fbd3b401cc..a69e65073050 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/octoprint.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/octoprint.nix @@ -68,7 +68,7 @@ in plugins = mkOption { default = plugins: []; defaultText = "plugins: []"; - example = literalExample "plugins: with plugins; [ m33-fio stlviewer ]"; + example = literalExample "plugins: with plugins; [ themeify stlviewer ]"; description = "Additional plugins to be used. Available plugins are passed through the plugins input."; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/safeeyes.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/safeeyes.nix index 6ecb0d13187c..1e748195e41a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/safeeyes.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/safeeyes.nix @@ -32,14 +32,14 @@ in wantedBy = [ "graphical-session.target" ]; partOf = [ "graphical-session.target" ]; + startLimitIntervalSec = 350; + startLimitBurst = 10; serviceConfig = { ExecStart = '' ${pkgs.safeeyes}/bin/safeeyes ''; Restart = "on-failure"; RestartSec = 3; - StartLimitInterval = 350; - StartLimitBurst = 10; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/siproxd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/siproxd.nix index 0e87fc461d3f..20fe0793b84b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/siproxd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/siproxd.nix @@ -39,7 +39,7 @@ in default = false; description = '' Whether to enable the Siproxd SIP - proxy/masquerading daemon. + proxy/masquerading daemon. ''; }; @@ -57,29 +57,29 @@ in hostsAllowReg = mkOption { type = types.listOf types.str; - default = [ ]; + default = [ ]; example = [ "192.168.1.0/24" "192.168.2.0/24" ]; - description = '' + description = '' Acess control list for incoming SIP registrations. ''; }; hostsAllowSip = mkOption { type = types.listOf types.str; - default = [ ]; + default = [ ]; example = [ "123.45.0.0/16" "123.46.0.0/16" ]; - description = '' + description = '' Acess control list for incoming SIP traffic. ''; }; hostsDenySip = mkOption { type = types.listOf types.str; - default = [ ]; + default = [ ]; example = [ "10.0.0.0/8" "11.0.0.0/8" ]; - description = '' + description = '' Acess control list for denying incoming - SIP registrations and traffic. + SIP registrations and traffic. ''; }; @@ -87,7 +87,7 @@ in type = types.int; default = 5060; description = '' - Port to listen for incoming SIP messages. + Port to listen for incoming SIP messages. ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/svnserve.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/svnserve.nix index 3335ed09d40e..f70e3ca7fef0 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/svnserve.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/svnserve.nix @@ -25,7 +25,7 @@ in svnBaseDir = mkOption { default = "/repos"; - description = "Base directory from which Subversion repositories are accessed."; + description = "Base directory from which Subversion repositories are accessed."; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/misc/zigbee2mqtt.nix b/infra/libkookie/nixpkgs/nixos/modules/services/misc/zigbee2mqtt.nix index 0957920f1a09..cd987eb76c76 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/misc/zigbee2mqtt.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/misc/zigbee2mqtt.nix @@ -70,6 +70,7 @@ in description = "Zigbee2mqtt Service"; wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; + environment.ZIGBEE2MQTT_DATA = cfg.dataDir; serviceConfig = { ExecStart = "${cfg.package}/bin/zigbee2mqtt"; User = "zigbee2mqtt"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/loki.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/loki.nix index f4eec7e0d284..51cabaa274a3 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/loki.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/loki.nix @@ -39,7 +39,7 @@ in { }; configuration = mkOption { - type = types.attrs; + type = (pkgs.formats.json {}).type; default = {}; description = '' Specify the configuration for Loki in Nix. @@ -78,6 +78,8 @@ in { ''; }]; + environment.systemPackages = [ pkgs.grafana-loki ]; # logcli + users.groups.${cfg.group} = { }; users.users.${cfg.user} = { description = "Loki Service User"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/mackerel-agent.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/mackerel-agent.nix new file mode 100644 index 000000000000..7046de9d403c --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/mackerel-agent.nix @@ -0,0 +1,111 @@ +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.mackerel-agent; + settingsFmt = pkgs.formats.toml {}; +in { + options.services.mackerel-agent = { + enable = mkEnableOption "mackerel.io agent"; + + # the upstream package runs as root, but doesn't seem to be strictly + # necessary for basic functionality + runAsRoot = mkEnableOption "Whether to run as root."; + + autoRetirement = mkEnableOption '' + Whether to automatically retire the host upon OS shutdown. + ''; + + apiKeyFile = mkOption { + type = types.path; + default = ""; + example = "/run/keys/mackerel-api-key"; + description = '' + Path to file containing the Mackerel API key. The file should contain a + single line of the following form: + + <literallayout>apikey = "EXAMPLE_API_KEY"</literallayout> + ''; + }; + + settings = mkOption { + description = '' + Options for mackerel-agent.conf. + + Documentation: + <link xlink:href="https://mackerel.io/docs/entry/spec/agent"/> + ''; + + default = {}; + example = { + verbose = false; + silent = false; + }; + + type = types.submodule { + freeformType = settingsFmt.type; + + options.host_status = { + on_start = mkOption { + type = types.enum [ "working" "standby" "maintenance" "poweroff" ]; + description = "Host status after agent startup."; + default = "working"; + }; + on_stop = mkOption { + type = types.enum [ "working" "standby" "maintenance" "poweroff" ]; + description = "Host status after agent shutdown."; + default = "poweroff"; + }; + }; + + options.diagnostic = + mkEnableOption "Collect memory usage for the agent itself"; + }; + }; + }; + + config = mkIf cfg.enable { + environment.systemPackages = with pkgs; [ mackerel-agent ]; + + environment.etc = { + "mackerel-agent/mackerel-agent.conf".source = + settingsFmt.generate "mackerel-agent.conf" cfg.settings; + "mackerel-agent/conf.d/api-key.conf".source = cfg.apiKeyFile; + }; + + services.mackerel-agent.settings = { + root = mkDefault "/var/lib/mackerel-agent"; + pidfile = mkDefault "/run/mackerel-agent/mackerel-agent.pid"; + + # conf.d stores the symlink to cfg.apiKeyFile + include = mkDefault "/etc/mackerel-agent/conf.d/*.conf"; + }; + + # upstream service file in https://git.io/JUt4Q + systemd.services.mackerel-agent = { + description = "mackerel.io agent"; + after = [ "network-online.target" "nss-lookup.target" ]; + wantedBy = [ "multi-user.target" ]; + environment = { + MACKEREL_PLUGIN_WORKDIR = mkDefault "%C/mackerel-agent"; + }; + serviceConfig = { + DynamicUser = !cfg.runAsRoot; + PrivateTmp = mkDefault true; + CacheDirectory = "mackerel-agent"; + ConfigurationDirectory = "mackerel-agent"; + RuntimeDirectory = "mackerel-agent"; + StateDirectory = "mackerel-agent"; + ExecStart = "${pkgs.mackerel-agent}/bin/mackerel-agent supervise"; + ExecStopPost = mkIf cfg.autoRetirement "${pkg.mackerel-agent}/bin/mackerel-agent retire -force"; + ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; + LimitNOFILE = mkDefault 65536; + LimitNPROC = mkDefault 65536; + }; + restartTriggers = [ + config.environment.etc."mackerel-agent/mackerel-agent.conf".source + ]; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/netdata.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/netdata.nix index 2e73e15d3a86..db51fdbd2c61 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/netdata.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/netdata.nix @@ -142,7 +142,7 @@ in { serviceConfig = { Environment="PYTHONPATH=${cfg.package}/libexec/netdata/python.d/python_modules"; ExecStart = "${cfg.package}/bin/netdata -P /run/netdata/netdata.pid -D -c ${configFile}"; - ExecReload = "${pkgs.utillinux}/bin/kill -s HUP -s USR1 -s USR2 $MAINPID"; + ExecReload = "${pkgs.util-linux}/bin/kill -s HUP -s USR1 -s USR2 $MAINPID"; TimeoutStopSec = 60; Restart = "on-failure"; # User and group diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/default.nix index 98aaa9c0f030..72428957109c 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/default.nix @@ -45,7 +45,7 @@ let cmdlineArgs = cfg.extraFlags ++ [ "--storage.tsdb.path=${workingDir}/data/" - "--config.file=${prometheusYml}" + "--config.file=/run/prometheus/prometheus-substituted.yaml" "--web.listen-address=${cfg.listenAddress}:${builtins.toString cfg.port}" "--alertmanager.notification-queue-capacity=${toString cfg.alertmanagerNotificationQueueCapacity}" "--alertmanager.timeout=${toString cfg.alertmanagerTimeout}s" @@ -522,6 +522,45 @@ in { ''; }; + environmentFile = mkOption { + type = types.nullOr types.path; + default = null; + example = "/root/prometheus.env"; + description = '' + Environment file as defined in <citerefentry> + <refentrytitle>systemd.exec</refentrytitle><manvolnum>5</manvolnum> + </citerefentry>. + + Secrets may be passed to the service without adding them to the + world-readable Nix store, by specifying placeholder variables as + the option value in Nix and setting these variables accordingly in the + environment file. + + Environment variables from this file will be interpolated into the + config file using envsubst with this syntax: + <literal>$ENVIRONMENT ''${VARIABLE}</literal> + + <programlisting> + # Example scrape config entry handling an OAuth bearer token + { + job_name = "home_assistant"; + metrics_path = "/api/prometheus"; + scheme = "https"; + bearer_token = "\''${HOME_ASSISTANT_BEARER_TOKEN}"; + [...] + } + </programlisting> + + <programlisting> + # Content of the environment file + HOME_ASSISTANT_BEARER_TOKEN=someoauthbearertoken + </programlisting> + + Note that this file needs to be available on the host on which + <literal>Prometheus</literal> is running. + ''; + }; + configText = mkOption { type = types.nullOr types.lines; default = null; @@ -662,12 +701,19 @@ in { systemd.services.prometheus = { wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; + preStart = '' + ${lib.getBin pkgs.envsubst}/bin/envsubst -o "/run/prometheus/prometheus-substituted.yaml" \ + -i "${prometheusYml}" + ''; serviceConfig = { ExecStart = "${cfg.package}/bin/prometheus" + optionalString (length cmdlineArgs != 0) (" \\\n " + concatStringsSep " \\\n " cmdlineArgs); User = "prometheus"; Restart = "always"; + EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ]; + RuntimeDirectory = "prometheus"; + RuntimeDirectoryMode = "0700"; WorkingDirectory = workingDir; StateDirectory = cfg.stateDir; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters.nix index 1233e5cdd1a9..995afca96ff8 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters.nix @@ -43,7 +43,9 @@ let "postgres" "redis" "rspamd" + "rtl_433" "snmp" + "sql" "surfboard" "tor" "unifi" @@ -217,6 +219,14 @@ in Please specify either 'services.prometheus.exporters.mail.configuration' or 'services.prometheus.exporters.mail.configFile'. ''; + } { + assertion = cfg.sql.enable -> ( + (cfg.sql.configFile == null) != (cfg.sql.configuration == null) + ); + message = '' + Please specify either 'services.prometheus.exporters.sql.configuration' or + 'services.prometheus.exporters.sql.configFile' + ''; } ]; }] ++ [(mkIf config.services.minio.enable { services.prometheus.exporters.minio.minioAddress = mkDefault "http://localhost:9000"; @@ -224,6 +234,8 @@ in services.prometheus.exporters.minio.minioAccessSecret = mkDefault config.services.minio.secretKey; })] ++ [(mkIf config.services.rspamd.enable { services.prometheus.exporters.rspamd.url = mkDefault "http://localhost:11334/stat"; + })] ++ [(mkIf config.services.prometheus.exporters.rtl_433.enable { + hardware.rtl-sdr.enable = mkDefault true; })] ++ [(mkIf config.services.nginx.enable { systemd.services.prometheus-nginx-exporter.after = [ "nginx.service" ]; systemd.services.prometheus-nginx-exporter.requires = [ "nginx.service" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/rtl_433.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/rtl_433.nix new file mode 100644 index 000000000000..01e420db3897 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/rtl_433.nix @@ -0,0 +1,78 @@ +{ config, lib, pkgs, options }: + +let + cfg = config.services.prometheus.exporters.rtl_433; +in +{ + port = 9550; + + extraOpts = let + mkMatcherOptionType = field: description: with lib.types; + listOf (submodule { + options = { + name = lib.mkOption { + type = str; + description = "Name to match."; + }; + "${field}" = lib.mkOption { + type = int; + inherit description; + }; + location = lib.mkOption { + type = str; + description = "Location to match."; + }; + }; + }); + in + { + rtl433Flags = lib.mkOption { + type = lib.types.str; + default = "-C si"; + example = "-C si -R 19"; + description = '' + Flags passed verbatim to rtl_433 binary. + Having <literal>-C si</literal> (the default) is recommended since only Celsius temperatures are parsed. + ''; + }; + channels = lib.mkOption { + type = mkMatcherOptionType "channel" "Channel to match."; + default = []; + example = [ + { name = "Acurite"; channel = 6543; location = "Kitchen"; } + ]; + description = '' + List of channel matchers to export. + ''; + }; + ids = lib.mkOption { + type = mkMatcherOptionType "id" "ID to match."; + default = []; + example = [ + { name = "Nexus"; id = 1; location = "Bedroom"; } + ]; + description = '' + List of ID matchers to export. + ''; + }; + }; + + serviceOpts = { + serviceConfig = { + # rtl-sdr udev rules make supported USB devices +rw by plugdev. + SupplementaryGroups = "plugdev"; + ExecStart = let + matchers = (map (m: + "--channel_matcher '${m.name},${toString m.channel},${m.location}'" + ) cfg.channels) ++ (map (m: + "--id_matcher '${m.name},${toString m.id},${m.location}'" + ) cfg.ids); in '' + ${pkgs.prometheus-rtl_433-exporter}/bin/rtl_433_prometheus \ + -listen ${cfg.listenAddress}:${toString cfg.port} \ + -subprocess "${pkgs.rtl_433}/bin/rtl_433 -F json ${cfg.rtl433Flags}" \ + ${lib.concatStringsSep " \\\n " matchers} \ + ${lib.concatStringsSep " \\\n " cfg.extraFlags} + ''; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/sql.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/sql.nix new file mode 100644 index 000000000000..d9be724ebc03 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/prometheus/exporters/sql.nix @@ -0,0 +1,104 @@ +{ config, lib, pkgs, options }: +with lib; +let + cfg = config.services.prometheus.exporters.sql; + cfgOptions = { + options = with types; { + jobs = mkOption { + type = attrsOf (submodule jobOptions); + default = { }; + description = "An attrset of metrics scraping jobs to run."; + }; + }; + }; + jobOptions = { + options = with types; { + interval = mkOption { + type = str; + description = '' + How often to run this job, specified in + <link xlink:href="https://golang.org/pkg/time/#ParseDuration">Go duration</link> format. + ''; + }; + connections = mkOption { + type = listOf str; + description = "A list of connection strings of the SQL servers to scrape metrics from"; + }; + startupSql = mkOption { + type = listOf str; + default = []; + description = "A list of SQL statements to execute once after making a connection."; + }; + queries = mkOption { + type = attrsOf (submodule queryOptions); + description = "SQL queries to run."; + }; + }; + }; + queryOptions = { + options = with types; { + help = mkOption { + type = nullOr str; + default = null; + description = "A human-readable description of this metric."; + }; + labels = mkOption { + type = listOf str; + default = [ ]; + description = "A set of columns that will be used as Prometheus labels."; + }; + query = mkOption { + type = str; + description = "The SQL query to run."; + }; + values = mkOption { + type = listOf str; + description = "A set of columns that will be used as values of this metric."; + }; + }; + }; + + configFile = + if cfg.configFile != null + then cfg.configFile + else + let + nameInline = mapAttrsToList (k: v: v // { name = k; }); + renameStartupSql = j: removeAttrs (j // { startup_sql = j.startupSql; }) [ "startupSql" ]; + configuration = { + jobs = map renameStartupSql + (nameInline (mapAttrs (k: v: (v // { queries = nameInline v.queries; })) cfg.configuration.jobs)); + }; + in + builtins.toFile "config.yaml" (builtins.toJSON configuration); +in +{ + extraOpts = { + configFile = mkOption { + type = with types; nullOr path; + default = null; + description = '' + Path to configuration file. + ''; + }; + configuration = mkOption { + type = with types; nullOr (submodule cfgOptions); + default = null; + description = '' + Exporter configuration as nix attribute set. Mutually exclusive with 'configFile' option. + ''; + }; + }; + + port = 9237; + serviceOpts = { + serviceConfig = { + ExecStart = '' + ${pkgs.prometheus-sql-exporter}/bin/sql_exporter \ + -web.listen-address ${cfg.listenAddress}:${toString cfg.port} \ + -config.file ${configFile} \ + ${concatStringsSep " \\\n " cfg.extraFlags} + ''; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/smartd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/smartd.nix index c72b4abfcdce..3ea254371142 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/smartd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/smartd.nix @@ -36,7 +36,7 @@ let $SMARTD_MESSAGE EOF - } | ${pkgs.utillinux}/bin/wall 2>/dev/null + } | ${pkgs.util-linux}/bin/wall 2>/dev/null ''} ${optionalString nx.enable '' export DISPLAY=${nx.display} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/teamviewer.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/teamviewer.nix index 8d781d82d086..ce9e57a187cd 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/teamviewer.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/teamviewer.nix @@ -31,14 +31,14 @@ in after = [ "NetworkManager-wait-online.service" "network.target" ]; preStart = "mkdir -pv /var/lib/teamviewer /var/log/teamviewer"; + startLimitIntervalSec = 60; + startLimitBurst = 10; serviceConfig = { Type = "forking"; ExecStart = "${pkgs.teamviewer}/bin/teamviewerd -d"; PIDFile = "/run/teamviewerd.pid"; ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; Restart = "on-abort"; - StartLimitInterval = "60"; - StartLimitBurst = "10"; }; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/telegraf.nix b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/telegraf.nix index 5d131557e8be..b341a9005c2a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/telegraf.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/monitoring/telegraf.nix @@ -5,14 +5,8 @@ with lib; let cfg = config.services.telegraf; - configFile = pkgs.runCommand "config.toml" { - buildInputs = [ pkgs.remarshal ]; - preferLocalBuild = true; - } '' - remarshal -if json -of toml \ - < ${pkgs.writeText "config.json" (builtins.toJSON cfg.extraConfig)} \ - > $out - ''; + settingsFormat = pkgs.formats.toml {}; + configFile = settingsFormat.generate "config.toml" cfg.extraConfig; in { ###### interface options = { @@ -26,22 +20,31 @@ in { type = types.package; }; + environmentFiles = mkOption { + type = types.listOf types.path; + default = []; + example = "/run/keys/telegraf.env"; + description = '' + File to load as environment file. Environment variables + from this file will be interpolated into the config file + using envsubst with this syntax: + <literal>$ENVIRONMENT ''${VARIABLE}</literal> + This is useful to avoid putting secrets into the nix store. + ''; + }; + extraConfig = mkOption { default = {}; description = "Extra configuration options for telegraf"; - type = types.attrs; + type = settingsFormat.type; example = { - outputs = { - influxdb = { - urls = ["http://localhost:8086"]; - database = "telegraf"; - }; + outputs.influxdb = { + urls = ["http://localhost:8086"]; + database = "telegraf"; }; - inputs = { - statsd = { - service_address = ":8125"; - delete_timings = true; - }; + inputs.statsd = { + service_address = ":8125"; + delete_timings = true; }; }; }; @@ -51,15 +54,28 @@ in { ###### implementation config = mkIf config.services.telegraf.enable { - systemd.services.telegraf = { + systemd.services.telegraf = let + finalConfigFile = if config.services.telegraf.environmentFiles == [] + then configFile + else "/var/run/telegraf/config.toml"; + in { description = "Telegraf Agent"; wantedBy = [ "multi-user.target" ]; after = [ "network-online.target" ]; serviceConfig = { - ExecStart=''${cfg.package}/bin/telegraf -config "${configFile}"''; + EnvironmentFile = config.services.telegraf.environmentFiles; + ExecStartPre = lib.optional (config.services.telegraf.environmentFiles != []) + (pkgs.writeShellScript "pre-start" '' + umask 077 + ${pkgs.envsubst}/bin/envsubst -i "${configFile}" > /var/run/telegraf/config.toml + ''); + ExecStart=''${cfg.package}/bin/telegraf -config ${finalConfigFile}''; ExecReload="${pkgs.coreutils}/bin/kill -HUP $MAINPID"; + RuntimeDirectory = "telegraf"; User = "telegraf"; Restart = "on-failure"; + # for ping probes + AmbientCapabilities = [ "CAP_NET_RAW" ]; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ceph.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ceph.nix index d17959a6a305..f2dc740fd88e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ceph.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ceph.nix @@ -28,6 +28,9 @@ let # Don't start services that are not yet initialized unitConfig.ConditionPathExists = "/var/lib/${stateDirectory}/keyring"; + startLimitBurst = + if daemonType == "osd" then 30 else if lib.elem daemonType ["mgr" "mds"] then 3 else 5; + startLimitIntervalSec = 60 * 30; # 30 mins serviceConfig = { LimitNOFILE = 1048576; @@ -39,8 +42,6 @@ let ProtectHome = "true"; ProtectSystem = "full"; Restart = "on-failure"; - StartLimitBurst = "5"; - StartLimitInterval = "30min"; StateDirectory = stateDirectory; User = "ceph"; Group = if daemonType == "osd" then "disk" else "ceph"; @@ -48,13 +49,10 @@ let -f --cluster ${clusterName} --id ${daemonId}''; } // optionalAttrs (daemonType == "osd") { ExecStartPre = ''${ceph.lib}/libexec/ceph/ceph-osd-prestart.sh --id ${daemonId} --cluster ${clusterName}''; - StartLimitBurst = "30"; RestartSec = "20s"; PrivateDevices = "no"; # osd needs disk access } // optionalAttrs ( daemonType == "mon") { RestartSec = "10"; - } // optionalAttrs (lib.elem daemonType ["mgr" "mds"]) { - StartLimitBurst = "3"; }; }); diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ipfs.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ipfs.nix index f298f831fa7b..2082d513161e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ipfs.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/ipfs.nix @@ -44,6 +44,13 @@ in { enable = mkEnableOption "Interplanetary File System (WARNING: may cause severe network degredation)"; + package = mkOption { + type = types.package; + default = pkgs.ipfs; + defaultText = "pkgs.ipfs"; + description = "Which IPFS package to use."; + }; + user = mkOption { type = types.str; default = "ipfs"; @@ -176,7 +183,7 @@ in { ###### implementation config = mkIf cfg.enable { - environment.systemPackages = [ pkgs.ipfs ]; + environment.systemPackages = [ cfg.package ]; environment.variables.IPFS_PATH = cfg.dataDir; programs.fuse = mkIf cfg.autoMount { @@ -207,14 +214,14 @@ in { "d '${cfg.ipnsMountDir}' - ${cfg.user} ${cfg.group} - -" ]; - systemd.packages = [ pkgs.ipfs ]; + systemd.packages = [ cfg.package ]; systemd.services.ipfs-init = { description = "IPFS Initializer"; environment.IPFS_PATH = cfg.dataDir; - path = [ pkgs.ipfs ]; + path = [ cfg.package ]; script = '' if [[ ! -f ${cfg.dataDir}/config ]]; then @@ -239,7 +246,7 @@ in { }; systemd.services.ipfs = { - path = [ "/run/wrappers" pkgs.ipfs ]; + path = [ "/run/wrappers" cfg.package ]; environment.IPFS_PATH = cfg.dataDir; wants = [ "ipfs-init.service" ]; @@ -267,7 +274,7 @@ in { cfg.extraConfig)) ); serviceConfig = { - ExecStart = ["" "${pkgs.ipfs}/bin/ipfs daemon ${ipfsFlags}"]; + ExecStart = ["" "${cfg.package}/bin/ipfs daemon ${ipfsFlags}"]; User = cfg.user; Group = cfg.group; } // optionalAttrs (cfg.serviceFdlimit != null) { LimitNOFILE = cfg.serviceFdlimit; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/netatalk.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/netatalk.nix index 7674c8f7fa8d..ca9d32311f5f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/netatalk.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/netatalk.nix @@ -108,10 +108,10 @@ in extmap = mkOption { type = types.lines; - default = ""; - description = '' - File name extension mappings. - See <literal>man extmap.conf</literal> for more information. + default = ""; + description = '' + File name extension mappings. + See <literal>man extmap.conf</literal> for more information. ''; }; @@ -132,10 +132,10 @@ in Type = "forking"; GuessMainPID = "no"; PIDFile = "/run/lock/netatalk"; - ExecStartPre = "${pkgs.coreutils}/bin/mkdir -m 0755 -p /var/lib/netatalk/CNID"; + ExecStartPre = "${pkgs.coreutils}/bin/mkdir -m 0755 -p /var/lib/netatalk/CNID"; ExecStart = "${pkgs.netatalk}/sbin/netatalk -F ${afpConfFile}"; ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; - ExecStop = "${pkgs.coreutils}/bin/kill -TERM $MAINPID"; + ExecStop = "${pkgs.coreutils}/bin/kill -TERM $MAINPID"; Restart = "always"; RestartSec = 1; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/client.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/client.nix index 677111814a01..03884cb72976 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/client.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/client.nix @@ -244,7 +244,7 @@ in # postStop, then we get a hang + kernel oops, because AFS can't be # stopped simply by sending signals to processes. preStop = '' - ${pkgs.utillinux}/bin/umount ${cfg.mountPoint} + ${pkgs.util-linux}/bin/umount ${cfg.mountPoint} ${openafsBin}/sbin/afsd -shutdown ${pkgs.kmod}/sbin/rmmod libafs ''; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/server.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/server.nix index 095024d2c8af..d782f7821656 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/server.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/openafs/server.nix @@ -251,7 +251,6 @@ in { wantedBy = [ "multi-user.target" ]; restartIfChanged = false; unitConfig.ConditionPathExists = [ - "|/etc/openafs/server/rxkad.keytab" "|/etc/openafs/server/KeyFileExt" ]; preStart = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/samba-wsdd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/samba-wsdd.nix new file mode 100644 index 000000000000..004d07064afd --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/samba-wsdd.nix @@ -0,0 +1,124 @@ +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.samba-wsdd; + +in { + options = { + services.samba-wsdd = { + enable = mkEnableOption '' + Enable Web Services Dynamic Discovery host daemon. This enables (Samba) hosts, like your local NAS device, + to be found by Web Service Discovery Clients like Windows. + <note> + <para>If you use the firewall consider adding the following:</para> + <programlisting> + networking.firewall.allowedTCPPorts = [ 5357 ]; + networking.firewall.allowedUDPPorts = [ 3702 ]; + </programlisting> + </note> + ''; + interface = mkOption { + type = types.nullOr types.str; + default = null; + example = "eth0"; + description = "Interface or address to use."; + }; + hoplimit = mkOption { + type = types.nullOr types.int; + default = null; + example = 2; + description = "Hop limit for multicast packets (default = 1)."; + }; + workgroup = mkOption { + type = types.nullOr types.str; + default = null; + example = "HOME"; + description = "Set workgroup name (default WORKGROUP)."; + }; + hostname = mkOption { + type = types.nullOr types.str; + default = null; + example = "FILESERVER"; + description = "Override (NetBIOS) hostname to be used (default hostname)."; + }; + domain = mkOption { + type = types.nullOr types.str; + default = null; + description = "Set domain name (disables workgroup)."; + }; + discovery = mkOption { + type = types.bool; + default = false; + description = "Enable discovery operation mode."; + }; + listen = mkOption { + type = types.str; + default = "/run/wsdd/wsdd.sock"; + description = "Listen on path or localhost port in discovery mode."; + }; + extraOptions = mkOption { + type = types.listOf types.str; + default = [ "--shortlog" ]; + example = [ "--verbose" "--no-http" "--ipv4only" "--no-host" ]; + description = "Additional wsdd options."; + }; + }; + }; + + config = mkIf cfg.enable { + + environment.systemPackages = [ pkgs.wsdd ]; + + systemd.services.samba-wsdd = { + description = "Web Services Dynamic Discovery host daemon"; + after = [ "network.target" ]; + wantedBy = [ "multi-user.target" ]; + serviceConfig = { + DynamicUser = true; + Type = "simple"; + ExecStart = '' + ${pkgs.wsdd}/bin/wsdd ${optionalString (cfg.interface != null) "--interface '${cfg.interface}'"} \ + ${optionalString (cfg.hoplimit != null) "--hoplimit '${toString cfg.hoplimit}'"} \ + ${optionalString (cfg.workgroup != null) "--workgroup '${cfg.workgroup}'"} \ + ${optionalString (cfg.hostname != null) "--hostname '${cfg.hostname}'"} \ + ${optionalString (cfg.domain != null) "--domain '${cfg.domain}'"} \ + ${optionalString cfg.discovery "--discovery --listen '${cfg.listen}'"} \ + ${escapeShellArgs cfg.extraOptions} + ''; + # Runtime directory and mode + RuntimeDirectory = "wsdd"; + RuntimeDirectoryMode = "0750"; + # Access write directories + UMask = "0027"; + # Capabilities + CapabilityBoundingSet = ""; + # Security + NoNewPrivileges = true; + # Sandboxing + ProtectSystem = "strict"; + ProtectHome = true; + PrivateTmp = true; + PrivateDevices = true; + PrivateUsers = false; + ProtectHostname = true; + ProtectClock = true; + ProtectKernelTunables = true; + ProtectKernelModules = true; + ProtectKernelLogs = true; + ProtectControlGroups = true; + RestrictAddressFamilies = [ "AF_UNIX" "AF_INET" "AF_INET6" ]; + RestrictNamespaces = true; + LockPersonality = true; + MemoryDenyWriteExecute = true; + RestrictRealtime = true; + RestrictSUIDSGID = true; + PrivateMounts = true; + # System Call Filtering + SystemCallArchitectures = "native"; + SystemCallFilter = "~@clock @cpu-emulation @debug @module @mount @obsolete @privileged @raw-io @reboot @resources @swap"; + }; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/xtreemfs.nix b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/xtreemfs.nix index b8f8c1d71174..27a9fe847c58 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/xtreemfs.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/network-filesystems/xtreemfs.nix @@ -112,7 +112,7 @@ in description = '' Must be set to a unique identifier, preferably a UUID according to RFC 4122. UUIDs can be generated with `uuidgen` command, found in - the `utillinux` package. + the `util-linux` package. ''; }; port = mkOption { @@ -232,7 +232,7 @@ in description = '' Must be set to a unique identifier, preferably a UUID according to RFC 4122. UUIDs can be generated with `uuidgen` command, found in - the `utillinux` package. + the `util-linux` package. ''; }; port = mkOption { @@ -370,7 +370,7 @@ in description = '' Must be set to a unique identifier, preferably a UUID according to RFC 4122. UUIDs can be generated with `uuidgen` command, found in - the `utillinux` package. + the `util-linux` package. ''; }; port = mkOption { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/avahi-daemon.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/avahi-daemon.nix index 2900c37f990f..0b7d5575c11f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/avahi-daemon.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/avahi-daemon.nix @@ -86,7 +86,8 @@ in ipv6 = mkOption { type = types.bool; - default = false; + default = config.networking.enableIPv6; + defaultText = "config.networking.enableIPv6"; description = "Whether to use IPv6."; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/babeld.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/babeld.nix index e62c74d0069d..90395dbd3c54 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/babeld.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/babeld.nix @@ -87,9 +87,37 @@ in description = "Babel routing daemon"; after = [ "network.target" ]; wantedBy = [ "multi-user.target" ]; - serviceConfig.ExecStart = "${pkgs.babeld}/bin/babeld -c ${configFile}"; + serviceConfig = { + ExecStart = "${pkgs.babeld}/bin/babeld -c ${configFile} -I /run/babeld/babeld.pid -S /var/lib/babeld/state"; + CapabilityBoundingSet = [ "CAP_NET_ADMIN" ]; + IPAddressAllow = [ "fe80::/64" "ff00::/8" "::1/128" "127.0.0.0/8" ]; + IPAddressDeny = "any"; + LockPersonality = true; + NoNewPrivileges = true; + MemoryDenyWriteExecute = true; + ProtectSystem = "strict"; + ProtectClock = true; + ProtectKernelTunables = false; # Couldn't write sysctl: Read-only file system + ProtectKernelModules = true; + ProtectKernelLogs = true; + ProtectControlGroups = true; + RestrictAddressFamilies = [ "AF_NETLINK" "AF_INET6" ]; + RestrictNamespaces = true; + RestrictRealtime = true; + RestrictSUIDSGID = true; + RemoveIPC = true; + ProtectHome = true; + ProtectHostname = true; + PrivateMounts = true; + PrivateTmp = true; + PrivateDevices = true; + PrivateUsers = false; # kernel_route(ADD): Operation not permitted + SystemCallArchitectures = "native"; + SystemCallFilter = [ "@system-service" ]; + UMask = "0177"; + RuntimeDirectory = "babeld"; + StateDirectory = "babeld"; + }; }; - }; - } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/blockbook-frontend.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/blockbook-frontend.nix index dde24522756a..ca323e495ec1 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/blockbook-frontend.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/blockbook-frontend.nix @@ -158,15 +158,21 @@ let type = types.attrs; default = {}; example = literalExample '' { - alternative_estimate_fee = "whatthefee-disabled"; - alternative_estimate_fee_params = "{\"url\": \"https://whatthefee.io/data.json\", \"periodSeconds\": 60}"; - fiat_rates = "coingecko"; - fiat_rates_params = "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"bitcoin\", \"periodSeconds\": 60}"; - coin_shortcut = "BTC"; - coin_label = "Bitcoin"; - xpub_magic = 76067358; - xpub_magic_segwit_p2sh = 77429938; - xpub_magic_segwit_native = 78792518; + "alternative_estimate_fee" = "whatthefee-disabled"; + "alternative_estimate_fee_params" = "{\"url\": \"https://whatthefee.io/data.json\", \"periodSeconds\": 60}"; + "fiat_rates" = "coingecko"; + "fiat_rates_params" = "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"bitcoin\", \"periodSeconds\": 60}"; + "coin_shortcut" = "BTC"; + "coin_label" = "Bitcoin"; + "parse" = true; + "subversion" = ""; + "address_format" = ""; + "xpub_magic" = 76067358; + "xpub_magic_segwit_p2sh" = 77429938; + "xpub_magic_segwit_native" = 78792518; + "mempool_workers" = 8; + "mempool_sub_workers" = 2; + "block_addresses_to_keep" = 300; }''; description = '' Additional configurations to be appended to <filename>coin.conf</filename>. diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/cjdns.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/cjdns.nix index 5f8ac96b2292..f116d6392ea7 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/cjdns.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/cjdns.nix @@ -264,10 +264,10 @@ in '' ); + startLimitIntervalSec = 0; serviceConfig = { Type = "forking"; Restart = "always"; - StartLimitInterval = 0; RestartSec = 1; CapabilityBoundingSet = "CAP_NET_ADMIN CAP_NET_RAW CAP_SETUID"; ProtectSystem = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/dhcpcd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/dhcpcd.nix index 0507b739d499..d10bffd91474 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/dhcpcd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/dhcpcd.nix @@ -69,6 +69,11 @@ let if-carrier-up = ""; }.${cfg.wait}} + ${optionalString (config.networking.enableIPv6 == false) '' + # Don't solicit or accept IPv6 Router Advertisements and DHCPv6 if disabled IPv6 + noipv6 + ''} + ${cfg.extraConfig} ''; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnscrypt-wrapper.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnscrypt-wrapper.nix index b9333cd19a2a..ee7e9b0454de 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnscrypt-wrapper.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnscrypt-wrapper.nix @@ -55,7 +55,10 @@ let rotateKeys = '' # check if keys are not expired keyValid() { - fingerprint=$(dnscrypt-wrapper --show-provider-publickey | awk '{print $(NF)}') + fingerprint=$(dnscrypt-wrapper \ + --show-provider-publickey \ + --provider-publickey-file=${publicKey} \ + | awk '{print $(NF)}') dnscrypt-proxy --test=${toString (cfg.keys.checkInterval + 1)} \ --resolver-address=127.0.0.1:${toString cfg.port} \ --provider-name=${cfg.providerName} \ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnsdist.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnsdist.nix index c48835e73612..05c2bdef83e7 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnsdist.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/dnsdist.nix @@ -41,6 +41,7 @@ in { systemd.services.dnsdist = { wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 0; serviceConfig = { DynamicUser = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/kresd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/kresd.nix index ccb34163d5f3..6f1c4c48b430 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/kresd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/kresd.nix @@ -23,18 +23,14 @@ let ''; configFile = pkgs.writeText "kresd.conf" ( - optionalString (cfg.listenDoH != []) '' - modules.load('http') - '' + "" + concatMapStrings (mkListen "dns") cfg.listenPlain + concatMapStrings (mkListen "tls") cfg.listenTLS - + concatMapStrings (mkListen "doh") cfg.listenDoH + + concatMapStrings (mkListen "doh2") cfg.listenDoH + cfg.extraConfig ); - package = if cfg.listenDoH == [] - then pkgs.knot-resolver # never force `extraFeatures = false` - else pkgs.knot-resolver.override { extraFeatures = true; }; + package = pkgs.knot-resolver; in { meta.maintainers = [ maintainers.vcunat /* upstream developer */ ]; @@ -92,7 +88,7 @@ in { default = []; example = [ "198.51.100.1:443" "[2001:db8::1]:443" "443" ]; description = '' - Addresses and ports on which kresd should provide DNS over HTTPS (see RFC 8484). + Addresses and ports on which kresd should provide DNS over HTTPS/2 (see RFC 8484). For detailed syntax see ListenStream in man systemd.socket. ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/morty.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/morty.nix index e3a6444c1163..e110a5c86101 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/morty.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/morty.nix @@ -29,9 +29,11 @@ in key = mkOption { type = types.str; default = ""; - description = "HMAC url validation key (hexadecimal encoded). - Leave blank to disable. Without validation key, anyone can - submit proxy requests. Leave blank to disable."; + description = '' + HMAC url validation key (hexadecimal encoded). + Leave blank to disable. Without validation key, anyone can + submit proxy requests. Leave blank to disable. + ''; defaultText = "No HMAC url validation. Generate with echo -n somevalue | openssl dgst -sha1 -hmac somekey"; }; @@ -85,10 +87,10 @@ in serviceConfig = { User = "morty"; ExecStart = ''${cfg.package}/bin/morty \ - -listen ${cfg.listenAddress}:${toString cfg.port} \ - ${optionalString cfg.ipv6 "-ipv6"} \ - ${optionalString (cfg.key != "") "-key " + cfg.key} \ - ''; + -listen ${cfg.listenAddress}:${toString cfg.port} \ + ${optionalString cfg.ipv6 "-ipv6"} \ + ${optionalString (cfg.key != "") "-key " + cfg.key} \ + ''; }; }; environment.systemPackages = [ cfg.package ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/mosquitto.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/mosquitto.nix index d2feb93e2b72..10b49d9b2206 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/mosquitto.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/mosquitto.nix @@ -123,12 +123,33 @@ in ''; }; + passwordFile = mkOption { + type = with types; uniq (nullOr str); + example = "/path/to/file"; + default = null; + description = '' + Specifies the path to a file containing the + clear text password for the MQTT user. + ''; + }; + hashedPassword = mkOption { type = with types; uniq (nullOr str); default = null; description = '' Specifies the hashed password for the MQTT User. - <option>hashedPassword</option> overrides <option>password</option>. + To generate hashed password install <literal>mosquitto</literal> + package and use <literal>mosquitto_passwd</literal>. + ''; + }; + + hashedPasswordFile = mkOption { + type = with types; uniq (nullOr str); + example = "/path/to/file"; + default = null; + description = '' + Specifies the path to a file containing the + hashed password for the MQTT user. To generate hashed password install <literal>mosquitto</literal> package and use <literal>mosquitto_passwd</literal>. ''; @@ -190,6 +211,13 @@ in config = mkIf cfg.enable { + assertions = mapAttrsToList (name: cfg: { + assertion = length (filter (s: s != null) (with cfg; [ + password passwordFile hashedPassword hashedPasswordFile + ])) <= 1; + message = "Cannot set more than one password option"; + }) cfg.users; + systemd.services.mosquitto = { description = "Mosquitto MQTT Broker Daemon"; wantedBy = [ "multi-user.target" ]; @@ -204,13 +232,27 @@ in Restart = "on-failure"; ExecStart = "${pkgs.mosquitto}/bin/mosquitto -c ${mosquittoConf}"; ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; + + ProtectSystem = "strict"; + ProtectHome = true; + PrivateDevices = true; + PrivateTmp = true; + ReadWritePaths = "${cfg.dataDir}"; + ProtectControlGroups = true; + ProtectKernelModules = true; + ProtectKernelTunables = true; + NoNewPrivileges = true; }; preStart = '' rm -f ${cfg.dataDir}/passwd touch ${cfg.dataDir}/passwd '' + concatStringsSep "\n" ( mapAttrsToList (n: c: - if c.hashedPassword != null then + if c.hashedPasswordFile != null then + "echo '${n}:'$(cat '${c.hashedPasswordFile}') >> ${cfg.dataDir}/passwd" + else if c.passwordFile != null then + "${pkgs.mosquitto}/bin/mosquitto_passwd -b ${cfg.dataDir}/passwd ${n} $(cat '${c.passwordFile}')" + else if c.hashedPassword != null then "echo '${n}:${c.hashedPassword}' >> ${cfg.dataDir}/passwd" else optionalString (c.password != null) "${pkgs.mosquitto}/bin/mosquitto_passwd -b ${cfg.dataDir}/passwd ${n} '${c.password}'" diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/mullvad-vpn.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/mullvad-vpn.nix index cc98414257ca..6f595ca4be2b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/mullvad-vpn.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/mullvad-vpn.nix @@ -15,6 +15,9 @@ with lib; config = mkIf cfg.enable { boot.kernelModules = [ "tun" ]; + # mullvad-daemon writes to /etc/iproute2/rt_tables + networking.iproute2.enable = true; + systemd.services.mullvad-daemon = { description = "Mullvad VPN daemon"; wantedBy = [ "multi-user.target" ]; @@ -29,9 +32,9 @@ with lib; # Needed for ping "/run/wrappers" ]; + startLimitBurst = 5; + startLimitIntervalSec = 20; serviceConfig = { - StartLimitBurst = 5; - StartLimitIntervalSec = 20; ExecStart = "${pkgs.mullvad-vpn}/bin/mullvad-daemon -v --disable-stdout-timestamps"; Restart = "always"; RestartSec = 1; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/murmur.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/murmur.nix index b384f436861d..c6e5649ec479 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/murmur.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/murmur.nix @@ -278,6 +278,10 @@ in home = "/var/lib/murmur"; createHome = true; uid = config.ids.uids.murmur; + group = "murmur"; + }; + users.groups.murmur = { + gid = config.ids.gids.murmur; }; systemd.services.murmur = { @@ -300,6 +304,7 @@ in RuntimeDirectory = "murmur"; RuntimeDirectoryMode = "0700"; User = "murmur"; + Group = "murmur"; }; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/namecoind.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/namecoind.nix index 16f85df2e77c..4966ed2cac8d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/namecoind.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/namecoind.nix @@ -165,6 +165,8 @@ in after = [ "network.target" ]; wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 120; + startLimitBurst = 5; serviceConfig = { User = "namecoin"; Group = "namecoin"; @@ -176,8 +178,6 @@ in TimeoutStopSec = "60s"; TimeoutStartSec = "2s"; Restart = "always"; - StartLimitInterval = "120s"; - StartLimitBurst = "5"; }; preStart = optionalString (cfg.wallet != "${dataDir}/wallet.dat") '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nar-serve.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nar-serve.nix new file mode 100644 index 000000000000..ddd42fa01073 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nar-serve.nix @@ -0,0 +1,55 @@ +{ config, pkgs, lib, ... }: + +with lib; +let + cfg = config.services.nar-serve; +in +{ + meta = { + maintainers = [ maintainers.rizary ]; + }; + options = { + services.nar-serve = { + enable = mkEnableOption "Serve NAR file contents via HTTP"; + + port = mkOption { + type = types.int; + default = 8383; + description = '' + Port number where nar-serve will listen on. + ''; + }; + + cacheURL = mkOption { + type = types.str; + default = "https://cache.nixos.org/"; + description = '' + Binary cache URL to connect to. + + The URL format is compatible with the nix remote url style, such as: + - http://, https:// for binary caches via HTTP or HTTPS + - s3:// for binary caches stored in Amazon S3 + - gs:// for binary caches stored in Google Cloud Storage + ''; + }; + }; + }; + + config = mkIf cfg.enable { + systemd.services.nar-serve = { + description = "NAR server"; + after = [ "network.target" ]; + wantedBy = [ "multi-user.target" ]; + + environment.PORT = toString cfg.port; + environment.NAR_CACHE_URL = cfg.cacheURL; + + serviceConfig = { + Restart = "always"; + RestartSec = "5s"; + ExecStart = "${pkgs.nar-serve}/bin/nar-serve"; + DynamicUser = true; + }; + }; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nat.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nat.nix index 21ae9eb8b6d4..45eb500fe8ce 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nat.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nat.nix @@ -9,7 +9,14 @@ with lib; let cfg = config.networking.nat; - dest = if cfg.externalIP == null then "-j MASQUERADE" else "-j SNAT --to-source ${cfg.externalIP}"; + mkDest = externalIP: if externalIP == null + then "-j MASQUERADE" + else "-j SNAT --to-source ${externalIP}"; + dest = mkDest cfg.externalIP; + destIPv6 = mkDest cfg.externalIPv6; + + # Whether given IP (plus optional port) is an IPv6. + isIPv6 = ip: builtins.length (lib.splitString ":" ip) > 2; helpers = import ./helpers.nix { inherit config lib; }; @@ -28,63 +35,80 @@ let ${cfg.extraStopCommands} ''; - setupNat = '' - ${helpers} - # Create subchain where we store rules - ip46tables -w -t nat -N nixos-nat-pre - ip46tables -w -t nat -N nixos-nat-post - ip46tables -w -t nat -N nixos-nat-out - + mkSetupNat = { iptables, dest, internalIPs, forwardPorts }: '' # We can't match on incoming interface in POSTROUTING, so # mark packets coming from the internal interfaces. ${concatMapStrings (iface: '' - iptables -w -t nat -A nixos-nat-pre \ + ${iptables} -w -t nat -A nixos-nat-pre \ -i '${iface}' -j MARK --set-mark 1 '') cfg.internalInterfaces} # NAT the marked packets. ${optionalString (cfg.internalInterfaces != []) '' - iptables -w -t nat -A nixos-nat-post -m mark --mark 1 \ + ${iptables} -w -t nat -A nixos-nat-post -m mark --mark 1 \ ${optionalString (cfg.externalInterface != null) "-o ${cfg.externalInterface}"} ${dest} ''} # NAT packets coming from the internal IPs. ${concatMapStrings (range: '' - iptables -w -t nat -A nixos-nat-post \ + ${iptables} -w -t nat -A nixos-nat-post \ -s '${range}' ${optionalString (cfg.externalInterface != null) "-o ${cfg.externalInterface}"} ${dest} - '') cfg.internalIPs} + '') internalIPs} # NAT from external ports to internal ports. ${concatMapStrings (fwd: '' - iptables -w -t nat -A nixos-nat-pre \ + ${iptables} -w -t nat -A nixos-nat-pre \ -i ${toString cfg.externalInterface} -p ${fwd.proto} \ --dport ${builtins.toString fwd.sourcePort} \ -j DNAT --to-destination ${fwd.destination} ${concatMapStrings (loopbackip: let - m = builtins.match "([0-9.]+):([0-9-]+)" fwd.destination; - destinationIP = if (m == null) then throw "bad ip:ports `${fwd.destination}'" else elemAt m 0; - destinationPorts = if (m == null) then throw "bad ip:ports `${fwd.destination}'" else builtins.replaceStrings ["-"] [":"] (elemAt m 1); + matchIP = if isIPv6 fwd.destination then "[[]([0-9a-fA-F:]+)[]]" else "([0-9.]+)"; + m = builtins.match "${matchIP}:([0-9-]+)" fwd.destination; + destinationIP = if m == null then throw "bad ip:ports `${fwd.destination}'" else elemAt m 0; + destinationPorts = if m == null then throw "bad ip:ports `${fwd.destination}'" else builtins.replaceStrings ["-"] [":"] (elemAt m 1); in '' # Allow connections to ${loopbackip}:${toString fwd.sourcePort} from the host itself - iptables -w -t nat -A nixos-nat-out \ + ${iptables} -w -t nat -A nixos-nat-out \ -d ${loopbackip} -p ${fwd.proto} \ --dport ${builtins.toString fwd.sourcePort} \ -j DNAT --to-destination ${fwd.destination} # Allow connections to ${loopbackip}:${toString fwd.sourcePort} from other hosts behind NAT - iptables -w -t nat -A nixos-nat-pre \ + ${iptables} -w -t nat -A nixos-nat-pre \ -d ${loopbackip} -p ${fwd.proto} \ --dport ${builtins.toString fwd.sourcePort} \ -j DNAT --to-destination ${fwd.destination} - iptables -w -t nat -A nixos-nat-post \ + ${iptables} -w -t nat -A nixos-nat-post \ -d ${destinationIP} -p ${fwd.proto} \ --dport ${destinationPorts} \ -j SNAT --to-source ${loopbackip} '') fwd.loopbackIPs} - '') cfg.forwardPorts} + '') forwardPorts} + ''; + + setupNat = '' + ${helpers} + # Create subchains where we store rules + ip46tables -w -t nat -N nixos-nat-pre + ip46tables -w -t nat -N nixos-nat-post + ip46tables -w -t nat -N nixos-nat-out + + ${mkSetupNat { + iptables = "iptables"; + inherit dest; + inherit (cfg) internalIPs; + forwardPorts = filter (x: !(isIPv6 x.destination)) cfg.forwardPorts; + }} + + ${optionalString cfg.enableIPv6 (mkSetupNat { + iptables = "ip6tables"; + dest = destIPv6; + internalIPs = cfg.internalIPv6s; + forwardPorts = filter (x: isIPv6 x.destination) cfg.forwardPorts; + })} ${optionalString (cfg.dmzHost != null) '' iptables -w -t nat -A nixos-nat-pre \ @@ -117,6 +141,15 @@ in ''; }; + networking.nat.enableIPv6 = mkOption { + type = types.bool; + default = false; + description = + '' + Whether to enable IPv6 NAT. + ''; + }; + networking.nat.internalInterfaces = mkOption { type = types.listOf types.str; default = []; @@ -141,6 +174,18 @@ in ''; }; + networking.nat.internalIPv6s = mkOption { + type = types.listOf types.str; + default = []; + example = [ "fc00::/64" ]; + description = + '' + The IPv6 address ranges for which to perform NAT. Packets + coming from these addresses (on any interface) and destined + for the external interface will be rewritten. + ''; + }; + networking.nat.externalInterface = mkOption { type = types.nullOr types.str; default = null; @@ -164,6 +209,19 @@ in ''; }; + networking.nat.externalIPv6 = mkOption { + type = types.nullOr types.str; + default = null; + example = "2001:dc0:2001:11::175"; + description = + '' + The public IPv6 address to which packets from the local + network are to be rewritten. If this is left empty, the + IP address associated with the external interface will be + used. + ''; + }; + networking.nat.forwardPorts = mkOption { type = with types; listOf (submodule { options = { @@ -176,7 +234,7 @@ in destination = mkOption { type = types.str; example = "10.0.0.1:80"; - description = "Forward connection to destination ip:port; to specify a port range, use ip:start-end"; + description = "Forward connection to destination ip:port (or [ipv6]:port); to specify a port range, use ip:start-end"; }; proto = mkOption { @@ -195,11 +253,15 @@ in }; }); default = []; - example = [ { sourcePort = 8080; destination = "10.0.0.1:80"; proto = "tcp"; } ]; + example = [ + { sourcePort = 8080; destination = "10.0.0.1:80"; proto = "tcp"; } + { sourcePort = 8080; destination = "[fc00::2]:80"; proto = "tcp"; } + ]; description = '' List of forwarded ports from the external interface to - internal destinations by using DNAT. + internal destinations by using DNAT. Destination can be + IPv6 if IPv6 NAT is enabled. ''; }; @@ -246,6 +308,9 @@ in (mkIf config.networking.nat.enable { assertions = [ + { assertion = cfg.enableIPv6 -> config.networking.enableIPv6; + message = "networking.nat.enableIPv6 requires networking.enableIPv6"; + } { assertion = (cfg.dmzHost != null) -> (cfg.externalInterface != null); message = "networking.nat.dmzHost requires networking.nat.externalInterface"; } @@ -261,6 +326,15 @@ in kernel.sysctl = { "net.ipv4.conf.all.forwarding" = mkOverride 99 true; "net.ipv4.conf.default.forwarding" = mkOverride 99 true; + } // optionalAttrs cfg.enableIPv6 { + # Do not prevent IPv6 autoconfiguration. + # See <http://strugglers.net/~andy/blog/2011/09/04/linux-ipv6-router-advertisements-and-forwarding/>. + "net.ipv6.conf.all.accept_ra" = mkOverride 99 2; + "net.ipv6.conf.default.accept_ra" = mkOverride 99 2; + + # Forward IPv6 packets. + "net.ipv6.conf.all.forwarding" = mkOverride 99 true; + "net.ipv6.conf.default.forwarding" = mkOverride 99 true; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/networkmanager.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/networkmanager.nix index 17c549d42c32..2e680544ec24 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/networkmanager.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/networkmanager.nix @@ -15,6 +15,7 @@ let networkmanager-openconnect networkmanager-openvpn networkmanager-vpnc + networkmanager-sstp ] ++ optional (!delegateWireless && !enableIwd) wpa_supplicant; delegateWireless = config.networking.wireless.enable == true && cfg.unmanaged != []; @@ -386,6 +387,9 @@ in { "NetworkManager/VPN/nm-iodine-service.name".source = "${networkmanager-iodine}/lib/NetworkManager/VPN/nm-iodine-service.name"; + + "NetworkManager/VPN/nm-sstp-service.name".source = + "${networkmanager-sstp}/lib/NetworkManager/VPN/nm-sstp-service.name"; } // optionalAttrs (cfg.appendNameservers != [] || cfg.insertNameservers != []) { @@ -461,7 +465,7 @@ in { restartTriggers = [ configFile overrideNameserversScript ]; # useful binaries for user-specified hooks - path = [ pkgs.iproute pkgs.utillinux pkgs.coreutils ]; + path = [ pkgs.iproute pkgs.util-linux pkgs.coreutils ]; aliases = [ "dbus-org.freedesktop.nm-dispatcher.service" ]; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nextdns.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nextdns.nix index a633bff62ec7..b070eeec894f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nextdns.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nextdns.nix @@ -28,9 +28,9 @@ in { environment = { SERVICE_RUN_MODE = "1"; }; + startLimitIntervalSec = 5; + startLimitBurst = 10; serviceConfig = { - StartLimitInterval = 5; - StartLimitBurst = 10; ExecStart = "${pkgs.nextdns}/bin/nextdns run ${escapeShellArgs config.services.nextdns.arguments}"; RestartSec = 120; LimitMEMLOCK = "infinity"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nix-store-gcs-proxy.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nix-store-gcs-proxy.nix index 3f2ce5bca4da..0012302db2e3 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nix-store-gcs-proxy.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nix-store-gcs-proxy.nix @@ -42,9 +42,9 @@ in description = "A HTTP nix store that proxies requests to Google Storage"; wantedBy = ["multi-user.target"]; + startLimitIntervalSec = 10; serviceConfig = { RestartSec = 5; - StartLimitInterval = 10; ExecStart = '' ${pkgs.nix-store-gcs-proxy}/bin/nix-store-gcs-proxy \ --bucket-name ${cfg.bucketName} \ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nsd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nsd.nix index 3ecbd06ee416..f33c350a257a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/nsd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/nsd.nix @@ -916,14 +916,14 @@ in after = [ "network.target" ]; wantedBy = [ "multi-user.target" ]; + startLimitBurst = 4; + startLimitIntervalSec = 5 * 60; # 5 mins serviceConfig = { ExecStart = "${nsdPkg}/sbin/nsd -d -c ${nsdEnv}/nsd.conf"; StandardError = "null"; PIDFile = pidFile; Restart = "always"; RestartSec = "4s"; - StartLimitBurst = 4; - StartLimitInterval = "5min"; }; preStart = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/ntp/chrony.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/ntp/chrony.nix index 78de50583f34..e6fa48daf46c 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/ntp/chrony.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/ntp/chrony.nix @@ -6,6 +6,7 @@ let cfg = config.services.chrony; stateDir = "/var/lib/chrony"; + driftFile = "${stateDir}/chrony.drift"; keyFile = "${stateDir}/chrony.keys"; configFile = pkgs.writeText "chrony.conf" '' @@ -16,7 +17,7 @@ let "initstepslew ${toString cfg.initstepslew.threshold} ${concatStringsSep " " cfg.servers}" } - driftfile ${stateDir}/chrony.drift + driftfile ${driftFile} keyfile ${keyFile} ${optionalString (!config.time.hardwareClockInLocalTime) "rtconutc"} @@ -95,6 +96,7 @@ in systemd.tmpfiles.rules = [ "d ${stateDir} 0755 chrony chrony - -" + "f ${driftFile} 0640 chrony chrony -" "f ${keyFile} 0640 chrony chrony -" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/pdns-recursor.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/pdns-recursor.nix index 6ff181377fcc..a326eccfd65d 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/pdns-recursor.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/pdns-recursor.nix @@ -3,9 +3,6 @@ with lib; let - dataDir = "/var/lib/pdns-recursor"; - username = "pdns-recursor"; - cfg = config.services.pdns-recursor; oneOrMore = type: with types; either type (listOf type); @@ -21,7 +18,7 @@ let else if builtins.isList val then (concatMapStringsSep "," serialize val) else ""; - configFile = pkgs.writeText "recursor.conf" + configDir = pkgs.writeTextDir "recursor.conf" (concatStringsSep "\n" (flip mapAttrsToList cfg.settings (name: val: "${name}=${serialize val}"))); @@ -173,45 +170,30 @@ in { serve-rfc1918 = cfg.serveRFC1918; lua-config-file = pkgs.writeText "recursor.lua" cfg.luaConfig; + daemon = false; + write-pid = false; log-timestamp = false; disable-syslog = true; }; - users.users.${username} = { - home = dataDir; - createHome = true; - uid = config.ids.uids.pdns-recursor; - description = "PowerDNS Recursor daemon user"; - }; + systemd.packages = [ pkgs.pdns-recursor ]; systemd.services.pdns-recursor = { - unitConfig.Documentation = "man:pdns_recursor(1) man:rec_control(1)"; - description = "PowerDNS recursive server"; wantedBy = [ "multi-user.target" ]; - after = [ "network.target" ]; serviceConfig = { - User = username; - Restart ="on-failure"; - RestartSec = "5"; - PrivateTmp = true; - PrivateDevices = true; - AmbientCapabilities = "cap_net_bind_service"; - ExecStart = ''${pkgs.pdns-recursor}/bin/pdns_recursor \ - --config-dir=${dataDir} \ - --socket-dir=${dataDir} - ''; + ExecStart = [ "" "${pkgs.pdns-recursor}/bin/pdns_recursor --config-dir=${configDir}" ]; }; + }; - preStart = '' - # Link configuration file into recursor home directory - configPath=${dataDir}/recursor.conf - if [ "$(realpath $configPath)" != "${configFile}" ]; then - rm -f $configPath - ln -s ${configFile} $configPath - fi - ''; + users.users.pdns-recursor = { + isSystemUser = true; + group = "pdns-recursor"; + description = "PowerDNS Recursor daemon user"; }; + + users.groups.pdns-recursor = {}; + }; imports = [ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/powerdns.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/powerdns.nix index ba05e15389f6..8cae61b83543 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/powerdns.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/powerdns.nix @@ -8,42 +8,40 @@ let in { options = { services.powerdns = { - enable = mkEnableOption "Powerdns domain name server"; + enable = mkEnableOption "PowerDNS domain name server"; extraConfig = mkOption { type = types.lines; default = "launch=bind"; description = '' - Extra lines to be added verbatim to pdns.conf. - Powerdns will chroot to /var/lib/powerdns. - So any file, powerdns is supposed to be read, - should be in /var/lib/powerdns and needs to specified - relative to the chroot. + PowerDNS configuration. Refer to + <link xlink:href="https://doc.powerdns.com/authoritative/settings.html"/> + for details on supported values. ''; }; }; }; - config = mkIf config.services.powerdns.enable { + config = mkIf cfg.enable { + + systemd.packages = [ pkgs.powerdns ]; + systemd.services.pdns = { - unitConfig.Documentation = "man:pdns_server(1) man:pdns_control(1)"; - description = "Powerdns name server"; wantedBy = [ "multi-user.target" ]; - after = ["network.target" "mysql.service" "postgresql.service" "openldap.service"]; + after = [ "network.target" "mysql.service" "postgresql.service" "openldap.service" ]; serviceConfig = { - Restart="on-failure"; - RestartSec="1"; - StartLimitInterval="0"; - PrivateDevices=true; - CapabilityBoundingSet="CAP_CHOWN CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID CAP_SYS_CHROOT"; - NoNewPrivileges=true; - ExecStartPre = "${pkgs.coreutils}/bin/mkdir -p /var/lib/powerdns"; - ExecStart = "${pkgs.powerdns}/bin/pdns_server --setuid=nobody --setgid=nogroup --chroot=/var/lib/powerdns --socket-dir=/ --daemon=no --guardian=no --disable-syslog --write-pid=no --config-dir=${configDir}"; - ProtectSystem="full"; - ProtectHome=true; - RestrictAddressFamilies="AF_UNIX AF_INET AF_INET6"; + ExecStart = [ "" "${pkgs.powerdns}/bin/pdns_server --config-dir=${configDir} --guardian=no --daemon=no --disable-syslog --log-timestamp=no --write-pid=no" ]; }; }; + + users.users.pdns = { + isSystemUser = true; + group = "pdns"; + description = "PowerDNS"; + }; + + users.groups.pdns = {}; + }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/prosody.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/prosody.nix index a6c1cb0f4797..e7a7aa700be6 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/prosody.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/prosody.nix @@ -261,7 +261,7 @@ let toLua = x: if builtins.isString x then ''"${x}"'' - else if builtins.isBool x then (if x == true then "true" else "false") + else if builtins.isBool x then boolToString x else if builtins.isInt x then toString x else if builtins.isList x then ''{ ${lib.concatStringsSep ", " (map (n: toLua n) x) } }'' else throw "Invalid Lua value"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/ssh/sshd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/ssh/sshd.nix index 5365b8b9b107..3cc77e4cb938 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/ssh/sshd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/ssh/sshd.nix @@ -269,6 +269,7 @@ in kexAlgorithms = mkOption { type = types.listOf types.str; default = [ + "curve25519-sha256" "curve25519-sha256@libssh.org" "diffie-hellman-group-exchange-sha256" ]; @@ -279,7 +280,7 @@ in Defaults to recommended settings from both <link xlink:href="https://stribika.github.io/2015/01/04/secure-secure-shell.html" /> and - <link xlink:href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Modern_.28OpenSSH_6.7.2B.29" /> + <link xlink:href="https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67" /> ''; }; @@ -300,7 +301,7 @@ in Defaults to recommended settings from both <link xlink:href="https://stribika.github.io/2015/01/04/secure-secure-shell.html" /> and - <link xlink:href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Modern_.28OpenSSH_6.7.2B.29" /> + <link xlink:href="https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67" /> ''; }; @@ -321,7 +322,7 @@ in Defaults to recommended settings from both <link xlink:href="https://stribika.github.io/2015/01/04/secure-secure-shell.html" /> and - <link xlink:href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Modern_.28OpenSSH_6.7.2B.29" /> + <link xlink:href="https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67" /> ''; }; @@ -476,7 +477,7 @@ in # https://github.com/NixOS/nixpkgs/pull/10155 # https://github.com/NixOS/nixpkgs/pull/41745 services.openssh.authorizedKeysFiles = - [ ".ssh/authorized_keys" ".ssh/authorized_keys2" "/etc/ssh/authorized_keys.d/%u" ]; + [ "%h/.ssh/authorized_keys" "%h/.ssh/authorized_keys2" "/etc/ssh/authorized_keys.d/%u" ]; services.openssh.extraConfig = mkOrder 0 '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/sslh.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/sslh.nix index 0921febba668..4c2740d20192 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/sslh.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/sslh.nix @@ -31,7 +31,7 @@ let { name: "openvpn"; host: "localhost"; port: "1194"; probe: "builtin"; }, { name: "xmpp"; host: "localhost"; port: "5222"; probe: "builtin"; }, { name: "http"; host: "localhost"; port: "80"; probe: "builtin"; }, - { name: "ssl"; host: "localhost"; port: "443"; probe: "builtin"; }, + { name: "tls"; host: "localhost"; port: "443"; probe: "builtin"; }, { name: "anyprot"; host: "localhost"; port: "443"; probe: "builtin"; } ); ''; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/module.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/module.nix index 0fec3ef00ad9..f67eedac2961 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/module.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/module.nix @@ -63,7 +63,7 @@ in { description = "strongSwan IPsec IKEv1/IKEv2 daemon using swanctl"; wantedBy = [ "multi-user.target" ]; after = [ "network-online.target" ]; - path = with pkgs; [ kmod iproute iptables utillinux ]; + path = with pkgs; [ kmod iproute iptables util-linux ]; environment = { STRONGSWAN_CONF = pkgs.writeTextFile { name = "strongswan.conf"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/swanctl-params.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/swanctl-params.nix index 808cb863a9cf..1d1e0bd1ca19 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/swanctl-params.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan-swanctl/swanctl-params.nix @@ -1173,20 +1173,20 @@ in { ppk = mkPrefixedAttrsOfParams { secret = mkOptionalStrParam '' - Value of the PPK. It may either be an ASCII string, a hex encoded string - if it has a <literal>0x</literal> prefix or a Base64 encoded string if - it has a <literal>0s</literal> prefix in its value. Should have at least - 256 bits of entropy for 128-bit security. + Value of the PPK. It may either be an ASCII string, a hex encoded string + if it has a <literal>0x</literal> prefix or a Base64 encoded string if + it has a <literal>0s</literal> prefix in its value. Should have at least + 256 bits of entropy for 128-bit security. ''; id = mkPrefixedAttrsOfParam (mkOptionalStrParam "") '' - PPK identity the PPK belongs to. Multiple unique identities may be - specified, each having an <literal>id</literal> prefix, if a secret is - shared between multiple peers. + PPK identity the PPK belongs to. Multiple unique identities may be + specified, each having an <literal>id</literal> prefix, if a secret is + shared between multiple peers. ''; } '' - Postquantum Preshared Key (PPK) section for a specific secret. Each PPK is - defined in a unique section having the <literal>ppk</literal> prefix. + Postquantum Preshared Key (PPK) section for a specific secret. Each PPK is + defined in a unique section having the <literal>ppk</literal> prefix. ''; private = mkPrefixedAttrsOfParams { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan.nix index 13a1a897c5ed..f6170b813654 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/strongswan.nix @@ -152,7 +152,7 @@ in systemd.services.strongswan = { description = "strongSwan IPSec Service"; wantedBy = [ "multi-user.target" ]; - path = with pkgs; [ kmod iproute iptables utillinux ]; # XXX Linux + path = with pkgs; [ kmod iproute iptables util-linux ]; # XXX Linux after = [ "network-online.target" ]; environment = { STRONGSWAN_CONF = strongswanConf { inherit setup connections ca secretsFile managePlugins enabledPlugins; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/stunnel.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/stunnel.nix index ab51bba2f6ac..fe1616f411f0 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/stunnel.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/stunnel.nix @@ -16,8 +16,12 @@ let serverConfig = { options = { accept = mkOption { - type = types.int; - description = "On which port stunnel should listen for incoming TLS connections."; + type = types.either types.str types.int; + description = '' + On which [host:]port stunnel should listen for incoming TLS connections. + Note that unlike other softwares stunnel ipv6 address need no brackets, + so to listen on all IPv6 addresses on port 1234 one would use ':::1234'. + ''; }; connect = mkOption { @@ -129,7 +133,6 @@ in type = with types; attrsOf (submodule serverConfig); example = { fancyWebserver = { - enable = true; accept = 443; connect = 8080; cert = "/path/to/pem/file"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/supybot.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/supybot.nix index dc9fb31ffd0b..7a62e04ec7c4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/supybot.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/supybot.nix @@ -103,6 +103,8 @@ in rm -f '${cfg.stateDir}/supybot.cfg.bak' ''; + startLimitIntervalSec = 5 * 60; # 5 min + startLimitBurst = 1; serviceConfig = { ExecStart = "${pyEnv}/bin/supybot ${cfg.stateDir}/supybot.cfg"; PIDFile = "/run/supybot.pid"; @@ -110,8 +112,6 @@ in Group = "supybot"; UMask = "0007"; Restart = "on-abort"; - StartLimitInterval = "5m"; - StartLimitBurst = "1"; NoNewPrivileges = true; PrivateDevices = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/tailscale.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/tailscale.nix index 4d6aeb75ebd1..1a1474595beb 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/tailscale.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/tailscale.nix @@ -14,36 +14,21 @@ in { default = 41641; description = "The port to listen on for tunnel traffic (0=autoselect)."; }; + + package = mkOption { + type = types.package; + default = pkgs.tailscale; + defaultText = "pkgs.tailscale"; + description = "The package to use for tailscale"; + }; }; config = mkIf cfg.enable { - systemd.services.tailscale = { - description = "Tailscale client daemon"; - - after = [ "network-pre.target" ]; - wants = [ "network-pre.target" ]; + environment.systemPackages = [ cfg.package ]; # for the CLI + systemd.packages = [ cfg.package ]; + systemd.services.tailscaled = { wantedBy = [ "multi-user.target" ]; - - unitConfig = { - StartLimitIntervalSec = 0; - StartLimitBurst = 0; - }; - - serviceConfig = { - ExecStart = - "${pkgs.tailscale}/bin/tailscaled --port ${toString cfg.port}"; - - RuntimeDirectory = "tailscale"; - RuntimeDirectoryMode = 755; - - StateDirectory = "tailscale"; - StateDirectoryMode = 750; - - CacheDirectory = "tailscale"; - CacheDirectoryMode = 750; - - Restart = "on-failure"; - }; + serviceConfig.Environment = "PORT=${toString cfg.port}"; }; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/unbound.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/unbound.nix index baed83591e1e..9a46fa3075fa 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/unbound.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/unbound.nix @@ -1,9 +1,7 @@ { config, lib, pkgs, ... }: with lib; - let - cfg = config.services.unbound; stateDir = "/var/lib/unbound"; @@ -17,12 +15,12 @@ let forward = optionalString (any isLocalAddress cfg.forwardAddresses) '' do-not-query-localhost: no - '' + - optionalString (cfg.forwardAddresses != []) '' + '' + + optionalString (cfg.forwardAddresses != []) '' forward-zone: name: . - '' + - concatMapStringsSep "\n" (x: " forward-addr: ${x}") cfg.forwardAddresses; + '' + + concatMapStringsSep "\n" (x: " forward-addr: ${x}") cfg.forwardAddresses; rootTrustAnchorFile = "${stateDir}/root.key"; @@ -31,19 +29,25 @@ let confFile = pkgs.writeText "unbound.conf" '' server: + ip-freebind: yes directory: "${stateDir}" username: unbound - chroot: "${stateDir}" + chroot: "" pidfile: "" + # when running under systemd there is no need to daemonize + do-daemonize: no ${interfaces} ${access} ${trustAnchor} + ${lib.optionalString (cfg.localControlSocketPath != null) '' + remote-control: + control-enable: yes + control-interface: ${cfg.localControlSocketPath} + ''} ${cfg.extraConfig} ${forward} ''; - in - { ###### interface @@ -55,8 +59,8 @@ in package = mkOption { type = types.package; - default = pkgs.unbound; - defaultText = "pkgs.unbound"; + default = pkgs.unbound-with-systemd; + defaultText = "pkgs.unbound-with-systemd"; description = "The unbound package to use"; }; @@ -69,11 +73,14 @@ in interfaces = mkOption { default = [ "127.0.0.1" ] ++ optional config.networking.enableIPv6 "::1"; type = types.listOf types.str; - description = "What addresses the server should listen on."; + description = '' + What addresses the server should listen on. This supports the interface syntax documented in + <citerefentry><refentrytitle>unbound.conf</refentrytitle><manvolnum>8</manvolnum></citerefentry>. + ''; }; forwardAddresses = mkOption { - default = [ ]; + default = []; type = types.listOf types.str; description = "What servers to forward queries to."; }; @@ -84,6 +91,28 @@ in description = "Use and update root trust anchor for DNSSEC validation."; }; + localControlSocketPath = mkOption { + default = null; + # FIXME: What is the proper type here so users can specify strings, + # paths and null? + # My guess would be `types.nullOr (types.either types.str types.path)` + # but I haven't verified yet. + type = types.nullOr types.str; + example = "/run/unbound/unbound.ctl"; + description = '' + When not set to <literal>null</literal> this option defines the path + at which the unbound remote control socket should be created at. The + socket will be owned by the unbound user (<literal>unbound</literal>) + and group will be <literal>nogroup</literal>. + + Users that should be permitted to access the socket must be in the + <literal>unbound</literal> group. + + If this option is <literal>null</literal> remote control will not be + configured at all. Unbounds default values apply. + ''; + }; + extraConfig = mkOption { default = ""; type = types.lines; @@ -106,43 +135,85 @@ in users.users.unbound = { description = "unbound daemon user"; isSystemUser = true; + group = lib.mkIf (cfg.localControlSocketPath != null) (lib.mkDefault "unbound"); + }; + + # We need a group so that we can give users access to the configured + # control socket. Unbound allows access to the socket only to the unbound + # user and the primary group. + users.groups = lib.mkIf (cfg.localControlSocketPath != null) { + unbound = {}; }; networking.resolvconf.useLocalResolver = mkDefault true; + + environment.etc."unbound/unbound.conf".source = confFile; + systemd.services.unbound = { description = "Unbound recursive Domain Name Server"; after = [ "network.target" ]; before = [ "nss-lookup.target" ]; - wants = [ "nss-lookup.target" ]; - wantedBy = [ "multi-user.target" ]; - - preStart = '' - mkdir -m 0755 -p ${stateDir}/dev/ - cp ${confFile} ${stateDir}/unbound.conf - ${optionalString cfg.enableRootTrustAnchor '' - ${cfg.package}/bin/unbound-anchor -a ${rootTrustAnchorFile} || echo "Root anchor updated!" - chown unbound ${stateDir} ${rootTrustAnchorFile} - ''} - touch ${stateDir}/dev/random - ${pkgs.utillinux}/bin/mount --bind -n /dev/urandom ${stateDir}/dev/random + wantedBy = [ "multi-user.target" "nss-lookup.target" ]; + + preStart = lib.mkIf cfg.enableRootTrustAnchor '' + ${cfg.package}/bin/unbound-anchor -a ${rootTrustAnchorFile} || echo "Root anchor updated!" ''; - serviceConfig = { - ExecStart = "${cfg.package}/bin/unbound -d -c ${stateDir}/unbound.conf"; - ExecStopPost="${pkgs.utillinux}/bin/umount ${stateDir}/dev/random"; + restartTriggers = [ + confFile + ]; - ProtectSystem = true; - ProtectHome = true; + serviceConfig = { + ExecStart = "${cfg.package}/bin/unbound -p -d -c /etc/unbound/unbound.conf"; + ExecReload = "+/run/current-system/sw/bin/kill -HUP $MAINPID"; + + NotifyAccess = "main"; + Type = "notify"; + + # FIXME: Which of these do we actualy need, can we drop the chroot flag? + AmbientCapabilities = [ + "CAP_NET_BIND_SERVICE" + "CAP_NET_RAW" + "CAP_SETGID" + "CAP_SETUID" + "CAP_SYS_CHROOT" + "CAP_SYS_RESOURCE" + ]; + + User = "unbound"; + Group = lib.mkIf (cfg.localControlSocketPath != null) (lib.mkDefault "unbound"); + + MemoryDenyWriteExecute = true; + NoNewPrivileges = true; PrivateDevices = true; - Restart = "always"; - RestartSec = "5s"; + PrivateTmp = true; + ProtectHome = true; + ProtectControlGroups = true; + ProtectKernelModules = true; + ProtectSystem = "strict"; + RuntimeDirectory = "unbound"; + ConfigurationDirectory = "unbound"; + StateDirectory = "unbound"; + RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ]; + RestrictRealtime = true; + SystemCallArchitectures = "native"; + SystemCallFilter = [ + "~@clock" + "@cpu-emulation" + "@debug" + "@keyring" + "@module" + "mount" + "@obsolete" + "@resources" + ]; + RestrictNamespaces = true; + LockPersonality = true; + RestrictSUIDSGID = true; }; }; - # If networkmanager is enabled, ask it to interface with unbound. networking.networkmanager.dns = "unbound"; - }; - } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/networking/wasabibackend.nix b/infra/libkookie/nixpkgs/nixos/modules/services/networking/wasabibackend.nix index 6eacffe709b0..8482823e197f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/networking/wasabibackend.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/networking/wasabibackend.nix @@ -21,7 +21,7 @@ let RegTestBitcoinCoreRpcEndPoint = "${cfg.rpc.ip}:${toString cfg.rpc.port}"; }; - configFile = pkgs.writeText "wasabibackend.conf" (builtins.toJSON confOptions); + configFile = pkgs.writeText "wasabibackend.conf" (builtins.toJSON confOptions); in { diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/security/fail2ban.nix b/infra/libkookie/nixpkgs/nixos/modules/services/security/fail2ban.nix index 3f84f9c2560c..cf0d72d5c531 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/security/fail2ban.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/security/fail2ban.nix @@ -282,12 +282,12 @@ in services.fail2ban.jails.DEFAULT = '' ${optionalString cfg.bantime-increment.enable '' # Bantime incremental - bantime.increment = ${if cfg.bantime-increment.enable then "true" else "false"} + bantime.increment = ${boolToString cfg.bantime-increment.enable} bantime.maxtime = ${cfg.bantime-increment.maxtime} bantime.factor = ${cfg.bantime-increment.factor} bantime.formula = ${cfg.bantime-increment.formula} bantime.multipliers = ${cfg.bantime-increment.multipliers} - bantime.overalljails = ${if cfg.bantime-increment.overalljails then "true" else "false"} + bantime.overalljails = ${boolToString cfg.bantime-increment.overalljails} ''} # Miscellaneous options ignoreip = 127.0.0.1/8 ${optionalString config.networking.enableIPv6 "::1"} ${concatStringsSep " " cfg.ignoreIP} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/security/oauth2_proxy.nix b/infra/libkookie/nixpkgs/nixos/modules/services/security/oauth2_proxy.nix index 2f9e94bd77ba..486f3ab05386 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/security/oauth2_proxy.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/security/oauth2_proxy.nix @@ -448,7 +448,7 @@ in default = false; description = '' In case when running behind a reverse proxy, controls whether headers - like <literal>X-Real-Ip</literal> are accepted. Usage behind a reverse + like <literal>X-Real-Ip</literal> are accepted. Usage behind a reverse proxy will require this flag to be set to avoid logging the reverse proxy IP address. ''; @@ -524,7 +524,7 @@ in type = types.nullOr types.str; default = null; description = '' - Profile access endpoint. + Profile access endpoint. ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/security/usbguard.nix b/infra/libkookie/nixpkgs/nixos/modules/services/security/usbguard.nix index 16a90da52314..71fd71a2cab2 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/security/usbguard.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/security/usbguard.nix @@ -19,13 +19,13 @@ let PresentDevicePolicy=${cfg.presentDevicePolicy} PresentControllerPolicy=${cfg.presentControllerPolicy} InsertedDevicePolicy=${cfg.insertedDevicePolicy} - RestoreControllerDeviceState=${if cfg.restoreControllerDeviceState then "true" else "false"} + RestoreControllerDeviceState=${boolToString cfg.restoreControllerDeviceState} # this does not seem useful for endusers to change DeviceManagerBackend=uevent IPCAllowedUsers=${concatStringsSep " " cfg.IPCAllowedUsers} IPCAllowedGroups=${concatStringsSep " " cfg.IPCAllowedGroups} IPCAccessControlFiles=/var/lib/usbguard/IPCAccessControl.d/ - DeviceRulesWithPort=${if cfg.deviceRulesWithPort then "true" else "false"} + DeviceRulesWithPort=${boolToString cfg.deviceRulesWithPort} # HACK: that way audit logs still land in the journal AuditFilePath=/dev/null ''; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/security/vault.nix b/infra/libkookie/nixpkgs/nixos/modules/services/security/vault.nix index 6a8a3a93327e..64622454b9de 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/security/vault.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/security/vault.nix @@ -131,6 +131,8 @@ in restartIfChanged = false; # do not restart on "nixos-rebuild switch". It would seal the storage and disrupt the clients. + startLimitIntervalSec = 60; + startLimitBurst = 3; serviceConfig = { User = "vault"; Group = "vault"; @@ -145,8 +147,6 @@ in KillSignal = "SIGINT"; TimeoutStopSec = "30s"; Restart = "on-failure"; - StartLimitInterval = "60s"; - StartLimitBurst = 3; }; unitConfig.RequiresMountsFor = optional (cfg.storagePath != null) cfg.storagePath; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/system/cloud-init.nix b/infra/libkookie/nixpkgs/nixos/modules/services/system/cloud-init.nix index 15fe822aec67..3518e0ee9dca 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/system/cloud-init.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/system/cloud-init.nix @@ -9,7 +9,7 @@ let cfg = config.services.cloud-init; nettools openssh shadow - utillinux + util-linux ] ++ optional cfg.btrfs.enable btrfs-progs ++ optional cfg.ext4.enable e2fsprogs ; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/system/dbus.nix b/infra/libkookie/nixpkgs/nixos/modules/services/system/dbus.nix index d9dd26f0f189..d4cacb85694b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/system/dbus.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/system/dbus.nix @@ -1,6 +1,6 @@ # D-Bus configuration and system bus daemon. -{ config, lib, pkgs, ... }: +{ config, lib, options, pkgs, ... }: with lib; @@ -11,6 +11,7 @@ let homeDir = "/run/dbus"; configDir = pkgs.makeDBusConf { + inherit (cfg) apparmor; suidHelper = "${config.security.wrapperDir}/dbus-daemon-launch-helper"; serviceDirectories = cfg.packages; }; @@ -18,13 +19,6 @@ let in { - - imports = [ - (mkRemovedOptionModule - [ "services" "dbus" "socketActivated" ] - "The user D-Bus session is now always socket activated and this option can safely be removed.") - ]; - ###### interface options = { @@ -57,12 +51,43 @@ in <filename><replaceable>pkg</replaceable>/share/dbus-1/services</filename> ''; }; + + apparmor = mkOption { + type = types.enum [ "enabled" "disabled" "required" ]; + description = '' + AppArmor mode for dbus. + + <literal>enabled</literal> enables mediation when it's + supported in the kernel, <literal>disabled</literal> + always disables AppArmor even with kernel support, and + <literal>required</literal> fails when AppArmor was not found + in the kernel. + ''; + default = "disabled"; + }; + + socketActivated = mkOption { + type = types.nullOr types.bool; + default = null; + visible = false; + description = '' + Removed option, do not use. + ''; + }; }; }; ###### implementation config = mkIf cfg.enable { + warnings = optional (cfg.socketActivated != null) ( + let + files = showFiles options.services.dbus.socketActivated.files; + in + "The option 'services.dbus.socketActivated' in ${files} no longer has" + + " any effect and can be safely removed: the user D-Bus session is" + + " now always socket activated." + ); environment.systemPackages = [ pkgs.dbus.daemon pkgs.dbus ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/torrent/transmission.nix b/infra/libkookie/nixpkgs/nixos/modules/services/torrent/transmission.nix index 014a22bb5a8d..7bec073e26f7 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/torrent/transmission.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/torrent/transmission.nix @@ -197,7 +197,7 @@ in install -D -m 600 -o '${cfg.user}' -g '${cfg.group}' /dev/stdin \ '${cfg.home}/${settingsDir}/settings.json' '')]; - ExecStart="${pkgs.transmission}/bin/transmission-daemon -f"; + ExecStart="${pkgs.transmission}/bin/transmission-daemon -f -g ${cfg.home}/${settingsDir}"; ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; User = cfg.user; Group = cfg.group; @@ -236,6 +236,7 @@ in # an AppArmor profile is provided to get a confinement based upon paths and rights. builtins.storeDir "/etc" + "/run" ] ++ optional (cfg.settings.script-torrent-done-enabled && cfg.settings.script-torrent-done-filename != "") @@ -396,9 +397,9 @@ in mr ${getLib pkgs.openssl}/lib/libcrypto*.so*, mr ${getLib pkgs.openssl}/lib/libssl*.so*, mr ${getLib pkgs.systemd}/lib/libsystemd*.so*, - mr ${getLib pkgs.utillinuxMinimal.out}/lib/libblkid.so*, - mr ${getLib pkgs.utillinuxMinimal.out}/lib/libmount.so*, - mr ${getLib pkgs.utillinuxMinimal.out}/lib/libuuid.so*, + mr ${getLib pkgs.util-linuxMinimal.out}/lib/libblkid.so*, + mr ${getLib pkgs.util-linuxMinimal.out}/lib/libmount.so*, + mr ${getLib pkgs.util-linuxMinimal.out}/lib/libuuid.so*, mr ${getLib pkgs.xz}/lib/liblzma*.so*, mr ${getLib pkgs.zlib}/lib/libz*.so*, @@ -408,6 +409,7 @@ in #r @{PROC}/@{pid}/environ, r @{PROC}/@{pid}/mounts, rwk /tmp/tr_session_id_*, + r /run/systemd/resolve/stub-resolv.conf, r ${pkgs.openssl.out}/etc/**, r ${config.systemd.services.transmission.environment.CURL_CA_BUNDLE}, diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/ttys/agetty.nix b/infra/libkookie/nixpkgs/nixos/modules/services/ttys/agetty.nix index f3a629f7af70..d07746be2377 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/ttys/agetty.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/ttys/agetty.nix @@ -5,7 +5,7 @@ with lib; let autologinArg = optionalString (config.services.mingetty.autologinUser != null) "--autologin ${config.services.mingetty.autologinUser}"; - gettyCmd = extraArgs: "@${pkgs.utillinux}/sbin/agetty agetty --login-program ${pkgs.shadow}/bin/login ${autologinArg} ${extraArgs}"; + gettyCmd = extraArgs: "@${pkgs.util-linux}/sbin/agetty agetty --login-program ${pkgs.shadow}/bin/login ${autologinArg} ${extraArgs}"; in diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/generate b/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/generate deleted file mode 100755 index 2940768b6d2c..000000000000 --- a/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/generate +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env -S nix-build --no-out-link - -# Script to generate default streaming configurations for EPGStation. There's -# no need to run this script directly since generate.sh in the EPGStation -# package directory would run this script for you. -# -# Usage: ./generate | xargs cat > streaming.json - -{ pkgs ? (import ../../../../.. {}) }: - -let - sampleConfigPath = "${pkgs.epgstation.src}/config/config.sample.json"; - sampleConfig = builtins.fromJSON (builtins.readFile sampleConfigPath); - streamingConfig = { - inherit (sampleConfig) - mpegTsStreaming - mpegTsViewer - liveHLS - liveMP4 - liveWebM - recordedDownloader - recordedStreaming - recordedViewer - recordedHLS; - }; -in -pkgs.runCommand "streaming.json" { nativeBuildInputs = [ pkgs.jq ]; } '' - jq . <<<'${builtins.toJSON streamingConfig}' > $out -'' - -# vim:set ft=nix: diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/streaming.json b/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/streaming.json index 37957f6cb6a2..8eb99cf85584 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/streaming.json +++ b/infra/libkookie/nixpkgs/nixos/modules/services/video/epgstation/streaming.json @@ -1,119 +1,119 @@ { "liveHLS": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 48k -ac 2 -c:v libx264 -vf yadif,scale=-2:180 -b:v 100k -preset veryfast -maxrate 110k -bufsize 1000k -flags +loop-global_header %OUTPUT%", - "name": "180p" + "name": "180p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 48k -ac 2 -c:v libx264 -vf yadif,scale=-2:180 -b:v 100k -preset veryfast -maxrate 110k -bufsize 1000k -flags +loop-global_header %OUTPUT%" } ], "liveMP4": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1" } ], "liveWebM": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -b:a 192k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 -b:v 3000k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -b:a 192k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 -b:v 3000k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -b:a 128k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:480 -b:v 1500k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -b:a 128k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:480 -b:v 1500k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1" } ], "mpegTsStreaming": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -y -f mpegts pipe:1", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -y -f mpegts pipe:1" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -y -f mpegts pipe:1", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -y -f mpegts pipe:1" }, { "name": "Original" } ], "mpegTsViewer": { - "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end", - "ios": "vlc-x-callback://x-callback-url/stream?url=http://ADDRESS" + "ios": "vlc-x-callback://x-callback-url/stream?url=http://ADDRESS", + "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end" }, "recordedDownloader": { - "android": "intent://ADDRESS#Intent;package=com.dv.adm;type=video;scheme=http;end", - "ios": "vlc-x-callback://x-callback-url/download?url=http://ADDRESS&filename=FILENAME" + "ios": "vlc-x-callback://x-callback-url/download?url=http://ADDRESS&filename=FILENAME", + "android": "intent://ADDRESS#Intent;package=com.dv.adm;type=video;scheme=http;end" }, - "recordedHLS": [ - { - "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "720p" - }, - { - "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "480p" - }, - { - "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_type fmp4 -hls_fmp4_init_filename stream%streamNum%-init.mp4 -hls_segment_filename stream%streamNum%-%09d.m4s -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx265 -vf yadif,scale=-2:480 -b:v 350k -preset veryfast -tag:v hvc1 %OUTPUT%", - "name": "480p(h265)" - } - ], "recordedStreaming": { - "mp4": [ + "webm": [ { - "ab": "192k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", "name": "720p", - "vb": "3000k" + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", + "vb": "3000k", + "ab": "192k" }, { - "ab": "128k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", "name": "360p", - "vb": "1500k" + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", + "vb": "1500k", + "ab": "128k" } ], - "mpegTs": [ + "mp4": [ { - "ab": "192k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", - "name": "720p (H.264)", - "vb": "3000k" + "name": "720p", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", + "vb": "3000k", + "ab": "192k" }, { - "ab": "128k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", - "name": "360p (H.264)", - "vb": "1500k" + "name": "360p", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", + "vb": "1500k", + "ab": "128k" } ], - "webm": [ + "mpegTs": [ { - "ab": "192k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "720p", - "vb": "3000k" + "name": "720p (H.264)", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", + "vb": "3000k", + "ab": "192k" }, { - "ab": "128k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "360p", - "vb": "1500k" + "name": "360p (H.264)", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", + "vb": "1500k", + "ab": "128k" } ] }, + "recordedHLS": [ + { + "name": "720p", + "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%" + }, + { + "name": "480p", + "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%" + }, + { + "name": "480p(h265)", + "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_type fmp4 -hls_fmp4_init_filename stream%streamNum%-init.mp4 -hls_segment_filename stream%streamNum%-%09d.m4s -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx265 -vf yadif,scale=-2:480 -b:v 350k -preset veryfast -tag:v hvc1 %OUTPUT%" + } + ], "recordedViewer": { - "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end", - "ios": "infuse://x-callback-url/play?url=http://ADDRESS" + "ios": "infuse://x-callback-url/play?url=http://ADDRESS", + "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end" } } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/codimd.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/codimd.nix index c787c36b877c..0fbc9ee820e6 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/codimd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/codimd.nix @@ -6,8 +6,10 @@ let cfg = config.services.codimd; prettyJSON = conf: - pkgs.runCommand "codimd-config.json" { preferLocalBuild = true; } '' - echo '${builtins.toJSON conf}' | ${pkgs.jq}/bin/jq \ + pkgs.runCommandLocal "codimd-config.json" { + nativeBuildInputs = [ pkgs.jq ]; + } '' + echo '${builtins.toJSON conf}' | jq \ '{production:del(.[]|nulls)|del(.[][]?|nulls)}' > $out ''; in @@ -878,7 +880,6 @@ in }; }; - environmentFile = mkOption { type = with types; nullOr path; default = null; @@ -908,6 +909,14 @@ in <literal>CodiMD</literal> is running. ''; }; + + package = mkOption { + type = types.package; + default = pkgs.codimd; + description = '' + Package that provides CodiMD. + ''; + }; }; config = mkIf cfg.enable { @@ -938,7 +947,7 @@ in ''; serviceConfig = { WorkingDirectory = cfg.workDir; - ExecStart = "${pkgs.codimd}/bin/codimd"; + ExecStart = "${cfg.package}/bin/codimd"; EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ]; Environment = [ "CMD_CONFIG_FILE=${cfg.workDir}/config.json" diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/engelsystem.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/engelsystem.nix index 899582a20304..2e755ae9d523 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/engelsystem.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/engelsystem.nix @@ -10,7 +10,7 @@ in { default = false; example = true; description = '' - Whether to enable engelsystem, an online tool for coordinating helpers + Whether to enable engelsystem, an online tool for coordinating volunteers and shifts on large events. ''; type = lib.types.bool; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/gerrit.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/gerrit.nix index 657b1a4fc5ba..864587aea565 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/gerrit.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/gerrit.nix @@ -143,7 +143,7 @@ in Set a UUID that uniquely identifies the server. This can be generated with - <literal>nix-shell -p utillinux --run uuidgen</literal>. + <literal>nix-shell -p util-linux --run uuidgen</literal>. ''; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/keycloak.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/keycloak.nix new file mode 100644 index 000000000000..bbb0c8d04831 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/keycloak.nix @@ -0,0 +1,692 @@ +{ config, pkgs, lib, ... }: + +let + cfg = config.services.keycloak; +in +{ + options.services.keycloak = { + + enable = lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = '' + Whether to enable the Keycloak identity and access management + server. + ''; + }; + + bindAddress = lib.mkOption { + type = lib.types.str; + default = "\${jboss.bind.address:0.0.0.0}"; + example = "127.0.0.1"; + description = '' + On which address Keycloak should accept new connections. + + A special syntax can be used to allow command line Java system + properties to override the value: ''${property.name:value} + ''; + }; + + httpPort = lib.mkOption { + type = lib.types.str; + default = "\${jboss.http.port:80}"; + example = "8080"; + description = '' + On which port Keycloak should listen for new HTTP connections. + + A special syntax can be used to allow command line Java system + properties to override the value: ''${property.name:value} + ''; + }; + + httpsPort = lib.mkOption { + type = lib.types.str; + default = "\${jboss.https.port:443}"; + example = "8443"; + description = '' + On which port Keycloak should listen for new HTTPS connections. + + A special syntax can be used to allow command line Java system + properties to override the value: ''${property.name:value} + ''; + }; + + frontendUrl = lib.mkOption { + type = lib.types.str; + example = "keycloak.example.com/auth"; + description = '' + The public URL used as base for all frontend requests. Should + normally include a trailing <literal>/auth</literal>. + + See <link xlink:href="https://www.keycloak.org/docs/latest/server_installation/#_hostname">the + Hostname section of the Keycloak server installation + manual</link> for more information. + ''; + }; + + forceBackendUrlToFrontendUrl = lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = '' + Whether Keycloak should force all requests to go through the + frontend URL configured in <xref + linkend="opt-services.keycloak.frontendUrl" />. By default, + Keycloak allows backend requests to instead use its local + hostname or IP address and may also advertise it to clients + through its OpenID Connect Discovery endpoint. + + See <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/#_hostname">the + Hostname section of the Keycloak server installation + manual</link> for more information. + ''; + }; + + certificatePrivateKeyBundle = lib.mkOption { + type = lib.types.nullOr lib.types.path; + default = null; + example = "/run/keys/ssl_cert"; + description = '' + The path to a PEM formatted bundle of the private key and + certificate to use for TLS connections. + + This should be a string, not a Nix path, since Nix paths are + copied into the world-readable Nix store. + ''; + }; + + databaseType = lib.mkOption { + type = lib.types.enum [ "mysql" "postgresql" ]; + default = "postgresql"; + example = "mysql"; + description = '' + The type of database Keycloak should connect to. + ''; + }; + + databaseHost = lib.mkOption { + type = lib.types.str; + default = "localhost"; + description = '' + Hostname of the database to connect to. + ''; + }; + + databasePort = + let + dbPorts = { + postgresql = 5432; + mysql = 3306; + }; + in + lib.mkOption { + type = lib.types.port; + default = dbPorts.${cfg.databaseType}; + description = '' + Port of the database to connect to. + ''; + }; + + databaseUseSSL = lib.mkOption { + type = lib.types.bool; + default = cfg.databaseHost != "localhost"; + description = '' + Whether the database connection should be secured by SSL / + TLS. + ''; + }; + + databaseCaCert = lib.mkOption { + type = lib.types.nullOr lib.types.path; + default = null; + description = '' + The SSL / TLS CA certificate that verifies the identity of the + database server. + + Required when PostgreSQL is used and SSL is turned on. + + For MySQL, if left at <literal>null</literal>, the default + Java keystore is used, which should suffice if the server + certificate is issued by an official CA. + ''; + }; + + databaseCreateLocally = lib.mkOption { + type = lib.types.bool; + default = true; + description = '' + Whether a database should be automatically created on the + local host. Set this to false if you plan on provisioning a + local database yourself. This has no effect if + services.keycloak.databaseHost is customized. + ''; + }; + + databaseUsername = lib.mkOption { + type = lib.types.str; + default = "keycloak"; + description = '' + Username to use when connecting to an external or manually + provisioned database; has no effect when a local database is + automatically provisioned. + ''; + }; + + databasePasswordFile = lib.mkOption { + type = lib.types.path; + example = "/run/keys/db_password"; + description = '' + File containing the database password. + + This should be a string, not a Nix path, since Nix paths are + copied into the world-readable Nix store. + ''; + }; + + package = lib.mkOption { + type = lib.types.package; + default = pkgs.keycloak; + description = '' + Keycloak package to use. + ''; + }; + + initialAdminPassword = lib.mkOption { + type = lib.types.str; + default = "changeme"; + description = '' + Initial password set for the <literal>admin</literal> + user. The password is not stored safely and should be changed + immediately in the admin panel. + ''; + }; + + extraConfig = lib.mkOption { + type = lib.types.attrs; + default = { }; + example = lib.literalExample '' + { + "subsystem=keycloak-server" = { + "spi=hostname" = { + "provider=default" = null; + "provider=fixed" = { + enabled = true; + properties.hostname = "keycloak.example.com"; + }; + default-provider = "fixed"; + }; + }; + } + ''; + description = '' + Additional Keycloak configuration options to set in + <literal>standalone.xml</literal>. + + Options are expressed as a Nix attribute set which matches the + structure of the jboss-cli configuration. The configuration is + effectively overlayed on top of the default configuration + shipped with Keycloak. To remove existing nodes and undefine + attributes from the default configuration, set them to + <literal>null</literal>. + + The example configuration does the equivalent of the following + script, which removes the hostname provider + <literal>default</literal>, adds the deprecated hostname + provider <literal>fixed</literal> and defines it the default: + + <programlisting> + /subsystem=keycloak-server/spi=hostname/provider=default:remove() + /subsystem=keycloak-server/spi=hostname/provider=fixed:add(enabled = true, properties = { hostname = "keycloak.example.com" }) + /subsystem=keycloak-server/spi=hostname:write-attribute(name=default-provider, value="fixed") + </programlisting> + + You can discover available options by using the <link + xlink:href="http://docs.wildfly.org/21/Admin_Guide.html#Command_Line_Interface">jboss-cli.sh</link> + program and by referring to the <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/index.html">Keycloak + Server Installation and Configuration Guide</link>. + ''; + }; + + }; + + config = + let + # We only want to create a database if we're actually going to connect to it. + databaseActuallyCreateLocally = cfg.databaseCreateLocally && cfg.databaseHost == "localhost"; + createLocalPostgreSQL = databaseActuallyCreateLocally && cfg.databaseType == "postgresql"; + createLocalMySQL = databaseActuallyCreateLocally && cfg.databaseType == "mysql"; + + mySqlCaKeystore = pkgs.runCommandNoCC "mysql-ca-keystore" {} '' + ${pkgs.jre}/bin/keytool -importcert -trustcacerts -alias MySQLCACert -file ${cfg.databaseCaCert} -keystore $out -storepass notsosecretpassword -noprompt + ''; + + keycloakConfig' = builtins.foldl' lib.recursiveUpdate { + "interface=public".inet-address = cfg.bindAddress; + "socket-binding-group=standard-sockets"."socket-binding=http".port = cfg.httpPort; + "subsystem=keycloak-server"."spi=hostname" = { + "provider=default" = { + enabled = true; + properties = { + inherit (cfg) frontendUrl forceBackendUrlToFrontendUrl; + }; + }; + }; + "subsystem=datasources"."data-source=KeycloakDS" = { + max-pool-size = "20"; + user-name = if databaseActuallyCreateLocally then "keycloak" else cfg.databaseUsername; + password = "@db-password@"; + }; + } [ + (lib.optionalAttrs (cfg.databaseType == "postgresql") { + "subsystem=datasources" = { + "jdbc-driver=postgresql" = { + driver-module-name = "org.postgresql"; + driver-name = "postgresql"; + driver-xa-datasource-class-name = "org.postgresql.xa.PGXADataSource"; + }; + "data-source=KeycloakDS" = { + connection-url = "jdbc:postgresql://${cfg.databaseHost}:${builtins.toString cfg.databasePort}/keycloak"; + driver-name = "postgresql"; + "connection-properties=ssl".value = lib.boolToString cfg.databaseUseSSL; + } // (lib.optionalAttrs (cfg.databaseCaCert != null) { + "connection-properties=sslrootcert".value = cfg.databaseCaCert; + "connection-properties=sslmode".value = "verify-ca"; + }); + }; + }) + (lib.optionalAttrs (cfg.databaseType == "mysql") { + "subsystem=datasources" = { + "jdbc-driver=mysql" = { + driver-module-name = "com.mysql"; + driver-name = "mysql"; + driver-class-name = "com.mysql.jdbc.Driver"; + }; + "data-source=KeycloakDS" = { + connection-url = "jdbc:mysql://${cfg.databaseHost}:${builtins.toString cfg.databasePort}/keycloak"; + driver-name = "mysql"; + "connection-properties=useSSL".value = lib.boolToString cfg.databaseUseSSL; + "connection-properties=requireSSL".value = lib.boolToString cfg.databaseUseSSL; + "connection-properties=verifyServerCertificate".value = lib.boolToString cfg.databaseUseSSL; + "connection-properties=characterEncoding".value = "UTF-8"; + valid-connection-checker-class-name = "org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"; + validate-on-match = true; + exception-sorter-class-name = "org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"; + } // (lib.optionalAttrs (cfg.databaseCaCert != null) { + "connection-properties=trustCertificateKeyStoreUrl".value = "file:${mySqlCaKeystore}"; + "connection-properties=trustCertificateKeyStorePassword".value = "notsosecretpassword"; + }); + }; + }) + (lib.optionalAttrs (cfg.certificatePrivateKeyBundle != null) { + "socket-binding-group=standard-sockets"."socket-binding=https".port = cfg.httpsPort; + "core-service=management"."security-realm=UndertowRealm"."server-identity=ssl" = { + keystore-path = "/run/keycloak/ssl/certificate_private_key_bundle.p12"; + keystore-password = "notsosecretpassword"; + }; + "subsystem=undertow"."server=default-server"."https-listener=https".security-realm = "UndertowRealm"; + }) + cfg.extraConfig + ]; + + + /* Produces a JBoss CLI script that creates paths and sets + attributes matching those described by `attrs`. When the + script is run, the existing settings are effectively overlayed + by those from `attrs`. Existing attributes can be unset by + defining them `null`. + + JBoss paths and attributes / maps are distinguished by their + name, where paths follow a `key=value` scheme. + + Example: + mkJbossScript { + "subsystem=keycloak-server"."spi=hostname" = { + "provider=fixed" = null; + "provider=default" = { + enabled = true; + properties = { + inherit frontendUrl; + forceBackendUrlToFrontendUrl = false; + }; + }; + }; + } + => '' + if (outcome != success) of /:read-resource() + /:add() + end-if + if (outcome != success) of /subsystem=keycloak-server:read-resource() + /subsystem=keycloak-server:add() + end-if + if (outcome != success) of /subsystem=keycloak-server/spi=hostname:read-resource() + /subsystem=keycloak-server/spi=hostname:add() + end-if + if (outcome != success) of /subsystem=keycloak-server/spi=hostname/provider=default:read-resource() + /subsystem=keycloak-server/spi=hostname/provider=default:add(enabled = true, properties = { forceBackendUrlToFrontendUrl = false, frontendUrl = "https://keycloak.example.com/auth" }) + end-if + if (result != true) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="enabled") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=enabled, value=true) + end-if + if (result != false) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.forceBackendUrlToFrontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.forceBackendUrlToFrontendUrl, value=false) + end-if + if (result != "https://keycloak.example.com/auth") of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.frontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.frontendUrl, value="https://keycloak.example.com/auth") + end-if + if (outcome != success) of /subsystem=keycloak-server/spi=hostname/provider=fixed:read-resource() + /subsystem=keycloak-server/spi=hostname/provider=fixed:remove() + end-if + '' + */ + mkJbossScript = attrs: + let + /* From a JBoss path and an attrset, produces a JBoss CLI + snippet that writes the corresponding attributes starting + at `path`. Recurses down into subattrsets as necessary, + producing the variable name from its full path in the + attrset. + + Example: + writeAttributes "/subsystem=keycloak-server/spi=hostname/provider=default" { + enabled = true; + properties = { + forceBackendUrlToFrontendUrl = false; + frontendUrl = "https://keycloak.example.com/auth"; + }; + } + => '' + if (result != true) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="enabled") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=enabled, value=true) + end-if + if (result != false) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.forceBackendUrlToFrontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.forceBackendUrlToFrontendUrl, value=false) + end-if + if (result != "https://keycloak.example.com/auth") of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.frontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.frontendUrl, value="https://keycloak.example.com/auth") + end-if + '' + */ + writeAttributes = path: set: + let + # JBoss expressions like `${var}` need to be prefixed + # with `expression` to evaluate. + prefixExpression = string: + let + match = (builtins.match ''"\$\{.*}"'' string); + in + if match != null then + "expression " + string + else + string; + + writeAttribute = attribute: value: + let + type = builtins.typeOf value; + in + if type == "set" then + let + names = builtins.attrNames value; + in + builtins.foldl' (text: name: text + (writeAttribute "${attribute}.${name}" value.${name})) "" names + else if value == null then '' + if (outcome == success) of ${path}:read-attribute(name="${attribute}") + ${path}:undefine-attribute(name="${attribute}") + end-if + '' + else if builtins.elem type [ "string" "path" "bool" ] then + let + value' = if type == "bool" then lib.boolToString value else ''"${value}"''; + in '' + if (result != ${prefixExpression value'}) of ${path}:read-attribute(name="${attribute}") + ${path}:write-attribute(name=${attribute}, value=${value'}) + end-if + '' + else throw "Unsupported type '${type}' for path '${path}'!"; + in + lib.concatStrings + (lib.mapAttrsToList + (attribute: value: (writeAttribute attribute value)) + set); + + + /* Produces an argument list for the JBoss `add()` function, + which adds a JBoss path and takes as its arguments the + required subpaths and attributes. + + Example: + makeArgList { + enabled = true; + properties = { + forceBackendUrlToFrontendUrl = false; + frontendUrl = "https://keycloak.example.com/auth"; + }; + } + => '' + enabled = true, properties = { forceBackendUrlToFrontendUrl = false, frontendUrl = "https://keycloak.example.com/auth" } + '' + */ + makeArgList = set: + let + makeArg = attribute: value: + let + type = builtins.typeOf value; + in + if type == "set" then + "${attribute} = { " + (makeArgList value) + " }" + else if builtins.elem type [ "string" "path" "bool" ] then + "${attribute} = ${if type == "bool" then lib.boolToString value else ''"${value}"''}" + else if value == null then + "" + else + throw "Unsupported type '${type}' for attribute '${attribute}'!"; + in + lib.concatStringsSep ", " (lib.mapAttrsToList makeArg set); + + + /* Recurses into the `attrs` attrset, beginning at the path + resolved from `state.path ++ node`; if `node` is `null`, + starts from `state.path`. Only subattrsets that are JBoss + paths, i.e. follows the `key=value` format, are recursed + into - the rest are considered JBoss attributes / maps. + */ + recurse = state: node: + let + path = state.path ++ (lib.optional (node != null) node); + isPath = name: + let + value = lib.getAttrFromPath (path ++ [ name ]) attrs; + in + if (builtins.match ".*([=]).*" name) == [ "=" ] then + if builtins.isAttrs value || value == null then + true + else + throw "Parsing path '${lib.concatStringsSep "." (path ++ [ name ])}' failed: JBoss attributes cannot contain '='!" + else + false; + jbossPath = "/" + (lib.concatStringsSep "/" path); + nodeValue = lib.getAttrFromPath path attrs; + children = if !builtins.isAttrs nodeValue then {} else nodeValue; + subPaths = builtins.filter isPath (builtins.attrNames children); + jbossAttrs = lib.filterAttrs (name: _: !(isPath name)) children; + in + state // { + text = state.text + ( + if nodeValue != null then '' + if (outcome != success) of ${jbossPath}:read-resource() + ${jbossPath}:add(${makeArgList jbossAttrs}) + end-if + '' + (writeAttributes jbossPath jbossAttrs) + else '' + if (outcome == success) of ${jbossPath}:read-resource() + ${jbossPath}:remove() + end-if + '') + (builtins.foldl' recurse { text = ""; inherit path; } subPaths).text; + }; + in + (recurse { text = ""; path = []; } null).text; + + + jbossCliScript = pkgs.writeText "jboss-cli-script" (mkJbossScript keycloakConfig'); + + keycloakConfig = pkgs.runCommandNoCC "keycloak-config" {} '' + export JBOSS_BASE_DIR="$(pwd -P)"; + export JBOSS_MODULEPATH="${cfg.package}/modules"; + export JBOSS_LOG_DIR="$JBOSS_BASE_DIR/log"; + + cp -r ${cfg.package}/standalone/configuration . + chmod -R u+rwX ./configuration + + mkdir -p {deployments,ssl} + + "${cfg.package}/bin/standalone.sh"& + + attempt=1 + max_attempts=30 + while ! ${cfg.package}/bin/jboss-cli.sh --connect ':read-attribute(name=server-state)'; do + if [[ "$attempt" == "$max_attempts" ]]; then + echo "ERROR: Could not connect to Keycloak after $attempt attempts! Failing.." >&2 + exit 1 + fi + echo "Keycloak not fully started yet, retrying.. ($attempt/$max_attempts)" + sleep 1 + (( attempt++ )) + done + + ${cfg.package}/bin/jboss-cli.sh --connect --file=${jbossCliScript} --echo-command + + cp configuration/standalone.xml $out + ''; + in + lib.mkIf cfg.enable { + + assertions = [ + { + assertion = (cfg.databaseUseSSL && cfg.databaseType == "postgresql") -> (cfg.databaseCaCert != null); + message = ''A CA certificate must be specified (in 'services.keycloak.databaseCaCert') when PostgreSQL is used with SSL''; + } + ]; + + environment.systemPackages = [ cfg.package ]; + + systemd.services.keycloakPostgreSQLInit = lib.mkIf createLocalPostgreSQL { + after = [ "postgresql.service" ]; + before = [ "keycloak.service" ]; + bindsTo = [ "postgresql.service" ]; + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + User = "postgres"; + Group = "postgres"; + }; + script = '' + set -eu + + PSQL=${config.services.postgresql.package}/bin/psql + + db_password="$(<'${cfg.databasePasswordFile}')" + $PSQL -tAc "SELECT 1 FROM pg_roles WHERE rolname='keycloak'" | grep -q 1 || $PSQL -tAc "CREATE ROLE keycloak WITH LOGIN PASSWORD '$db_password' CREATEDB" + $PSQL -tAc "SELECT 1 FROM pg_database WHERE datname = 'keycloak'" | grep -q 1 || $PSQL -tAc 'CREATE DATABASE "keycloak" OWNER "keycloak"' + ''; + }; + + systemd.services.keycloakMySQLInit = lib.mkIf createLocalMySQL { + after = [ "mysql.service" ]; + before = [ "keycloak.service" ]; + bindsTo = [ "mysql.service" ]; + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + User = config.services.mysql.user; + Group = config.services.mysql.group; + }; + script = '' + set -eu + + db_password="$(<'${cfg.databasePasswordFile}')" + ( echo "CREATE USER IF NOT EXISTS 'keycloak'@'localhost' IDENTIFIED BY '$db_password';" + echo "CREATE DATABASE keycloak CHARACTER SET utf8 COLLATE utf8_unicode_ci;" + echo "GRANT ALL PRIVILEGES ON keycloak.* TO 'keycloak'@'localhost';" + ) | ${config.services.mysql.package}/bin/mysql -N + ''; + }; + + systemd.services.keycloak = + let + databaseServices = + if createLocalPostgreSQL then [ + "keycloakPostgreSQLInit.service" "postgresql.service" + ] + else if createLocalMySQL then [ + "keycloakMySQLInit.service" "mysql.service" + ] + else [ ]; + in { + after = databaseServices; + bindsTo = databaseServices; + wantedBy = [ "multi-user.target" ]; + environment = { + JBOSS_LOG_DIR = "/var/log/keycloak"; + JBOSS_BASE_DIR = "/run/keycloak"; + JBOSS_MODULEPATH = "${cfg.package}/modules"; + }; + serviceConfig = { + ExecStartPre = let + startPreFullPrivileges = '' + set -eu + + install -T -m 0400 -o keycloak -g keycloak '${cfg.databasePasswordFile}' /run/keycloak/secrets/db_password + '' + lib.optionalString (cfg.certificatePrivateKeyBundle != null) '' + install -T -m 0400 -o keycloak -g keycloak '${cfg.certificatePrivateKeyBundle}' /run/keycloak/secrets/ssl_cert_pk_bundle + ''; + startPre = '' + set -eu + + install -m 0600 ${cfg.package}/standalone/configuration/*.properties /run/keycloak/configuration + install -T -m 0600 ${keycloakConfig} /run/keycloak/configuration/standalone.xml + + db_password="$(</run/keycloak/secrets/db_password)" + ${pkgs.replace}/bin/replace-literal -fe '@db-password@' "$db_password" /run/keycloak/configuration/standalone.xml + + export JAVA_OPTS=-Djboss.server.config.user.dir=/run/keycloak/configuration + ${cfg.package}/bin/add-user-keycloak.sh -u admin -p '${cfg.initialAdminPassword}' + '' + lib.optionalString (cfg.certificatePrivateKeyBundle != null) '' + pushd /run/keycloak/ssl/ + cat /run/keycloak/secrets/ssl_cert_pk_bundle <(echo) /etc/ssl/certs/ca-certificates.crt > allcerts.pem + ${pkgs.openssl}/bin/openssl pkcs12 -export -in /run/keycloak/secrets/ssl_cert_pk_bundle -chain \ + -name "${cfg.frontendUrl}" -out certificate_private_key_bundle.p12 \ + -CAfile allcerts.pem -passout pass:notsosecretpassword + popd + ''; + in [ + "+${pkgs.writeShellScript "keycloak-start-pre-full-privileges" startPreFullPrivileges}" + "${pkgs.writeShellScript "keycloak-start-pre" startPre}" + ]; + ExecStart = "${cfg.package}/bin/standalone.sh"; + User = "keycloak"; + Group = "keycloak"; + DynamicUser = true; + RuntimeDirectory = map (p: "keycloak/" + p) [ + "secrets" + "configuration" + "deployments" + "data" + "ssl" + "log" + "tmp" + ]; + RuntimeDirectoryMode = 0700; + LogsDirectory = "keycloak"; + AmbientCapabilities = "CAP_NET_BIND_SERVICE"; + }; + }; + + services.postgresql.enable = lib.mkDefault createLocalPostgreSQL; + services.mysql.enable = lib.mkDefault createLocalMySQL; + services.mysql.package = lib.mkIf createLocalMySQL pkgs.mysql; + }; + + meta.doc = ./keycloak.xml; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/keycloak.xml b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/keycloak.xml new file mode 100644 index 000000000000..ca5e223eee46 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/keycloak.xml @@ -0,0 +1,205 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="module-services-keycloak"> + <title>Keycloak</title> + <para> + <link xlink:href="https://www.keycloak.org/">Keycloak</link> is an + open source identity and access management server with support for + <link xlink:href="https://openid.net/connect/">OpenID + Connect</link>, <link xlink:href="https://oauth.net/2/">OAUTH + 2.0</link> and <link + xlink:href="https://en.wikipedia.org/wiki/SAML_2.0">SAML + 2.0</link>. + </para> + <section xml:id="module-services-keycloak-admin"> + <title>Administration</title> + <para> + An administrative user with the username + <literal>admin</literal> is automatically created in the + <literal>master</literal> realm. Its initial password can be + configured by setting <xref linkend="opt-services.keycloak.initialAdminPassword" /> + and defaults to <literal>changeme</literal>. The password is + not stored safely and should be changed immediately in the + admin panel. + </para> + + <para> + Refer to the <link + xlink:href="https://www.keycloak.org/docs/latest/server_admin/index.html#admin-console">Admin + Console section of the Keycloak Server Administration Guide</link> for + information on how to administer your + <productname>Keycloak</productname> instance. + </para> + </section> + + <section xml:id="module-services-keycloak-database"> + <title>Database access</title> + <para> + <productname>Keycloak</productname> can be used with either + <productname>PostgreSQL</productname> or + <productname>MySQL</productname>. Which one is used can be + configured in <xref + linkend="opt-services.keycloak.databaseType" />. The selected + database will automatically be enabled and a database and role + created unless <xref + linkend="opt-services.keycloak.databaseHost" /> is changed from + its default of <literal>localhost</literal> or <xref + linkend="opt-services.keycloak.databaseCreateLocally" /> is set + to <literal>false</literal>. + </para> + + <para> + External database access can also be configured by setting + <xref linkend="opt-services.keycloak.databaseHost" />, <xref + linkend="opt-services.keycloak.databaseUsername" />, <xref + linkend="opt-services.keycloak.databaseUseSSL" /> and <xref + linkend="opt-services.keycloak.databaseCaCert" /> as + appropriate. Note that you need to manually create a database + called <literal>keycloak</literal> and allow the configured + database user full access to it. + </para> + + <para> + <xref linkend="opt-services.keycloak.databasePasswordFile" /> + must be set to the path to a file containing the password used + to log in to the database. If <xref linkend="opt-services.keycloak.databaseHost" /> + and <xref linkend="opt-services.keycloak.databaseCreateLocally" /> + are kept at their defaults, the database role + <literal>keycloak</literal> with that password is provisioned + on the local database instance. + </para> + + <warning> + <para> + The path should be provided as a string, not a Nix path, since Nix + paths are copied into the world readable Nix store. + </para> + </warning> + </section> + + <section xml:id="module-services-keycloak-frontendurl"> + <title>Frontend URL</title> + <para> + The frontend URL is used as base for all frontend requests and + must be configured through <xref linkend="opt-services.keycloak.frontendUrl" />. + It should normally include a trailing <literal>/auth</literal> + (the default web context). + </para> + + <para> + <xref linkend="opt-services.keycloak.forceBackendUrlToFrontendUrl" /> + determines whether Keycloak should force all requests to go + through the frontend URL. By default, + <productname>Keycloak</productname> allows backend requests to + instead use its local hostname or IP address and may also + advertise it to clients through its OpenID Connect Discovery + endpoint. + </para> + + <para> + See the <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/#_hostname">Hostname + section of the Keycloak Server Installation and Configuration + Guide</link> for more information. + </para> + </section> + + <section xml:id="module-services-keycloak-tls"> + <title>Setting up TLS/SSL</title> + <para> + By default, <productname>Keycloak</productname> won't accept + unsecured HTTP connections originating from outside its local + network. + </para> + + <para> + For HTTPS support, a TLS certificate and private key is + required. They should be <link + xlink:href="https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail">PEM + formatted</link> and concatenated into a single file. The path + to this file should be configured in + <xref linkend="opt-services.keycloak.certificatePrivateKeyBundle" />. + </para> + + <warning> + <para> + The path should be provided as a string, not a Nix path, + since Nix paths are copied into the world readable Nix store. + </para> + </warning> + </section> + + <section xml:id="module-services-keycloak-extra-config"> + <title>Additional configuration</title> + <para> + Additional Keycloak configuration options, for which no + explicit <productname>NixOS</productname> options are provided, + can be set in <xref linkend="opt-services.keycloak.extraConfig" />. + </para> + + <para> + Options are expressed as a Nix attribute set which matches the + structure of the jboss-cli configuration. The configuration is + effectively overlayed on top of the default configuration + shipped with Keycloak. To remove existing nodes and undefine + attributes from the default configuration, set them to + <literal>null</literal>. + </para> + <para> + For example, the following script, which removes the hostname + provider <literal>default</literal>, adds the deprecated + hostname provider <literal>fixed</literal> and defines it the + default: + +<programlisting> +/subsystem=keycloak-server/spi=hostname/provider=default:remove() +/subsystem=keycloak-server/spi=hostname/provider=fixed:add(enabled = true, properties = { hostname = "keycloak.example.com" }) +/subsystem=keycloak-server/spi=hostname:write-attribute(name=default-provider, value="fixed") +</programlisting> + + would be expressed as + +<programlisting> +services.keycloak.extraConfig = { + "subsystem=keycloak-server" = { + "spi=hostname" = { + "provider=default" = null; + "provider=fixed" = { + enabled = true; + properties.hostname = "keycloak.example.com"; + }; + default-provider = "fixed"; + }; + }; +}; +</programlisting> + </para> + <para> + You can discover available options by using the <link + xlink:href="http://docs.wildfly.org/21/Admin_Guide.html#Command_Line_Interface">jboss-cli.sh</link> + program and by referring to the <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/index.html">Keycloak + Server Installation and Configuration Guide</link>. + </para> + </section> + + <section xml:id="module-services-keycloak-example-config"> + <title>Example configuration</title> + <para> + A basic configuration with some custom settings could look like this: +<programlisting> +services.keycloak = { + <link linkend="opt-services.keycloak.enable">enable</link> = true; + <link linkend="opt-services.keycloak.initialAdminPassword">initialAdminPassword</link> = "e6Wcm0RrtegMEHl"; # change on first login + <link linkend="opt-services.keycloak.frontendUrl">frontendUrl</link> = "https://keycloak.example.com/auth"; + <link linkend="opt-services.keycloak.forceBackendUrlToFrontendUrl">forceBackendUrlToFrontendUrl</link> = true; + <link linkend="opt-services.keycloak.certificatePrivateKeyBundle">certificatePrivateKeyBundle</link> = "/run/keys/ssl_cert"; + <link linkend="opt-services.keycloak.databasePasswordFile">databasePasswordFile</link> = "/run/keys/db_password"; +}; +</programlisting> + </para> + + </section> + </chapter> diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/moinmoin.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/moinmoin.nix index dc7abce2a5cb..3a876f75f4a4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/moinmoin.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/moinmoin.nix @@ -224,6 +224,8 @@ in chmod -R u+w ${dataDir}/${wikiIdent}/underlay ''; + startLimitIntervalSec = 30; + serviceConfig = { User = user; Group = group; @@ -237,7 +239,6 @@ in Restart = "on-failure"; RestartSec = "2s"; - StartLimitIntervalSec = "30s"; StateDirectory = "moin/${wikiIdent}"; StateDirectoryMode = "0750"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/nextcloud.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/nextcloud.nix index bad1bd9c767d..53c2ab76fdfa 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/nextcloud.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/nextcloud.nix @@ -549,9 +549,7 @@ in { }; "/" = { priority = 900; - extraConfig = if major < 20 - then "rewrite ^ /index.php;" - else "try_files $uri $uri/ /index.php$request_uri;"; + extraConfig = "rewrite ^ /index.php;"; }; "~ ^/store-apps" = { priority = 201; @@ -575,7 +573,7 @@ in { "~ ^/(?:\\.|autotest|occ|issue|indie|db_|console)".extraConfig = '' return 404; ''; - ${if major < 20 then "~ ^\\/(?:index|remote|public|cron|core\\/ajax\\/update|status|ocs\\/v[12]|updater\\/.+|oc[ms]-provider\\/.+|.+\\/richdocumentscode\\/proxy)\\.php(?:$|\\/)" else "~ \\.php(?:$|/)"} = { + "~ ^\\/(?:index|remote|public|cron|core\\/ajax\\/update|status|ocs\\/v[12]|updater\\/.+|oc[ms]-provider\\/.+|.+\\/richdocumentscode\\/proxy)\\.php(?:$|\\/)" = { priority = 500; extraConfig = '' include ${config.services.nginx.package}/conf/fastcgi.conf; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/shiori.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/shiori.nix index 1817a2039352..9083ddfa2206 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/shiori.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-apps/shiori.nix @@ -37,11 +37,60 @@ in { description = "Shiori simple bookmarks manager"; wantedBy = [ "multi-user.target" ]; + environment.SHIORI_DIR = "/var/lib/shiori"; + serviceConfig = { ExecStart = "${package}/bin/shiori serve --address '${address}' --port '${toString port}'"; + DynamicUser = true; - Environment = "SHIORI_DIR=/var/lib/shiori"; StateDirectory = "shiori"; + # As the RootDirectory + RuntimeDirectory = "shiori"; + + # Security options + + BindReadOnlyPaths = [ + "/nix/store" + + # For SSL certificates, and the resolv.conf + "/etc" + ]; + + CapabilityBoundingSet = ""; + + DeviceAllow = ""; + + LockPersonality = true; + + MemoryDenyWriteExecute = true; + + PrivateDevices = true; + PrivateUsers = true; + + ProtectClock = true; + ProtectControlGroups = true; + ProtectHome = true; + ProtectHostname = true; + ProtectKernelLogs = true; + ProtectKernelModules = true; + ProtectKernelTunables = true; + + RestrictNamespaces = true; + RestrictAddressFamilies = [ "AF_INET" "AF_INET6" ]; + RestrictRealtime = true; + RestrictSUIDSGID = true; + + RootDirectory = "/run/shiori"; + + SystemCallArchitectures = "native"; + SystemCallErrorNumber = "EPERM"; + SystemCallFilter = [ + "@system-service" + + "~@chown" "~@cpu-emulation" "~@debug" "~@ipc" "~@keyring" "~@memlock" + "~@module" "~@obsolete" "~@privileged" "~@process" "~@raw-io" + "~@resources" "~@setuid" + ]; }; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/apache-httpd/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/apache-httpd/default.nix index 6ffda3d63614..dc78728d6636 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/apache-httpd/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/apache-httpd/default.nix @@ -750,8 +750,8 @@ in # Get rid of old semaphores. These tend to accumulate across # server restarts, eventually preventing it from restarting # successfully. - for i in $(${pkgs.utillinux}/bin/ipcs -s | grep ' ${cfg.user} ' | cut -f2 -d ' '); do - ${pkgs.utillinux}/bin/ipcrm -s $i + for i in $(${pkgs.util-linux}/bin/ipcs -s | grep ' ${cfg.user} ' | cut -f2 -d ' '); do + ${pkgs.util-linux}/bin/ipcrm -s $i done ''; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/caddy.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/caddy.nix index 72bf9a9a1f44..297b73273392 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/caddy.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/caddy.nix @@ -24,6 +24,10 @@ let ${pkgs.jq}/bin/jq -s '.[0] * .[1]' ${adaptedConfig} ${tlsJSON} > $out ''; in { + imports = [ + (mkRemovedOptionModule [ "services" "caddy" "agree" ] "this option is no longer necessary for Caddy 2") + ]; + options.services.caddy = { enable = mkEnableOption "Caddy web server"; @@ -66,12 +70,6 @@ in { description = "Email address (for Let's Encrypt certificate)"; }; - agree = mkOption { - default = false; - type = types.bool; - description = "Agree to Let's Encrypt Subscriber Agreement"; - }; - dataDir = mkOption { default = "/var/lib/caddy"; type = types.path; @@ -103,6 +101,8 @@ in { after = [ "network-online.target" ]; wants = [ "network-online.target" ]; # systemd-networkd-wait-online.service wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 14400; + startLimitBurst = 10; serviceConfig = { ExecStart = "${cfg.package}/bin/caddy run --config ${configJSON}"; ExecReload = "${cfg.package}/bin/caddy reload --config ${configJSON}"; @@ -110,8 +110,6 @@ in { User = "caddy"; Group = "caddy"; Restart = "on-abnormal"; - StartLimitIntervalSec = 14400; - StartLimitBurst = 10; AmbientCapabilities = "cap_net_bind_service"; CapabilityBoundingSet = "cap_net_bind_service"; NoNewPrivileges = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/default.nix index 39bcb14e5afe..e9630d379f36 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/default.nix @@ -34,7 +34,6 @@ let proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; - proxy_set_header Accept-Encoding ""; ''; upstreamConfig = toString (flip mapAttrsToList cfg.upstreams (name: upstream: '' @@ -87,7 +86,7 @@ let ''} ssl_protocols ${cfg.sslProtocols}; - ssl_ciphers ${cfg.sslCiphers}; + ${optionalString (cfg.sslCiphers != null) "ssl_ciphers ${cfg.sslCiphers};"} ${optionalString (cfg.sslDhparam != null) "ssl_dhparam ${cfg.sslDhparam};"} ${optionalString (cfg.recommendedTlsSettings) '' @@ -262,10 +261,7 @@ let ssl_trusted_certificate ${vhost.sslTrustedCertificate}; ''} - ${optionalString (vhost.basicAuthFile != null || vhost.basicAuth != {}) '' - auth_basic secured; - auth_basic_user_file ${if vhost.basicAuthFile != null then vhost.basicAuthFile else mkHtpasswd vhostName vhost.basicAuth}; - ''} + ${mkBasicAuth vhostName vhost} ${mkLocations vhost.locations} @@ -294,9 +290,19 @@ let ${optionalString (config.return != null) "return ${config.return};"} ${config.extraConfig} ${optionalString (config.proxyPass != null && cfg.recommendedProxySettings) "include ${recommendedProxyConfig};"} + ${mkBasicAuth "sublocation" config} } '') (sortProperties (mapAttrsToList (k: v: v // { location = k; }) locations))); - mkHtpasswd = vhostName: authDef: pkgs.writeText "${vhostName}.htpasswd" ( + + mkBasicAuth = name: zone: optionalString (zone.basicAuthFile != null || zone.basicAuth != {}) (let + auth_file = if zone.basicAuthFile != null + then zone.basicAuthFile + else mkHtpasswd name zone.basicAuth; + in '' + auth_basic secured; + auth_basic_user_file ${auth_file}; + ''); + mkHtpasswd = name: authDef: pkgs.writeText "${name}.htpasswd" ( concatStringsSep "\n" (mapAttrsToList (user: password: '' ${user}:{PLAIN}${password} '') authDef) @@ -488,7 +494,7 @@ in }; sslCiphers = mkOption { - type = types.str; + type = types.nullOr types.str; # Keep in sync with https://ssl-config.mozilla.org/#server=nginx&config=intermediate default = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384"; description = "Ciphers to choose from when negotiating TLS handshakes."; @@ -694,6 +700,8 @@ in ${cfg.preStart} ${execCommand} -t ''; + + startLimitIntervalSec = 60; serviceConfig = { ExecStart = execCommand; ExecReload = [ @@ -702,7 +710,6 @@ in ]; Restart = "always"; RestartSec = "10s"; - StartLimitInterval = "1min"; # User and group User = cfg.user; Group = cfg.group; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/location-options.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/location-options.nix index 3d9e391ecf20..f2fc07255725 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/location-options.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/location-options.nix @@ -9,6 +9,34 @@ with lib; { options = { + basicAuth = mkOption { + type = types.attrsOf types.str; + default = {}; + example = literalExample '' + { + user = "password"; + }; + ''; + description = '' + Basic Auth protection for a vhost. + + WARNING: This is implemented to store the password in plain text in the + Nix store. + ''; + }; + + basicAuthFile = mkOption { + type = types.nullOr types.path; + default = null; + description = '' + Basic Auth password file for a vhost. + Can be created via: <command>htpasswd -c <filename> <username></command>. + + WARNING: The generate file contains the users' passwords in a + non-cryptographically-securely hashed way. + ''; + }; + proxyPass = mkOption { type = types.nullOr types.str; default = null; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/vhost-options.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/vhost-options.nix index 455854e2a965..cf211ea9a71b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/vhost-options.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/nginx/vhost-options.nix @@ -198,7 +198,7 @@ with lib; Basic Auth protection for a vhost. WARNING: This is implemented to store the password in plain text in the - nix store. + Nix store. ''; }; @@ -207,7 +207,10 @@ with lib; default = null; description = '' Basic Auth password file for a vhost. - Can be created via: <command>htpasswd -c <filename> <username></command> + Can be created via: <command>htpasswd -c <filename> <username></command>. + + WARNING: The generate file contains the users' passwords in a + non-cryptographically-securely hashed way. ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/traefik.nix b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/traefik.nix index 4ab7307c3b67..3d29199dd454 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/traefik.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/web-servers/traefik.nix @@ -136,6 +136,8 @@ in { description = "Traefik web server"; after = [ "network-online.target" ]; wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 86400; + startLimitBurst = 5; serviceConfig = { ExecStart = "${cfg.package}/bin/traefik --configfile=${staticConfigFile}"; @@ -143,8 +145,6 @@ in { User = "traefik"; Group = cfg.group; Restart = "on-failure"; - StartLimitInterval = 86400; - StartLimitBurst = 5; AmbientCapabilities = "cap_net_bind_service"; CapabilityBoundingSet = "cap_net_bind_service"; NoNewPrivileges = true; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/gnome3.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/gnome3.nix index acccbdb9950a..68a65d77d62f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/gnome3.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/gnome3.nix @@ -17,6 +17,11 @@ let ''; }; + defaultFavoriteAppsOverride = '' + [org.gnome.shell] + favorite-apps=[ 'org.gnome.Geary.desktop', 'org.gnome.Calendar.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop' ] + ''; + nixos-gsettings-desktop-schemas = let defaultPackages = with pkgs; [ gsettings-desktop-schemas gnome3.gnome-shell ]; in @@ -42,8 +47,7 @@ let [org.gnome.desktop.screensaver] picture-uri='file://${pkgs.nixos-artwork.wallpapers.simple-dark-gray-bottom.gnomeFilePath}' - [org.gnome.shell] - favorite-apps=[ 'org.gnome.Epiphany.desktop', 'org.gnome.Geary.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop', 'org.gnome.Software.desktop' ] + ${cfg.favoriteAppsOverride} ${cfg.extraGSettingsOverrides} EOF @@ -69,6 +73,7 @@ in core-os-services.enable = mkEnableOption "essential services for GNOME3"; core-shell.enable = mkEnableOption "GNOME Shell services"; core-utilities.enable = mkEnableOption "GNOME core utilities"; + core-developer-tools.enable = mkEnableOption "GNOME core developer tools"; games.enable = mkEnableOption "GNOME games"; experimental-features = { @@ -123,6 +128,17 @@ in apply = list: list ++ [ pkgs.gnome3.gnome-shell pkgs.gnome3.gnome-shell-extensions ]; }; + favoriteAppsOverride = mkOption { + internal = true; # this is messy + default = defaultFavoriteAppsOverride; + type = types.lines; + example = literalExample '' + [org.gnome.shell] + favorite-apps=[ 'firefox.desktop', 'org.gnome.Calendar.desktop' ] + ''; + description = "List of desktop files to put as favorite apps into gnome-shell. These need to be installed somehow globally."; + }; + extraGSettingsOverrides = mkOption { default = ""; type = types.lines; @@ -179,6 +195,14 @@ in config = mkMerge [ (mkIf (cfg.enable || flashbackEnabled) { + # Seed our configuration into nixos-generate-config + system.nixos-generate-config.desktopConfiguration = '' + # Enable the GNOME 3 Desktop Environment. + services.xserver.enable = true; + services.xserver.displayManager.gdm.enable = true; + services.xserver.desktopManager.gnome3.enable = true; + ''; + services.gnome3.core-os-services.enable = true; services.gnome3.core-shell.enable = true; services.gnome3.core-utilities.enable = mkDefault true; @@ -207,6 +231,11 @@ in # If gnome3 is installed, build vim for gtk3 too. nixpkgs.config.vim.gui = "gtk3"; + + # Install gnome-software if flatpak is enabled + services.flatpak.guiPackages = [ + pkgs.gnome3.gnome-software + ]; }) (mkIf flashbackEnabled { @@ -294,6 +323,12 @@ in gnome-shell ]; + services.udev.packages = with pkgs.gnome3; [ + # Force enable KMS modifiers for devices that require them. + # https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1443 + mutter + ]; + services.avahi.enable = mkDefault true; xdg.portal.extraPortals = [ @@ -323,7 +358,7 @@ in source-sans-pro ]; - # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-36/elements/core/meta-gnome-core-shell.bst + # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-38/elements/core/meta-gnome-core-shell.bst environment.systemPackages = with pkgs.gnome3; [ adwaita-icon-theme gnome-backgrounds @@ -368,7 +403,7 @@ in }; }) - # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-36/elements/core/meta-gnome-core-utilities.bst + # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-38/elements/core/meta-gnome-core-utilities.bst (mkIf serviceCfg.core-utilities.enable { environment.systemPackages = (with pkgs.gnome3; removePackagesByName [ baobab @@ -387,17 +422,15 @@ in gnome-logs gnome-maps gnome-music - gnome-photos + pkgs.gnome-photos gnome-screenshot - gnome-software gnome-system-monitor gnome-weather nautilus + pkgs.gnome-connections simple-scan totem yelp - # Unsure if sensible for NixOS - /* gnome-boxes */ ] config.environment.gnome3.excludePackages); # Enable default program modules @@ -426,12 +459,43 @@ in (mkIf serviceCfg.games.enable { environment.systemPackages = (with pkgs.gnome3; removePackagesByName [ - aisleriot atomix five-or-more four-in-a-row gnome-chess gnome-klotski - gnome-mahjongg gnome-mines gnome-nibbles gnome-robots gnome-sudoku - gnome-taquin gnome-tetravex hitori iagno lightsoff quadrapassel - swell-foop tali + aisleriot + atomix + five-or-more + four-in-a-row + gnome-chess + gnome-klotski + gnome-mahjongg + gnome-mines + gnome-nibbles + gnome-robots + gnome-sudoku + gnome-taquin + gnome-tetravex + hitori + iagno + lightsoff + quadrapassel + swell-foop + tali ] config.environment.gnome3.excludePackages); }) + + # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/-/blob/3.38.0/elements/core/meta-gnome-core-developer-tools.bst + (mkIf serviceCfg.core-developer-tools.enable { + environment.systemPackages = (with pkgs.gnome3; removePackagesByName [ + dconf-editor + devhelp + pkgs.gnome-builder + # boxes would make sense in this option, however + # it doesn't function well enough to be included + # in default configurations. + # https://github.com/NixOS/nixpkgs/issues/60908 + /* gnome-boxes */ + ] config.environment.gnome3.excludePackages); + + services.sysprof.enable = true; + }) ]; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/pantheon.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/pantheon.nix index e67e216f90d9..cf02a71248b1 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/pantheon.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/pantheon.nix @@ -180,7 +180,6 @@ in gtk3.out hicolor-icon-theme lightlocker - nixos-artwork.wallpapers.simple-dark-gray onboard qgnomeplatform shared-mime-info diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/plasma5.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/plasma5.nix index e48b5f23b58f..8cc579af2ca0 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/plasma5.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/desktop-managers/plasma5.nix @@ -184,6 +184,14 @@ in config = mkMerge [ (mkIf cfg.enable { + # Seed our configuration into nixos-generate-config + system.nixos-generate-config.desktopConfiguration = '' + # Enable the Plasma 5 Desktop Environment. + services.xserver.enable = true; + services.xserver.displayManager.sddm.enable = true; + services.xserver.desktopManager.plasma5.enable = true; + ''; + services.xserver.desktopManager.session = singleton { name = "plasma5"; bgSupport = true; @@ -192,7 +200,7 @@ in security.wrappers = { kcheckpass.source = "${lib.getBin plasma5.kscreenlocker}/libexec/kcheckpass"; - start_kdeinit.source = "${lib.getBin pkgs.kinit}/libexec/kf5/start_kdeinit"; + start_kdeinit.source = "${lib.getBin pkgs.kdeFrameworks.kinit}/libexec/kf5/start_kdeinit"; kwin_wayland = { source = "${lib.getBin plasma5.kwin}/bin/kwin_wayland"; capabilities = "cap_sys_nice+ep"; @@ -359,7 +367,7 @@ in security.pam.services.sddm.enableKwallet = true; xdg.portal.enable = true; - xdg.portal.extraPortals = [ pkgs.xdg-desktop-portal-kde ]; + xdg.portal.extraPortals = [ plasma5.xdg-desktop-portal-kde ]; # Update the start menu for each user that is currently logged in system.userActivationScripts.plasmaSetup = activationScript; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/default.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/default.nix index 568aeaceef75..6945a241f92f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/default.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/default.nix @@ -474,6 +474,12 @@ in ) [dms wms] ); + + # Make xsessions and wayland sessions available in XDG_DATA_DIRS + # as some programs have behavior that depends on them being present + environment.sessionVariables.XDG_DATA_DIRS = [ + "${cfg.displayManager.sessionData.desktops}/share" + ]; }; imports = [ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/gdm.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/gdm.nix index eae70a57c781..e3c5adb9737f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/gdm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/gdm.nix @@ -160,7 +160,7 @@ in ]; # Otherwise GDM will not be able to start correctly and display Wayland sessions - systemd.packages = with pkgs.gnome3; [ gnome-session gnome-shell ]; + systemd.packages = with pkgs.gnome3; [ gdm gnome-session gnome-shell ]; environment.systemPackages = [ pkgs.gnome3.adwaita-icon-theme ]; systemd.services.display-manager.wants = [ @@ -264,7 +264,7 @@ in # presented and there's a little delay. environment.etc."gdm/custom.conf".text = '' [daemon] - WaylandEnable=${if cfg.gdm.wayland then "true" else "false"} + WaylandEnable=${boolToString cfg.gdm.wayland} ${optionalString cfg.autoLogin.enable ( if cfg.gdm.autoLogin.delay > 0 then '' TimedLoginEnable=true diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/lightdm.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/lightdm.nix index 143785db0b4f..2dafee9e36e3 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/lightdm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/lightdm.nix @@ -308,6 +308,7 @@ in home = "/var/lib/lightdm"; group = "lightdm"; uid = config.ids.uids.lightdm; + shell = pkgs.bash; }; systemd.tmpfiles.rules = [ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/sddm.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/sddm.nix index e63bb2e44539..a39bb55b38c4 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/sddm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/display-managers/sddm.nix @@ -9,7 +9,12 @@ let cfg = dmcfg.sddm; xEnv = config.systemd.services.display-manager.environment; - inherit (pkgs) sddm; + sddm = if config.services.xserver.desktopManager.lxqt.enable then + # TODO: Move lxqt to libsForQt515 + pkgs.libsForQt514.sddm + else + pkgs.libsForQt5.sddm + ; xserverWrapper = pkgs.writeScript "xserver-wrapper" '' #!/bin/sh @@ -55,10 +60,10 @@ let XauthPath=${pkgs.xorg.xauth}/bin/xauth DisplayCommand=${Xsetup} DisplayStopCommand=${Xstop} - EnableHidpi=${if cfg.enableHidpi then "true" else "false"} + EnableHidpi=${boolToString cfg.enableHidpi} [Wayland] - EnableHidpi=${if cfg.enableHidpi then "true" else "false"} + EnableHidpi=${boolToString cfg.enableHidpi} SessionDir=${dmcfg.sessionData.desktops}/share/wayland-sessions ${optionalString dmcfg.autoLogin.enable '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/redshift.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/redshift.nix index 21b0b33553ac..60d80a28762b 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/redshift.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/redshift.nix @@ -82,6 +82,15 @@ in { ''; }; + executable = mkOption { + type = types.str; + default = "/bin/redshift"; + example = "/bin/redshift-gtk"; + description = '' + Redshift executable to use within the package. + ''; + }; + extraOptions = mkOption { type = types.listOf types.str; default = []; @@ -114,7 +123,7 @@ in { partOf = [ "graphical-session.target" ]; serviceConfig = { ExecStart = '' - ${cfg.package}/bin/redshift \ + ${cfg.package}${cfg.executable} \ -l ${providerString} \ -t ${toString cfg.temperature.day}:${toString cfg.temperature.night} \ -b ${toString cfg.brightness.day}:${toString cfg.brightness.night} \ diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/terminal-server.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/terminal-server.nix index 503c14c9b624..e6b50c21a952 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/terminal-server.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/terminal-server.nix @@ -32,7 +32,7 @@ with lib; path = [ pkgs.xorg.xorgserver.out pkgs.gawk pkgs.which pkgs.openssl pkgs.xorg.xauth - pkgs.nettools pkgs.shadow pkgs.procps pkgs.utillinux pkgs.bash + pkgs.nettools pkgs.shadow pkgs.procps pkgs.util-linux pkgs.bash ]; environment.FD_GEOM = "1024x786x24"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/evilwm.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/evilwm.nix index 6e19e3572c79..6f1db2110f87 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/evilwm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/evilwm.nix @@ -16,8 +16,8 @@ in services.xserver.windowManager.session = singleton { name = "evilwm"; start = '' - ${pkgs.evilwm}/bin/evilwm & - waitPID=$! + ${pkgs.evilwm}/bin/evilwm & + waitPID=$! ''; }; environment.systemPackages = [ pkgs.evilwm ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/exwm.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/exwm.nix index dc1d957c1709..88e13f4dbfb0 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/exwm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/window-managers/exwm.nix @@ -5,7 +5,7 @@ with lib; let cfg = config.services.xserver.windowManager.exwm; loadScript = pkgs.writeText "emacs-exwm-load" '' - (require 'exwm) + ${cfg.loadScript} ${optionalString cfg.enableDefaultConfig '' (require 'exwm-config) (exwm-config-default) @@ -19,6 +19,18 @@ in options = { services.xserver.windowManager.exwm = { enable = mkEnableOption "exwm"; + loadScript = mkOption { + default = "(require 'exwm)"; + example = literalExample '' + (require 'exwm) + (exwm-enable) + ''; + description = '' + Emacs lisp code to be run after loading the user's init + file. If enableDefaultConfig is true, this will be run + before loading the default config. + ''; + }; enableDefaultConfig = mkOption { default = true; type = lib.types.bool; diff --git a/infra/libkookie/nixpkgs/nixos/modules/services/x11/xserver.nix b/infra/libkookie/nixpkgs/nixos/modules/services/x11/xserver.nix index 55d3e742ef7c..9e971671c474 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/services/x11/xserver.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/services/x11/xserver.nix @@ -678,14 +678,14 @@ in script = "${cfg.displayManager.job.execCmd}"; + # Stop restarting if the display manager stops (crashes) 2 times + # in one minute. Starting X typically takes 3-4s. + startLimitIntervalSec = 30; + startLimitBurst = 3; serviceConfig = { Restart = "always"; RestartSec = "200ms"; SyslogIdentifier = "display-manager"; - # Stop restarting if the display manager stops (crashes) 2 times - # in one minute. Starting X typically takes 3-4s. - StartLimitInterval = "30s"; - StartLimitBurst = "3"; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/activation/activation-script.nix b/infra/libkookie/nixpkgs/nixos/modules/system/activation/activation-script.nix index ddfd1af4a319..3a6930314b1a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/activation/activation-script.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/activation/activation-script.nix @@ -25,9 +25,23 @@ let stdenv.cc.libc # nscd in update-users-groups.pl shadow nettools # needed for hostname - utillinux # needed for mount and mountpoint + util-linux # needed for mount and mountpoint ]; + scriptType = with types; + let scriptOptions = + { deps = mkOption + { type = types.listOf types.str; + default = [ ]; + description = "List of dependencies. The script will run after these."; + }; + text = mkOption + { type = types.lines; + description = "The content of the script."; + }; + }; + in either str (submodule { options = scriptOptions; }); + in { @@ -40,16 +54,14 @@ in default = {}; example = literalExample '' - { stdio = { - text = ''' - # Needed by some programs. - ln -sfn /proc/self/fd /dev/fd - ln -sfn /proc/self/fd/0 /dev/stdin - ln -sfn /proc/self/fd/1 /dev/stdout - ln -sfn /proc/self/fd/2 /dev/stderr - '''; - deps = []; - }; + { stdio.text = + ''' + # Needed by some programs. + ln -sfn /proc/self/fd /dev/fd + ln -sfn /proc/self/fd/0 /dev/stdin + ln -sfn /proc/self/fd/1 /dev/stdout + ln -sfn /proc/self/fd/2 /dev/stderr + '''; } ''; @@ -62,7 +74,7 @@ in idempotent and fast. ''; - type = types.attrsOf types.unspecified; # FIXME + type = types.attrsOf scriptType; apply = set: { script = @@ -125,7 +137,7 @@ in idempotent and fast. ''; - type = types.attrsOf types.unspecified; + type = with types; attrsOf scriptType; apply = set: { script = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/activation/top-level.nix b/infra/libkookie/nixpkgs/nixos/modules/system/activation/top-level.nix index 2724d9f9cb6f..03d7e7493230 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/activation/top-level.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/activation/top-level.nix @@ -97,10 +97,11 @@ let allowSubstitutes = false; buildCommand = systemBuilder; - inherit (pkgs) utillinux coreutils; + inherit (pkgs) coreutils; systemd = config.systemd.package; shell = "${pkgs.bash}/bin/sh"; su = "${pkgs.shadow.su}/bin/su"; + utillinux = pkgs.util-linux; kernelParams = config.boot.kernelParams; installBootLoader = diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/grow-partition.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/grow-partition.nix index be70c4ad9c8d..87c981b24cec 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/grow-partition.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/grow-partition.nix @@ -20,8 +20,8 @@ with lib; boot.initrd.extraUtilsCommands = '' copy_bin_and_libs ${pkgs.gawk}/bin/gawk copy_bin_and_libs ${pkgs.gnused}/bin/sed - copy_bin_and_libs ${pkgs.utillinux}/sbin/sfdisk - copy_bin_and_libs ${pkgs.utillinux}/sbin/lsblk + copy_bin_and_libs ${pkgs.util-linux}/sbin/sfdisk + copy_bin_and_libs ${pkgs.util-linux}/sbin/lsblk substitute "${pkgs.cloud-utils.guest}/bin/.growpart-wrapped" "$out/bin/growpart" \ --replace "${pkgs.bash}/bin/sh" "/bin/sh" \ diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-network.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-network.nix index ec794d6eb014..2a7417ed3715 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-network.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-network.nix @@ -32,8 +32,8 @@ let fi if [ -n "$dns" ]; then rm -f /etc/resolv.conf - for i in $dns; do - echo "nameserver $dns" >> /etc/resolv.conf + for server in $dns; do + echo "nameserver $server" >> /etc/resolv.conf done fi fi diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-ssh.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-ssh.nix index f7ef26103709..00ac83a18972 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-ssh.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/initrd-ssh.nix @@ -159,9 +159,14 @@ in boot.initrd.extraUtilsCommandsTest = '' # sshd requires a host key to check config, so we pass in the test's + tmpkey="$(mktemp initrd-ssh-testkey.XXXXXXXXXX)" + cp "${../../../tests/initrd-network-ssh/ssh_host_ed25519_key}" "$tmpkey" + # keys from Nix store are world-readable, which sshd doesn't like + chmod 600 "$tmpkey" echo -n ${escapeShellArg sshdConfig} | $out/bin/sshd -t -f /dev/stdin \ - -h ${../../../tests/initrd-network-ssh/ssh_host_ed25519_key} + -h "$tmpkey" + rm "$tmpkey" ''; boot.initrd.network.postCommands = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/kernel.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/kernel.nix index 43871f439f7f..ed7226331d70 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/kernel.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/kernel.nix @@ -227,7 +227,7 @@ in "xhci_pci" "usbhid" "hid_generic" "hid_lenovo" "hid_apple" "hid_roccat" - "hid_logitech_hidpp" "hid_logitech_dj" + "hid_logitech_hidpp" "hid_logitech_dj" "hid_microsoft" ] ++ optionals (pkgs.stdenv.isi686 || pkgs.stdenv.isx86_64) [ # Misc. x86 keyboard stuff. diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/loader/grub/grub.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/loader/grub/grub.nix index 20e39628eabb..df5dfaa554bc 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/loader/grub/grub.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/loader/grub/grub.nix @@ -66,7 +66,7 @@ let extraEntriesBeforeNixOS extraPrepareConfig configurationLimit copyKernels default fsIdentifier efiSupport efiInstallAsRemovable gfxmodeEfi gfxmodeBios gfxpayloadEfi gfxpayloadBios; path = with pkgs; makeBinPath ( - [ coreutils gnused gnugrep findutils diffutils btrfs-progs utillinux mdadm ] + [ coreutils gnused gnugrep findutils diffutils btrfs-progs util-linux mdadm ] ++ optional (cfg.efiSupport && (cfg.version == 2)) efibootmgr ++ optionals cfg.useOSProber [ busybox os-prober ]); font = if cfg.font == null then "" @@ -705,7 +705,7 @@ in let install-grub-pl = pkgs.substituteAll { src = ./install-grub.pl; - inherit (pkgs) utillinux; + utillinux = pkgs.util-linux; btrfsprogs = pkgs.btrfs-progs; }; in pkgs.writeScript "install-grub.sh" ('' @@ -741,7 +741,7 @@ in + "'boot.loader.grub.mirroredBoots' to make the system bootable."; } { - assertion = cfg.efiSupport || all (c: c < 2) (mapAttrsToList (_: c: c) bootDeviceCounters); + assertion = cfg.efiSupport || all (c: c < 2) (mapAttrsToList (n: c: if n == "nodev" then 0 else c) bootDeviceCounters); message = "You cannot have duplicated devices in mirroredBoots"; } { diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/luksroot.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/luksroot.nix index 88190e8200b1..8dd2ea20519a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/luksroot.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/luksroot.nix @@ -404,7 +404,7 @@ let echo "Please move your mouse to create needed randomness." ''} echo "Waiting for your FIDO2 device..." - fido2luks -i open ${device} ${name} ${fido2.credential} --await-dev ${toString fido2.gracePeriod} --salt string:$passphrase + fido2luks open ${device} ${name} ${fido2.credential} --await-dev ${toString fido2.gracePeriod} --salt string:$passphrase if [ $? -ne 0 ]; then echo "No FIDO2 key found, falling back to normal open procedure" open_normally diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/networkd.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/networkd.nix index 47689b2a4700..3b01bc00bafa 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/networkd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/networkd.nix @@ -648,11 +648,13 @@ let "RapidCommit" "ForceDHCPv6PDOtherInformation" "PrefixDelegationHint" + "RouteMetric" ]) (assertValueOneOf "UseDNS" boolValues) (assertValueOneOf "UseNTP" boolValues) (assertValueOneOf "RapidCommit" boolValues) (assertValueOneOf "ForceDHCPv6PDOtherInformation" boolValues) + (assertInt "RouteMetric") ]; sectionDHCPServer = checkUnitConfig "DHCPServer" [ diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/pbkdf2-sha512.c b/infra/libkookie/nixpkgs/nixos/modules/system/boot/pbkdf2-sha512.c index b40c383ac023..67e989957ba6 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/pbkdf2-sha512.c +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/pbkdf2-sha512.c @@ -35,4 +35,4 @@ int main(int argc, char** argv) fwrite(key, 1, key_length, stdout); return 0; -}
\ No newline at end of file +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/plymouth.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/plymouth.nix index 55e5b07ed615..ddf5ef8a0a6a 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/plymouth.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/plymouth.nix @@ -9,7 +9,7 @@ let cfg = config.boot.plymouth; - nixosBreezePlymouth = pkgs.breeze-plymouth.override { + nixosBreezePlymouth = pkgs.plasma5.breeze-plymouth.override { logoFile = cfg.logo; logoName = "nixos"; osName = "NixOS"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/resolved.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/resolved.nix index b024f9cf5ee9..84bc9b78076c 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/resolved.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/resolved.nix @@ -136,7 +136,7 @@ in } ]; - users.users.resolved.group = "systemd-resolve"; + users.users.systemd-resolve.group = "systemd-resolve"; # add resolve to nss hosts database if enabled and nscd enabled # system.nssModules is configured in nixos/modules/system/boot/systemd.nix diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/shutdown.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/shutdown.nix index 11041066e07c..8cda7b3aabe8 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/shutdown.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/shutdown.nix @@ -18,7 +18,7 @@ with lib; serviceConfig = { Type = "oneshot"; - ExecStart = "${pkgs.utillinux}/sbin/hwclock --systohc ${if config.time.hardwareClockInLocalTime then "--localtime" else "--utc"}"; + ExecStart = "${pkgs.util-linux}/sbin/hwclock --systohc ${if config.time.hardwareClockInLocalTime then "--localtime" else "--utc"}"; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1-init.sh b/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1-init.sh index f7c2940049e5..abc1a0af48a6 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1-init.sh +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1-init.sh @@ -120,7 +120,7 @@ eval "exec $logOutFd>&1 $logErrFd>&2" if test -w /dev/kmsg; then tee -i < /tmp/stage-1-init.log.fifo /proc/self/fd/"$logOutFd" | while read -r line; do if test -n "$line"; then - echo "<7>stage-1-init: $line" > /dev/kmsg + echo "<7>stage-1-init: [$(date)] $line" > /dev/kmsg fi done & else @@ -219,6 +219,9 @@ done @preDeviceCommands@ echo "running udev..." ln -sfn /proc/self/fd /dev/fd +ln -sfn /proc/self/fd/0 /dev/stdin +ln -sfn /proc/self/fd/1 /dev/stdout +ln -sfn /proc/self/fd/2 /dev/stderr mkdir -p /etc/systemd ln -sfn @linkUnits@ /etc/systemd/network mkdir -p /etc/udev @@ -356,6 +359,7 @@ mountFS() { case $options in *x-nixos.autoresize*) if [ "$fsType" = ext2 -o "$fsType" = ext3 -o "$fsType" = ext4 ]; then + modprobe "$fsType" echo "resizing $device..." e2fsck -fp "$device" resize2fs "$device" diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1.nix index 6823e12847c2..0f5787a19210 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-1.nix @@ -107,8 +107,8 @@ let copy_bin_and_libs $BIN done - # Copy some utillinux stuff. - copy_bin_and_libs ${pkgs.utillinux}/sbin/blkid + # Copy some util-linux stuff. + copy_bin_and_libs ${pkgs.util-linux}/sbin/blkid # Copy dmsetup and lvm. copy_bin_and_libs ${getBin pkgs.lvm2}/bin/dmsetup @@ -235,7 +235,7 @@ let --replace scsi_id ${extraUtils}/bin/scsi_id \ --replace cdrom_id ${extraUtils}/bin/cdrom_id \ --replace ${pkgs.coreutils}/bin/basename ${extraUtils}/bin/basename \ - --replace ${pkgs.utillinux}/bin/blkid ${extraUtils}/bin/blkid \ + --replace ${pkgs.util-linux}/bin/blkid ${extraUtils}/bin/blkid \ --replace ${getBin pkgs.lvm2}/bin ${extraUtils}/bin \ --replace ${pkgs.mdadm}/sbin ${extraUtils}/sbin \ --replace ${pkgs.bash}/bin/sh ${extraUtils}/bin/sh \ diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-2.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-2.nix index dd6d83ee0094..94bc34fea0db 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-2.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/stage-2.nix @@ -17,7 +17,7 @@ let inherit (config.system.build) earlyMountScript; path = lib.makeBinPath ([ pkgs.coreutils - pkgs.utillinux + pkgs.util-linux ] ++ lib.optional useHostResolvConf pkgs.openresolv); fsPackagesPath = lib.makeBinPath config.system.fsPackages; postBootCommands = pkgs.writeText "local-cmds" diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd-unit-options.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd-unit-options.nix index 5addc6f9ca44..4154389b2ce5 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd-unit-options.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd-unit-options.nix @@ -210,12 +210,21 @@ in rec { ''; }; + startLimitBurst = mkOption { + type = types.int; + description = '' + Configure unit start rate limiting. Units which are started + more than startLimitBurst times within an interval time + interval are not permitted to start any more. + ''; + }; + startLimitIntervalSec = mkOption { type = types.int; description = '' Configure unit start rate limiting. Units which are started - more than burst times within an interval time interval are - not permitted to start any more. + more than startLimitBurst times within an interval time + interval are not permitted to start any more. ''; }; @@ -245,8 +254,7 @@ in rec { serviceConfig = mkOption { default = {}; example = - { StartLimitInterval = 10; - RestartSec = 5; + { RestartSec = 5; }; type = types.addCheck (types.attrsOf unitOption) checkService; description = '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd.nix b/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd.nix index 74d6957678f5..cbf9e7b49d36 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/system/boot/systemd.nix @@ -243,6 +243,8 @@ let OnFailure = toString config.onFailure; } // optionalAttrs (options.startLimitIntervalSec.isDefined) { StartLimitIntervalSec = toString config.startLimitIntervalSec; + } // optionalAttrs (options.startLimitBurst.isDefined) { + StartLimitBurst = toString config.startLimitBurst; }; }; }; @@ -548,6 +550,14 @@ in ''; }; + systemd.enableUnifiedCgroupHierarchy = mkOption { + default = true; + type = types.bool; + description = '' + Whether to enable the unified cgroup hierarchy (cgroupsv2). + ''; + }; + systemd.coredump.enable = mkOption { default = true; type = types.bool; @@ -884,14 +894,25 @@ in config = { - warnings = concatLists (mapAttrsToList (name: service: - let - type = service.serviceConfig.Type or ""; - restart = service.serviceConfig.Restart or "no"; - in optional - (type == "oneshot" && (restart == "always" || restart == "on-success")) - "Service '${name}.service' with 'Type=oneshot' cannot have 'Restart=always' or 'Restart=on-success'") - cfg.services); + warnings = concatLists ( + mapAttrsToList + (name: service: + let + type = service.serviceConfig.Type or ""; + restart = service.serviceConfig.Restart or "no"; + hasDeprecated = builtins.hasAttr "StartLimitInterval" service.serviceConfig; + in + concatLists [ + (optional (type == "oneshot" && (restart == "always" || restart == "on-success")) + "Service '${name}.service' with 'Type=oneshot' cannot have 'Restart=always' or 'Restart=on-success'" + ) + (optional hasDeprecated + "Service '${name}.service' uses the attribute 'StartLimitInterval' in the Service section, which is deprecated. See https://github.com/NixOS/nixpkgs/issues/45786." + ) + ] + ) + cfg.services + ); system.build.units = cfg.units; @@ -1165,6 +1186,7 @@ in boot.kernel.sysctl = mkIf (!cfg.coredump.enable) { "kernel.core_pattern" = "core"; }; + boot.kernelParams = optional (!cfg.enableUnifiedCgroupHierarchy) "systemd.unified_cgroup_hierarchy=0"; }; # FIXME: Remove these eventually. diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/auto-upgrade.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/auto-upgrade.nix index 69385e5f2fe0..b19b688a1fb8 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/auto-upgrade.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/auto-upgrade.nix @@ -109,9 +109,8 @@ in { ''; }]; - system.autoUpgrade.flags = [ "--no-build-output" ] - ++ (if cfg.flake == null then - (if cfg.channel == null then + system.autoUpgrade.flags = (if cfg.flake == null then + [ "--no-build-output" ] ++ (if cfg.channel == null then [ "--upgrade" ] else [ "-I" diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems.nix index 3ea67dac7146..a055072f9c96 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems.nix @@ -286,7 +286,7 @@ in before = [ mountPoint' "systemd-fsck@${device'}.service" ]; requires = [ device'' ]; after = [ device'' ]; - path = [ pkgs.utillinux ] ++ config.system.fsPackages; + path = [ pkgs.util-linux ] ++ config.system.fsPackages; script = '' if ! [ -e "${fs.device}" ]; then exit 1; fi diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/nfs.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/nfs.nix index ddcc0ed8f5a4..fd35c35d32ad 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/nfs.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/nfs.nix @@ -10,20 +10,9 @@ let rpcMountpoint = "${nfsStateDir}/rpc_pipefs"; - idmapdConfFile = pkgs.writeText "idmapd.conf" '' - [General] - Pipefs-Directory = ${rpcMountpoint} - ${optionalString (config.networking.domain != null) - "Domain = ${config.networking.domain}"} - - [Mapping] - Nobody-User = nobody - Nobody-Group = nogroup - - [Translation] - Method = nsswitch - ''; + format = pkgs.formats.ini {}; + idmapdConfFile = format.generate "idmapd.conf" cfg.idmapd.settings; nfsConfFile = pkgs.writeText "nfs.conf" cfg.extraConfig; requestKeyConfFile = pkgs.writeText "request-key.conf" '' create id_resolver * * ${pkgs.nfs-utils}/bin/nfsidmap -t 600 %k %d @@ -38,6 +27,25 @@ in options = { services.nfs = { + idmapd.settings = mkOption { + type = format.type; + default = {}; + description = '' + libnfsidmap configuration. Refer to + <link xlink:href="https://linux.die.net/man/5/idmapd.conf"/> + for details. + ''; + example = literalExample '' + { + Translation = { + GSS-Methods = "static,nsswitch"; + }; + Static = { + "root/hostname.domain.com@REALM.COM" = "root"; + }; + } + ''; + }; extraConfig = mkOption { type = types.lines; default = ""; @@ -54,6 +62,20 @@ in services.rpcbind.enable = true; + services.nfs.idmapd.settings = { + General = mkMerge [ + { Pipefs-Directory = rpcMountpoint; } + (mkIf (config.networking.domain != null) { Domain = config.networking.domain; }) + ]; + Mapping = { + Nobody-User = "nobody"; + Nobody-Group = "nogroup"; + }; + Translation = { + Method = "nsswitch"; + }; + }; + system.fsPackages = [ pkgs.nfs-utils ]; boot.initrd.kernelModules = mkIf inInitrd [ "nfs" ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/unionfs-fuse.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/unionfs-fuse.nix index 1dcc4c87e3ce..f54f3559c341 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/unionfs-fuse.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/unionfs-fuse.nix @@ -18,9 +18,9 @@ boot.initrd.postDeviceCommands = '' # Hacky!!! fuse hard-codes the path to mount - mkdir -p /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-${pkgs.utillinux.name}-bin/bin - ln -s $(which mount) /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-${pkgs.utillinux.name}-bin/bin - ln -s $(which umount) /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-${pkgs.utillinux.name}-bin/bin + mkdir -p /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-${pkgs.util-linux.name}-bin/bin + ln -s $(which mount) /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-${pkgs.util-linux.name}-bin/bin + ln -s $(which umount) /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-${pkgs.util-linux.name}-bin/bin ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/zfs.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/zfs.nix index 9ca7c6fb3431..6becc6962735 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/zfs.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/filesystems/zfs.nix @@ -175,14 +175,10 @@ in forceImportAll = mkOption { type = types.bool; - default = true; + default = false; description = '' Forcibly import all ZFS pool(s). - This is enabled by default for backwards compatibility purposes, but it is highly - recommended to disable this option, as it bypasses some of the safeguards ZFS uses - to protect your ZFS pools. - If you set this option to <literal>false</literal> and NixOS subsequently fails to import your non-root ZFS pool(s), you should manually import each pool with "zpool import -f <pool-name>", and then reboot. You should only need to do @@ -444,7 +440,7 @@ in pkgs.gnugrep pkgs.gnused pkgs.nettools - pkgs.utillinux + pkgs.util-linux ]; }; @@ -507,6 +503,7 @@ in Type = "oneshot"; RemainAfterExit = true; }; + environment.ZFS_FORCE = optionalString cfgZfs.forceImportAll "-f"; script = (importLib { # See comments at importLib definition. zpoolCmd="${packages.zfsUser}/sbin/zpool"; diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/lvm.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/lvm.nix index 2c3cc4c5467d..98a0e2ddef90 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/lvm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/lvm.nix @@ -21,6 +21,10 @@ in { }; config = mkMerge [ + ({ + # minimal configuration file to make lvmconfig/lvm2-activation-generator happy + environment.etc."lvm/lvm.conf".text = "config {}"; + }) (mkIf (!config.boot.isContainer) { systemd.tmpfiles.packages = [ cfg.package.out ]; environment.systemPackages = [ cfg.package ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/tasks/network-interfaces.nix b/infra/libkookie/nixpkgs/nixos/modules/tasks/network-interfaces.nix index e5bd57753683..afb9c5404169 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/tasks/network-interfaces.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/tasks/network-interfaces.nix @@ -1062,7 +1062,6 @@ in ]; boot.kernelModules = [ ] - ++ optional cfg.enableIPv6 "ipv6" ++ optional hasVirtuals "tun" ++ optional hasSits "sit" ++ optional hasBonds "bonding"; @@ -1247,7 +1246,7 @@ in ''; # Udev attributes for systemd to name the device and to create a .device target. - systemdAttrs = n: ''NAME:="${n}", ENV{INTERFACE}:="${n}", ENV{SYSTEMD_ALIAS}:="/sys/subsystem/net/devices/${n}", TAG+="systemd"''; + systemdAttrs = n: ''NAME:="${n}", ENV{INTERFACE}="${n}", ENV{SYSTEMD_ALIAS}="/sys/subsystem/net/devices/${n}", TAG+="systemd"''; in flip (concatMapStringsSep "\n") (attrNames wlanDeviceInterfaces) (device: let diff --git a/infra/libkookie/nixpkgs/nixos/modules/testing/test-instrumentation.nix b/infra/libkookie/nixpkgs/nixos/modules/testing/test-instrumentation.nix index c0ec76e8a3a3..be5fa88b8ade 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/testing/test-instrumentation.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/testing/test-instrumentation.nix @@ -45,13 +45,22 @@ with import ../../lib/qemu-flags.nix { inherit pkgs; }; systemd.services."serial-getty@${qemuSerialDevice}".enable = false; systemd.services."serial-getty@hvc0".enable = false; - # Only use a serial console, no TTY. - # NOTE: optionalAttrs - # test-instrumentation.nix appears to be used without qemu-vm.nix, so - # we avoid defining consoles if not possible. - # TODO: refactor such that test-instrumentation can import qemu-vm - # or declare virtualisation.qemu.console option in a module that's always imported - virtualisation = lib.optionalAttrs (options ? virtualisation.qemu.consoles) { qemu.consoles = [ qemuSerialDevice ]; }; + # Only set these settings when the options exist. Some tests (e.g. those + # that do not specify any nodes, or an empty attr set as nodes) will not + # have the QEMU module loaded and thuse these options can't and should not + # be set. + virtualisation = lib.optionalAttrs (options ? virtualisation.qemu) { + qemu = { + # Only use a serial console, no TTY. + # NOTE: optionalAttrs + # test-instrumentation.nix appears to be used without qemu-vm.nix, so + # we avoid defining consoles if not possible. + # TODO: refactor such that test-instrumentation can import qemu-vm + # or declare virtualisation.qemu.console option in a module that's always imported + consoles = [ qemuSerialDevice ]; + package = lib.mkDefault pkgs.qemu_test; + }; + }; boot.initrd.preDeviceCommands = '' @@ -116,6 +125,10 @@ with import ../../lib/qemu-flags.nix { inherit pkgs; }; users.users.root.initialHashedPassword = mkOverride 150 ""; services.xserver.displayManager.job.logToJournal = true; + + # Make sure we use the Guest Agent from the QEMU package for testing + # to reduce the closure size required for the tests. + services.qemuGuest.package = pkgs.qemu_test.ga; }; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/amazon-image.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/amazon-image.nix index 20d48add7129..26297a7d0f1f 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/amazon-image.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/amazon-image.nix @@ -11,6 +11,7 @@ with lib; let cfg = config.ec2; metadataFetcher = import ./ec2-metadata-fetcher.nix { + inherit (pkgs) curl; targetRoot = "$targetRoot/"; wgetExtraOptions = "-q"; }; @@ -48,7 +49,7 @@ in ]; boot.initrd.kernelModules = [ "xen-blkfront" "xen-netfront" ]; boot.initrd.availableKernelModules = [ "ixgbevf" "ena" "nvme" ]; - boot.kernelParams = mkIf cfg.hvm [ "console=ttyS0" ]; + boot.kernelParams = mkIf cfg.hvm [ "console=ttyS0" "random.trust_cpu=on" ]; # Prevent the nouveau kernel module from being loaded, as it # interferes with the nvidia/nvidia-uvm modules needed for CUDA. @@ -123,7 +124,7 @@ in boot.initrd.extraUtilsCommands = '' # We need swapon in the initrd. - copy_bin_and_libs ${pkgs.utillinux}/sbin/swapon + copy_bin_and_libs ${pkgs.util-linux}/sbin/swapon ''; # Don't put old configurations in the GRUB menu. The user has no diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/azure-agent.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/azure-agent.nix index e85482af8392..81413792eda0 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/azure-agent.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/azure-agent.nix @@ -22,7 +22,7 @@ let nettools # for hostname procps # for pidof shadow # for useradd, usermod - utillinux # for (u)mount, fdisk, sfdisk, mkswap + util-linux # for (u)mount, fdisk, sfdisk, mkswap parted ]; pythonPath = [ pythonPackages.pyasn1 ]; diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/brightbox-image.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/brightbox-image.nix index d0efbcc808aa..4498e3a73618 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/brightbox-image.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/brightbox-image.nix @@ -27,7 +27,7 @@ in popd ''; diskImageBase = "nixos-image-${config.system.nixos.label}-${pkgs.stdenv.hostPlatform.system}.raw"; - buildInputs = [ pkgs.utillinux pkgs.perl ]; + buildInputs = [ pkgs.util-linux pkgs.perl ]; exportReferencesGraph = [ "closure" config.system.build.toplevel ]; } diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/docker.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/docker.nix index d87ada35a0ae..ec257801b330 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/docker.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/docker.nix @@ -155,6 +155,9 @@ in users.groups.docker.gid = config.ids.gids.docker; systemd.packages = [ cfg.package ]; + # TODO: remove once docker 20.10 is released + systemd.enableUnifiedCgroupHierarchy = false; + systemd.services.docker = { wantedBy = optional cfg.enableOnBoot "multi-user.target"; environment = proxy_env; diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-amis.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-amis.nix index 24de8cf1afbf..3da63078a214 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-amis.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-amis.nix @@ -329,5 +329,24 @@ let self = { "20.03".ap-east-1.hvm-ebs = "ami-0d18fdd309cdefa86"; "20.03".sa-east-1.hvm-ebs = "ami-09859378158ae971d"; - latest = self."20.03"; + # 20.09.1632.a6a3a368dda + "20.09".eu-west-1.hvm-ebs = "ami-01a79d5ce435f4db3"; + "20.09".eu-west-2.hvm-ebs = "ami-0cbe14f32904e6331"; + "20.09".eu-west-3.hvm-ebs = "ami-07f493412d6213de6"; + "20.09".eu-central-1.hvm-ebs = "ami-01d4a0c2248cbfe38"; + "20.09".eu-north-1.hvm-ebs = "ami-0003f54dd99d68e0f"; + "20.09".us-east-1.hvm-ebs = "ami-068a62d478710462d"; + "20.09".us-east-2.hvm-ebs = "ami-01ac677ff61399caa"; + "20.09".us-west-1.hvm-ebs = "ami-04befdb203b4b17f6"; + "20.09".us-west-2.hvm-ebs = "ami-0fb7bd4a43261c6b2"; + "20.09".ca-central-1.hvm-ebs = "ami-06d5ee429f153f856"; + "20.09".ap-southeast-1.hvm-ebs = "ami-0db0304e23c535b2a"; + "20.09".ap-southeast-2.hvm-ebs = "ami-045983c4db7e36447"; + "20.09".ap-northeast-1.hvm-ebs = "ami-0beb18d632cf64e5a"; + "20.09".ap-northeast-2.hvm-ebs = "ami-0dd0316af578862db"; + "20.09".ap-south-1.hvm-ebs = "ami-008d15ced81c88aed"; + "20.09".ap-east-1.hvm-ebs = "ami-071f49713f86ea965"; + "20.09".sa-east-1.hvm-ebs = "ami-05ded1ae35209b5a8"; + + latest = self."20.09"; }; in self diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-metadata-fetcher.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-metadata-fetcher.nix index b531787c31a2..dca5c2abd4e0 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-metadata-fetcher.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/ec2-metadata-fetcher.nix @@ -1,23 +1,77 @@ -{ targetRoot, wgetExtraOptions }: +{ curl, targetRoot, wgetExtraOptions }: +# Note: be very cautious about dependencies, each dependency grows +# the closure of the initrd. Ideally we would not even require curl, +# but there is no reasonable way to send an HTTP PUT request without +# it. Note: do not be fooled: the wget referenced in this script +# is busybox's wget, not the fully featured one with --method support. +# +# Make sure that every package you depend on here is already listed as +# a channel blocker for both the full-sized and small channels. +# Otherwise, we risk breaking user deploys in released channels. +# +# Also note: OpenStack's metadata service for its instances aims to be +# compatible with the EC2 IMDS. Where possible, try to keep the set of +# fetched metadata in sync with ./openstack-metadata-fetcher.nix . '' metaDir=${targetRoot}etc/ec2-metadata mkdir -m 0755 -p "$metaDir" + rm -f "$metaDir/*" - echo "getting EC2 instance metadata..." + get_imds_token() { + # retry-delay of 1 selected to give the system a second to get going, + # but not add a lot to the bootup time + ${curl}/bin/curl \ + -v \ + --retry 3 \ + --retry-delay 1 \ + --fail \ + -X PUT \ + --connect-timeout 1 \ + -H "X-aws-ec2-metadata-token-ttl-seconds: 600" \ + http://169.254.169.254/latest/api/token + } - if ! [ -e "$metaDir/ami-manifest-path" ]; then - wget ${wgetExtraOptions} -O "$metaDir/ami-manifest-path" http://169.254.169.254/1.0/meta-data/ami-manifest-path - fi + preflight_imds_token() { + # retry-delay of 1 selected to give the system a second to get going, + # but not add a lot to the bootup time + ${curl}/bin/curl \ + -v \ + --retry 3 \ + --retry-delay 1 \ + --fail \ + --connect-timeout 1 \ + -H "X-aws-ec2-metadata-token: $IMDS_TOKEN" \ + http://169.254.169.254/1.0/meta-data/instance-id + } - if ! [ -e "$metaDir/user-data" ]; then - wget ${wgetExtraOptions} -O "$metaDir/user-data" http://169.254.169.254/1.0/user-data && chmod 600 "$metaDir/user-data" - fi + try=1 + while [ $try -le 3 ]; do + echo "(attempt $try/3) getting an EC2 instance metadata service v2 token..." + IMDS_TOKEN=$(get_imds_token) && break + try=$((try + 1)) + sleep 1 + done - if ! [ -e "$metaDir/hostname" ]; then - wget ${wgetExtraOptions} -O "$metaDir/hostname" http://169.254.169.254/1.0/meta-data/hostname + if [ "x$IMDS_TOKEN" == "x" ]; then + echo "failed to fetch an IMDS2v token." fi - if ! [ -e "$metaDir/public-keys-0-openssh-key" ]; then - wget ${wgetExtraOptions} -O "$metaDir/public-keys-0-openssh-key" http://169.254.169.254/1.0/meta-data/public-keys/0/openssh-key - fi + try=1 + while [ $try -le 10 ]; do + echo "(attempt $try/10) validating the EC2 instance metadata service v2 token..." + preflight_imds_token && break + try=$((try + 1)) + sleep 1 + done + + echo "getting EC2 instance metadata..." + + wget_imds() { + wget ${wgetExtraOptions} --header "X-aws-ec2-metadata-token: $IMDS_TOKEN" "$@"; + } + + wget_imds -O "$metaDir/ami-manifest-path" http://169.254.169.254/1.0/meta-data/ami-manifest-path + wget_imds -O "$metaDir/user-data" http://169.254.169.254/1.0/user-data && chmod 600 "$metaDir/user-data" + wget_imds -O "$metaDir/hostname" http://169.254.169.254/1.0/meta-data/hostname + wget_imds -O "$metaDir/public-keys-0-openssh-key" http://169.254.169.254/1.0/meta-data/public-keys/0/openssh-key '' diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/nixos-containers.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/nixos-containers.nix index 8fbb4efd2019..26398afb3cf5 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/nixos-containers.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/nixos-containers.nix @@ -614,17 +614,17 @@ in ''; }; - timeoutStartSec = mkOption { - type = types.str; - default = "1min"; - description = '' - Time for the container to start. In case of a timeout, - the container processes get killed. - See <citerefentry><refentrytitle>systemd.time</refentrytitle> - <manvolnum>7</manvolnum></citerefentry> - for more information about the format. - ''; - }; + timeoutStartSec = mkOption { + type = types.str; + default = "1min"; + description = '' + Time for the container to start. In case of a timeout, + the container processes get killed. + See <citerefentry><refentrytitle>systemd.time</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> + for more information about the format. + ''; + }; bindMounts = mkOption { type = with types; attrsOf (submodule bindMountOpts); diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-config.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-config.nix index c2da5d0d2301..d01e0f23aba1 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-config.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-config.nix @@ -3,7 +3,7 @@ with lib; let - metadataFetcher = import ./ec2-metadata-fetcher.nix { + metadataFetcher = import ./openstack-metadata-fetcher.nix { targetRoot = "/"; wgetExtraOptions = "--retry-connrefused"; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-metadata-fetcher.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-metadata-fetcher.nix new file mode 100644 index 000000000000..8c191397cf9a --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/openstack-metadata-fetcher.nix @@ -0,0 +1,21 @@ +{ targetRoot, wgetExtraOptions }: + +# OpenStack's metadata service aims to be EC2-compatible. Where +# possible, try to keep the set of fetched metadata in sync with +# ./ec2-metadata-fetcher.nix . +'' + metaDir=${targetRoot}etc/ec2-metadata + mkdir -m 0755 -p "$metaDir" + rm -f "$metaDir/*" + + echo "getting instance metadata..." + + wget_imds() { + wget ${wgetExtraOptions} "$@" + } + + wget_imds -O "$metaDir/ami-manifest-path" http://169.254.169.254/1.0/meta-data/ami-manifest-path + wget_imds -O "$metaDir/user-data" http://169.254.169.254/1.0/user-data && chmod 600 "$metaDir/user-data" + wget_imds -O "$metaDir/hostname" http://169.254.169.254/1.0/meta-data/hostname + wget_imds -O "$metaDir/public-keys-0-openssh-key" http://169.254.169.254/1.0/meta-data/public-keys/0/openssh-key +'' diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-guest-agent.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-guest-agent.nix index 665224e35d8c..6a735f451a7e 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-guest-agent.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-guest-agent.nix @@ -12,6 +12,11 @@ in { default = false; description = "Whether to enable the qemu guest agent."; }; + package = mkOption { + type = types.package; + default = pkgs.qemu.ga; + description = "The QEMU guest agent package."; + }; }; config = mkIf cfg.enable ( @@ -25,7 +30,7 @@ in { systemd.services.qemu-guest-agent = { description = "Run the QEMU Guest Agent"; serviceConfig = { - ExecStart = "${pkgs.qemu.ga}/bin/qemu-ga"; + ExecStart = "${cfg.package}/bin/qemu-ga"; Restart = "always"; RestartSec = 0; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix index 42e43f5ee023..447d1f091c8c 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/qemu-vm.nix @@ -14,10 +14,11 @@ with import ../../lib/qemu-flags.nix { inherit pkgs; }; let - qemu = config.system.build.qemu or pkgs.qemu_test; cfg = config.virtualisation; + qemu = cfg.qemu.package; + consoles = lib.concatMapStringsSep " " (c: "console=${c}") cfg.qemu.consoles; driveOpts = { ... }: { @@ -189,7 +190,7 @@ let '' else '' ''} ''; - buildInputs = [ pkgs.utillinux ]; + buildInputs = [ pkgs.util-linux ]; QEMU_OPTS = "-nographic -serial stdio -monitor none" + lib.optionalString cfg.useEFIBoot ( " -drive if=pflash,format=raw,unit=0,readonly=on,file=${efiFirmware}" @@ -401,6 +402,14 @@ in }; virtualisation.qemu = { + package = + mkOption { + type = types.package; + default = pkgs.qemu; + example = "pkgs.qemu_test"; + description = "QEMU package to use."; + }; + options = mkOption { type = types.listOf types.unspecified; @@ -735,16 +744,19 @@ in (isEnabled "VIRTIO_PCI") (isEnabled "VIRTIO_NET") (isEnabled "EXT4_FS") + (isEnabled "NET_9P_VIRTIO") + (isEnabled "9P_FS") (isYes "BLK_DEV") (isYes "PCI") - (isYes "EXPERIMENTAL") (isYes "NETDEVICES") (isYes "NET_CORE") (isYes "INET") (isYes "NETWORK_FILESYSTEMS") - ] ++ optional (!cfg.graphics) [ + ] ++ optionals (!cfg.graphics) [ (isYes "SERIAL_8250_CONSOLE") (isYes "SERIAL_8250") + ] ++ optionals (cfg.writableStore) [ + (isEnabled "OVERLAY_FS") ]; }; diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/vagrant-guest.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/vagrant-guest.nix new file mode 100644 index 000000000000..263b1ebca086 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/vagrant-guest.nix @@ -0,0 +1,58 @@ +# Minimal configuration that vagrant depends on + +{ config, pkgs, ... }: +let + # Vagrant uses an insecure shared private key by default, but we + # don't use the authorizedKeys attribute under users because it should be + # removed on first boot and replaced with a random one. This script sets + # the correct permissions and installs the temporary key if no + # ~/.ssh/authorized_keys exists. + install-vagrant-ssh-key = pkgs.writeScriptBin "install-vagrant-ssh-key" '' + #!${pkgs.runtimeShell} + if [ ! -e ~/.ssh/authorized_keys ]; then + mkdir -m 0700 -p ~/.ssh + echo "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key" >> ~/.ssh/authorized_keys + chmod 0600 ~/.ssh/authorized_keys + fi + ''; +in +{ + # Enable the OpenSSH daemon. + services.openssh.enable = true; + + # Packages used by Vagrant + environment.systemPackages = with pkgs; [ + findutils + iputils + nettools + netcat + nfs-utils + rsync + ]; + + users.extraUsers.vagrant = { + isNormalUser = true; + createHome = true; + description = "Vagrant user account"; + extraGroups = [ "users" "wheel" ]; + home = "/home/vagrant"; + password = "vagrant"; + useDefaultShell = true; + uid = 1000; + }; + + systemd.services.install-vagrant-ssh-key = { + description = "Vagrant SSH key install (if needed)"; + after = [ "fs.target" ]; + wants = [ "fs.target" ]; + wantedBy = [ "multi-user.target" ]; + serviceConfig = { + ExecStart = "${install-vagrant-ssh-key}/bin/install-vagrant-ssh-key"; + User = "vagrant"; + # So it won't be (needlessly) restarted: + RemainAfterExit = true; + }; + }; + + security.sudo.wheelNeedsPassword = false; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/vagrant-virtualbox-image.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/vagrant-virtualbox-image.nix new file mode 100644 index 000000000000..2a921894ab61 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/vagrant-virtualbox-image.nix @@ -0,0 +1,60 @@ +# Vagrant + VirtualBox + +{ config, pkgs, ... }: + +{ + imports = [ + ./vagrant-guest.nix + ./virtualbox-image.nix + ]; + + virtualbox.params = { + audio = "none"; + audioin = "off"; + audioout = "off"; + usb = "off"; + usbehci = "off"; + }; + sound.enable = false; + documentation.man.enable = false; + documentation.nixos.enable = false; + + users.extraUsers.vagrant.extraGroups = [ "vboxsf" ]; + + # generate the box v1 format which is much easier to generate + # https://www.vagrantup.com/docs/boxes/format.html + system.build.vagrantVirtualbox = pkgs.runCommand + "virtualbox-vagrant.box" + {} + '' + mkdir workdir + cd workdir + + # 1. create that metadata.json file + echo '{"provider":"virtualbox"}' > metadata.json + + # 2. create a default Vagrantfile config + cat <<VAGRANTFILE > Vagrantfile + Vagrant.configure("2") do |config| + config.vm.base_mac = "0800275F0936" + end + VAGRANTFILE + + # 3. add the exported VM files + tar xvf ${config.system.build.virtualBoxOVA}/*.ova + + # 4. move the ovf to the fixed location + mv *.ovf box.ovf + + # 5. generate OVF manifest file + rm *.mf + touch box.mf + for fname in *; do + checksum=$(sha256sum $fname | cut -d' ' -f 1) + echo "SHA256($fname)= $checksum" >> box.mf + done + + # 6. compress everything back together + tar --owner=0 --group=0 --sort=name --numeric-owner -czf $out . + ''; +} diff --git a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/xen-dom0.nix b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/xen-dom0.nix index 7b2a66c43489..5ad647769bbd 100644 --- a/infra/libkookie/nixpkgs/nixos/modules/virtualisation/xen-dom0.nix +++ b/infra/libkookie/nixpkgs/nixos/modules/virtualisation/xen-dom0.nix @@ -201,8 +201,8 @@ in '' if [ -d /proc/xen ]; then ${pkgs.kmod}/bin/modprobe xenfs 2> /dev/null - ${pkgs.utillinux}/bin/mountpoint -q /proc/xen || \ - ${pkgs.utillinux}/bin/mount -t xenfs none /proc/xen + ${pkgs.util-linux}/bin/mountpoint -q /proc/xen || \ + ${pkgs.util-linux}/bin/mount -t xenfs none /proc/xen fi ''; diff --git a/infra/libkookie/nixpkgs/nixos/release-combined.nix b/infra/libkookie/nixpkgs/nixos/release-combined.nix index 945ba90e3454..5f8a64702a82 100644 --- a/infra/libkookie/nixpkgs/nixos/release-combined.nix +++ b/infra/libkookie/nixpkgs/nixos/release-combined.nix @@ -49,6 +49,7 @@ in rec { [ "nixos.channel" ] (onFullSupported "nixos.dummy") (onAllSupported "nixos.iso_minimal") + (onSystems ["x86_64-linux" "aarch64-linux"] "nixos.amazonImage") (onSystems ["x86_64-linux"] "nixos.iso_plasma5") (onSystems ["x86_64-linux"] "nixos.iso_gnome") (onFullSupported "nixos.manual") @@ -71,7 +72,6 @@ in rec { (onFullSupported "nixos.tests.fontconfig-default-fonts") (onFullSupported "nixos.tests.gnome3") (onFullSupported "nixos.tests.gnome3-xorg") - (onFullSupported "nixos.tests.hardened") (onSystems ["x86_64-linux"] "nixos.tests.hibernate") (onFullSupported "nixos.tests.i3wm") (onSystems ["x86_64-linux"] "nixos.tests.installer.btrfsSimple") @@ -93,12 +93,12 @@ in rec { (onFullSupported "nixos.tests.keymap.dvp") (onFullSupported "nixos.tests.keymap.neo") (onFullSupported "nixos.tests.keymap.qwertz") - (onFullSupported "nixos.tests.latestKernel.hardened") (onFullSupported "nixos.tests.latestKernel.login") (onFullSupported "nixos.tests.lightdm") (onFullSupported "nixos.tests.login") (onFullSupported "nixos.tests.misc") (onFullSupported "nixos.tests.mutableUsers") + (onFullSupported "nixos.tests.nano") (onFullSupported "nixos.tests.nat.firewall-conntrack") (onFullSupported "nixos.tests.nat.firewall") (onFullSupported "nixos.tests.nat.standalone") diff --git a/infra/libkookie/nixpkgs/nixos/release-small.nix b/infra/libkookie/nixpkgs/nixos/release-small.nix index 6da2c59cedd4..996db54c9a40 100644 --- a/infra/libkookie/nixpkgs/nixos/release-small.nix +++ b/infra/libkookie/nixpkgs/nixos/release-small.nix @@ -28,7 +28,7 @@ let in rec { nixos = { - inherit (nixos') channel manual options iso_minimal dummy; + inherit (nixos') channel manual options iso_minimal amazonImage dummy; tests = { inherit (nixos'.tests) containers-imperative @@ -92,6 +92,7 @@ in rec { [ "nixos.channel" "nixos.dummy.x86_64-linux" "nixos.iso_minimal.x86_64-linux" + "nixos.amazonImage.x86_64-linux" "nixos.manual.x86_64-linux" "nixos.tests.boot.biosCdrom.x86_64-linux" "nixos.tests.containers-imperative.x86_64-linux" diff --git a/infra/libkookie/nixpkgs/nixos/tests/all-tests.nix b/infra/libkookie/nixpkgs/nixos/tests/all-tests.nix index 6564a958d5b4..c58203cc481f 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/all-tests.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/all-tests.nix @@ -24,10 +24,13 @@ in _3proxy = handleTest ./3proxy.nix {}; acme = handleTest ./acme.nix {}; agda = handleTest ./agda.nix {}; + ammonite = handleTest ./ammonite.nix {}; atd = handleTest ./atd.nix {}; avahi = handleTest ./avahi.nix {}; avahi-with-resolved = handleTest ./avahi.nix { networkd = true; }; + awscli = handleTest ./awscli.nix { }; babeld = handleTest ./babeld.nix {}; + bat = handleTest ./bat.nix {}; bazarr = handleTest ./bazarr.nix {}; bcachefs = handleTestOn ["x86_64-linux"] ./bcachefs.nix {}; # linux-4.18.2018.10.12 is unsupported on aarch64 beanstalkd = handleTest ./beanstalkd.nix {}; @@ -37,18 +40,18 @@ in bittorrent = handleTest ./bittorrent.nix {}; bitwarden = handleTest ./bitwarden.nix {}; blockbook-frontend = handleTest ./blockbook-frontend.nix {}; - buildkite-agents = handleTest ./buildkite-agents.nix {}; boot = handleTestOn ["x86_64-linux"] ./boot.nix {}; # syslinux is unsupported on aarch64 boot-stage1 = handleTest ./boot-stage1.nix {}; borgbackup = handleTest ./borgbackup.nix {}; buildbot = handleTest ./buildbot.nix {}; + buildkite-agents = handleTest ./buildkite-agents.nix {}; caddy = handleTest ./caddy.nix {}; cadvisor = handleTestOn ["x86_64-linux"] ./cadvisor.nix {}; cage = handleTest ./cage.nix {}; cagebreak = handleTest ./cagebreak.nix {}; cassandra = handleTest ./cassandra.nix {}; - ceph-single-node = handleTestOn ["x86_64-linux"] ./ceph-single-node.nix {}; ceph-multi-node = handleTestOn ["x86_64-linux"] ./ceph-multi-node.nix {}; + ceph-single-node = handleTestOn ["x86_64-linux"] ./ceph-single-node.nix {}; certmgr = handleTest ./certmgr.nix {}; cfssl = handleTestOn ["x86_64-linux"] ./cfssl.nix {}; charliecloud = handleTest ./charliecloud.nix {}; @@ -56,9 +59,9 @@ in cjdns = handleTest ./cjdns.nix {}; clickhouse = handleTest ./clickhouse.nix {}; cloud-init = handleTest ./cloud-init.nix {}; + cockroachdb = handleTestOn ["x86_64-linux"] ./cockroachdb.nix {}; codimd = handleTest ./codimd.nix {}; consul = handleTest ./consul.nix {}; - cockroachdb = handleTestOn ["x86_64-linux"] ./cockroachdb.nix {}; containers-bridge = handleTest ./containers-bridge.nix {}; containers-custom-pkgs.nix = handleTest ./containers-custom-pkgs.nix {}; containers-ephemeral = handleTest ./containers-ephemeral.nix {}; @@ -82,7 +85,6 @@ in dnscrypt-wrapper = handleTestOn ["x86_64-linux"] ./dnscrypt-wrapper {}; doas = handleTest ./doas.nix {}; docker = handleTestOn ["x86_64-linux"] ./docker.nix {}; - oci-containers = handleTestOn ["x86_64-linux"] ./oci-containers.nix {}; docker-edge = handleTestOn ["x86_64-linux"] ./docker-edge.nix {}; docker-registry = handleTest ./docker-registry.nix {}; docker-tools = handleTestOn ["x86_64-linux"] ./docker-tools.nix {}; @@ -102,6 +104,7 @@ in etcd = handleTestOn ["x86_64-linux"] ./etcd.nix {}; etcd-cluster = handleTestOn ["x86_64-linux"] ./etcd-cluster.nix {}; fancontrol = handleTest ./fancontrol.nix {}; + fcitx = handleTest ./fcitx {}; ferm = handleTest ./ferm.nix {}; firefox = handleTest ./firefox.nix {}; firefox-esr = handleTest ./firefox.nix { esr = true; }; @@ -115,24 +118,23 @@ in fsck = handleTest ./fsck.nix {}; ft2-clone = handleTest ./ft2-clone.nix {}; gerrit = handleTest ./gerrit.nix {}; - gotify-server = handleTest ./gotify-server.nix {}; - grocy = handleTest ./grocy.nix {}; gitdaemon = handleTest ./gitdaemon.nix {}; gitea = handleTest ./gitea.nix {}; gitlab = handleTest ./gitlab.nix {}; gitolite = handleTest ./gitolite.nix {}; gitolite-fcgiwrap = handleTest ./gitolite-fcgiwrap.nix {}; glusterfs = handleTest ./glusterfs.nix {}; - gnome3-xorg = handleTest ./gnome3-xorg.nix {}; gnome3 = handleTest ./gnome3.nix {}; - installed-tests = pkgs.recurseIntoAttrs (handleTest ./installed-tests {}); + gnome3-xorg = handleTest ./gnome3-xorg.nix {}; + go-neb = handleTest ./go-neb.nix {}; gocd-agent = handleTest ./gocd-agent.nix {}; gocd-server = handleTest ./gocd-server.nix {}; - go-neb = handleTest ./go-neb.nix {}; google-oslogin = handleTest ./google-oslogin {}; + gotify-server = handleTest ./gotify-server.nix {}; grafana = handleTest ./grafana.nix {}; graphite = handleTest ./graphite.nix {}; graylog = handleTest ./graylog.nix {}; + grocy = handleTest ./grocy.nix {}; grub = handleTest ./grub.nix {}; gvisor = handleTest ./gvisor.nix {}; hadoop.hdfs = handleTestOn [ "x86_64-linux" ] ./hadoop/hdfs.nix {}; @@ -140,6 +142,8 @@ in handbrake = handleTestOn ["x86_64-linux"] ./handbrake.nix {}; haproxy = handleTest ./haproxy.nix {}; hardened = handleTest ./hardened.nix {}; + installed-tests = pkgs.recurseIntoAttrs (handleTest ./installed-tests {}); + oci-containers = handleTestOn ["x86_64-linux"] ./oci-containers.nix {}; # 9pnet_virtio used to mount /nix partition doesn't support # hibernation. This test happens to work on x86_64-linux but # not on other platforms. @@ -150,15 +154,14 @@ in hostname = handleTest ./hostname.nix {}; hound = handleTest ./hound.nix {}; hydra = handleTest ./hydra {}; - hydra-db-migration = handleTest ./hydra/db-migration.nix {}; i3wm = handleTest ./i3wm.nix {}; icingaweb2 = handleTest ./icingaweb2.nix {}; iftop = handleTest ./iftop.nix {}; ihatemoney = handleTest ./ihatemoney.nix {}; incron = handleTest ./incron.nix {}; influxdb = handleTest ./influxdb.nix {}; - initrd-network-ssh = handleTest ./initrd-network-ssh {}; initrd-network-openvpn = handleTest ./initrd-network-openvpn {}; + initrd-network-ssh = handleTest ./initrd-network-ssh {}; initrdNetwork = handleTest ./initrd-network.nix {}; installer = handleTest ./installer.nix {}; iodine = handleTest ./iodine.nix {}; @@ -169,6 +172,7 @@ in jenkins = handleTest ./jenkins.nix {}; jirafeau = handleTest ./jirafeau.nix {}; jitsi-meet = handleTest ./jitsi-meet.nix {}; + jq = handleTest ./jq.nix {}; k3s = handleTest ./k3s.nix {}; kafka = handleTest ./kafka.nix {}; keepalived = handleTest ./keepalived.nix {}; @@ -176,6 +180,7 @@ in kernel-latest = handleTest ./kernel-latest.nix {}; kernel-lts = handleTest ./kernel-lts.nix {}; kernel-testing = handleTest ./kernel-testing.nix {}; + keycloak = discoverTests (import ./keycloak.nix); keymap = handleTest ./keymap.nix {}; knot = handleTest ./knot.nix {}; krb5 = discoverTests (import ./krb5 {}); @@ -191,12 +196,13 @@ in limesurvey = handleTest ./limesurvey.nix {}; login = handleTest ./login.nix {}; loki = handleTest ./loki.nix {}; + lsd = handleTest ./lsd.nix {}; lxd = handleTest ./lxd.nix {}; lxd-nftables = handleTest ./lxd-nftables.nix {}; #logstash = handleTest ./logstash.nix {}; lorri = handleTest ./lorri/default.nix {}; - magnetico = handleTest ./magnetico.nix {}; magic-wormhole-mailbox-server = handleTest ./magic-wormhole-mailbox-server.nix {}; + magnetico = handleTest ./magnetico.nix {}; mailcatcher = handleTest ./mailcatcher.nix {}; mariadb-galera-mariabackup = handleTest ./mysql/mariadb-galera-mariabackup.nix {}; mariadb-galera-rsync = handleTest ./mysql/mariadb-galera-rsync.nix {}; @@ -205,9 +211,11 @@ in mediawiki = handleTest ./mediawiki.nix {}; memcached = handleTest ./memcached.nix {}; metabase = handleTest ./metabase.nix {}; + minecraft = handleTest ./minecraft.nix {}; + minecraft-server = handleTest ./minecraft-server.nix {}; + minidlna = handleTest ./minidlna.nix {}; miniflux = handleTest ./miniflux.nix {}; minio = handleTest ./minio.nix {}; - minidlna = handleTest ./minidlna.nix {}; misc = handleTest ./misc.nix {}; moinmoin = handleTest ./moinmoin.nix {}; mongodb = handleTest ./mongodb.nix {}; @@ -223,17 +231,20 @@ in mysql-autobackup = handleTest ./mysql/mysql-autobackup.nix {}; mysql-backup = handleTest ./mysql/mysql-backup.nix {}; mysql-replication = handleTest ./mysql/mysql-replication.nix {}; + n8n = handleTest ./n8n.nix {}; nagios = handleTest ./nagios.nix {}; + nano = handleTest ./nano.nix {}; + nar-serve = handleTest ./nar-serve.nix {}; nat.firewall = handleTest ./nat.nix { withFirewall = true; }; nat.firewall-conntrack = handleTest ./nat.nix { withFirewall = true; withConntrackHelpers = true; }; nat.standalone = handleTest ./nat.nix { withFirewall = false; }; ncdns = handleTest ./ncdns.nix {}; ndppd = handleTest ./ndppd.nix {}; neo4j = handleTest ./neo4j.nix {}; - specialisation = handleTest ./specialisation.nix {}; netdata = handleTest ./netdata.nix {}; networking.networkd = handleTest ./networking.nix { networkd = true; }; networking.scripted = handleTest ./networking.nix { networkd = false; }; + specialisation = handleTest ./specialisation.nix {}; # TODO: put in networking.nix after the test becomes more complete networkingProxy = handleTest ./networking-proxy.nix {}; nextcloud = handleTest ./nextcloud {}; @@ -243,6 +254,7 @@ in nfs4 = handleTest ./nfs { version = 4; }; nghttpx = handleTest ./nghttpx.nix {}; nginx = handleTest ./nginx.nix {}; + nginx-auth = handleTest ./nginx-auth.nix {}; nginx-etag = handleTest ./nginx-etag.nix {}; nginx-pubhtml = handleTest ./nginx-pubhtml.nix {}; nginx-sandbox = handleTestOn ["x86_64-linux"] ./nginx-sandbox.nix {}; @@ -253,12 +265,13 @@ in novacomd = handleTestOn ["x86_64-linux"] ./novacomd.nix {}; nsd = handleTest ./nsd.nix {}; nzbget = handleTest ./nzbget.nix {}; + oh-my-zsh = handleTest ./oh-my-zsh.nix {}; openarena = handleTest ./openarena.nix {}; openldap = handleTest ./openldap.nix {}; opensmtpd = handleTest ./opensmtpd.nix {}; openssh = handleTest ./openssh.nix {}; - openstack-image-userdata = (handleTestOn ["x86_64-linux"] ./openstack-image.nix {}).userdata or {}; openstack-image-metadata = (handleTestOn ["x86_64-linux"] ./openstack-image.nix {}).metadata or {}; + openstack-image-userdata = (handleTestOn ["x86_64-linux"] ./openstack-image.nix {}).userdata or {}; orangefs = handleTest ./orangefs.nix {}; os-prober = handleTestOn ["x86_64-linux"] ./os-prober.nix {}; osrm-backend = handleTest ./osrm-backend.nix {}; @@ -268,6 +281,7 @@ in pam-u2f = handleTest ./pam-u2f.nix {}; pantheon = handleTest ./pantheon.nix {}; paperless = handleTest ./paperless.nix {}; + pdns-recursor = handleTest ./pdns-recursor.nix {}; peerflix = handleTest ./peerflix.nix {}; pgjwt = handleTest ./pgjwt.nix {}; pgmanage = handleTest ./pgmanage.nix {}; @@ -309,7 +323,11 @@ in runInMachine = handleTest ./run-in-machine.nix {}; rxe = handleTest ./rxe.nix {}; samba = handleTest ./samba.nix {}; + samba-wsdd = handleTest ./samba-wsdd.nix {}; sanoid = handleTest ./sanoid.nix {}; + sbt = handleTest ./sbt.nix {}; + sbt-extras = handleTest ./sbt-extras.nix {}; + scala = handleTest ./scala.nix {}; sddm = handleTest ./sddm.nix {}; service-runner = handleTest ./service-runner.nix {}; shadowsocks = handleTest ./shadowsocks {}; @@ -323,9 +341,9 @@ in snapper = handleTest ./snapper.nix {}; sogo = handleTest ./sogo.nix {}; solr = handleTest ./solr.nix {}; + sonarr = handleTest ./sonarr.nix {}; spacecookie = handleTest ./spacecookie.nix {}; spike = handleTest ./spike.nix {}; - sonarr = handleTest ./sonarr.nix {}; sslh = handleTest ./sslh.nix {}; sssd = handleTestOn ["x86_64-linux"] ./sssd.nix {}; sssd-ldap = handleTestOn ["x86_64-linux"] ./sssd-ldap.nix {}; @@ -341,13 +359,13 @@ in systemd-binfmt = handleTestOn ["x86_64-linux"] ./systemd-binfmt.nix {}; systemd-boot = handleTest ./systemd-boot.nix {}; systemd-confinement = handleTest ./systemd-confinement.nix {}; - systemd-timesyncd = handleTest ./systemd-timesyncd.nix {}; - systemd-networkd-vrf = handleTest ./systemd-networkd-vrf.nix {}; + systemd-journal = handleTest ./systemd-journal.nix {}; systemd-networkd = handleTest ./systemd-networkd.nix {}; systemd-networkd-dhcpserver = handleTest ./systemd-networkd-dhcpserver.nix {}; systemd-networkd-ipv6-prefix-delegation = handleTest ./systemd-networkd-ipv6-prefix-delegation.nix {}; + systemd-networkd-vrf = handleTest ./systemd-networkd-vrf.nix {}; systemd-nspawn = handleTest ./systemd-nspawn.nix {}; - pdns-recursor = handleTest ./pdns-recursor.nix {}; + systemd-timesyncd = handleTest ./systemd-timesyncd.nix {}; taskserver = handleTest ./taskserver.nix {}; telegraf = handleTest ./telegraf.nix {}; tiddlywiki = handleTest ./tiddlywiki.nix {}; @@ -355,19 +373,22 @@ in tinydns = handleTest ./tinydns.nix {}; tor = handleTest ./tor.nix {}; # traefik test relies on docker-containers + trac = handleTest ./trac.nix {}; traefik = handleTestOn ["x86_64-linux"] ./traefik.nix {}; transmission = handleTest ./transmission.nix {}; - trac = handleTest ./trac.nix {}; - trilium-server = handleTestOn ["x86_64-linux"] ./trilium-server.nix {}; trezord = handleTest ./trezord.nix {}; trickster = handleTest ./trickster.nix {}; + trilium-server = handleTestOn ["x86_64-linux"] ./trilium-server.nix {}; tuptime = handleTest ./tuptime.nix {}; + ucg = handleTest ./ucg.nix {}; udisks2 = handleTest ./udisks2.nix {}; + unbound = handleTest ./unbound.nix {}; unit-php = handleTest ./web-servers/unit-php.nix {}; upnp = handleTest ./upnp.nix {}; uwsgi = handleTest ./uwsgi.nix {}; v2ray = handleTest ./v2ray.nix {}; vault = handleTest ./vault.nix {}; + vector = handleTest ./vector.nix {}; victoriametrics = handleTest ./victoriametrics.nix {}; virtualbox = handleTestOn ["x86_64-linux"] ./virtualbox.nix {}; wasabibackend = handleTest ./wasabibackend.nix {}; @@ -379,6 +400,7 @@ in xmonad = handleTest ./xmonad.nix {}; xrdp = handleTest ./xrdp.nix {}; xss-lock = handleTest ./xss-lock.nix {}; + xterm = handleTest ./xterm.nix {}; yabar = handleTest ./yabar.nix {}; yggdrasil = handleTest ./yggdrasil.nix {}; zfs = handleTest ./zfs.nix {}; diff --git a/infra/libkookie/nixpkgs/nixos/tests/ammonite.nix b/infra/libkookie/nixpkgs/nixos/tests/ammonite.nix index 1955e42be5f0..e9f06358e13f 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/ammonite.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/ammonite.nix @@ -8,7 +8,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { amm = { pkgs, ... }: { - environment.systemPackages = [ pkgs.ammonite ]; + environment.systemPackages = [ (pkgs.ammonite.override { jre = pkgs.jre8; }) ]; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/avahi.nix b/infra/libkookie/nixpkgs/nixos/tests/avahi.nix index 66cff3009f7d..c1a9114a40f6 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/avahi.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/avahi.nix @@ -2,10 +2,11 @@ , config ? {} , pkgs ? import ../.. { inherit system config; } # bool: whether to use networkd in the tests -, networkd ? false }: +, networkd ? false +} @ args: # Test whether `avahi-daemon' and `libnss-mdns' work as expected. -import ./make-test-python.nix ({ ... } : { +import ./make-test-python.nix { name = "avahi"; meta = with pkgs.stdenv.lib.maintainers; { maintainers = [ eelco ]; @@ -75,4 +76,4 @@ import ./make-test-python.nix ({ ... } : { two.succeed("avahi-browse -r -t _ssh._tcp | tee out >&2") two.succeed("test `wc -l < out` -gt 0") ''; -}) +} args diff --git a/infra/libkookie/nixpkgs/nixos/tests/awscli.nix b/infra/libkookie/nixpkgs/nixos/tests/awscli.nix new file mode 100644 index 000000000000..35bdd6d99b1a --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/awscli.nix @@ -0,0 +1,17 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "awscli"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + environment.systemPackages = [ pkgs.awscli ]; + }; + + testScript = + '' + assert "${pkgs.python3Packages.botocore.version}" in machine.succeed("aws --version") + assert "${pkgs.awscli.version}" in machine.succeed("aws --version") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/bat.nix b/infra/libkookie/nixpkgs/nixos/tests/bat.nix new file mode 100644 index 000000000000..8e65e235d94f --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/bat.nix @@ -0,0 +1,12 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "bat"; + meta = with pkgs.stdenv.lib.maintainers; { maintainers = [ nequissimus ]; }; + + machine = { pkgs, ... }: { environment.systemPackages = [ pkgs.bat ]; }; + + testScript = '' + machine.succeed("echo 'Foobar\n\n\n42' > /tmp/foo") + assert "Foobar" in machine.succeed("bat -p /tmp/foo") + assert "42" in machine.succeed("bat -p /tmp/foo -r 4:4") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/bitcoind.nix b/infra/libkookie/nixpkgs/nixos/tests/bitcoind.nix index 09f3e4a6ec07..9068b29b8e5c 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/bitcoind.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/bitcoind.nix @@ -31,16 +31,16 @@ import ./make-test-python.nix ({ pkgs, ... }: { machine.wait_for_unit("bitcoind-testnet.service") machine.wait_until_succeeds( - 'curl --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' + 'curl --fail --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' ) machine.wait_until_succeeds( - 'curl --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' + 'curl --fail --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' ) machine.wait_until_succeeds( - 'curl --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' + 'curl --fail --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' ) machine.wait_until_succeeds( - 'curl --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' + 'curl --fail --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/caddy.nix b/infra/libkookie/nixpkgs/nixos/tests/caddy.nix index 445a7fa6b0b4..a21dbec248ab 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/caddy.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/caddy.nix @@ -1,7 +1,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { name = "caddy"; meta = with pkgs.stdenv.lib.maintainers; { - maintainers = [ xfix filalex77 ]; + maintainers = [ xfix Br1ght0ne ]; }; nodes = { @@ -57,11 +57,13 @@ import ./make-test-python.nix ({ pkgs, ... }: { def check_etag(url): etag = webserver.succeed( - "curl -v '{}' 2>&1 | sed -n -e \"s/^< [Ee][Tt][Aa][Gg]: *//p\"".format(url) + "curl --fail -v '{}' 2>&1 | sed -n -e \"s/^< [Ee][Tt][Aa][Gg]: *//p\"".format( + url + ) ) etag = etag.replace("\r\n", " ") http_code = webserver.succeed( - "curl --silent --show-error -o /dev/null -w \"%{{http_code}}\" --head -H 'If-None-Match: {}' {}".format( + "curl --fail --silent --show-error -o /dev/null -w \"%{{http_code}}\" --head -H 'If-None-Match: {}' {}".format( etag, url ) ) diff --git a/infra/libkookie/nixpkgs/nixos/tests/cadvisor.nix b/infra/libkookie/nixpkgs/nixos/tests/cadvisor.nix index 60c04f147800..664aa3ad876a 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/cadvisor.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/cadvisor.nix @@ -19,16 +19,16 @@ import ./make-test-python.nix ({ pkgs, ... } : { testScript = '' start_all() machine.wait_for_unit("cadvisor.service") - machine.succeed("curl http://localhost:8080/containers/") + machine.succeed("curl -f http://localhost:8080/containers/") influxdb.wait_for_unit("influxdb.service") # create influxdb database influxdb.succeed( - 'curl -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE root"' + 'curl -f -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE root"' ) influxdb.wait_for_unit("cadvisor.service") - influxdb.succeed("curl http://localhost:8080/containers/") + influxdb.succeed("curl -f http://localhost:8080/containers/") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/certmgr.nix b/infra/libkookie/nixpkgs/nixos/tests/certmgr.nix index ef32f54400e3..8f5b89487793 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/certmgr.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/certmgr.nix @@ -11,7 +11,7 @@ let file = { group = "nginx"; owner = "nginx"; - path = "/tmp/${host}-ca.pem"; + path = "/var/ssl/${host}-ca.pem"; }; label = "www_ca"; profile = "three-month"; @@ -20,13 +20,13 @@ let certificate = { group = "nginx"; owner = "nginx"; - path = "/tmp/${host}-cert.pem"; + path = "/var/ssl/${host}-cert.pem"; }; private_key = { group = "nginx"; mode = "0600"; owner = "nginx"; - path = "/tmp/${host}-key.pem"; + path = "/var/ssl/${host}-key.pem"; }; request = { CN = host; @@ -57,6 +57,8 @@ let services.cfssl.enable = true; systemd.services.cfssl.after = [ "cfssl-init.service" "networking.target" ]; + systemd.tmpfiles.rules = [ "d /var/ssl 777 root root" ]; + systemd.services.cfssl-init = { description = "Initialize the cfssl CA"; wantedBy = [ "multi-user.target" ]; @@ -87,8 +89,8 @@ let enable = true; virtualHosts = lib.mkMerge (map (host: { ${host} = { - sslCertificate = "/tmp/${host}-cert.pem"; - sslCertificateKey = "/tmp/${host}-key.pem"; + sslCertificate = "/var/ssl/${host}-cert.pem"; + sslCertificateKey = "/var/ssl/${host}-key.pem"; extraConfig = '' ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ''; @@ -124,16 +126,18 @@ in }; testScript = '' machine.wait_for_unit("cfssl.service") - machine.wait_until_succeeds("ls /tmp/decl.example.org-ca.pem") - machine.wait_until_succeeds("ls /tmp/decl.example.org-key.pem") - machine.wait_until_succeeds("ls /tmp/decl.example.org-cert.pem") - machine.wait_until_succeeds("ls /tmp/imp.example.org-ca.pem") - machine.wait_until_succeeds("ls /tmp/imp.example.org-key.pem") - machine.wait_until_succeeds("ls /tmp/imp.example.org-cert.pem") + machine.wait_until_succeeds("ls /var/ssl/decl.example.org-ca.pem") + machine.wait_until_succeeds("ls /var/ssl/decl.example.org-key.pem") + machine.wait_until_succeeds("ls /var/ssl/decl.example.org-cert.pem") + machine.wait_until_succeeds("ls /var/ssl/imp.example.org-ca.pem") + machine.wait_until_succeeds("ls /var/ssl/imp.example.org-key.pem") + machine.wait_until_succeeds("ls /var/ssl/imp.example.org-cert.pem") machine.wait_for_unit("nginx.service") assert 1 < int(machine.succeed('journalctl -u nginx | grep "Starting Nginx" | wc -l')) - machine.succeed("curl --cacert /tmp/imp.example.org-ca.pem https://imp.example.org") - machine.succeed("curl --cacert /tmp/decl.example.org-ca.pem https://decl.example.org") + machine.succeed("curl --cacert /var/ssl/imp.example.org-ca.pem https://imp.example.org") + machine.succeed( + "curl --cacert /var/ssl/decl.example.org-ca.pem https://decl.example.org" + ) ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/cfssl.nix b/infra/libkookie/nixpkgs/nixos/tests/cfssl.nix index e291fc285fba..170f09d9b76c 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/cfssl.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/cfssl.nix @@ -38,7 +38,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { testScript = let cfsslrequest = with pkgs; writeScript "cfsslrequest" '' - curl -X POST -H "Content-Type: application/json" -d @${csr} \ + curl -f -X POST -H "Content-Type: application/json" -d @${csr} \ http://localhost:8888/api/v1/cfssl/newkey | ${cfssl}/bin/cfssljson /tmp/certificate ''; csr = pkgs.writeText "csr.json" (builtins.toJSON { diff --git a/infra/libkookie/nixpkgs/nixos/tests/cloud-init.nix b/infra/libkookie/nixpkgs/nixos/tests/cloud-init.nix index a127be6dd85f..d59d222974b5 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/cloud-init.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/cloud-init.nix @@ -51,29 +51,31 @@ in makeTest { networking.hostName = ""; }; testScript = '' - machine.wait_for_unit("cloud-init.service") - machine.succeed("cat /tmp/cloudinit-write-file | grep -q 'cloudinit'") + # To wait until cloud-init terminates its run + unnamed.wait_for_unit("cloud-final.service") + + unnamed.succeed("cat /tmp/cloudinit-write-file | grep -q 'cloudinit'") # install snakeoil ssh key and provision .ssh/config file - machine.succeed("mkdir -p ~/.ssh") - machine.succeed( + unnamed.succeed("mkdir -p ~/.ssh") + unnamed.succeed( "cat ${snakeOilPrivateKey} > ~/.ssh/id_snakeoil" ) - machine.succeed("chmod 600 ~/.ssh/id_snakeoil") + unnamed.succeed("chmod 600 ~/.ssh/id_snakeoil") - machine.wait_for_unit("sshd.service") + unnamed.wait_for_unit("sshd.service") # we should be able to log in as the root user, as well as the created nixos user - machine.succeed( + unnamed.succeed( "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentityFile=~/.ssh/id_snakeoil root@localhost 'true'" ) - machine.succeed( + unnamed.succeed( "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentityFile=~/.ssh/id_snakeoil nixos@localhost 'true'" ) # test changing hostname via cloud-init worked assert ( - machine.succeed( + unnamed.succeed( "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentityFile=~/.ssh/id_snakeoil nixos@localhost 'hostname'" ).strip() == "test" diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/README.md b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/README.md new file mode 100644 index 000000000000..9de2b2c71029 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/README.md @@ -0,0 +1,21 @@ +# Fake Certificate Authority for ACME testing + +This will set up a test node running [pebble](https://github.com/letsencrypt/pebble) +to serve ACME certificate requests. + +## "Snake oil" certs + +The snake oil certs are hard coded into the repo for reasons explained [here](https://github.com/NixOS/nixpkgs/pull/91121#discussion_r505410235). +The root of the issue is that Nix will hash the derivation based on the arguments +to mkDerivation, not the output. [Minica](https://github.com/jsha/minica) will +always generate a random certificate even if the arguments are unchanged. As a +result, it's possible to end up in a situation where the cached and local +generated certs mismatch and cause issues with testing. + +To generate new certificates, run the following commands: + +```bash +nix-build generate-certs.nix +cp result/* . +rm result +``` diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/acme.test.cert.pem b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/acme.test.cert.pem new file mode 100644 index 000000000000..76b0d916a817 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/acme.test.cert.pem @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDLDCCAhSgAwIBAgIIRDAN3FHH//IwDQYJKoZIhvcNAQELBQAwIDEeMBwGA1UE +AxMVbWluaWNhIHJvb3QgY2EgNzg3NDZmMB4XDTIwMTAyMTEzMjgzNloXDTIyMTEy +MDEzMjgzNlowFDESMBAGA1UEAxMJYWNtZS50ZXN0MIIBIjANBgkqhkiG9w0BAQEF +AAOCAQ8AMIIBCgKCAQEAo8XjMVUaljcaqQ5MFhfPuQgSwdyXEUbpSHz+5yPkE0h9 +Z4Xu5BJF1Oq7h5ggCtadVsIspiY6Jm6aWDOjlh4myzW5UNBNUG3OPEk50vmmHFeH +pImHO/d8yb33QoF9VRcTZs4tuJYg7l9bSs4jNG72vYvv2YiGAcmjJcsmAZIfniCN +Xf/LjIm+Cxykn+Vo3UuzO1w5/iuofdgWO/aZxMezmXUivlL3ih4cNzCJei8WlB/l +EnHrkcy3ogRmmynP5zcz7vmGIJX2ji6dhCa4Got5B7eZK76o2QglhQXqPatG0AOY +H+RfQfzKemqPG5om9MgJtwFtTOU1LoaiBw//jXKESQIDAQABo3YwdDAOBgNVHQ8B +Af8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB +/wQCMAAwHwYDVR0jBBgwFoAU+8IZlLV/Qp5CXqpXMLvtxWlxcJwwFAYDVR0RBA0w +C4IJYWNtZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQB0pe8I5/VDkB5VMgQB2GJV +GKzyigfWbVez9uLmqMj9PPP/zzYKSYeq+91aMuOZrnH7NqBxSTwanULkmqAmhbJJ +YkXw+FlFekf9FyxcuArzwzzNZDSGcjcdXpN8S2K1qkBd00iSJF9kU7pdZYCIKR20 +QirdBrELEfsJ3GU62a6N3a2YsrisZUvq5TbjGJDcytAtt+WG3gmV7RInLdFfPwbw +bEHPCnx0uiV0nxLjd/aVT+RceVrFQVt4hR99jLoMlBitSKluZ1ljsrpIyroBhQT0 +pp/pVi6HJdijG0fsPrC325NEGAwcpotLUhczoeM/rffKJd54wLhDkfYxOyRZXivs +-----END CERTIFICATE----- diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/acme.test.key.pem b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/acme.test.key.pem new file mode 100644 index 000000000000..741df99a372e --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/acme.test.key.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAo8XjMVUaljcaqQ5MFhfPuQgSwdyXEUbpSHz+5yPkE0h9Z4Xu +5BJF1Oq7h5ggCtadVsIspiY6Jm6aWDOjlh4myzW5UNBNUG3OPEk50vmmHFeHpImH +O/d8yb33QoF9VRcTZs4tuJYg7l9bSs4jNG72vYvv2YiGAcmjJcsmAZIfniCNXf/L +jIm+Cxykn+Vo3UuzO1w5/iuofdgWO/aZxMezmXUivlL3ih4cNzCJei8WlB/lEnHr +kcy3ogRmmynP5zcz7vmGIJX2ji6dhCa4Got5B7eZK76o2QglhQXqPatG0AOYH+Rf +QfzKemqPG5om9MgJtwFtTOU1LoaiBw//jXKESQIDAQABAoIBADox/2FwVFo8ioS4 +R+Ex5OZjMAcjU6sX/516jTmlT05q2+UFerYgqB/YqXqtW/V9/brulN8VhmRRuRbO +grq9TBu5o3hMDK0f18EkZB/MBnLbx594H033y6gEkPBZAyhRYtuNOEH3VwxdZhtW +1Lu1EoiYSUqLcNMBy6+KWJ8GRaXyacMYBlj2lMHmyzkA/t1+2mwTGC3lT6zN0F5Y +E5umXOxsn6Tb6q3KM9O5IvtmMMKpgj4HIHZLZ6j40nNgHwGRaAv4Sha/vx0DeBw3 +6VlNiTTPdShEkhESlM5/ocqTfI92VHJpM5gkqTYOWBi2aKIPfAopXoqoJdWl4pQ/ +NCFIu2ECgYEAzntNKIcQtf0ewe0/POo07SIFirvz6jVtYNMTzeQfL6CoEjYArJeu +Vzc4wEQfA4ZFVerBb1/O6M449gI3zex1PH4AX0h8q8DSjrppK1Jt2TnpVh97k7Gg +Tnat/M/yW3lWYkcMVJJ3AYurXLFTT1dYP0HvBwZN04yInrEcPNXKfmcCgYEAywyJ +51d4AE94PrANathKqSI/gk8sP+L1gzylZCcUEAiGk/1r45iYB4HN2gvWbS+CvSdp +F7ShlDWrTaNh2Bm1dgTjc4pWb4J+CPy/KN2sgLwIuM4+ZWIZmEDcio6khrM/gNqK +aR7xUsvWsqU26O84woY/xR8IHjSNF7cFWE1H2c8CgYEAt6SSi2kVQ8dMg84uYE8t +o3qO00U3OycpkOQqyQQLeKC62veMwfRl6swCfX4Y11mkcTXJtPTRYd2Ia8StPUkB +PDwUuKoPt/JXUvoYb59wc7M+BIsbrdBdc2u6cw+/zfutCNuH6/AYSBeg4WAVaIuW +wSwzG1xP+8cR+5IqOzEqWCECgYATweeVTCyQEyuHJghYMi2poXx+iIesu7/aAkex +pB/Oo5W8xrb90XZRnK7UHbzCqRHWqAQQ23Gxgztk9ZXqui2vCzC6qGZauV7cLwPG +zTMg36sVmHP314DYEM+k59ZYiQ6P0jQPoIQo407D2VGrfsOOIhQIcUmP7tsfyJ5L +hlGMfwKBgGq4VNnnuX8I5kl03NpaKfG+M8jEHmVwtI9RkPTCCX9bMjeG0cDxqPTF +TRkf3r8UWQTZ5QfAfAXYAOlZvmGhHjSembRbXMrMdi3rGsYRSrQL6n5NHnORUaMy +FCWo4gyAnniry7tx9dVNgmHmbjEHuQnf8AC1r3dibRCjvJWUiQ8H +-----END RSA PRIVATE KEY----- diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/ca.cert.pem b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/ca.cert.pem new file mode 100644 index 000000000000..5c33e879b675 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/ca.cert.pem @@ -0,0 +1,20 @@ +-----BEGIN CERTIFICATE----- +MIIDSzCCAjOgAwIBAgIIeHRvRrNvbGQwDQYJKoZIhvcNAQELBQAwIDEeMBwGA1UE +AxMVbWluaWNhIHJvb3QgY2EgNzg3NDZmMCAXDTIwMTAyMTEzMjgzNloYDzIxMjAx +MDIxMTMyODM2WjAgMR4wHAYDVQQDExVtaW5pY2Egcm9vdCBjYSA3ODc0NmYwggEi +MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCrNTzVLDJOKtGYGLU98EEcLKps +tXHCLC6G54LKbEcU80fn+ArX8qsPSHyhdXQkcYjq6Vh/EDJ1TctyRSnvAjwyG4Aa +1Zy1QFc/JnjMjvzimCkUc9lQ+wkLwHSM/KGwR1cGjmtQ/EMClZTA0NwulJsXMKVz +bd5asXbq/yJTQ5Ww25HtdNjwRQXTvB7r3IKcY+DsED9CvFvC9oG/ZhtZqZuyyRdC +kFUrrv8WNUDkWSN+lMR6xMx8v0583IN6f11IhX0b+svK98G81B2eswBdkzvVyv9M +unZBO0JuJG8sdM502KhWLmzBC1ZbvgUBF9BumDRpMFH4DCj7+qQ2taWeGyc7AgMB +AAGjgYYwgYMwDgYDVR0PAQH/BAQDAgKEMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr +BgEFBQcDAjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdDgQWBBT7whmUtX9CnkJe +qlcwu+3FaXFwnDAfBgNVHSMEGDAWgBT7whmUtX9CnkJeqlcwu+3FaXFwnDANBgkq +hkiG9w0BAQsFAAOCAQEARMe1wKmF33GjEoLLw0oDDS4EdAv26BzCwtrlljsEtwQN +95oSzUNd6o4Js7WCG2o543OX6cxzM+yju8TES3+vJKDgsbNMU0bWCv//tdrb0/G8 +OkU3Kfi5q4fOauZ1pqGv/pXdfYhZ5ieB/zwis3ykANe5JfB0XqwCb1Vd0C3UCIS2 +NPKngRwNSzphIsbzfvxGDkdM1enuGl5CVyDhrwTMqGaJGDSOv6U5jKFxKRvigqTN +Ls9lPmT5NXYETduWLBR3yUIdH6kZXrcozZ02B9vjOB2Cv4RMDc+9eM30CLIWpf1I +097e7JkhzxFhfC/bMMt3P1FeQc+fwH91wdBmNi7tQw== +-----END CERTIFICATE----- diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/ca.key.pem b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/ca.key.pem new file mode 100644 index 000000000000..ed46f5dccf46 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/ca.key.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAqzU81SwyTirRmBi1PfBBHCyqbLVxwiwuhueCymxHFPNH5/gK +1/KrD0h8oXV0JHGI6ulYfxAydU3LckUp7wI8MhuAGtWctUBXPyZ4zI784pgpFHPZ +UPsJC8B0jPyhsEdXBo5rUPxDApWUwNDcLpSbFzClc23eWrF26v8iU0OVsNuR7XTY +8EUF07we69yCnGPg7BA/QrxbwvaBv2YbWambsskXQpBVK67/FjVA5FkjfpTEesTM +fL9OfNyDen9dSIV9G/rLyvfBvNQdnrMAXZM71cr/TLp2QTtCbiRvLHTOdNioVi5s +wQtWW74FARfQbpg0aTBR+Awo+/qkNrWlnhsnOwIDAQABAoIBAA3ykVkgd5ysmlSU +trcsCnHcJaojgff6l3PACoSpG4VWaGY6a8+54julgRm6MtMBONFCX0ZCsImj484U +Wl0xRmwil2YYPuL5MeJgJPktMObY1IfpBCw3tz3w2M3fiuCMf0d2dMGtO1xLiUnH ++hgFXTkfamsj6ThkOrbcQBSebeRxbKM5hqyCaQoieV+0IJnyxUVq/apib8N50VsH +SHd4oqLUuEZgg6N70+l5DpzedJUb4nrwS/KhUHUBgnoPItYBCiGPmrwLk7fUhPs6 +kTDqJDtc/xW/JbjmzhWEpVvtumcC/OEKULss7HLdeQqwVBrRQkznb0M9AnSra3d0 +X11/Y4ECgYEA3FC8SquLPFb2lHK4+YbJ4Ac6QVWeYFEHiZ0Rj+CmONmjcAvOGLPE +SblRLm3Nbrkxbm8FF6/AfXa/rviAKEVPs5xqGfSDw/3n1uInPcmShiBCLwM/jHH5 +NeVG+R5mTg5zyQ/pQMLWRcs+Ail+ZAnZuoGpW3Cdc8OtCUYFQ7XB6nsCgYEAxvBJ +zFxcTtsDzWbMWXejugQiUqJcEbKWwEfkRbf3J2rAVO2+EFr7LxdRfN2VwPiTQcWc +LnN2QN+ouOjqBMTh3qm5oQY+TLLHy86k9g1k0gXWkMRQgP2ZdfWH1HyrwjLUgLe1 +VezFN7N1azgy6xFkInAAvuA4loxElZNvkGBgekECgYA/Xw26ILvNIGqO6qzgQXAh ++5I7JsiGheg4IjDiBMlrQtbrLMoceuD0H9UFGNplhel9DXwWgxxIOncKejpK2x0A +2fX+/0FDh+4+9hA5ipiV8gN3iGSoHkSDxy5yC9d7jlapt+TtFt4Rd1OfxZWwatDw +/8jaH3t6yAcmyrhK8KYVrwKBgAE5KwsBqmOlvyE9N5Z5QN189wUREIXfVkP6bTHs +jq2EX4hmKdwJ4y+H8i1VY31bSfSGlY5HkXuWpH/2lrHO0CDBZG3UDwADvWzIaYVF +0c/kz0v2mRQh+xaZmus4lQnNrDbaalgL666LAPbW0qFVaws3KxoBYPe0BxvwWyhF +H3LBAoGBAKRRNsq2pWQ8Gqxc0rVoH0FlexU9U2ci3lsLmgEB0A/o/kQkSyAxaRM+ +VdKp3sWfO8o8lX5CVQslCNBSjDTNcat3Co4NEBLg6Xv1yKN/WN1GhusnchP9szsP +oU47gC89QhUyWSd6vvr2z2NG9C3cACxe4dhDSHQcE4nHSldzCKv2 +-----END RSA PRIVATE KEY----- diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/default.nix b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/default.nix index cea10c16900d..1c3bfdf76b7e 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/default.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/default.nix @@ -51,10 +51,7 @@ # that it has to be started _before_ the ACME service. { config, pkgs, lib, ... }: let - testCerts = import ./snakeoil-certs.nix { - minica = pkgs.minica; - mkDerivation = pkgs.stdenv.mkDerivation; - }; + testCerts = import ./snakeoil-certs.nix; domain = testCerts.domain; resolver = let diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/generate-certs.nix b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/generate-certs.nix new file mode 100644 index 000000000000..cd8fe0dffca1 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/generate-certs.nix @@ -0,0 +1,29 @@ +# Minica can provide a CA key and cert, plus a key +# and cert for our fake CA server's Web Front End (WFE). +{ + pkgs ? import <nixpkgs> {}, + minica ? pkgs.minica, + mkDerivation ? pkgs.stdenv.mkDerivation +}: +let + conf = import ./snakeoil-certs.nix; + domain = conf.domain; +in mkDerivation { + name = "test-certs"; + buildInputs = [ minica ]; + phases = [ "buildPhase" "installPhase" ]; + + buildPhase = '' + minica \ + --ca-key ca.key.pem \ + --ca-cert ca.cert.pem \ + --domains ${domain} + ''; + + installPhase = '' + mkdir -p $out + mv ca.*.pem $out/ + mv ${domain}/key.pem $out/${domain}.key.pem + mv ${domain}/cert.pem $out/${domain}.cert.pem + ''; +} diff --git a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/snakeoil-certs.nix b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/snakeoil-certs.nix index 4b6a38b8fa30..11c3f7fc9290 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/snakeoil-certs.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/common/acme/server/snakeoil-certs.nix @@ -1,37 +1,13 @@ -# Minica can provide a CA key and cert, plus a key -# and cert for our fake CA server's Web Front End (WFE). -{ minica, mkDerivation }: let domain = "acme.test"; - - selfSignedCertData = mkDerivation { - name = "test-certs"; - buildInputs = [ minica ]; - phases = [ "buildPhase" "installPhase" ]; - - buildPhase = '' - mkdir ca - minica \ - --ca-key ca/key.pem \ - --ca-cert ca/cert.pem \ - --domains ${domain} - chmod 600 ca/* - chmod 640 ${domain}/*.pem - ''; - - installPhase = '' - mkdir -p $out - mv ${domain} ca $out/ - ''; - }; in { inherit domain; ca = { - cert = "${selfSignedCertData}/ca/cert.pem"; - key = "${selfSignedCertData}/ca/key.pem"; + cert = ./ca.cert.pem; + key = ./ca.key.pem; }; "${domain}" = { - cert = "${selfSignedCertData}/${domain}/cert.pem"; - key = "${selfSignedCertData}/${domain}/key.pem"; + cert = ./. + "/${domain}.cert.pem"; + key = ./. + "/${domain}.key.pem"; }; } diff --git a/infra/libkookie/nixpkgs/nixos/tests/convos.nix b/infra/libkookie/nixpkgs/nixos/tests/convos.nix index b4ff1188fd8b..af2758c857d0 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/convos.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/convos.nix @@ -25,6 +25,6 @@ in machine.wait_for_unit("convos") machine.wait_for_open_port("${toString port}") machine.succeed("journalctl -u convos | grep -q 'Listening at.*${toString port}'") - machine.succeed("curl http://localhost:${toString port}/") + machine.succeed("curl -f http://localhost:${toString port}/") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/corerad.nix b/infra/libkookie/nixpkgs/nixos/tests/corerad.nix index 37a1e90477a8..638010f92f44 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/corerad.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/corerad.nix @@ -80,7 +80,7 @@ import ./make-test-python.nix ( ), "SLAAC temporary address was not configured on client after router advertisement" with subtest("Verify HTTP debug server is configured"): - out = router.succeed("curl localhost:9430/metrics") + out = router.succeed("curl -f localhost:9430/metrics") assert ( "corerad_build_info" in out diff --git a/infra/libkookie/nixpkgs/nixos/tests/docker-edge.nix b/infra/libkookie/nixpkgs/nixos/tests/docker-edge.nix index 96de885a554a..703179eef195 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/docker-edge.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/docker-edge.nix @@ -43,7 +43,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { docker.fail("sudo -u noprivs docker ps") docker.succeed("docker stop sleeping") - # Must match version twice to ensure client and server versions are correct - docker.succeed('[ $(docker version | grep ${pkgs.docker-edge.version} | wc -l) = "2" ]') + # Must match version 4 times to ensure client and server git commits and versions are correct + docker.succeed('[ $(docker version | grep ${pkgs.docker-edge.version} | wc -l) = "4" ]') ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/docker-tools.nix b/infra/libkookie/nixpkgs/nixos/tests/docker-tools.nix index edb9aec62db3..3d1e39a379c1 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/docker-tools.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/docker-tools.nix @@ -115,9 +115,10 @@ import ./make-test-python.nix ({ pkgs, ... }: { "docker load --input='${examples.nginx}'", "docker run --name nginx -d -p 8000:80 ${examples.nginx.imageName}", ) - docker.wait_until_succeeds("curl http://localhost:8000/") + docker.wait_until_succeeds("curl -f http://localhost:8000/") docker.succeed( - "docker rm --force nginx", "docker rmi '${examples.nginx.imageName}'", + "docker rm --force nginx", + "docker rmi '${examples.nginx.imageName}'", ) with subtest("A pulled image can be used as base image"): @@ -234,5 +235,17 @@ import ./make-test-python.nix ({ pkgs, ... }: { "docker run --rm file-in-store nix-store --verify --check-contents", "docker run --rm file-in-store |& grep 'some data'", ) + + with subtest("Ensure cross compiled image can be loaded and has correct arch."): + docker.succeed( + "docker load --input='${pkgs.dockerTools.examples.cross}'", + ) + assert ( + docker.succeed( + "docker inspect ${pkgs.dockerTools.examples.cross.imageName} " + + "| ${pkgs.jq}/bin/jq -r .[].Architecture" + ).strip() + == "${if pkgs.system == "aarch64-linux" then "amd64" else "arm64v8"}" + ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/fcitx/config b/infra/libkookie/nixpkgs/nixos/tests/fcitx/config new file mode 100644 index 000000000000..169768994e28 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/fcitx/config @@ -0,0 +1,12 @@ +[Hotkey] +SwitchKey=Disabled +IMSwitchHotkey=ALT_SHIFT +TimeInterval=240 + +[Program] +DelayStart=5 + +[Output] + +[Appearance] + diff --git a/infra/libkookie/nixpkgs/nixos/tests/fcitx/default.nix b/infra/libkookie/nixpkgs/nixos/tests/fcitx/default.nix new file mode 100644 index 000000000000..cbeb95d33b0c --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/fcitx/default.nix @@ -0,0 +1,142 @@ +import ../make-test-python.nix ( + { + pkgs, ... + }: + # copy_from_host works only for store paths + rec { + name = "fcitx"; + machine = + { + pkgs, + ... + }: + { + virtualisation.memorySize = 1024; + + imports = [ + ../common/user-account.nix + ]; + + environment.systemPackages = [ + # To avoid clashing with xfce4-terminal + pkgs.alacritty + ]; + + + services.xserver = + { + enable = true; + + displayManager = { + lightdm.enable = true; + autoLogin = { + enable = true; + user = "alice"; + }; + }; + + desktopManager.xfce.enable = true; + }; + + i18n = { + inputMethod = { + enabled = "fcitx"; + fcitx.engines = [ + pkgs.fcitx-engines.m17n + pkgs.fcitx-engines.table-extra + ]; + }; + }; + } + ; + + testScript = { nodes, ... }: + let + user = nodes.machine.config.users.users.alice; + userName = user.name; + userHome = user.home; + xauth = "${userHome}/.Xauthority"; + fcitx_confdir = "${userHome}/.config/fcitx"; + in + '' + # We need config files before login session + # So copy first thing + + # Point and click would be expensive, + # So configure using files + machine.copy_from_host( + "${./profile}", + "${fcitx_confdir}/profile", + ) + machine.copy_from_host( + "${./config}", + "${fcitx_confdir}/config", + ) + + start_all() + + machine.wait_for_file("${xauth}") + machine.succeed("xauth merge ${xauth}") + + machine.sleep(5) + + machine.succeed("su - ${userName} -c 'alacritty&'") + machine.succeed("su - ${userName} -c 'fcitx&'") + machine.sleep(10) + + ### Type on terminal + machine.send_chars("echo ") + machine.sleep(1) + + ### Start fcitx Unicode input + machine.send_key("ctrl-alt-shift-u") + machine.sleep(5) + machine.sleep(1) + + ### Search for smiling face + machine.send_chars("smil") + machine.sleep(1) + + ### Navigate to the second one + machine.send_key("tab") + machine.sleep(1) + + ### Choose it + machine.send_key("\n") + machine.sleep(1) + + ### Start fcitx language input + machine.send_key("ctrl-spc") + machine.sleep(1) + + ### Default zhengma, enter 一下 + machine.send_chars("a2") + machine.sleep(1) + + ### Switch to Harvard Kyoto + machine.send_key("alt-shift") + machine.sleep(1) + + ### Enter क + machine.send_chars("ka ") + machine.sleep(1) + + machine.send_key("alt-shift") + machine.sleep(1) + + ### Turn off Fcitx + machine.send_key("ctrl-spc") + machine.sleep(1) + + ### Redirect typed characters to a file + machine.send_chars(" > fcitx_test.out\n") + machine.sleep(1) + machine.screenshot("terminal_chars") + + ### Verify that file contents are as expected + file_content = machine.succeed("cat ${userHome}/fcitx_test.out") + assert file_content == "☺一下क\n" + '' + ; + } +) diff --git a/infra/libkookie/nixpkgs/nixos/tests/fcitx/profile b/infra/libkookie/nixpkgs/nixos/tests/fcitx/profile new file mode 100644 index 000000000000..77497a1496bd --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/fcitx/profile @@ -0,0 +1,4 @@ +[Profile] +IMName=zhengma-large +EnabledIMList=fcitx-keyboard-us:True,zhengma-large:True,m17n_sa_harvard-kyoto:True +PreeditStringInClientWindow=False diff --git a/infra/libkookie/nixpkgs/nixos/tests/ferm.nix b/infra/libkookie/nixpkgs/nixos/tests/ferm.nix index a73c9ce739cf..112b5f19a7de 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/ferm.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/ferm.nix @@ -56,6 +56,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { start_all() client.wait_for_unit("network-online.target") + server.wait_for_unit("network-online.target") server.wait_for_unit("ferm.service") server.wait_for_unit("nginx.service") server.wait_until_succeeds("ss -ntl | grep -q 80") diff --git a/infra/libkookie/nixpkgs/nixos/tests/firefox.nix b/infra/libkookie/nixpkgs/nixos/tests/firefox.nix index 7071baceba73..07e25bd4ca72 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/firefox.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/firefox.nix @@ -11,26 +11,105 @@ import ./make-test-python.nix ({ pkgs, esr ? false, ... }: { environment.systemPackages = (if esr then [ pkgs.firefox-esr ] else [ pkgs.firefox ]) ++ [ pkgs.xdotool ]; + + # Need some more memory to record audio. + virtualisation.memorySize = "500"; + + # Create a virtual sound device, with mixing + # and all, for recording audio. + boot.kernelModules = [ "snd-aloop" ]; + sound.enable = true; + sound.extraConfig = '' + pcm.!default { + type plug + slave.pcm pcm.dmixer + } + pcm.dmixer { + type dmix + ipc_key 1 + slave { + pcm "hw:Loopback,0,0" + rate 48000 + periods 128 + period_time 0 + period_size 1024 + buffer_size 8192 + } + } + pcm.recorder { + type hw + card "Loopback" + device 1 + subdevice 0 + } + ''; + + systemd.services.audio-recorder = { + description = "Record NixOS test audio to /tmp/record.wav"; + script = "${pkgs.alsaUtils}/bin/arecord -D recorder -f S16_LE -r48000 /tmp/record.wav"; + }; + }; testScript = '' + from contextlib import contextmanager + + + @contextmanager + def audio_recording(machine: Machine) -> None: + """ + Perform actions while recording the + machine audio output. + """ + machine.systemctl("start audio-recorder") + yield + machine.systemctl("stop audio-recorder") + + + def wait_for_sound(machine: Machine) -> None: + """ + Wait until any sound has been emitted. + """ + machine.wait_for_file("/tmp/record.wav") + while True: + # Get at most 2M of the recording + machine.execute("tail -c 2M /tmp/record.wav > /tmp/last") + # Get the exact size + size = int(machine.succeed("stat -c '%s' /tmp/last").strip()) + # Compare it against /dev/zero using `cmp` (skipping 50B of WAVE header). + # If some non-NULL bytes are found it returns 1. + status, output = machine.execute( + f"cmp -i 50 -n {size - 50} /tmp/last /dev/zero 2>&1" + ) + if status == 1: + break + machine.sleep(2) + + machine.wait_for_x() - with subtest("wait until Firefox has finished loading the Valgrind docs page"): + with subtest("Wait until Firefox has finished loading the Valgrind docs page"): machine.execute( "xterm -e 'firefox file://${pkgs.valgrind.doc}/share/doc/valgrind/html/index.html' &" ) machine.wait_for_window("Valgrind") machine.sleep(40) + with subtest("Check whether Firefox can play sound"): + with audio_recording(machine): + machine.succeed( + "firefox file://${pkgs.sound-theme-freedesktop}/share/sounds/freedesktop/stereo/phone-incoming-call.oga &" + ) + wait_for_sound(machine) + machine.copy_from_vm("/tmp/record.wav") + + with subtest("Close sound test tab"): + machine.execute("xdotool key ctrl+w") + with subtest("Close default browser prompt"): machine.execute("xdotool key space") - with subtest("Hide default browser window"): - machine.sleep(2) - machine.execute("xdotool key F12") - - with subtest("wait until Firefox draws the developer tool panel"): + with subtest("Wait until Firefox draws the developer tool panel"): machine.sleep(10) machine.succeed("xwininfo -root -tree | grep Valgrind") machine.screenshot("screen") diff --git a/infra/libkookie/nixpkgs/nixos/tests/firejail.nix b/infra/libkookie/nixpkgs/nixos/tests/firejail.nix index a723cb01664f..5f122c3fa94d 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/firejail.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/firejail.nix @@ -11,6 +11,10 @@ import ./make-test-python.nix ({ pkgs, ...} : { enable = true; wrappedBinaries = { bash-jailed = "${pkgs.bash}/bin/bash"; + bash-jailed2 = { + executable = "${pkgs.bash}/bin/bash"; + extraArgs = [ "--private=~/firejail-home" ]; + }; }; }; @@ -53,6 +57,11 @@ import ./make-test-python.nix ({ pkgs, ...} : { ) machine.fail("sudo -u alice bash-jailed -c 'cat ~/my-secrets/secret' | grep -q s3cret") + # Test extraArgs + machine.succeed("sudo -u alice mkdir /home/alice/firejail-home") + machine.succeed("sudo -u alice bash-jailed2 -c 'echo test > /home/alice/foo'") + machine.fail("sudo -u alice cat /home/alice/foo") + machine.succeed("sudo -u alice cat /home/alice/firejail-home/foo | grep test") # Test path acl with firejail executable machine.succeed("sudo -u alice firejail -- bash -c 'cat ~/public' | grep -q publ1c") diff --git a/infra/libkookie/nixpkgs/nixos/tests/gitea.nix b/infra/libkookie/nixpkgs/nixos/tests/gitea.nix index aaed2486421f..1fb27593f056 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/gitea.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/gitea.nix @@ -14,6 +14,7 @@ let nodes = { server = { config, pkgs, ... }: { + virtualisation.memorySize = 2048; services.gitea = { enable = true; database = { inherit type; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/gitlab.nix b/infra/libkookie/nixpkgs/nixos/tests/gitlab.nix index 7e4e8bcef92d..1214cddd0937 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/gitlab.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/gitlab.nix @@ -33,9 +33,9 @@ import ./make-test-python.nix ({ pkgs, lib, ...} : with lib; { initialRootPasswordFile = pkgs.writeText "rootPassword" initialRootPassword; smtp.enable = true; secrets = { - secretFile = pkgs.writeText "secret" "Aig5zaic"; - otpFile = pkgs.writeText "otpsecret" "Riew9mue"; - dbFile = pkgs.writeText "dbsecret" "we2quaeZ"; + secretFile = pkgs.writeText "secret" "r8X9keSKynU7p4aKlh4GO1Bo77g5a7vj"; + otpFile = pkgs.writeText "otpsecret" "Zu5hGx3YvQx40DvI8WoZJQpX2paSDOlG"; + dbFile = pkgs.writeText "dbsecret" "lsGltKWTejOf6JxCVa7nLDenzkO9wPLR"; jwsFile = pkgs.runCommand "oidcKeyBase" {} "${pkgs.openssl}/bin/openssl genrsa 2048 > $out"; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/go-neb.nix b/infra/libkookie/nixpkgs/nixos/tests/go-neb.nix index d9e5db0b4a53..531ab5a66714 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/go-neb.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/go-neb.nix @@ -34,7 +34,7 @@ import ./make-test-python.nix ({ pkgs, ... }: start_all() server.wait_for_unit("go-neb.service") server.wait_until_succeeds( - "curl -L http://localhost:4050/services/hooks/d2lraXBlZGlhX3NlcnZpY2U" + "curl -fL http://localhost:4050/services/hooks/d2lraXBlZGlhX3NlcnZpY2U" ) server.wait_until_succeeds( "journalctl -eu go-neb -o cat | grep -q service_id=wikipedia_service" diff --git a/infra/libkookie/nixpkgs/nixos/tests/hadoop/hdfs.nix b/infra/libkookie/nixpkgs/nixos/tests/hadoop/hdfs.nix index 85aaab34b158..f1f98ed42eb3 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/hadoop/hdfs.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/hadoop/hdfs.nix @@ -48,7 +48,7 @@ import ../make-test-python.nix ({...}: { datanode.wait_for_open_port(9866) datanode.wait_for_open_port(9867) - namenode.succeed("curl http://namenode:9870") - datanode.succeed("curl http://datanode:9864") + namenode.succeed("curl -f http://namenode:9870") + datanode.succeed("curl -f http://datanode:9864") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/hadoop/yarn.nix b/infra/libkookie/nixpkgs/nixos/tests/hadoop/yarn.nix index 2264ecaff155..01077245d397 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/hadoop/yarn.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/hadoop/yarn.nix @@ -40,7 +40,7 @@ import ../make-test-python.nix ({...}: { nodemanager.wait_for_open_port(8042) nodemanager.wait_for_open_port(8041) - resourcemanager.succeed("curl http://localhost:8088") - nodemanager.succeed("curl http://localhost:8042") + resourcemanager.succeed("curl -f http://localhost:8088") + nodemanager.succeed("curl -f http://localhost:8042") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/haproxy.nix b/infra/libkookie/nixpkgs/nixos/tests/haproxy.nix index ffb77c052a24..2c3878131b68 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/haproxy.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/haproxy.nix @@ -39,9 +39,9 @@ import ./make-test-python.nix ({ pkgs, ...}: { machine.wait_for_unit("multi-user.target") machine.wait_for_unit("haproxy.service") machine.wait_for_unit("httpd.service") - assert "We are all good!" in machine.succeed("curl -k http://localhost:80/index.txt") + assert "We are all good!" in machine.succeed("curl -fk http://localhost:80/index.txt") assert "haproxy_process_pool_allocated_bytes" in machine.succeed( - "curl -k http://localhost:80/metrics" + "curl -fk http://localhost:80/metrics" ) with subtest("reload"): @@ -49,7 +49,7 @@ import ./make-test-python.nix ({ pkgs, ...}: { # wait some time to ensure the following request hits the reloaded haproxy machine.sleep(5) assert "We are all good!" in machine.succeed( - "curl -k http://localhost:80/index.txt" + "curl -fk http://localhost:80/index.txt" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/hitch/default.nix b/infra/libkookie/nixpkgs/nixos/tests/hitch/default.nix index 904d12619d70..8a2193e75f2a 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/hitch/default.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/hitch/default.nix @@ -28,6 +28,6 @@ import ../make-test-python.nix ({ pkgs, ... }: machine.wait_for_unit("multi-user.target") machine.wait_for_unit("hitch.service") machine.wait_for_open_port(443) - assert "We are all good!" in machine.succeed("curl -k https://localhost:443/index.txt") + assert "We are all good!" in machine.succeed("curl -fk https://localhost:443/index.txt") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/hound.nix b/infra/libkookie/nixpkgs/nixos/tests/hound.nix index 27c65abdf27c..b8b10022bd92 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/hound.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/hound.nix @@ -53,7 +53,7 @@ import ./make-test-python.nix ({ pkgs, ... } : { machine.wait_for_unit("hound.service") machine.wait_for_open_port(6080) machine.wait_until_succeeds( - "curl http://127.0.0.1:6080/api/v1/search\?stats\=fosho\&repos\=\*\&rng=%3A20\&q\=hi\&files\=\&i=nope | grep 'Filename' | grep 'hello'" + "curl -f http://127.0.0.1:6080/api/v1/search\?stats\=fosho\&repos\=\*\&rng=%3A20\&q\=hi\&files\=\&i=nope | grep 'Filename' | grep 'hello'" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/hydra/db-migration.nix b/infra/libkookie/nixpkgs/nixos/tests/hydra/db-migration.nix deleted file mode 100644 index ca65e2e66aa1..000000000000 --- a/infra/libkookie/nixpkgs/nixos/tests/hydra/db-migration.nix +++ /dev/null @@ -1,92 +0,0 @@ -{ system ? builtins.currentSystem -, pkgs ? import ../../.. { inherit system; } -, ... -}: - -let inherit (import ./common.nix { inherit system; }) baseConfig; in - -with import ../../lib/testing-python.nix { inherit system pkgs; }; -with pkgs.lib; - -{ mig = makeTest { - name = "hydra-db-migration"; - meta = with pkgs.stdenv.lib.maintainers; { - maintainers = [ ma27 ]; - }; - - nodes = { - original = { pkgs, lib, ... }: { - imports = [ baseConfig ]; - - # An older version of Hydra before the db change - # for testing purposes. - services.hydra.package = pkgs.hydra-migration.overrideAttrs (old: { - inherit (old) pname; - version = "2020-02-06"; - src = pkgs.fetchFromGitHub { - owner = "NixOS"; - repo = "hydra"; - rev = "2b4f14963b16b21ebfcd6b6bfa7832842e9b2afc"; - sha256 = "16q0cffcsfx5pqd91n9k19850c1nbh4vvbd9h8yi64ihn7v8bick"; - }; - }); - }; - - migration_phase1 = { pkgs, lib, ... }: { - imports = [ baseConfig ]; - services.hydra.package = pkgs.hydra-migration; - }; - - finished = { pkgs, lib, ... }: { - imports = [ baseConfig ]; - services.hydra.package = pkgs.hydra-unstable; - }; - }; - - testScript = { nodes, ... }: let - next = nodes.migration_phase1.config.system.build.toplevel; - finished = nodes.finished.config.system.build.toplevel; - in '' - original.start() - original.wait_for_unit("multi-user.target") - original.wait_for_unit("postgresql.service") - original.wait_for_unit("hydra-init.service") - original.require_unit_state("hydra-queue-runner.service") - original.require_unit_state("hydra-evaluator.service") - original.require_unit_state("hydra-notify.service") - original.succeed("hydra-create-user admin --role admin --password admin") - original.wait_for_open_port(3000) - original.succeed("create-trivial-project.sh") - original.wait_until_succeeds( - 'curl -L -s http://localhost:3000/build/1 -H "Accept: application/json" | jq .buildstatus | xargs test 0 -eq' - ) - - out = original.succeed("su -l postgres -c 'psql -d hydra <<< \"\\d+ builds\" -A'") - assert "jobset_id" not in out - - original.succeed( - "${next}/bin/switch-to-configuration test >&2" - ) - original.wait_for_unit("hydra-init.service") - - out = original.succeed("su -l postgres -c 'psql -d hydra <<< \"\\d+ builds\" -A'") - assert "jobset_id|integer|||" in out - - original.succeed("hydra-backfill-ids") - - original.succeed( - "${finished}/bin/switch-to-configuration test >&2" - ) - original.wait_for_unit("hydra-init.service") - - out = original.succeed("su -l postgres -c 'psql -d hydra <<< \"\\d+ builds\" -A'") - assert "jobset_id|integer||not null|" in out - - original.wait_until_succeeds( - 'curl -L -s http://localhost:3000/build/1 -H "Accept: application/json" | jq .buildstatus | xargs test 0 -eq' - ) - - original.shutdown() - ''; - }; -} diff --git a/infra/libkookie/nixpkgs/nixos/tests/hydra/default.nix b/infra/libkookie/nixpkgs/nixos/tests/hydra/default.nix index 2336e4033d6d..e91a1cd3359d 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/hydra/default.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/hydra/default.nix @@ -11,7 +11,7 @@ let inherit (import ./common.nix { inherit system; }) baseConfig; hydraPkgs = { - inherit (pkgs) hydra-migration hydra-unstable; + inherit (pkgs) hydra-unstable; }; makeHydraTest = with pkgs.lib; name: package: makeTest { diff --git a/infra/libkookie/nixpkgs/nixos/tests/initrd-network-ssh/default.nix b/infra/libkookie/nixpkgs/nixos/tests/initrd-network-ssh/default.nix index 017de6882081..0ad0563b0ce1 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/initrd-network-ssh/default.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/initrd-network-ssh/default.nix @@ -22,6 +22,10 @@ import ../make-test-python.nix ({ lib, ... }: hostKeys = [ ./ssh_host_ed25519_key ]; }; }; + boot.initrd.extraUtilsCommands = '' + mkdir -p $out/secrets/etc/ssh + cat "${./ssh_host_ed25519_key}" > $out/secrets/etc/ssh/sh_host_ed25519_key + ''; boot.initrd.preLVMCommands = '' while true; do if [ -f fnord ]; then diff --git a/infra/libkookie/nixpkgs/nixos/tests/installed-tests/default.nix b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/default.nix index 889a00d4b568..e5d7009bb7b9 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/installed-tests/default.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/default.nix @@ -94,6 +94,7 @@ in glib-networking = callInstalledTest ./glib-networking.nix {}; gnome-photos = callInstalledTest ./gnome-photos.nix {}; graphene = callInstalledTest ./graphene.nix {}; + gsconnect = callInstalledTest ./gsconnect.nix {}; ibus = callInstalledTest ./ibus.nix {}; libgdata = callInstalledTest ./libgdata.nix {}; glib-testing = callInstalledTest ./glib-testing.nix {}; @@ -101,5 +102,6 @@ in libxmlb = callInstalledTest ./libxmlb.nix {}; malcontent = callInstalledTest ./malcontent.nix {}; ostree = callInstalledTest ./ostree.nix {}; + pipewire = callInstalledTest ./pipewire.nix {}; xdg-desktop-portal = callInstalledTest ./xdg-desktop-portal.nix {}; } diff --git a/infra/libkookie/nixpkgs/nixos/tests/installed-tests/fwupd.nix b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/fwupd.nix index 6a0ceb57dda4..a8a683a1af7b 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/installed-tests/fwupd.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/fwupd.nix @@ -5,7 +5,7 @@ makeInstalledTest { testConfig = { services.fwupd.enable = true; - services.fwupd.blacklistPlugins = lib.mkForce []; # don't blacklist test plugin + services.fwupd.disabledPlugins = lib.mkForce []; # don't disable test plugin services.fwupd.enableTestRemote = true; virtualisation.memorySize = 768; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/installed-tests/gsconnect.nix b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/gsconnect.nix new file mode 100644 index 000000000000..ac39f7435786 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/gsconnect.nix @@ -0,0 +1,7 @@ +{ pkgs, makeInstalledTest, ... }: + +makeInstalledTest { + tested = pkgs.gnomeExtensions.gsconnect; + + withX11 = true; +} diff --git a/infra/libkookie/nixpkgs/nixos/tests/installed-tests/pipewire.nix b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/pipewire.nix new file mode 100644 index 000000000000..f4154b5d2fd7 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/installed-tests/pipewire.nix @@ -0,0 +1,5 @@ +{ pkgs, lib, makeInstalledTest, ... }: + +makeInstalledTest { + tested = pkgs.pipewire; +} diff --git a/infra/libkookie/nixpkgs/nixos/tests/jq.nix b/infra/libkookie/nixpkgs/nixos/tests/jq.nix new file mode 100644 index 000000000000..20b67522ee6e --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/jq.nix @@ -0,0 +1,10 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "jq"; + meta = with pkgs.stdenv.lib.maintainers; { maintainers = [ nequissimus ]; }; + + nodes.jq = { pkgs, ... }: { environment.systemPackages = [ pkgs.jq ]; }; + + testScript = '' + assert "world" in jq.succeed('echo \'{"values":["hello","world"]}\'| jq \'.values[1]\''') + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/keycloak.nix b/infra/libkookie/nixpkgs/nixos/tests/keycloak.nix new file mode 100644 index 000000000000..f448a0f7095f --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/keycloak.nix @@ -0,0 +1,144 @@ +# This tests Keycloak: it starts the service, creates a realm with an +# OIDC client and a user, and simulates the user logging in to the +# client using their Keycloak login. + +let + frontendUrl = "http://keycloak/auth"; + initialAdminPassword = "h4IhoJFnt2iQIR9"; + + keycloakTest = import ./make-test-python.nix ( + { pkgs, databaseType, ... }: + { + name = "keycloak"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ talyz ]; + }; + + nodes = { + keycloak = { ... }: { + virtualisation.memorySize = 1024; + services.keycloak = { + enable = true; + inherit frontendUrl databaseType initialAdminPassword; + databasePasswordFile = pkgs.writeText "dbPassword" "wzf6vOCbPp6cqTH"; + }; + environment.systemPackages = with pkgs; [ + xmlstarlet + libtidy + jq + ]; + }; + }; + + testScript = + let + client = { + clientId = "test-client"; + name = "test-client"; + redirectUris = [ "urn:ietf:wg:oauth:2.0:oob" ]; + }; + + user = { + firstName = "Chuck"; + lastName = "Testa"; + username = "chuck.testa"; + email = "chuck.testa@example.com"; + }; + + password = "password1234"; + + realm = { + enabled = true; + realm = "test-realm"; + clients = [ client ]; + users = [( + user // { + enabled = true; + credentials = [{ + type = "password"; + temporary = false; + value = password; + }]; + } + )]; + }; + + realmDataJson = pkgs.writeText "realm-data.json" (builtins.toJSON realm); + + jqCheckUserinfo = pkgs.writeText "check-userinfo.jq" '' + if { + "firstName": .given_name, + "lastName": .family_name, + "username": .preferred_username, + "email": .email + } != ${builtins.toJSON user} then + error("Wrong user info!") + else + empty + end + ''; + in '' + keycloak.start() + keycloak.wait_for_unit("keycloak.service") + keycloak.wait_until_succeeds("curl -sSf ${frontendUrl}") + + + ### Realm Setup ### + + # Get an admin interface access token + keycloak.succeed( + "curl -sSf -d 'client_id=admin-cli' -d 'username=admin' -d 'password=${initialAdminPassword}' -d 'grant_type=password' '${frontendUrl}/realms/master/protocol/openid-connect/token' | jq -r '\"Authorization: bearer \" + .access_token' >admin_auth_header" + ) + + # Publish the realm, including a test OIDC client and user + keycloak.succeed( + "curl -sSf -H @admin_auth_header -X POST -H 'Content-Type: application/json' -d @${realmDataJson} '${frontendUrl}/admin/realms/'" + ) + + # Generate and save the client secret. To do this we need + # Keycloak's internal id for the client. + keycloak.succeed( + "curl -sSf -H @admin_auth_header '${frontendUrl}/admin/realms/${realm.realm}/clients?clientId=${client.name}' | jq -r '.[].id' >client_id", + "curl -sSf -H @admin_auth_header -X POST '${frontendUrl}/admin/realms/${realm.realm}/clients/'$(<client_id)'/client-secret' | jq -r .value >client_secret", + ) + + + ### Authentication Testing ### + + # Start the login process by sending an initial request to the + # OIDC authentication endpoint, saving the returned page. Tidy + # up the HTML (XmlStarlet is picky) and extract the login form + # post url. + keycloak.succeed( + "curl -sSf -c cookie '${frontendUrl}/realms/${realm.realm}/protocol/openid-connect/auth?client_id=${client.name}&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=openid+email&response_type=code&response_mode=query&nonce=qw4o89g3qqm' >login_form", + "tidy -q -m login_form || true", + "xml sel -T -t -m \"_:html/_:body/_:div/_:div/_:div/_:div/_:div/_:div/_:form[@id='kc-form-login']\" -v @action login_form >form_post_url", + ) + + # Post the login form and save the response. Once again tidy up + # the HTML, then extract the authorization code. + keycloak.succeed( + "curl -sSf -L -b cookie -d 'username=${user.username}' -d 'password=${password}' -d 'credentialId=' \"$(<form_post_url)\" >auth_code_html", + "tidy -q -m auth_code_html || true", + "xml sel -T -t -m \"_:html/_:body/_:div/_:div/_:div/_:div/_:div/_:input[@id='code']\" -v @value auth_code_html >auth_code", + ) + + # Exchange the authorization code for an access token. + keycloak.succeed( + "curl -sSf -d grant_type=authorization_code -d code=$(<auth_code) -d client_id=${client.name} -d client_secret=$(<client_secret) -d redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob '${frontendUrl}/realms/${realm.realm}/protocol/openid-connect/token' | jq -r '\"Authorization: bearer \" + .access_token' >auth_header" + ) + + # Use the access token on the OIDC userinfo endpoint and check + # that the returned user info matches what we initialized the + # realm with. + keycloak.succeed( + "curl -sSf -H @auth_header '${frontendUrl}/realms/${realm.realm}/protocol/openid-connect/userinfo' | jq -f ${jqCheckUserinfo}" + ) + ''; + } + ); +in +{ + postgres = keycloakTest { databaseType = "postgresql"; }; + mysql = keycloakTest { databaseType = "mysql"; }; +} diff --git a/infra/libkookie/nixpkgs/nixos/tests/leaps.nix b/infra/libkookie/nixpkgs/nixos/tests/leaps.nix index ac0c602d4450..ec5b69a76290 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/leaps.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/leaps.nix @@ -26,7 +26,7 @@ import ./make-test-python.nix ({ pkgs, ... }: server.wait_for_open_port(6666) client.wait_for_unit("network.target") assert "leaps" in client.succeed( - "${pkgs.curl}/bin/curl http://server:6666/leaps/" + "${pkgs.curl}/bin/curl -f http://server:6666/leaps/" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/limesurvey.nix b/infra/libkookie/nixpkgs/nixos/tests/limesurvey.nix index 7228fcb83315..dad807fb7330 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/limesurvey.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/limesurvey.nix @@ -20,7 +20,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { machine.wait_for_unit("phpfpm-limesurvey.service") assert "The following surveys are available" in machine.succeed( - "curl http://example.local/" + "curl -f http://example.local/" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/loki.nix b/infra/libkookie/nixpkgs/nixos/tests/loki.nix index dbf1e8a650f5..bede775b7d3c 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/loki.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/loki.nix @@ -12,15 +12,28 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: enable = true; configFile = "${pkgs.grafana-loki.src}/cmd/loki/loki-local-config.yaml"; }; - systemd.services.promtail = { - description = "Promtail service for Loki test"; - wantedBy = [ "multi-user.target" ]; - - serviceConfig = { - ExecStart = '' - ${pkgs.grafana-loki}/bin/promtail --config.file ${pkgs.grafana-loki.src}/cmd/promtail/promtail-local-config.yaml - ''; - DynamicUser = true; + services.promtail = { + enable = true; + configuration = { + server = { + http_listen_port = 9080; + grpc_listen_port = 0; + }; + clients = [ { url = "http://localhost:3100/loki/api/v1/push"; } ]; + scrape_configs = [ + { + job_name = "system"; + static_configs = [ + { + targets = [ "localhost" ]; + labels = { + job = "varlogs"; + __path__ = "/var/log/*log"; + }; + } + ]; + } + ]; }; }; }; @@ -32,6 +45,8 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: machine.wait_for_open_port(3100) machine.wait_for_open_port(9080) machine.succeed("echo 'Loki Ingestion Test' > /var/log/testlog") + # should not have access to journal unless specified + machine.fail("systemctl show --property=SupplementaryGroups promtail | grep -q systemd-journal") machine.wait_until_succeeds( "${pkgs.grafana-loki}/bin/logcli --addr='http://localhost:3100' query --no-labels '{job=\"varlogs\",filename=\"/var/log/testlog\"}' | grep -q 'Loki Ingestion Test'" ) diff --git a/infra/libkookie/nixpkgs/nixos/tests/lsd.nix b/infra/libkookie/nixpkgs/nixos/tests/lsd.nix new file mode 100644 index 000000000000..fee8e95e14ff --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/lsd.nix @@ -0,0 +1,12 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "lsd"; + meta = with pkgs.stdenv.lib.maintainers; { maintainers = [ nequissimus ]; }; + + nodes.lsd = { pkgs, ... }: { environment.systemPackages = [ pkgs.lsd ]; }; + + testScript = '' + lsd.succeed('echo "abc" > /tmp/foo') + assert "4 B /tmp/foo" in lsd.succeed('lsd --classic --blocks "size,name" /tmp/foo') + assert "lsd ${pkgs.lsd.version}" in lsd.succeed("lsd --version") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/mailcatcher.nix b/infra/libkookie/nixpkgs/nixos/tests/mailcatcher.nix index 2ef38544fe0a..a55fba8a9950 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/mailcatcher.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/mailcatcher.nix @@ -24,7 +24,7 @@ import ./make-test-python.nix ({ lib, ... }: 'echo "this is the body of the email" | mail -s "subject" root@example.org' ) assert "this is the body of the email" in machine.succeed( - "curl http://localhost:1080/messages/1.source" + "curl -f http://localhost:1080/messages/1.source" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/matrix-synapse.nix b/infra/libkookie/nixpkgs/nixos/tests/matrix-synapse.nix index 9ca808721763..6c8f1e188d52 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/matrix-synapse.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/matrix-synapse.nix @@ -77,12 +77,12 @@ in { start_all() serverpostgres.wait_for_unit("matrix-synapse.service") serverpostgres.wait_until_succeeds( - "curl -L --cacert ${ca_pem} https://localhost:8448/" + "curl --fail -L --cacert ${ca_pem} https://localhost:8448/" ) serverpostgres.require_unit_state("postgresql.service") serversqlite.wait_for_unit("matrix-synapse.service") serversqlite.wait_until_succeeds( - "curl -L --cacert ${ca_pem} https://localhost:8448/" + "curl --fail -L --cacert ${ca_pem} https://localhost:8448/" ) serversqlite.succeed("[ -e /var/lib/matrix-synapse/homeserver.db ]") ''; diff --git a/infra/libkookie/nixpkgs/nixos/tests/mediawiki.nix b/infra/libkookie/nixpkgs/nixos/tests/mediawiki.nix index 008682310cf6..702fefefa161 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/mediawiki.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/mediawiki.nix @@ -22,7 +22,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: { machine.wait_for_unit("phpfpm-mediawiki.service") - page = machine.succeed("curl -L http://localhost/") + page = machine.succeed("curl -fL http://localhost/") assert "MediaWiki has been installed" in page ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/metabase.nix b/infra/libkookie/nixpkgs/nixos/tests/metabase.nix index 1450a4e9086f..65619cc793a7 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/metabase.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/metabase.nix @@ -15,6 +15,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { start_all() machine.wait_for_unit("metabase.service") machine.wait_for_open_port(3000) - machine.wait_until_succeeds("curl -L http://localhost:3000/setup | grep Metabase") + machine.wait_until_succeeds("curl -fL http://localhost:3000/setup | grep Metabase") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/minecraft-server.nix b/infra/libkookie/nixpkgs/nixos/tests/minecraft-server.nix new file mode 100644 index 000000000000..53780e4636ca --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/minecraft-server.nix @@ -0,0 +1,37 @@ +let + seed = "2151901553968352745"; + rcon-pass = "foobar"; + rcon-port = 43000; +in import ./make-test-python.nix ({ pkgs, ... }: { + name = "minecraft-server"; + meta = with pkgs.stdenv.lib.maintainers; { maintainers = [ nequissimus ]; }; + + nodes.server = { ... }: { + environment.systemPackages = [ pkgs.mcrcon ]; + + nixpkgs.config.allowUnfree = true; + + services.minecraft-server = { + declarative = true; + enable = true; + eula = true; + serverProperties = { + enable-rcon = true; + level-seed = seed; + online-mode = false; + "rcon.password" = rcon-pass; + "rcon.port" = rcon-port; + }; + }; + + virtualisation.memorySize = 2048; + }; + + testScript = '' + server.wait_for_unit("minecraft-server") + server.wait_for_open_port(${toString rcon-port}) + assert "${seed}" in server.succeed( + "mcrcon -H localhost -P ${toString rcon-port} -p '${rcon-pass}' -c 'seed'" + ) + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/minecraft.nix b/infra/libkookie/nixpkgs/nixos/tests/minecraft.nix new file mode 100644 index 000000000000..e0c35f2d2769 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/minecraft.nix @@ -0,0 +1,28 @@ +import ./make-test-python.nix ({ pkgs, lib, ... }: { + name = "minecraft"; + meta = with lib.maintainers; { maintainers = [ nequissimus ]; }; + + nodes.client = { nodes, ... }: + let user = nodes.client.config.users.users.alice; + in { + imports = [ ./common/user-account.nix ./common/x11.nix ]; + + environment.systemPackages = [ pkgs.minecraft ]; + + nixpkgs.config.allowUnfree = true; + + test-support.displayManager.auto.user = user.name; + }; + + enableOCR = true; + + testScript = { nodes, ... }: + let user = nodes.client.config.users.users.alice; + in '' + client.wait_for_x() + client.execute("su - alice -c minecraft-launcher &") + client.wait_for_text("CONTINUE WITHOUT LOGIN") + client.sleep(10) + client.screenshot("launcher") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/misc.nix b/infra/libkookie/nixpkgs/nixos/tests/misc.nix index ae1505532734..40661cdca0a1 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/misc.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/misc.nix @@ -88,8 +88,8 @@ import ./make-test-python.nix ({ pkgs, ...} : rec { with subtest("whether kernel.poweroff_cmd is set"): machine.succeed('[ -x "$(cat /proc/sys/kernel/poweroff_cmd)" ]') - with subtest("whether the blkio controller is properly enabled"): - machine.succeed("[ -e /sys/fs/cgroup/blkio/blkio.reset_stats ]") + with subtest("whether the io cgroupv2 controller is properly enabled"): + machine.succeed("grep -q '\\bio\\b' /sys/fs/cgroup/cgroup.controllers") with subtest("whether we have a reboot record in wtmp"): machine.shutdown diff --git a/infra/libkookie/nixpkgs/nixos/tests/morty.nix b/infra/libkookie/nixpkgs/nixos/tests/morty.nix index 64c5a27665d6..924dce2717e3 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/morty.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/morty.nix @@ -12,9 +12,9 @@ import ./make-test-python.nix ({ pkgs, ... }: { ... }: { services.morty = { enable = true; - key = "78a9cd0cfee20c672f78427efb2a2a96036027f0"; - port = 3001; - }; + key = "78a9cd0cfee20c672f78427efb2a2a96036027f0"; + port = 3001; + }; }; }; @@ -24,7 +24,7 @@ import ./make-test-python.nix ({ pkgs, ... }: '' mortyProxyWithKey.wait_for_unit("default.target") mortyProxyWithKey.wait_for_open_port(3001) - mortyProxyWithKey.succeed("curl -L 127.0.0.1:3001 | grep MortyProxy") + mortyProxyWithKey.succeed("curl -fL 127.0.0.1:3001 | grep MortyProxy") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/n8n.nix b/infra/libkookie/nixpkgs/nixos/tests/n8n.nix new file mode 100644 index 000000000000..ed93639f2a42 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/n8n.nix @@ -0,0 +1,25 @@ +import ./make-test-python.nix ({ lib, ... }: + +with lib; + +let + port = 5678; +in +{ + name = "n8n"; + meta.maintainers = with maintainers; [ freezeboy ]; + + nodes.machine = + { pkgs, ... }: + { + services.n8n = { + enable = true; + }; + }; + + testScript = '' + machine.wait_for_unit("n8n.service") + machine.wait_for_open_port("${toString port}") + machine.succeed("curl --fail http://localhost:${toString port}/") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/nano.nix b/infra/libkookie/nixpkgs/nixos/tests/nano.nix new file mode 100644 index 000000000000..9e0a9e147f2c --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/nano.nix @@ -0,0 +1,44 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "nano"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { lib, ... }: { + environment.systemPackages = [ pkgs.nano ]; + }; + + testScript = { ... }: '' + start_all() + + with subtest("Create user and log in"): + machine.wait_for_unit("multi-user.target") + machine.wait_until_succeeds("pgrep -f 'agetty.*tty1'") + machine.succeed("useradd -m alice") + machine.succeed("(echo foobar; echo foobar) | passwd alice") + machine.wait_until_tty_matches(1, "login: ") + machine.send_chars("alice\n") + machine.wait_until_tty_matches(1, "login: alice") + machine.wait_until_succeeds("pgrep login") + machine.wait_until_tty_matches(1, "Password: ") + machine.send_chars("foobar\n") + machine.wait_until_succeeds("pgrep -u alice bash") + machine.screenshot("prompt") + + with subtest("Use nano"): + machine.send_chars("nano /tmp/foo") + machine.send_key("ret") + machine.sleep(2) + machine.send_chars("42") + machine.sleep(1) + machine.send_key("ctrl-x") + machine.sleep(1) + machine.send_key("y") + machine.sleep(1) + machine.screenshot("nano") + machine.sleep(1) + machine.send_key("ret") + machine.wait_for_file("/tmp/foo") + assert "42" in machine.succeed("cat /tmp/foo") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/nar-serve.nix b/infra/libkookie/nixpkgs/nixos/tests/nar-serve.nix new file mode 100644 index 000000000000..9ee738ffb170 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/nar-serve.nix @@ -0,0 +1,48 @@ +import ./make-test-python.nix ( + { pkgs, lib, ... }: + { + name = "nar-serve"; + meta.maintainers = [ lib.maintainers.rizary ]; + nodes = + { + server = { pkgs, ... }: { + services.nginx = { + enable = true; + virtualHosts.default.root = "/var/www"; + }; + services.nar-serve = { + enable = true; + # Connect to the localhost nginx instead of the default + # https://cache.nixos.org + cacheURL = "http://localhost/"; + }; + environment.systemPackages = [ + pkgs.hello + pkgs.curl + ]; + + networking.firewall.allowedTCPPorts = [ 8383 ]; + + # virtualisation.diskSize = 2 * 1024; + }; + }; + testScript = '' + start_all() + + # Create a fake cache with Nginx service the static files + server.succeed( + "nix copy --to file:///var/www ${pkgs.hello}" + ) + server.wait_for_unit("nginx.service") + server.wait_for_open_port(80) + + # Check that nar-serve can return the content of the derivation + drvName = os.path.basename("${pkgs.hello}") + drvHash = drvName.split("-")[0] + server.wait_for_unit("nar-serve.service") + server.succeed( + "curl -o hello -f http://localhost:8383/nix/store/{}/bin/hello".format(drvHash) + ) + ''; + } +) diff --git a/infra/libkookie/nixpkgs/nixos/tests/neo4j.nix b/infra/libkookie/nixpkgs/nixos/tests/neo4j.nix index 32ee7f501b8b..8329e5630d7a 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/neo4j.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/neo4j.nix @@ -15,6 +15,6 @@ import ./make-test-python.nix { master.wait_for_unit("neo4j") master.wait_for_open_port(7474) - master.succeed("curl http://localhost:7474/") + master.succeed("curl -f http://localhost:7474/") ''; } diff --git a/infra/libkookie/nixpkgs/nixos/tests/networking.nix b/infra/libkookie/nixpkgs/nixos/tests/networking.nix index 83d4f6465b68..4fc5d48e0e17 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/networking.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/networking.nix @@ -499,8 +499,8 @@ let list, targetList ) with subtest("Test MTU and MAC Address are configured"): - assert "mtu 1342" in machine.succeed("ip link show dev tap0") - assert "mtu 1343" in machine.succeed("ip link show dev tun0") + machine.wait_until_succeeds("ip link show dev tap0 | grep 'mtu 1342'") + machine.wait_until_succeeds("ip link show dev tun0 | grep 'mtu 1343'") assert "02:de:ad:be:ef:01" in machine.succeed("ip link show dev tap0") '' # network-addresses-* only exist in scripted networking + optionalString (!networkd) '' diff --git a/infra/libkookie/nixpkgs/nixos/tests/nginx-auth.nix b/infra/libkookie/nixpkgs/nixos/tests/nginx-auth.nix new file mode 100644 index 000000000000..c0d24a20ddbc --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/nginx-auth.nix @@ -0,0 +1,47 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "nginx-auth"; + + nodes = { + webserver = { pkgs, lib, ... }: { + services.nginx = let + root = pkgs.runCommand "testdir" {} '' + mkdir "$out" + echo hello world > "$out/index.html" + ''; + in { + enable = true; + + virtualHosts.lockedroot = { + inherit root; + basicAuth.alice = "jane"; + }; + + virtualHosts.lockedsubdir = { + inherit root; + locations."/sublocation/" = { + alias = "${root}/"; + basicAuth.bob = "john"; + }; + }; + }; + }; + }; + + testScript = '' + webserver.wait_for_unit("nginx") + webserver.wait_for_open_port(80) + + webserver.fail("curl --fail --resolve lockedroot:80:127.0.0.1 http://lockedroot") + webserver.succeed( + "curl --fail --resolve lockedroot:80:127.0.0.1 http://alice:jane@lockedroot" + ) + + webserver.succeed("curl --fail --resolve lockedsubdir:80:127.0.0.1 http://lockedsubdir") + webserver.fail( + "curl --fail --resolve lockedsubdir:80:127.0.0.1 http://lockedsubdir/sublocation/index.html" + ) + webserver.succeed( + "curl --fail --resolve lockedsubdir:80:127.0.0.1 http://bob:john@lockedsubdir/sublocation/index.html" + ) + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/nixos-generate-config.nix b/infra/libkookie/nixpkgs/nixos/tests/nixos-generate-config.nix index 6c83ccecc70a..5daa55a8abbe 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/nixos-generate-config.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/nixos-generate-config.nix @@ -7,8 +7,16 @@ import ./make-test-python.nix ({ lib, ... } : { { config, pkgs, ... }: { imports = [ ./hardware-configuration.nix ]; $bootLoaderConfig + $desktopConfiguration } ''; + + system.nixos-generate-config.desktopConfiguration = '' + # DESKTOP + # services.xserver.enable = true; + # services.xserver.displayManager.gdm.enable = true; + # services.xserver.desktopManager.gnome3.enable = true; + ''; }; testScript = '' start_all() @@ -18,9 +26,17 @@ import ./make-test-python.nix ({ lib, ... } : { # Test if the configuration really is overridden machine.succeed("grep 'OVERRIDDEN' /etc/nixos/configuration.nix") + # Test if desktop configuration really is overridden + machine.succeed("grep 'DESKTOP' /etc/nixos/configuration.nix") + # Test of if the Perl variable $bootLoaderConfig is spliced correctly: machine.succeed( "grep 'boot\\.loader\\.grub\\.enable = true;' /etc/nixos/configuration.nix" ) + + # Test if the Perl variable $desktopConfiguration is spliced correctly + machine.succeed( + "grep 'services\\.xserver\\.desktopManager\\.gnome3\\.enable = true;' /etc/nixos/configuration.nix" + ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/nzbget.nix b/infra/libkookie/nixpkgs/nixos/tests/nzbget.nix index 12d8ed6ea8da..b39c9b035e61 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/nzbget.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/nzbget.nix @@ -21,7 +21,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { server.wait_for_unit("network.target") server.wait_for_open_port(6789) assert "This file is part of nzbget" in server.succeed( - "curl -s -u nzbget:tegbzn6789 http://127.0.0.1:6789" + "curl -f -s -u nzbget:tegbzn6789 http://127.0.0.1:6789" ) server.succeed( "${pkgs.nzbget}/bin/nzbget -n -o Control_iP=127.0.0.1 -o Control_port=6789 -o Control_password=tegbzn6789 -V" diff --git a/infra/libkookie/nixpkgs/nixos/tests/oci-containers.nix b/infra/libkookie/nixpkgs/nixos/tests/oci-containers.nix index bb6c019f07c9..0dfc7ffb276b 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/oci-containers.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/oci-containers.nix @@ -32,7 +32,7 @@ let start_all() ${backend}.wait_for_unit("${backend}-nginx.service") ${backend}.wait_for_open_port(8181) - ${backend}.wait_until_succeeds("curl http://localhost:8181 | grep Hello") + ${backend}.wait_until_succeeds("curl -f http://localhost:8181 | grep Hello") ''; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/oh-my-zsh.nix b/infra/libkookie/nixpkgs/nixos/tests/oh-my-zsh.nix new file mode 100644 index 000000000000..57a073b086e8 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/oh-my-zsh.nix @@ -0,0 +1,18 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "oh-my-zsh"; + + machine = { pkgs, ... }: + + { + programs.zsh = { + enable = true; + ohMyZsh.enable = true; + }; + }; + + testScript = '' + start_all() + machine.succeed("touch ~/.zshrc") + machine.succeed("zsh -c 'source /etc/zshrc && echo $ZSH | grep oh-my-zsh-${pkgs.oh-my-zsh.version}'") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/openldap.nix b/infra/libkookie/nixpkgs/nixos/tests/openldap.nix index f8321a2c522d..392fae243467 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/openldap.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/openldap.nix @@ -1,33 +1,125 @@ -import ./make-test-python.nix { - name = "openldap"; - - machine = { pkgs, ... }: { - services.openldap = { - enable = true; - suffix = "dc=example"; - rootdn = "cn=root,dc=example"; - rootpw = "notapassword"; - database = "bdb"; - extraDatabaseConfig = '' - directory /var/db/openldap - ''; - declarativeContents = '' - dn: dc=example - objectClass: domain - dc: example - - dn: ou=users,dc=example - objectClass: organizationalUnit - ou: users - ''; - }; - }; +{ pkgs, system ? builtins.currentSystem, ... }: let + dbContents = '' + dn: dc=example + objectClass: domain + dc: example + dn: ou=users,dc=example + objectClass: organizationalUnit + ou: users + ''; testScript = '' machine.wait_for_unit("openldap.service") machine.succeed( - "systemctl status openldap.service", 'ldapsearch -LLL -D "cn=root,dc=example" -w notapassword -b "dc=example"', ) ''; +in { + # New-style configuration + current = import ./make-test-python.nix { + inherit testScript; + name = "openldap"; + + machine = { pkgs, ... }: { + environment.etc."openldap/root_password".text = "notapassword"; + services.openldap = { + enable = true; + settings = { + children = { + "cn=schema".includes = [ + "${pkgs.openldap}/etc/schema/core.ldif" + "${pkgs.openldap}/etc/schema/cosine.ldif" + "${pkgs.openldap}/etc/schema/inetorgperson.ldif" + "${pkgs.openldap}/etc/schema/nis.ldif" + ]; + "olcDatabase={1}mdb" = { + # This tests string, base64 and path values, as well as lists of string values + attrs = { + objectClass = [ "olcDatabaseConfig" "olcMdbConfig" ]; + olcDatabase = "{1}mdb"; + olcDbDirectory = "/var/db/openldap"; + olcSuffix = "dc=example"; + olcRootDN = { + # cn=root,dc=example + base64 = "Y249cm9vdCxkYz1leGFtcGxl"; + }; + olcRootPW = { + path = "/etc/openldap/root_password"; + }; + }; + }; + }; + }; + declarativeContents."dc=example" = dbContents; + }; + }; + }; + + # Old-style configuration + oldOptions = import ./make-test-python.nix { + inherit testScript; + name = "openldap"; + + machine = { pkgs, ... }: { + services.openldap = { + enable = true; + logLevel = "stats acl"; + defaultSchemas = true; + database = "mdb"; + suffix = "dc=example"; + rootdn = "cn=root,dc=example"; + rootpw = "notapassword"; + declarativeContents."dc=example" = dbContents; + }; + }; + }; + + # Manually managed configDir, for example if dynamic config is essential + manualConfigDir = import ./make-test-python.nix { + name = "openldap"; + + machine = { pkgs, ... }: { + services.openldap = { + enable = true; + configDir = "/var/db/slapd.d"; + }; + }; + + testScript = let + contents = pkgs.writeText "data.ldif" dbContents; + config = pkgs.writeText "config.ldif" '' + dn: cn=config + cn: config + objectClass: olcGlobal + olcLogLevel: stats + olcPidFile: /run/slapd/slapd.pid + + dn: cn=schema,cn=config + cn: schema + objectClass: olcSchemaConfig + + include: file://${pkgs.openldap}/etc/schema/core.ldif + include: file://${pkgs.openldap}/etc/schema/cosine.ldif + include: file://${pkgs.openldap}/etc/schema/inetorgperson.ldif + + dn: olcDatabase={1}mdb,cn=config + objectClass: olcDatabaseConfig + objectClass: olcMdbConfig + olcDatabase: {1}mdb + olcDbDirectory: /var/db/openldap + olcDbIndex: objectClass eq + olcSuffix: dc=example + olcRootDN: cn=root,dc=example + olcRootPW: notapassword + ''; + in '' + machine.succeed( + "mkdir -p /var/db/slapd.d /var/db/openldap", + "slapadd -F /var/db/slapd.d -n0 -l ${config}", + "slapadd -F /var/db/slapd.d -n1 -l ${contents}", + "chown -R openldap:openldap /var/db/slapd.d /var/db/openldap", + "systemctl restart openldap", + ) + '' + testScript; + }; } diff --git a/infra/libkookie/nixpkgs/nixos/tests/os-prober.nix b/infra/libkookie/nixpkgs/nixos/tests/os-prober.nix index be0235a41753..f778d30bdc06 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/os-prober.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/os-prober.nix @@ -9,7 +9,7 @@ let ${parted}/sbin/parted --script /dev/vda -- mkpart primary ext2 1M -1s mkdir /mnt ${e2fsprogs}/bin/mkfs.ext4 /dev/vda1 - ${utillinux}/bin/mount -t ext4 /dev/vda1 /mnt + ${util-linux}/bin/mount -t ext4 /dev/vda1 /mnt if test -e /mnt/.debug; then exec ${bash}/bin/sh diff --git a/infra/libkookie/nixpkgs/nixos/tests/osrm-backend.nix b/infra/libkookie/nixpkgs/nixos/tests/osrm-backend.nix index db67a5a589f9..4067d5b1a239 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/osrm-backend.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/osrm-backend.nix @@ -48,10 +48,10 @@ in { machine.wait_for_unit("osrm.service") machine.wait_for_open_port(${toString port}) assert "Boulevard Rainier III" in machine.succeed( - "curl --silent '${query}' | jq .waypoints[0].name" + "curl --fail --silent '${query}' | jq .waypoints[0].name" ) assert "Avenue de la Costa" in machine.succeed( - "curl --silent '${query}' | jq .waypoints[1].name" + "curl --fail --silent '${query}' | jq .waypoints[1].name" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/paperless.nix b/infra/libkookie/nixpkgs/nixos/tests/paperless.nix index 355e7041d3fe..fb83e6f976de 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/paperless.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/paperless.nix @@ -23,14 +23,14 @@ import ./make-test-python.nix ({ lib, ... } : { with subtest("Service gets ready"): machine.wait_for_unit("paperless-server.service") # Wait until server accepts connections - machine.wait_until_succeeds("curl -s localhost:28981") + machine.wait_until_succeeds("curl -fs localhost:28981") with subtest("Test document is consumed"): machine.wait_until_succeeds( - "(($(curl -s localhost:28981/api/documents/ | jq .count) == 1))" + "(($(curl -fs localhost:28981/api/documents/ | jq .count) == 1))" ) assert "2005-10-16" in machine.succeed( - "curl -s localhost:28981/api/documents/ | jq '.results | .[0] | .created'" + "curl -fs localhost:28981/api/documents/ | jq '.results | .[0] | .created'" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/peerflix.nix b/infra/libkookie/nixpkgs/nixos/tests/peerflix.nix index 37628604d49b..6e534dedc471 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/peerflix.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/peerflix.nix @@ -18,6 +18,6 @@ import ./make-test-python.nix ({ pkgs, ...} : { start_all() peerflix.wait_for_unit("peerflix.service") - peerflix.wait_until_succeeds("curl localhost:9000") + peerflix.wait_until_succeeds("curl -f localhost:9000") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/php/fpm.nix b/infra/libkookie/nixpkgs/nixos/tests/php/fpm.nix index 513abd943737..9ad515ebdde0 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/php/fpm.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/php/fpm.nix @@ -43,7 +43,7 @@ import ../make-test-python.nix ({pkgs, lib, ...}: { machine.wait_for_unit("phpfpm-foobar.service") # Check so we get an evaluated PHP back - response = machine.succeed("curl -vvv -s http://127.0.0.1:80/") + response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/") assert "PHP Version ${pkgs.php.version}" in response, "PHP version not detected" # Check so we have database and some other extensions loaded diff --git a/infra/libkookie/nixpkgs/nixos/tests/php/httpd.nix b/infra/libkookie/nixpkgs/nixos/tests/php/httpd.nix index 1092e0ecadd3..27ea7a24e3a9 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/php/httpd.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/php/httpd.nix @@ -21,7 +21,7 @@ import ../make-test-python.nix ({pkgs, lib, ...}: { machine.wait_for_unit("httpd.service") # Check so we get an evaluated PHP back - response = machine.succeed("curl -vvv -s http://127.0.0.1:80/") + response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/") assert "PHP Version ${pkgs.php.version}" in response, "PHP version not detected" # Check so we have database and some other extensions loaded diff --git a/infra/libkookie/nixpkgs/nixos/tests/php/pcre.nix b/infra/libkookie/nixpkgs/nixos/tests/php/pcre.nix index 3dd0964e60fb..3ea19304bffd 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/php/pcre.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/php/pcre.nix @@ -32,7 +32,7 @@ in import ../make-test-python.nix ({lib, ...}: { '' machine.wait_for_unit("httpd.service") # Ensure php evaluation by matching on the var_dump syntax - response = machine.succeed("curl -vvv -s http://127.0.0.1:80/index.php") + response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/index.php") expected = 'string(${toString (builtins.stringLength testString)}) "${testString}"' assert expected in response, "Does not appear to be able to use subgroups." ''; diff --git a/infra/libkookie/nixpkgs/nixos/tests/podman.nix b/infra/libkookie/nixpkgs/nixos/tests/podman.nix index cd8c2b4308c8..dd28563dc4c1 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/podman.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/podman.nix @@ -34,7 +34,6 @@ import ./make-test-python.nix ( podman.wait_for_unit("sockets.target") start_all() - with subtest("Run container as root with runc"): podman.succeed("tar cv --files-from /dev/null | podman import - scratchimg") podman.succeed( @@ -53,22 +52,38 @@ import ./make-test-python.nix ( podman.succeed("podman stop sleeping") podman.succeed("podman rm sleeping") - with subtest("Run container rootless with runc"): + with subtest("Run container as root with the default backend"): + podman.succeed("tar cv --files-from /dev/null | podman import - scratchimg") + podman.succeed( + "podman run -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10" + ) + podman.succeed("podman ps | grep sleeping") + podman.succeed("podman stop sleeping") + podman.succeed("podman rm sleeping") + + + podman.succeed( + "mkdir -p /tmp/podman-run-1000/libpod && chown alice -R /tmp/podman-run-1000" + ) + + + with subtest("Run container rootless with crun"): podman.succeed(su_cmd("tar cv --files-from /dev/null | podman import - scratchimg")) podman.succeed( su_cmd( - "podman run --runtime=runc -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10" + "podman run --runtime=crun -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10" ) ) podman.succeed(su_cmd("podman ps | grep sleeping")) podman.succeed(su_cmd("podman stop sleeping")) podman.succeed(su_cmd("podman rm sleeping")) + # As of 2020-11-20, the runc backend doesn't work with cgroupsv2 yet, so we don't run that test. - with subtest("Run container rootless with crun"): + with subtest("Run container rootless with the default backend"): podman.succeed(su_cmd("tar cv --files-from /dev/null | podman import - scratchimg")) podman.succeed( su_cmd( - "podman run --runtime=crun -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10" + "podman run -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10" ) ) podman.succeed(su_cmd("podman ps | grep sleeping")) diff --git a/infra/libkookie/nixpkgs/nixos/tests/postfix.nix b/infra/libkookie/nixpkgs/nixos/tests/postfix.nix index 37ae76afec10..6d22b4edba0a 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/postfix.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/postfix.nix @@ -11,9 +11,9 @@ import ./make-test-python.nix { enable = true; enableSubmission = true; enableSubmissions = true; - sslCACert = certs.ca.cert; - sslCert = certs.${domain}.cert; - sslKey = certs.${domain}.key; + tlsTrustedAuthorities = "${certs.ca.cert}"; + sslCert = "${certs.${domain}.cert}"; + sslKey = "${certs.${domain}.key}"; submissionsOptions = { smtpd_sasl_auth_enable = "yes"; smtpd_client_restrictions = "permit"; diff --git a/infra/libkookie/nixpkgs/nixos/tests/powerdns.nix b/infra/libkookie/nixpkgs/nixos/tests/powerdns.nix index 75d71315e644..d025934ad2b3 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/powerdns.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/powerdns.nix @@ -1,13 +1,65 @@ -import ./make-test-python.nix ({ pkgs, ... }: { +# This test runs PowerDNS authoritative server with the +# generic MySQL backend (gmysql) to connect to a +# MariaDB server using UNIX sockets authentication. + +import ./make-test-python.nix ({ pkgs, lib, ... }: { name = "powerdns"; nodes.server = { ... }: { services.powerdns.enable = true; - environment.systemPackages = [ pkgs.dnsutils ]; + services.powerdns.extraConfig = '' + launch=gmysql + gmysql-user=pdns + ''; + + services.mysql = { + enable = true; + package = pkgs.mariadb; + ensureDatabases = [ "powerdns" ]; + ensureUsers = lib.singleton + { name = "pdns"; + ensurePermissions = { "powerdns.*" = "ALL PRIVILEGES"; }; + }; + }; + + environment.systemPackages = with pkgs; + [ dnsutils powerdns mariadb ]; }; testScript = '' - server.wait_for_unit("pdns") - server.succeed("dig version.bind txt chaos \@127.0.0.1") + import re + + with subtest("PowerDNS database exists"): + server.wait_for_unit("mysql") + server.succeed("echo 'SHOW DATABASES;' | sudo -u pdns mysql -u pdns >&2") + + with subtest("Loading the MySQL schema works"): + server.succeed( + "sudo -u pdns mysql -u pdns -D powerdns <" + "${pkgs.powerdns}/share/doc/pdns/schema.mysql.sql" + ) + + with subtest("PowerDNS server starts"): + server.wait_for_unit("pdns") + server.succeed("dig version.bind txt chaos @127.0.0.1 >&2") + + with subtest("Adding an example zone works"): + # Extract configuration file needed by pdnsutil + unit = server.succeed("systemctl cat pdns") + conf = re.search("(--config-dir=[^ ]+)", unit).group(1) + pdnsutil = "sudo -u pdns pdnsutil " + conf + server.succeed(f"{pdnsutil} create-zone example.com ns1.example.com") + server.succeed(f"{pdnsutil} add-record example.com ns1 A 192.168.1.2") + + with subtest("Querying the example zone works"): + reply = server.succeed("dig +noall +answer ns1.example.com @127.0.0.1") + assert ( + "192.168.1.2" in reply + ), f"""" + The reply does not contain the expected IP address: + Expected: + ns1.example.com. 3600 IN A 192.168.1.2 + Reply: + {reply}""" ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/prometheus-exporters.nix b/infra/libkookie/nixpkgs/nixos/tests/prometheus-exporters.nix index ad2fff2b01f6..0b9957404f3b 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/prometheus-exporters.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/prometheus-exporters.nix @@ -563,6 +563,37 @@ let ''; }; + rtl_433 = { + exporterConfig = { + enable = true; + }; + metricProvider = { + # Mock rtl_433 binary to return a dummy metric stream. + nixpkgs.overlays = [ (self: super: { + rtl_433 = self.runCommand "rtl_433" {} '' + mkdir -p "$out/bin" + cat <<EOF > "$out/bin/rtl_433" + #!/bin/sh + while true; do + printf '{"time" : "2020-04-26 13:37:42", "model" : "zopieux", "id" : 55, "channel" : 3, "temperature_C" : 18.000}\n' + sleep 4 + done + EOF + chmod +x "$out/bin/rtl_433" + ''; + }) ]; + }; + exporterTest = '' + wait_for_unit("prometheus-rtl_433-exporter.service") + wait_for_open_port(9550) + wait_until_succeeds( + "curl -sSf localhost:9550/metrics | grep -q '{}'".format( + 'rtl_433_temperature_celsius{channel="3",id="55",location="",model="zopieux"} 18' + ) + ) + ''; + }; + snmp = { exporterConfig = { enable = true; @@ -578,6 +609,50 @@ let ''; }; + sql = { + exporterConfig = { + configuration.jobs.points = { + interval = "1m"; + connections = [ + "postgres://prometheus-sql-exporter@/data?host=/run/postgresql&sslmode=disable" + ]; + queries = { + points = { + labels = [ "name" ]; + help = "Amount of points accumulated per person"; + values = [ "amount" ]; + query = "SELECT SUM(amount) as amount, name FROM points GROUP BY name"; + }; + }; + }; + enable = true; + user = "prometheus-sql-exporter"; + }; + metricProvider = { + services.postgresql = { + enable = true; + initialScript = builtins.toFile "init.sql" '' + CREATE DATABASE data; + \c data; + CREATE TABLE points (amount INT, name TEXT); + INSERT INTO points(amount, name) VALUES (1, 'jack'); + INSERT INTO points(amount, name) VALUES (2, 'jill'); + INSERT INTO points(amount, name) VALUES (3, 'jack'); + + CREATE USER "prometheus-sql-exporter"; + GRANT ALL PRIVILEGES ON DATABASE data TO "prometheus-sql-exporter"; + GRANT SELECT ON points TO "prometheus-sql-exporter"; + ''; + }; + systemd.services.prometheus-sql-exporter.after = [ "postgresql.service" ]; + }; + exporterTest = '' + wait_for_unit("prometheus-sql-exporter.service") + wait_for_open_port(9237) + succeed("curl http://localhost:9237/metrics | grep -c 'sql_points{' | grep -q 2") + ''; + }; + surfboard = { exporterConfig = { enable = true; diff --git a/infra/libkookie/nixpkgs/nixos/tests/prometheus.nix b/infra/libkookie/nixpkgs/nixos/tests/prometheus.nix index af2aa66a5526..6881c659e6d0 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/prometheus.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/prometheus.nix @@ -19,7 +19,6 @@ let secret_key = s3.secretKey; insecure = true; signature_version2 = false; - encrypt_sse = false; put_user_metadata = {}; http_config = { idle_conn_timeout = "0s"; @@ -193,13 +192,13 @@ in import ./make-test-python.nix { # Check if prometheus responds to requests: prometheus.wait_for_unit("prometheus.service") prometheus.wait_for_open_port(${toString queryPort}) - prometheus.succeed("curl -s http://127.0.0.1:${toString queryPort}/metrics") + prometheus.succeed("curl -sf http://127.0.0.1:${toString queryPort}/metrics") # Let's test if pushing a metric to the pushgateway succeeds: prometheus.wait_for_unit("pushgateway.service") prometheus.succeed( "echo 'some_metric 3.14' | " - + "curl --data-binary \@- " + + "curl -f --data-binary \@- " + "http://127.0.0.1:${toString pushgwPort}/metrics/job/some_job" ) diff --git a/infra/libkookie/nixpkgs/nixos/tests/quorum.nix b/infra/libkookie/nixpkgs/nixos/tests/quorum.nix index 846d2a930188..d5906806a0a2 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/quorum.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/quorum.nix @@ -55,7 +55,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { parentHash = "0x0000000000000000000000000000000000000000000000000000000000000000"; timestamp = "0x5cffc201"; - }; + }; }; }; }; diff --git a/infra/libkookie/nixpkgs/nixos/tests/riak.nix b/infra/libkookie/nixpkgs/nixos/tests/riak.nix index 6915779e7e9c..3dd4e333d669 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/riak.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/riak.nix @@ -1,7 +1,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { name = "riak"; meta = with lib.maintainers; { - maintainers = [ filalex77 ]; + maintainers = [ Br1ght0ne ]; }; machine = { diff --git a/infra/libkookie/nixpkgs/nixos/tests/rspamd.nix b/infra/libkookie/nixpkgs/nixos/tests/rspamd.nix index bf3f0de62044..7f41e1a79566 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/rspamd.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/rspamd.nix @@ -13,10 +13,12 @@ let machine.succeed("id rspamd >/dev/null") ''; checkSocket = socket: user: group: mode: '' - machine.succeed("ls ${socket} >/dev/null") - machine.succeed('[[ "$(stat -c %U ${socket})" == "${user}" ]]') - machine.succeed('[[ "$(stat -c %G ${socket})" == "${group}" ]]') - machine.succeed('[[ "$(stat -c %a ${socket})" == "${mode}" ]]') + machine.succeed( + "ls ${socket} >/dev/null", + '[[ "$(stat -c %U ${socket})" == "${user}" ]]', + '[[ "$(stat -c %G ${socket})" == "${group}" ]]', + '[[ "$(stat -c %a ${socket})" == "${mode}" ]]', + ) ''; simple = name: enableIPv6: makeTest { name = "rspamd-${name}"; @@ -54,33 +56,35 @@ in services.rspamd = { enable = true; workers.normal.bindSockets = [{ - socket = "/run/rspamd.sock"; + socket = "/run/rspamd/rspamd.sock"; mode = "0600"; - owner = "root"; - group = "root"; + owner = "rspamd"; + group = "rspamd"; }]; workers.controller.bindSockets = [{ - socket = "/run/rspamd-worker.sock"; + socket = "/run/rspamd/rspamd-worker.sock"; mode = "0666"; - owner = "root"; - group = "root"; + owner = "rspamd"; + group = "rspamd"; }]; }; }; testScript = '' ${initMachine} - machine.wait_for_file("/run/rspamd.sock") - ${checkSocket "/run/rspamd.sock" "root" "root" "600" } - ${checkSocket "/run/rspamd-worker.sock" "root" "root" "666" } + machine.wait_for_file("/run/rspamd/rspamd.sock") + ${checkSocket "/run/rspamd/rspamd.sock" "rspamd" "rspamd" "600" } + ${checkSocket "/run/rspamd/rspamd-worker.sock" "rspamd" "rspamd" "666" } machine.log(machine.succeed("cat /etc/rspamd/rspamd.conf")) machine.log( machine.succeed("grep 'CONFDIR/worker-controller.inc' /etc/rspamd/rspamd.conf") ) machine.log(machine.succeed("grep 'CONFDIR/worker-normal.inc' /etc/rspamd/rspamd.conf")) - machine.log(machine.succeed("rspamc -h /run/rspamd-worker.sock stat")) + machine.log(machine.succeed("rspamc -h /run/rspamd/rspamd-worker.sock stat")) machine.log( - machine.succeed("curl --unix-socket /run/rspamd-worker.sock http://localhost/ping") + machine.succeed( + "curl --unix-socket /run/rspamd/rspamd-worker.sock http://localhost/ping" + ) ) ''; }; @@ -91,16 +95,16 @@ in services.rspamd = { enable = true; workers.normal.bindSockets = [{ - socket = "/run/rspamd.sock"; + socket = "/run/rspamd/rspamd.sock"; mode = "0600"; - owner = "root"; - group = "root"; + owner = "rspamd"; + group = "rspamd"; }]; workers.controller.bindSockets = [{ - socket = "/run/rspamd-worker.sock"; + socket = "/run/rspamd/rspamd-worker.sock"; mode = "0666"; - owner = "root"; - group = "root"; + owner = "rspamd"; + group = "rspamd"; }]; workers.controller2 = { type = "controller"; @@ -116,9 +120,9 @@ in testScript = '' ${initMachine} - machine.wait_for_file("/run/rspamd.sock") - ${checkSocket "/run/rspamd.sock" "root" "root" "600" } - ${checkSocket "/run/rspamd-worker.sock" "root" "root" "666" } + machine.wait_for_file("/run/rspamd/rspamd.sock") + ${checkSocket "/run/rspamd/rspamd.sock" "rspamd" "rspamd" "600" } + ${checkSocket "/run/rspamd/rspamd-worker.sock" "rspamd" "rspamd" "666" } machine.log(machine.succeed("cat /etc/rspamd/rspamd.conf")) machine.log( machine.succeed("grep 'CONFDIR/worker-controller.inc' /etc/rspamd/rspamd.conf") @@ -137,9 +141,11 @@ in machine.wait_until_succeeds( "journalctl -u rspamd | grep -i 'starting controller process' >&2" ) - machine.log(machine.succeed("rspamc -h /run/rspamd-worker.sock stat")) + machine.log(machine.succeed("rspamc -h /run/rspamd/rspamd-worker.sock stat")) machine.log( - machine.succeed("curl --unix-socket /run/rspamd-worker.sock http://localhost/ping") + machine.succeed( + "curl --unix-socket /run/rspamd/rspamd-worker.sock http://localhost/ping" + ) ) machine.log(machine.succeed("curl http://localhost:11335/ping")) ''; @@ -209,7 +215,7 @@ in return false end, score = 5.0, - description = 'Allow no cows', + description = 'Allow no cows', group = "cows", } rspamd_logger.infox(rspamd_config, 'Work dammit!!!') diff --git a/infra/libkookie/nixpkgs/nixos/tests/samba-wsdd.nix b/infra/libkookie/nixpkgs/nixos/tests/samba-wsdd.nix new file mode 100644 index 000000000000..1edef6c0056d --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/samba-wsdd.nix @@ -0,0 +1,44 @@ +import ./make-test-python.nix ({ pkgs, ... }: + +{ + name = "samba-wsdd"; + meta.maintainers = with pkgs.stdenv.lib.maintainers; [ izorkin ]; + + nodes = { + client_wsdd = { pkgs, ... }: { + services.samba-wsdd = { + enable = true; + interface = "eth1"; + workgroup = "WORKGROUP"; + hostname = "CLIENT-WSDD"; + discovery = true; + extraOptions = [ "--no-host" ]; + }; + networking.firewall.allowedTCPPorts = [ 5357 ]; + networking.firewall.allowedUDPPorts = [ 3702 ]; + }; + + server_wsdd = { ... }: { + services.samba-wsdd = { + enable = true; + interface = "eth1"; + workgroup = "WORKGROUP"; + hostname = "SERVER-WSDD"; + }; + networking.firewall.allowedTCPPorts = [ 5357 ]; + networking.firewall.allowedUDPPorts = [ 3702 ]; + }; + }; + + testScript = '' + client_wsdd.start() + client_wsdd.wait_for_unit("samba-wsdd") + + server_wsdd.start() + server_wsdd.wait_for_unit("samba-wsdd") + + client_wsdd.wait_until_succeeds( + "echo list | ${pkgs.libressl.nc}/bin/nc -U /run/wsdd/wsdd.sock | grep -i SERVER-WSDD" + ) + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/sanoid.nix b/infra/libkookie/nixpkgs/nixos/tests/sanoid.nix index 284b38932cce..66ddaad60ea2 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/sanoid.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/sanoid.nix @@ -38,7 +38,7 @@ in { services.syncoid = { enable = true; - sshKey = "/root/.ssh/id_ecdsa"; + sshKey = "/var/lib/syncoid/id_ecdsa"; commonArgs = [ "--no-sync-snap" ]; commands."pool/test".target = "root@target:pool/test"; }; @@ -69,11 +69,12 @@ in { "udevadm settle", ) - source.succeed("mkdir -m 700 /root/.ssh") source.succeed( - "cat '${snakeOilPrivateKey}' > /root/.ssh/id_ecdsa" + "mkdir -m 700 -p /var/lib/syncoid", + "cat '${snakeOilPrivateKey}' > /var/lib/syncoid/id_ecdsa", + "chmod 600 /var/lib/syncoid/id_ecdsa", + "chown -R syncoid:syncoid /var/lib/syncoid/", ) - source.succeed("chmod 600 /root/.ssh/id_ecdsa") source.succeed("touch /tmp/mnt/test.txt") source.systemctl("start --wait sanoid.service") diff --git a/infra/libkookie/nixpkgs/nixos/tests/sbt-extras.nix b/infra/libkookie/nixpkgs/nixos/tests/sbt-extras.nix new file mode 100644 index 000000000000..d63113f943e4 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/sbt-extras.nix @@ -0,0 +1,16 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "sbt-extras"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + environment.systemPackages = [ pkgs.sbt-extras ]; + }; + + testScript = + '' + machine.succeed("(sbt -h)") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/sbt.nix b/infra/libkookie/nixpkgs/nixos/tests/sbt.nix new file mode 100644 index 000000000000..004d9c2e140a --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/sbt.nix @@ -0,0 +1,18 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "sbt"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + environment.systemPackages = [ pkgs.sbt ]; + }; + + testScript = + '' + machine.succeed( + "(sbt --offline --version 2>&1 || true) | grep 'getting org.scala-sbt sbt ${pkgs.sbt.version} (this may take some time)'" + ) + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/scala.nix b/infra/libkookie/nixpkgs/nixos/tests/scala.nix new file mode 100644 index 000000000000..f99d9e563ffe --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/scala.nix @@ -0,0 +1,33 @@ +{ system ? builtins.currentSystem, + config ? {}, + pkgs ? import ../.. { inherit system config; } +}: + +with pkgs.lib; + +let + common = name: package: (import ./make-test-python.nix ({ + inherit name; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + nodes = { + scala = { ... }: { + environment.systemPackages = [ package ]; + }; + }; + + testScript = '' + start_all() + + scala.succeed("scalac -version 2>&1 | grep '^Scala compiler version ${package.version}'") + ''; + }) { inherit system; }); + +in with pkgs; { + scala_2_10 = common "scala_2_10" scala_2_10; + scala_2_11 = common "scala_2_11" scala_2_11; + scala_2_12 = common "scala_2_12" scala_2_12; + scala_2_13 = common "scala_2_13" scala_2_13; +} diff --git a/infra/libkookie/nixpkgs/nixos/tests/service-runner.nix b/infra/libkookie/nixpkgs/nixos/tests/service-runner.nix index 39ae66fe1116..55fbbb729344 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/service-runner.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/service-runner.nix @@ -29,7 +29,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { """ ) machine.wait_for_open_port(80) - machine.succeed(f"curl {url}") + machine.succeed(f"curl -f {url}") machine.succeed("kill -INT $(cat my-nginx.pid)") machine.wait_for_closed_port(80) ''; diff --git a/infra/libkookie/nixpkgs/nixos/tests/spacecookie.nix b/infra/libkookie/nixpkgs/nixos/tests/spacecookie.nix index 6eff32a2e75d..5b5022a74278 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/spacecookie.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/spacecookie.nix @@ -32,7 +32,7 @@ in ${gopherHost}.wait_for_unit("spacecookie.service") client.wait_for_unit("network.target") - fileResponse = client.succeed("curl -s gopher://${gopherHost}//${fileName}") + fileResponse = client.succeed("curl -f -s gopher://${gopherHost}//${fileName}") # the file response should return our created file exactly if not (fileResponse == "${fileContent}\n"): @@ -41,7 +41,7 @@ in # sanity check on the directory listing: we serve a directory and a file # via gopher, so the directory listing should have exactly two entries, # one with gopher file type 0 (file) and one with file type 1 (directory). - dirResponse = client.succeed("curl -s gopher://${gopherHost}") + dirResponse = client.succeed("curl -f -s gopher://${gopherHost}") dirEntries = [l[0] for l in dirResponse.split("\n") if len(l) > 0] dirEntries.sort() diff --git a/infra/libkookie/nixpkgs/nixos/tests/sslh.nix b/infra/libkookie/nixpkgs/nixos/tests/sslh.nix index 2a800aa52d0a..17094606e8e6 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/sslh.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/sslh.nix @@ -78,6 +78,6 @@ import ./make-test-python.nix { server.succeed(f"grep '{ip}' /tmp/foo{arg}") # check that http through sslh works - assert client.succeed(f"curl {arg} http://server:443").strip() == "hello world" + assert client.succeed(f"curl -f {arg} http://server:443").strip() == "hello world" ''; } diff --git a/infra/libkookie/nixpkgs/nixos/tests/sssd-ldap.nix b/infra/libkookie/nixpkgs/nixos/tests/sssd-ldap.nix index b68403a0102a..4831eaa4ba20 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/sssd-ldap.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/sssd-ldap.nix @@ -1,4 +1,4 @@ -import ./make-test-python.nix ({ pkgs, ... }: +({ pkgs, ... }: let dbDomain = "example.org"; dbSuffix = "dc=example,dc=org"; @@ -7,8 +7,7 @@ import ./make-test-python.nix ({ pkgs, ... }: ldapRootPassword = "foobar"; testUser = "alice"; - in - { + in import ./make-test-python.nix { name = "sssd-ldap"; meta = with pkgs.stdenv.lib.maintainers; { @@ -18,34 +17,53 @@ import ./make-test-python.nix ({ pkgs, ... }: machine = { pkgs, ... }: { services.openldap = { enable = true; - rootdn = "cn=${ldapRootUser},${dbSuffix}"; - rootpw = ldapRootPassword; - suffix = dbSuffix; - declarativeContents = '' - dn: ${dbSuffix} - objectClass: top - objectClass: dcObject - objectClass: organization - o: ${dbDomain} + settings = { + children = { + "cn=schema".includes = [ + "${pkgs.openldap}/etc/schema/core.ldif" + "${pkgs.openldap}/etc/schema/cosine.ldif" + "${pkgs.openldap}/etc/schema/inetorgperson.ldif" + "${pkgs.openldap}/etc/schema/nis.ldif" + ]; + "olcDatabase={1}mdb" = { + attrs = { + objectClass = [ "olcDatabaseConfig" "olcMdbConfig" ]; + olcDatabase = "{1}mdb"; + olcDbDirectory = "/var/db/openldap"; + olcSuffix = dbSuffix; + olcRootDN = "cn=${ldapRootUser},${dbSuffix}"; + olcRootPW = ldapRootPassword; + }; + }; + }; + }; + declarativeContents = { + ${dbSuffix} = '' + dn: ${dbSuffix} + objectClass: top + objectClass: dcObject + objectClass: organization + o: ${dbDomain} - dn: ou=posix,${dbSuffix} - objectClass: top - objectClass: organizationalUnit + dn: ou=posix,${dbSuffix} + objectClass: top + objectClass: organizationalUnit - dn: ou=accounts,ou=posix,${dbSuffix} - objectClass: top - objectClass: organizationalUnit + dn: ou=accounts,ou=posix,${dbSuffix} + objectClass: top + objectClass: organizationalUnit - dn: uid=${testUser},ou=accounts,ou=posix,${dbSuffix} - objectClass: person - objectClass: posixAccount - # userPassword: somePasswordHash - homeDirectory: /home/${testUser} - uidNumber: 1234 - gidNumber: 1234 - cn: "" - sn: "" - ''; + dn: uid=${testUser},ou=accounts,ou=posix,${dbSuffix} + objectClass: person + objectClass: posixAccount + # userPassword: somePasswordHash + homeDirectory: /home/${testUser} + uidNumber: 1234 + gidNumber: 1234 + cn: "" + sn: "" + ''; + }; }; services.sssd = { diff --git a/infra/libkookie/nixpkgs/nixos/tests/sympa.nix b/infra/libkookie/nixpkgs/nixos/tests/sympa.nix index 280691f7cb40..eb38df180a78 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/sympa.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/sympa.nix @@ -30,7 +30,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: { machine.wait_for_unit("sympa.service") machine.wait_for_unit("wwsympa.service") assert "Mailing lists service" in machine.succeed( - "curl --insecure -L http://localhost/" + "curl --fail --insecure -L http://localhost/" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/syncthing-relay.nix b/infra/libkookie/nixpkgs/nixos/tests/syncthing-relay.nix index cd72ef1cbe1d..c144bf7fca37 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/syncthing-relay.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/syncthing-relay.nix @@ -19,7 +19,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { machine.wait_for_open_port(12346) out = machine.succeed( - "curl -sS http://localhost:12346/status | jq -r '.options.\"provided-by\"'" + "curl -sSf http://localhost:12346/status | jq -r '.options.\"provided-by\"'" ) assert "nixos-test" in out ''; diff --git a/infra/libkookie/nixpkgs/nixos/tests/syncthing.nix b/infra/libkookie/nixpkgs/nixos/tests/syncthing.nix index 9e2a8e01e3fb..ac9df5e50c8c 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/syncthing.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/syncthing.nix @@ -25,7 +25,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { "xmllint --xpath 'string(configuration/gui/apikey)' %s/config.xml" % confdir ).strip() oldConf = host.succeed( - "curl -Ss -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config" % APIKey + "curl -Ssf -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config" % APIKey ) conf = json.loads(oldConf) conf["devices"].append({"deviceID": deviceID, "id": name}) @@ -39,7 +39,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { ) newConf = json.dumps(conf) host.succeed( - "curl -Ss -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config -d %s" + "curl -Ssf -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config -d %s" % (APIKey, shlex.quote(newConf)) ) diff --git a/infra/libkookie/nixpkgs/nixos/tests/systemd-journal.nix b/infra/libkookie/nixpkgs/nixos/tests/systemd-journal.nix new file mode 100644 index 000000000000..c50c151ae10d --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/systemd-journal.nix @@ -0,0 +1,20 @@ +import ./make-test-python.nix ({ pkgs, ... }: + +{ + name = "systemd-journal"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ lewo ]; + }; + + machine = { pkgs, lib, ... }: { + services.journald.enableHttpGateway = true; + }; + + testScript = '' + machine.wait_for_unit("multi-user.target") + + machine.succeed( + "${pkgs.curl}/bin/curl -s localhost:19531/machine | ${pkgs.jq}/bin/jq -e '.hostname == \"machine\"'" + ) + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/systemd.nix b/infra/libkookie/nixpkgs/nixos/tests/systemd.nix index dfa16eecfad2..f7c13a587c58 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/systemd.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/systemd.nix @@ -26,7 +26,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { systemd.shutdown.test = pkgs.writeScript "test.shutdown" '' #!${pkgs.runtimeShell} - PATH=${lib.makeBinPath (with pkgs; [ utillinux coreutils ])} + PATH=${lib.makeBinPath (with pkgs; [ util-linux coreutils ])} mount -t 9p shared -o trans=virtio,version=9p2000.L /tmp/shared touch /tmp/shared/shutdown-test umount /tmp/shared @@ -82,6 +82,10 @@ import ./make-test-python.nix ({ pkgs, ... }: { "systemd-run --pty --property=Type=oneshot --property=DynamicUser=yes --property=User=iamatest whoami" ) + with subtest("regression test for https://bugs.freedesktop.org/show_bug.cgi?id=77507"): + retcode, output = machine.execute("systemctl status testservice1.service") + assert retcode in [0, 3] # https://bugs.freedesktop.org/show_bug.cgi?id=77507 + # Regression test for https://github.com/NixOS/nixpkgs/issues/35268 with subtest("file system with x-initrd.mount is not unmounted"): machine.succeed("mountpoint -q /test-x-initrd-mount") @@ -122,17 +126,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { machine.wait_for_unit("multi-user.target") assert "fq_codel" in machine.succeed("sysctl net.core.default_qdisc") - # Test cgroup accounting is enabled - with subtest("systemd cgroup accounting is enabled"): - machine.wait_for_unit("multi-user.target") - assert "yes" in machine.succeed( - "systemctl show testservice1.service -p IOAccounting" - ) - - retcode, output = machine.execute("systemctl status testservice1.service") - assert retcode in [0, 3] # https://bugs.freedesktop.org/show_bug.cgi?id=77507 - assert "CPU:" in output - # Test systemd is configured to manage a watchdog with subtest("systemd manages hardware watchdog"): machine.wait_for_unit("multi-user.target") @@ -168,5 +161,25 @@ import ./make-test-python.nix ({ pkgs, ... }: { machine.succeed("systemctl status systemd-cryptsetup@luks1.service") machine.succeed("mkdir -p /tmp/luks1") machine.succeed("mount /dev/mapper/luks1 /tmp/luks1") + + # Do some IP traffic + output_ping = machine.succeed( + "systemd-run --wait -- /run/wrappers/bin/ping -c 1 127.0.0.1 2>&1" + ) + + with subtest("systemd reports accounting data on system.slice"): + output = machine.succeed("systemctl status system.slice") + assert "CPU:" in output + assert "Memory:" in output + + assert "IP:" in output + assert "0B in, 0B out" not in output + + assert "IO:" in output + assert "0B read, 0B written" not in output + + with subtest("systemd per-unit accounting works"): + assert "IP traffic received: 84B" in output_ping + assert "IP traffic sent: 84B" in output_ping ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/telegraf.nix b/infra/libkookie/nixpkgs/nixos/tests/telegraf.nix index 73f741b11357..7f4b36752582 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/telegraf.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/telegraf.nix @@ -6,12 +6,15 @@ import ./make-test-python.nix ({ pkgs, ...} : { machine = { ... }: { services.telegraf.enable = true; + services.telegraf.environmentFiles = [(pkgs.writeText "secrets" '' + SECRET=example + '')]; services.telegraf.extraConfig = { agent.interval = "1s"; agent.flush_interval = "1s"; inputs.exec = { commands = [ - "${pkgs.runtimeShell} -c 'echo example,tag=a i=42i'" + "${pkgs.runtimeShell} -c 'echo $SECRET,tag=a i=42i'" ]; timeout = "5s"; data_format = "influx"; diff --git a/infra/libkookie/nixpkgs/nixos/tests/trac.nix b/infra/libkookie/nixpkgs/nixos/tests/trac.nix index 7953f8d41f77..af7182d1e185 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/trac.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/trac.nix @@ -14,6 +14,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { start_all() machine.wait_for_unit("trac.service") machine.wait_for_open_port(8000) - machine.wait_until_succeeds("curl -L http://localhost:8000/ | grep 'Trac Powered'") + machine.wait_until_succeeds("curl -fL http://localhost:8000/ | grep 'Trac Powered'") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/trezord.nix b/infra/libkookie/nixpkgs/nixos/tests/trezord.nix index b7b3dd31942b..7c8370f409ed 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/trezord.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/trezord.nix @@ -14,6 +14,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { start_all() machine.wait_for_unit("trezord.service") machine.wait_for_open_port(21325) - machine.wait_until_succeeds("curl -L http://localhost:21325/status/ | grep Version") + machine.wait_until_succeeds("curl -fL http://localhost:21325/status/ | grep Version") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/trickster.nix b/infra/libkookie/nixpkgs/nixos/tests/trickster.nix index 713ac8f0b2fa..e32f919a1ada 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/trickster.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/trickster.nix @@ -19,19 +19,19 @@ import ./make-test-python.nix ({ pkgs, ... }: { prometheus.wait_for_unit("prometheus.service") prometheus.wait_for_open_port(9090) prometheus.wait_until_succeeds( - "curl -L http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) trickster.wait_for_unit("trickster.service") trickster.wait_for_open_port(8082) trickster.wait_for_open_port(9090) trickster.wait_until_succeeds( - "curl -L http://localhost:8082/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://localhost:8082/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) trickster.wait_until_succeeds( - "curl -L http://prometheus:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://prometheus:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) trickster.wait_until_succeeds( - "curl -L http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/ucg.nix b/infra/libkookie/nixpkgs/nixos/tests/ucg.nix new file mode 100644 index 000000000000..47507aee07c1 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/ucg.nix @@ -0,0 +1,18 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "ucg"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ AndersonTorres ]; + }; + + machine = { pkgs, ... }: { + environment.systemPackages = [ pkgs.ucg ]; + }; + + testScript = '' + machine.succeed("echo 'Lorem ipsum dolor sit amet\n2.7182818284590' > /tmp/foo") + assert "dolor" in machine.succeed("ucg 'dolor' /tmp/foo") + assert "Lorem" in machine.succeed("ucg --ignore-case 'lorem' /tmp/foo") + machine.fail("ucg --word-regexp '2718' /tmp/foo") + machine.fail("ucg 'pisum' /tmp/foo") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/unbound.nix b/infra/libkookie/nixpkgs/nixos/tests/unbound.nix new file mode 100644 index 000000000000..dc8e5a9d3ed8 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/unbound.nix @@ -0,0 +1,278 @@ +/* + Test that our unbound module indeed works as most users would expect. + There are a few settings that we must consider when modifying the test. The + ususal use-cases for unbound are + * running a recursive DNS resolver on the local machine + * running a recursive DNS resolver on the local machine, forwarding to a local DNS server via UDP/53 & TCP/53 + * running a recursive DNS resolver on the local machine, forwarding to a local DNS server via TCP/853 (DoT) + * running a recursive DNS resolver on a machine in the network awaiting input from clients over TCP/53 & UDP/53 + * running a recursive DNS resolver on a machine in the network awaiting input from clients over TCP/853 (DoT) + + In the below test setup we are trying to implement all of those use cases. + + Another aspect that we cover is access to the local control UNIX socket. It + can optionally be enabled and users can optionally be in a group to gain + access. Users that are not in the group (except for root) should not have + access to that socket. Also, when there is no socket configured, users + shouldn't be able to access the control socket at all. Not even root. +*/ +import ./make-test-python.nix ({ pkgs, lib, ... }: + let + # common client configuration that we can just use for the multitude of + # clients we are constructing + common = { lib, pkgs, ... }: { + config = { + environment.systemPackages = [ pkgs.knot-dns ]; + + # disable the root anchor update as we do not have internet access during + # the test execution + services.unbound.enableRootTrustAnchor = false; + }; + }; + + cert = pkgs.runCommandNoCC "selfSignedCerts" { buildInputs = [ pkgs.openssl ]; } '' + openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -nodes -subj '/CN=dns.example.local' + mkdir -p $out + cp key.pem cert.pem $out + ''; + in + { + name = "unbound"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ andir ]; + }; + + nodes = { + + # The server that actually serves our zones, this tests unbounds authoriative mode + authoritative = { lib, pkgs, config, ... }: { + imports = [ common ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ + { address = "192.168.0.1"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = lib.mkForce [ + { address = "fd21::1"; prefixLength = 64; } + ]; + networking.firewall.allowedTCPPorts = [ 53 ]; + networking.firewall.allowedUDPPorts = [ 53 ]; + + services.unbound = { + enable = true; + interfaces = [ "192.168.0.1" "fd21::1" "::1" "127.0.0.1" ]; + allowedAccess = [ "192.168.0.0/24" "fd21::/64" "::1" "127.0.0.0/8" ]; + extraConfig = '' + server: + local-data: "example.local. IN A 1.2.3.4" + local-data: "example.local. IN AAAA abcd::eeff" + ''; + }; + }; + + # The resolver that knows that fowards (only) to the authoritative server + # and listens on UDP/53, TCP/53 & TCP/853. + resolver = { lib, nodes, ... }: { + imports = [ common ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ + { address = "192.168.0.2"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = lib.mkForce [ + { address = "fd21::2"; prefixLength = 64; } + ]; + networking.firewall.allowedTCPPorts = [ + 53 # regular DNS + 853 # DNS over TLS + ]; + networking.firewall.allowedUDPPorts = [ 53 ]; + + services.unbound = { + enable = true; + allowedAccess = [ "192.168.0.0/24" "fd21::/64" "::1" "127.0.0.0/8" ]; + interfaces = [ "::1" "127.0.0.1" "192.168.0.2" "fd21::2" "192.168.0.2@853" "fd21::2@853" "::1@853" "127.0.0.1@853" ]; + forwardAddresses = [ + (lib.head nodes.authoritative.config.networking.interfaces.eth1.ipv6.addresses).address + (lib.head nodes.authoritative.config.networking.interfaces.eth1.ipv4.addresses).address + ]; + extraConfig = '' + server: + tls-service-pem: ${cert}/cert.pem + tls-service-key: ${cert}/key.pem + ''; + }; + }; + + # machine that runs a local unbound that will be reconfigured during test execution + local_resolver = { lib, nodes, config, ... }: { + imports = [ common ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ + { address = "192.168.0.3"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = lib.mkForce [ + { address = "fd21::3"; prefixLength = 64; } + ]; + networking.firewall.allowedTCPPorts = [ + 53 # regular DNS + ]; + networking.firewall.allowedUDPPorts = [ 53 ]; + + services.unbound = { + enable = true; + allowedAccess = [ "::1" "127.0.0.0/8" ]; + interfaces = [ "::1" "127.0.0.1" ]; + localControlSocketPath = "/run/unbound/unbound.ctl"; + extraConfig = '' + include: "/etc/unbound/extra*.conf" + ''; + }; + + users.users = { + # user that is permitted to access the unix socket + someuser.extraGroups = [ + config.users.users.unbound.group + ]; + + # user that is not permitted to access the unix socket + unauthorizeduser = {}; + }; + + environment.etc = { + "unbound-extra1.conf".text = '' + forward-zone: + name: "example.local." + forward-addr: ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address} + forward-addr: ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address} + ''; + "unbound-extra2.conf".text = '' + auth-zone: + name: something.local. + zonefile: ${pkgs.writeText "zone" '' + something.local. IN A 3.4.5.6 + ''} + ''; + }; + }; + + + # plain node that only has network access and doesn't run any part of the + # resolver software locally + client = { lib, nodes, ... }: { + imports = [ common ]; + networking.nameservers = [ + (lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address + (lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address + ]; + networking.interfaces.eth1.ipv4.addresses = [ + { address = "192.168.0.10"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = [ + { address = "fd21::10"; prefixLength = 64; } + ]; + }; + }; + + testScript = { nodes, ... }: '' + import typing + import json + + zone = "example.local." + records = [("AAAA", "abcd::eeff"), ("A", "1.2.3.4")] + + + def query( + machine, + host: str, + query_type: str, + query: str, + expected: typing.Optional[str] = None, + args: typing.Optional[typing.List[str]] = None, + ): + """ + Execute a single query and compare the result with expectation + """ + text_args = "" + if args: + text_args = " ".join(args) + + out = machine.succeed( + f"kdig {text_args} {query} {query_type} @{host} +short" + ).strip() + machine.log(f"{host} replied with {out}") + if expected: + assert expected == out, f"Expected `{expected}` but got `{out}`" + + + def test(machine, remotes, /, doh=False, zone=zone, records=records, args=[]): + """ + Run queries for the given remotes on the given machine. + """ + for query_type, expected in records: + for remote in remotes: + query(machine, remote, query_type, zone, expected, args) + query(machine, remote, query_type, zone, expected, ["+tcp"] + args) + if doh: + query( + machine, + remote, + query_type, + zone, + expected, + ["+tcp", "+tls"] + args, + ) + + + client.start() + authoritative.wait_for_unit("unbound.service") + + # verify that we can resolve locally + with subtest("test the authoritative servers local responses"): + test(authoritative, ["::1", "127.0.0.1"]) + + resolver.wait_for_unit("unbound.service") + + with subtest("root is unable to use unbounc-control when the socket is not configured"): + resolver.succeed("which unbound-control") # the binary must exist + resolver.fail("unbound-control list_forwards") # the invocation must fail + + # verify that the resolver is able to resolve on all the local protocols + with subtest("test that the resolver resolves on all protocols and transports"): + test(resolver, ["::1", "127.0.0.1"], doh=True) + + resolver.wait_for_unit("multi-user.target") + + with subtest("client should be able to query the resolver"): + test(client, ["${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address}", "${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address}"], doh=True) + + # discard the client we do not need anymore + client.shutdown() + + local_resolver.wait_for_unit("multi-user.target") + + # link a new config file to /etc/unbound/extra.conf + local_resolver.succeed("ln -s /etc/unbound-extra1.conf /etc/unbound/extra1.conf") + + # reload the server & ensure the forwarding works + with subtest("test that the local resolver resolves on all protocols and transports"): + local_resolver.succeed("systemctl reload unbound") + print(local_resolver.succeed("journalctl -u unbound -n 1000")) + test(local_resolver, ["::1", "127.0.0.1"], args=["+timeout=60"]) + + with subtest("test that we can use the unbound control socket"): + out = local_resolver.succeed( + "sudo -u someuser -- unbound-control list_forwards" + ).strip() + + # Thank you black! Can't really break this line into a readable version. + expected = "example.local. IN forward ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address} ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address}" + assert out == expected, f"Expected `{expected}` but got `{out}` instead." + local_resolver.fail("sudo -u unauthorizeduser -- unbound-control list_forwards") + + + # link a new config file to /etc/unbound/extra.conf + local_resolver.succeed("ln -sf /etc/unbound-extra2.conf /etc/unbound/extra2.conf") + + # reload the server & ensure the new local zone works + with subtest("test that we can query the new local zone"): + local_resolver.succeed("unbound-control reload") + r = [("A", "3.4.5.6")] + test(local_resolver, ["::1", "127.0.0.1"], zone="something.local.", records=r) + ''; + }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/upnp.nix b/infra/libkookie/nixpkgs/nixos/tests/upnp.nix index a7d837ea0708..046c0a56b2a7 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/upnp.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/upnp.nix @@ -90,7 +90,7 @@ in client1.succeed("upnpc -a ${internalClient1Address} 9000 9000 TCP") client1.wait_for_unit("httpd") - client2.wait_until_succeeds("curl http://${externalRouterAddress}:9000/") + client2.wait_until_succeeds("curl -f http://${externalRouterAddress}:9000/") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/uwsgi.nix b/infra/libkookie/nixpkgs/nixos/tests/uwsgi.nix index 78a87147f55c..7f4945a88030 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/uwsgi.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/uwsgi.nix @@ -33,6 +33,6 @@ import ./make-test-python.nix ({ pkgs, ... }: machine.wait_for_unit("multi-user.target") machine.wait_for_unit("uwsgi.service") machine.wait_for_open_port(8000) - assert "Hello World" in machine.succeed("curl -v 127.0.0.1:8000") + assert "Hello World" in machine.succeed("curl -fv 127.0.0.1:8000") ''; }) diff --git a/infra/libkookie/nixpkgs/nixos/tests/vector.nix b/infra/libkookie/nixpkgs/nixos/tests/vector.nix new file mode 100644 index 000000000000..e96c3ad152f3 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/vector.nix @@ -0,0 +1,37 @@ +{ system ? builtins.currentSystem, config ? { } +, pkgs ? import ../.. { inherit system config; } }: + +with import ../lib/testing-python.nix { inherit system pkgs; }; +with pkgs.lib; + +{ + test1 = makeTest { + name = "vector-test1"; + meta.maintainers = [ pkgs.stdenv.lib.maintainers.happysalada ]; + + machine = { config, pkgs, ... }: { + services.vector = { + enable = true; + journaldAccess = true; + settings = { + sources.journald.type = "journald"; + + sinks = { + file = { + type = "file"; + inputs = [ "journald" ]; + path = "/var/lib/vector/logs.log"; + encoding = { codec = "ndjson"; }; + }; + }; + }; + }; + }; + + # ensure vector is forwarding the messages appropriately + testScript = '' + machine.wait_for_unit("vector.service") + machine.succeed("test -f /var/lib/vector/logs.log") + ''; + }; +} diff --git a/infra/libkookie/nixpkgs/nixos/tests/victoriametrics.nix b/infra/libkookie/nixpkgs/nixos/tests/victoriametrics.nix index 73ef8b728615..fff8d7005da1 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/victoriametrics.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/victoriametrics.nix @@ -19,9 +19,11 @@ import ./make-test-python.nix ({ pkgs, ...} : { # write some points and run simple query out = one.succeed( - "curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'" + "curl -f -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'" + ) + cmd = ( + """curl -f -s -G 'http://localhost:8428/api/v1/export' -d 'match={__name__!=""}'""" ) - cmd = """curl -s -G 'http://localhost:8428/api/v1/export' -d 'match={__name__!=""}'""" # data takes a while to appear one.wait_until_succeeds(f"[[ $({cmd} | wc -l) -ne 0 ]]") out = one.succeed(cmd) diff --git a/infra/libkookie/nixpkgs/nixos/tests/virtualbox.nix b/infra/libkookie/nixpkgs/nixos/tests/virtualbox.nix index 0d9eafa4a20f..900ee610a70b 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/virtualbox.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/virtualbox.nix @@ -24,7 +24,7 @@ let miniInit = '' #!${pkgs.runtimeShell} -xe - export PATH="${lib.makeBinPath [ pkgs.coreutils pkgs.utillinux ]}" + export PATH="${lib.makeBinPath [ pkgs.coreutils pkgs.util-linux ]}" mkdir -p /run/dbus cat > /etc/passwd <<EOF @@ -72,7 +72,7 @@ let boot.initrd.extraUtilsCommands = '' copy_bin_and_libs "${guestAdditions}/bin/mount.vboxsf" - copy_bin_and_libs "${pkgs.utillinux}/bin/unshare" + copy_bin_and_libs "${pkgs.util-linux}/bin/unshare" ${(attrs.extraUtilsCommands or (const "")) pkgs} ''; @@ -122,7 +122,7 @@ let "$diskImage" "$out/disk.vdi" ''; - buildInputs = [ pkgs.utillinux pkgs.perl ]; + buildInputs = [ pkgs.util-linux pkgs.perl ]; } '' ${pkgs.parted}/sbin/parted --script /dev/vda mklabel msdos ${pkgs.parted}/sbin/parted --script /dev/vda -- mkpart primary ext2 1M -1s diff --git a/infra/libkookie/nixpkgs/nixos/tests/web-servers/unit-php.nix b/infra/libkookie/nixpkgs/nixos/tests/web-servers/unit-php.nix index 2a0a5bdaa5d5..033036ee7667 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/web-servers/unit-php.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/web-servers/unit-php.nix @@ -47,7 +47,7 @@ in { machine.wait_for_unit("unit.service") # Check so we get an evaluated PHP back - response = machine.succeed("curl -vvv -s http://127.0.0.1:9074/") + response = machine.succeed("curl -f -vvv -s http://127.0.0.1:9074/") assert "PHP Version ${pkgs.unit.usedPhp74.version}" in response, "PHP version not detected" # Check so we have database and some other extensions loaded diff --git a/infra/libkookie/nixpkgs/nixos/tests/wordpress.nix b/infra/libkookie/nixpkgs/nixos/tests/wordpress.nix index b7449859f7e6..5d740502bb57 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/wordpress.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/wordpress.nix @@ -40,7 +40,7 @@ import ./make-test-python.nix ({ pkgs, ... }: with subtest("website returns welcome screen"): for site_name in site_names: - assert "Welcome to the famous" in machine.succeed(f"curl -L {site_name}") + assert "Welcome to the famous" in machine.succeed(f"curl -fL {site_name}") with subtest("wordpress-init went through"): for site_name in site_names: diff --git a/infra/libkookie/nixpkgs/nixos/tests/xterm.nix b/infra/libkookie/nixpkgs/nixos/tests/xterm.nix new file mode 100644 index 000000000000..9f30543bf385 --- /dev/null +++ b/infra/libkookie/nixpkgs/nixos/tests/xterm.nix @@ -0,0 +1,23 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "xterm"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + imports = [ ./common/x11.nix ]; + services.xserver.desktopManager.xterm.enable = false; + }; + + testScript = + '' + machine.wait_for_x() + machine.succeed("DISPLAY=:0 xterm -title testterm -class testterm -fullscreen &") + machine.sleep(2) + machine.send_chars("echo $XTERM_VERSION >> /tmp/xterm_version\n") + machine.wait_for_file("/tmp/xterm_version") + assert "${pkgs.xterm.version}" in machine.succeed("cat /tmp/xterm_version") + machine.screenshot("window") + ''; +}) diff --git a/infra/libkookie/nixpkgs/nixos/tests/zfs.nix b/infra/libkookie/nixpkgs/nixos/tests/zfs.nix index 87e6c900c98e..e05cd540227a 100644 --- a/infra/libkookie/nixpkgs/nixos/tests/zfs.nix +++ b/infra/libkookie/nixpkgs/nixos/tests/zfs.nix @@ -18,7 +18,7 @@ let maintainers = [ adisbladis ]; }; - machine = { pkgs, ... }: { + machine = { pkgs, lib, ... }: { virtualisation.emptyDiskImages = [ 4096 ]; networking.hostId = "deadbeef"; boot.kernelPackages = kernelPackage; @@ -26,6 +26,24 @@ let boot.zfs.enableUnstable = enableUnstable; environment.systemPackages = [ pkgs.parted ]; + + # Setup regular fileSystems machinery to ensure forceImportAll can be + # tested via the regular service units. + fileSystems = lib.mkVMOverride { + "/forcepool" = { + device = "forcepool"; + fsType = "zfs"; + options = [ "noauto" ]; + }; + }; + + # forcepool doesn't exist at first boot, and we need to manually test + # the import after tweaking the hostId. + systemd.services.zfs-import-forcepool.wantedBy = lib.mkVMOverride []; + systemd.targets.zfs.wantedBy = lib.mkVMOverride []; + boot.zfs.forceImportAll = true; + # /dev/disk/by-id doesn't get populated in the NixOS test framework + boot.zfs.devNodes = "/dev/disk/by-uuid"; }; testScript = '' @@ -57,6 +75,21 @@ let "zpool destroy rpool", "udevadm settle", ) + + with subtest("boot.zfs.forceImportAll works"): + machine.succeed( + "rm /etc/hostid", + "zgenhostid deadcafe", + "zpool create forcepool /dev/vdb1 -O mountpoint=legacy", + ) + machine.shutdown() + machine.start() + machine.succeed("udevadm settle") + machine.fail("zpool import forcepool") + machine.succeed( + "systemctl start zfs-import-forcepool.service", + "mount -t zfs forcepool /tmp/mnt", + ) '' + extraTest; }; |