diff options
Diffstat (limited to 'infra/libkookie/nixpkgs/doc/languages-frameworks')
24 files changed, 1255 insertions, 2306 deletions
diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/beam.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/beam.section.md new file mode 100644 index 000000000000..ad3b94880b5c --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/beam.section.md @@ -0,0 +1,84 @@ +# BEAM Languages (Erlang, Elixir & LFE) {#sec-beam} + +## Introduction {#beam-introduction} + +In this document and related Nix expressions, we use the term, *BEAM*, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction. + +## Structure {#beam-structure} + +All BEAM-related expressions are available via the top-level `beam` attribute, which includes: + + - `interpreters`: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (`beam.interpreters.erlangR19`, etc), Elixir (`beam.interpreters.elixir`) and LFE (`beam.interpreters.lfe`). + + - `packages`: a set of package builders (Mix and rebar3), each compiled with a specific Erlang/OTP version, e.g. `beam.packages.erlangR19`. + +The default Erlang compiler, defined by `beam.interpreters.erlang`, is aliased as `erlang`. The default BEAM package set is defined by `beam.packages.erlang` and aliased at the top level as `beamPackages`. + +To create a package builder built with a custom Erlang version, use the lambda, `beam.packagesWith`, which accepts an Erlang/OTP derivation and produces a package builder similar to `beam.packages.erlang`. + +Many Erlang/OTP distributions available in `beam.interpreters` have versions with ODBC and/or Java enabled or without wx (no observer support). For example, there's `beam.interpreters.erlangR22_odbc_javac`, which corresponds to `beam.interpreters.erlangR22` and `beam.interpreters.erlangR22_nox`, which corresponds to `beam.interpreters.erlangR22`. + +## Build Tools {#build-tools} + +### Rebar3 {#build-tools-rebar3} + +We provide a version of Rebar3, under `rebar3`. We also provide a helper to fetch Rebar3 dependencies from a lockfile under `fetchRebar3Deps`. + +### Mix & Erlang.mk {#build-tools-other} + +Both Mix and Erlang.mk work exactly as expected. There is a bootstrap process that needs to be run for both, however, which is supported by the `buildMix` and `buildErlangMk` derivations, respectively. + +## How to Install BEAM Packages {#how-to-install-beam-packages} + +BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. To install any of those builders into your profile, refer to them by their attribute path `beamPackages.rebar3`: + +```ShellSession +$ nix-env -f "<nixpkgs>" -iA beamPackages.rebar3 +``` + +## Packaging BEAM Applications {#packaging-beam-applications} + +### Erlang Applications {#packaging-erlang-applications} + +#### Rebar3 Packages {#rebar3-packages} + +The Nix function, `buildRebar3`, defined in `beam.packages.erlang.buildRebar3` and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. + +If a package needs to compile native code via Rebar3's port compilation mechanism, add `compilePort = true;` to the derivation. + +#### Erlang.mk Packages {#erlang-mk-packages} + +Erlang.mk functions similarly to Rebar3, except we use `buildErlangMk` instead of `buildRebar3`. + +#### Mix Packages {#mix-packages} + +Mix functions similarly to Rebar3, except we use `buildMix` instead of `buildRebar3`. + +Alternatively, we can use `buildHex` as a shortcut: + +## How to Develop {#how-to-develop} + +### Creating a Shell {#creating-a-shell} + +Usually, we need to create a `shell.nix` file and do our development inside of the environment specified therein. Just install your version of erlang and other interpreter, and then user your normal build tools. As an example with elixir: + +```nix +{ pkgs ? import "<nixpkgs"> {} }: + +with pkgs; + +let + + elixir = beam.packages.erlangR22.elixir_1_9; + +in +mkShell { + buildInputs = [ elixir ]; + + ERL_INCLUDE_PATH="${erlang}/lib/erlang/usr/include"; +} +``` + +#### Building in a Shell (for Mix Projects) {#building-in-a-shell} + +Using a `shell.nix` as described (see <xref linkend="creating-a-shell"/>) should just work. diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/beam.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/beam.xml deleted file mode 100644 index addab24f7f6d..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/beam.xml +++ /dev/null @@ -1,159 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-beam"> - <title>BEAM Languages (Erlang, Elixir & LFE)</title> - - <section xml:id="beam-introduction"> - <title>Introduction</title> - - <para> - In this document and related Nix expressions, we use the term, <emphasis>BEAM</emphasis>, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction. - </para> - </section> - - <section xml:id="beam-structure"> - <title>Structure</title> - - <para> - All BEAM-related expressions are available via the top-level <literal>beam</literal> attribute, which includes: - </para> - - <itemizedlist> - <listitem> - <para> - <literal>interpreters</literal>: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (<literal>beam.interpreters.erlangR19</literal>, etc), Elixir (<literal>beam.interpreters.elixir</literal>) and LFE (<literal>beam.interpreters.lfe</literal>). - </para> - </listitem> - <listitem> - <para> - <literal>packages</literal>: a set of package builders (Mix and rebar3), each compiled with a specific Erlang/OTP version, e.g. <literal>beam.packages.erlangR19</literal>. - </para> - </listitem> - </itemizedlist> - - <para> - The default Erlang compiler, defined by <literal>beam.interpreters.erlang</literal>, is aliased as <literal>erlang</literal>. The default BEAM package set is defined by <literal>beam.packages.erlang</literal> and aliased at the top level as <literal>beamPackages</literal>. - </para> - - <para> - To create a package builder built with a custom Erlang version, use the lambda, <literal>beam.packagesWith</literal>, which accepts an Erlang/OTP derivation and produces a package builder similar to <literal>beam.packages.erlang</literal>. - </para> - - <para> - Many Erlang/OTP distributions available in <literal>beam.interpreters</literal> have versions with ODBC and/or Java enabled or without wx (no observer support). For example, there's <literal>beam.interpreters.erlangR22_odbc_javac</literal>, which corresponds to <literal>beam.interpreters.erlangR22</literal> and <literal>beam.interpreters.erlangR22_nox</literal>, which corresponds to <literal>beam.interpreters.erlangR22</literal>. - </para> - </section> - - <section xml:id="build-tools"> - <title>Build Tools</title> - - <section xml:id="build-tools-rebar3"> - <title>Rebar3</title> - - <para> - We provide a version of Rebar3, under <literal>rebar3</literal>. We also provide a helper to fetch Rebar3 dependencies from a lockfile under <literal>fetchRebar3Deps</literal>. - </para> - </section> - - <section xml:id="build-tools-other"> - <title>Mix & Erlang.mk</title> - - <para> - Both Mix and Erlang.mk work exactly as expected. There is a bootstrap process that needs to be run for both, however, which is supported by the <literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations, respectively. - </para> - </section> - </section> - - <section xml:id="how-to-install-beam-packages"> - <title>How to Install BEAM Packages</title> - - <para> - BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. - To install any of those builders into your profile, refer to them by their attribute path <literal>beamPackages.rebar3</literal>: - </para> - -<screen> -<prompt>$ </prompt>nix-env -f "<nixpkgs>" -iA beamPackages.rebar3 -</screen> -</section> - - <section xml:id="packaging-beam-applications"> - <title>Packaging BEAM Applications</title> - - <section xml:id="packaging-erlang-applications"> - <title>Erlang Applications</title> - - <section xml:id="rebar3-packages"> - <title>Rebar3 Packages</title> - - <para> - The Nix function, <literal>buildRebar3</literal>, defined in <literal>beam.packages.erlang.buildRebar3</literal> and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. - </para> - - <para> - If a package needs to compile native code via Rebar3's port compilation mechanism, add <literal>compilePort = true;</literal> to the derivation. - </para> - </section> - - <section xml:id="erlang-mk-packages"> - <title>Erlang.mk Packages</title> - - <para> - Erlang.mk functions similarly to Rebar3, except we use <literal>buildErlangMk</literal> instead of <literal>buildRebar3</literal>. - </para> - - </section> - - <section xml:id="mix-packages"> - <title>Mix Packages</title> - - <para> - Mix functions similarly to Rebar3, except we use <literal>buildMix</literal> instead of <literal>buildRebar3</literal>. - </para> - - <para> - Alternatively, we can use <literal>buildHex</literal> as a shortcut: - </para> - </section> - </section> - </section> - - <section xml:id="how-to-develop"> - <title>How to Develop</title> - - <section xml:id="creating-a-shell"> - <title>Creating a Shell</title> - - <para> - Usually, we need to create a <literal>shell.nix</literal> file and do our development inside of the environment specified therein. Just install your version of erlang and other interpreter, and then user your normal build tools. - As an example with elixir: - </para> - -<programlisting> -{ pkgs ? import "<nixpkgs"> {} }: - -with pkgs; - -let - - elixir = beam.packages.erlangR22.elixir_1_9; - -in -mkShell { - buildInputs = [ elixir ]; - - ERL_INCLUDE_PATH="${erlang}/lib/erlang/usr/include"; -} -</programlisting> - - <section xml:id="building-in-a-shell"> - <title>Building in a Shell (for Mix Projects)</title> - - <para> - Using a <literal>shell.nix</literal> as described (see <xref - linkend="creating-a-shell"/>) should just work. - </para> - </section> - </section> - </section> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/coq.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/coq.section.md new file mode 100644 index 000000000000..714e84efc8db --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/coq.section.md @@ -0,0 +1,40 @@ +# Coq {#sec-language-coq} + +Coq libraries should be installed in `$(out)/lib/coq/${coq.coq-version}/user-contrib/`. Such directories are automatically added to the `$COQPATH` environment variable by the hook defined in the Coq derivation. + +Some extensions (plugins) might require OCaml and sometimes other OCaml packages. The `coq.ocamlPackages` attribute can be used to depend on the same package set Coq was built against. + +Coq libraries may be compatible with some specific versions of Coq only. The `compatibleCoqVersions` attribute is used to precisely select those versions of Coq that are compatible with this derivation. + +Here is a simple package example. It is a pure Coq library, thus it depends on Coq. It builds on the Mathematical Components library, thus it also takes `mathcomp` as `buildInputs`. Its `Makefile` has been generated using `coq_makefile` so we only have to set the `$COQLIB` variable at install time. + +```nix +{ stdenv, fetchFromGitHub, coq, mathcomp }: + +stdenv.mkDerivation rec { + name = "coq${coq.coq-version}-multinomials-${version}"; + version = "1.0"; + src = fetchFromGitHub { + owner = "math-comp"; + repo = "multinomials"; + rev = version; + sha256 = "1qmbxp1h81cy3imh627pznmng0kvv37k4hrwi2faa101s6bcx55m"; + }; + + buildInputs = [ coq ]; + propagatedBuildInputs = [ mathcomp ]; + + installFlags = "COQLIB=$(out)/lib/coq/${coq.coq-version}/"; + + meta = { + description = "A Coq/SSReflect Library for Monoidal Rings and Multinomials"; + inherit (src.meta) homepage; + license = stdenv.lib.licenses.cecill-b; + inherit (coq.meta) platforms; + }; + + passthru = { + compatibleCoqVersions = v: builtins.elem v [ "8.5" "8.6" "8.7" ]; + }; +} +``` diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/coq.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/coq.xml deleted file mode 100644 index 86d9226166f5..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/coq.xml +++ /dev/null @@ -1,52 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-coq"> - <title>Coq</title> - - <para> - Coq libraries should be installed in <literal>$(out)/lib/coq/${coq.coq-version}/user-contrib/</literal>. Such directories are automatically added to the <literal>$COQPATH</literal> environment variable by the hook defined in the Coq derivation. - </para> - - <para> - Some extensions (plugins) might require OCaml and sometimes other OCaml packages. The <literal>coq.ocamlPackages</literal> attribute can be used to depend on the same package set Coq was built against. - </para> - - <para> - Coq libraries may be compatible with some specific versions of Coq only. The <literal>compatibleCoqVersions</literal> attribute is used to precisely select those versions of Coq that are compatible with this derivation. - </para> - - <para> - Here is a simple package example. It is a pure Coq library, thus it depends on Coq. It builds on the Mathematical Components library, thus it also takes <literal>mathcomp</literal> as <literal>buildInputs</literal>. Its <literal>Makefile</literal> has been generated using <literal>coq_makefile</literal> so we only have to set the <literal>$COQLIB</literal> variable at install time. - </para> - -<programlisting> -{ stdenv, fetchFromGitHub, coq, mathcomp }: - -stdenv.mkDerivation rec { - name = "coq${coq.coq-version}-multinomials-${version}"; - version = "1.0"; - src = fetchFromGitHub { - owner = "math-comp"; - repo = "multinomials"; - rev = version; - sha256 = "1qmbxp1h81cy3imh627pznmng0kvv37k4hrwi2faa101s6bcx55m"; - }; - - buildInputs = [ coq ]; - propagatedBuildInputs = [ mathcomp ]; - - installFlags = "COQLIB=$(out)/lib/coq/${coq.coq-version}/"; - - meta = { - description = "A Coq/SSReflect Library for Monoidal Rings and Multinomials"; - inherit (src.meta) homepage; - license = stdenv.lib.licenses.cecill-b; - inherit (coq.meta) platforms; - }; - - passthru = { - compatibleCoqVersions = v: builtins.elem v [ "8.5" "8.6" "8.7" ]; - }; -} -</programlisting> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/dotnet.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/dotnet.section.md index c56f4728bed8..88fd74db8256 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/dotnet.section.md +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/dotnet.section.md @@ -64,9 +64,9 @@ $ dotnet --info The `dotnetCorePackages.sdk_X_Y` is preferred over the old dotnet-sdk as both major and minor version are very important for a dotnet environment. If a given minor version isn't present (or was changed), then this will likely break your ability to build a project. -## dotnetCorePackages.sdk vs dotnetCorePackages.netcore vs dotnetCorePackages.aspnetcore +## dotnetCorePackages.sdk vs vs dotnetCorePackages.net vs dotnetCorePackages.netcore vs dotnetCorePackages.aspnetcore -The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. +The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `net`, `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. For runtime versions >= .NET 5 `net` is used while `netcore` is used for older .NET Core runtime version. ## Packaging a Dotnet Application diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/gnome.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/gnome.xml index 159216ca981f..f555cacbd2c4 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/gnome.xml +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/gnome.xml @@ -28,6 +28,22 @@ </para> </section> + <section xml:id="ssec-gnome-gdk-pixbuf-loaders"> + <title>GdkPixbuf loaders</title> + + <para> + GTK applications typically use <link xlink:href="https://developer.gnome.org/gdk-pixbuf/stable/">GdkPixbuf</link> to load images. But <package>gdk-pixbuf</package> package only supports basic bitmap formats like JPEG, PNG or TIFF, requiring to use third-party loader modules for other formats. This is especially painful since GTK itself includes SVG icons, which cannot be rendered without a loader provided by <package>librsvg</package>. + </para> + + <para> + Unlike other libraries mentioned in this section, GdkPixbuf only supports a single value in its controlling environment variable <envar>GDK_PIXBUF_MODULE_FILE</envar>. It is supposed to point to a cache file containing information about the available loaders. Each loader package will contain a <filename>lib/gdk-pixbuf-2.0/2.10.0/loaders.cache</filename> file describing the default loaders in <package>gdk-pixbuf</package> package plus the loader contained in the package itself. If you want to use multiple third-party loaders, you will need to create your own cache file manually. Fortunately, this is pretty rare as <link xlink:href="https://gitlab.gnome.org/federico/gdk-pixbuf-survey/blob/master/src/modules.md">not many loaders exist</link>. + </para> + + <para> + <package>gdk-pixbuf</package> contains <link linkend="ssec-gnome-hooks-gdk-pixbuf">a setup hook</link> that sets <envar>GDK_PIXBUF_MODULE_FILE</envar> from dependencies but as mentioned in further section, it is pretty limited. Loaders should propagate this setup hook. + </para> + </section> + <section xml:id="ssec-gnome-icons"> <title>Icons</title> @@ -100,9 +116,16 @@ preFixup = '' done ''; </programlisting> - Fortunately, there is <package>wrapGAppsHook</package>, that does the wrapping for us. In particular, it works in conjunction with other setup hooks that will populate the variable: + </para> + <para> + Fortunately, there is <package xml:id="ssec-gnome-hooks-wrapgappshook">wrapGAppsHook</package>. It works in conjunction with other setup hooks that populate environment variables, and it will then wrap all executables in <filename>bin</filename> and <filename>libexec</filename> directories using said variables. + </para> + <para> + For convenience, it also adds <package>dconf.lib</package> for a GIO module implementing a GSettings backend using <package>dconf</package>, <package>gtk3</package> for GSettings schemas, and <package>librsvg</package> for GdkPixbuf loader to the closure. In case you are packaging a program without a graphical interface, you might want to use <package xml:id="ssec-gnome-hooks-wrapgappsnoguihook">wrapGAppsNoGuiHook</package>, which runs the same script as <package>wrapGAppsHook</package> but does not bring <package>gtk3</package> and <package>librsvg</package> into the closure. + </para> + <para> <itemizedlist> - <listitem xml:id="ssec-gnome-hooks-wrapgappshook"> + <listitem> <para> <package>wrapGAppsHook</package> itself will add the package’s <filename>share</filename> directory to <envar>XDG_DATA_DIRS</envar>. </para> @@ -112,6 +135,11 @@ preFixup = '' <package>glib</package> setup hook will populate <envar>GSETTINGS_SCHEMAS_PATH</envar> and then <package>wrapGAppsHook</package> will prepend it to <envar>XDG_DATA_DIRS</envar>. </para> </listitem> + <listitem xml:id="ssec-gnome-hooks-gdk-pixbuf"> + <para> + <package>gdk-pixbuf</package> setup hook will populate <envar>GDK_PIXBUF_MODULE_FILE</envar> with the path to biggest <filename>loaders.cache</filename> file from the dependencies containing <link xlink:href="ssec-gnome-gdk-pixbuf-loaders">GdkPixbuf loaders</link>. This works fine when there are only two packages containing loaders (<package>gdk-pixbuf</package> and e.g. <package>librsvg</package>) – it will choose the second one, reasonably expecting that it will be bigger since it describes extra loader in addition to the default ones. But when there are more than two loader packages, this logic will break. One possible solution would be constructing a custom cache file for each package containing a program like <filename>services/x11/gdk-pixbuf.nix</filename> NixOS module does. <package>wrapGAppsHook</package> copies the <envar>GDK_PIXBUF_MODULE_FILE</envar> environment variable into the produced wrapper. + </para> + </listitem> <listitem xml:id="ssec-gnome-hooks-gtk-drop-icon-theme-cache"> <para> One of <package>gtk3</package>’s setup hooks will remove <filename>icon-theme.cache</filename> files from package’s icon theme directories to avoid conflicts. Icon theme packages should prevent this with <code>dontDropIconThemeCache = true;</code>. @@ -178,7 +206,7 @@ preFixup = '' </term> <listitem> <para> - There are no schemas avalable in <envar>XDG_DATA_DIRS</envar>. Temporarily add a random package containing schemas like <package>gsettings-desktop-schemas</package> to <literal>buildInputs</literal>. <link linkend="ssec-gnome-hooks-glib"><package>glib</package></link> and <link linkend="ssec-gnome-hooks-wrapgappshook"><package>wrapGAppsHook</package></link> setup hooks will take care of making the schemas available to application and you will see the actual missing schemas with the <link linkend="ssec-gnome-common-issues-missing-schema">next error</link>. Or you can try looking through the source code for the actual schemas used. + There are no schemas available in <envar>XDG_DATA_DIRS</envar>. Temporarily add a random package containing schemas like <package>gsettings-desktop-schemas</package> to <literal>buildInputs</literal>. <link linkend="ssec-gnome-hooks-glib"><package>glib</package></link> and <link linkend="ssec-gnome-hooks-wrapgappshook"><package>wrapGAppsHook</package></link> setup hooks will take care of making the schemas available to application and you will see the actual missing schemas with the <link linkend="ssec-gnome-common-issues-missing-schema">next error</link>. Or you can try looking through the source code for the actual schemas used. </para> </listitem> </varlistentry> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/go.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/go.section.md new file mode 100644 index 000000000000..b4228d9d313d --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/go.section.md @@ -0,0 +1,140 @@ +# Go {#sec-language-go} + +## Go modules {#ssec-language-go} + +The function `buildGoModule` builds Go programs managed with Go modules. It builds a [Go Modules](https://github.com/golang/go/wiki/Modules) through a two phase build: + +- An intermediate fetcher derivation. This derivation will be used to fetch all of the dependencies of the Go module. +- A final derivation will use the output of the intermediate derivation to build the binaries and produce the final output. + +### Example for `buildGoModule` {#ex-buildGoModule} + +In the following is an example expression using `buildGoModule`, the following arguments are of special significance to the function: + +- `vendorSha256`: is the hash of the output of the intermediate fetcher derivation. `vendorSha256` can also take `null` as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set `vendorSha256 = null;` +- `runVend`: runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build. + +```nix +pet = buildGoModule rec { + pname = "pet"; + version = "0.3.4"; + + src = fetchFromGitHub { + owner = "knqyf263"; + repo = "pet"; + rev = "v${version}"; + sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s"; + }; + + vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; + + runVend = true; + + meta = with lib; { + description = "Simple command-line snippet manager, written in Go"; + homepage = "https://github.com/knqyf263/pet"; + license = licenses.mit; + maintainers = with maintainers; [ kalbasit ]; + platforms = platforms.linux ++ platforms.darwin; + }; +} +``` + +## `buildGoPackage` (legacy) {#ssec-go-legacy} + +The function `buildGoPackage` builds legacy Go programs, not supporting Go modules. + +### Example for `buildGoPackage` + +In the following is an example expression using buildGoPackage, the following arguments are of special significance to the function: + +- `goPackagePath` specifies the package's canonical Go import path. +- `goDeps` is where the Go dependencies of a Go program are listed as a list of package source identified by Go import path. It could be imported as a separate `deps.nix` file for readability. The dependency data structure is described below. + +```nix +deis = buildGoPackage rec { + pname = "deis"; + version = "1.13.0"; + + goPackagePath = "github.com/deis/deis"; + + src = fetchFromGitHub { + owner = "deis"; + repo = "deis"; + rev = "v${version}"; + sha256 = "1qv9lxqx7m18029lj8cw3k7jngvxs4iciwrypdy0gd2nnghc68sw"; + }; + + goDeps = ./deps.nix; +} +``` + +The `goDeps` attribute can be imported from a separate `nix` file that defines which Go libraries are needed and should be included in `GOPATH` for `buildPhase`: + +```nix +# deps.nix +[ # goDeps is a list of Go dependencies. + { + # goPackagePath specifies Go package import path. + goPackagePath = "gopkg.in/yaml.v2"; + fetch = { + # `fetch type` that needs to be used to get package source. + # If `git` is used there should be `url`, `rev` and `sha256` defined next to it. + type = "git"; + url = "https://gopkg.in/yaml.v2"; + rev = "a83829b6f1293c91addabc89d0571c246397bbf4"; + sha256 = "1m4dsmk90sbi17571h6pld44zxz7jc4lrnl4f27dpd1l8g5xvjhh"; + }; + } + { + goPackagePath = "github.com/docopt/docopt-go"; + fetch = { + type = "git"; + url = "https://github.com/docopt/docopt-go"; + rev = "784ddc588536785e7299f7272f39101f7faccc3f"; + sha256 = "0wwz48jl9fvl1iknvn9dqr4gfy1qs03gxaikrxxp9gry6773v3sj"; + }; + } +] +``` + +To extract dependency information from a Go package in automated way use [go2nix](https://github.com/kamilchm/go2nix). It can produce complete derivation and `goDeps` file for Go programs. + +You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc: + +```bash +for p in $NIX_PROFILES; do + GOPATH="$p/share/go:$GOPATH" +done +``` + +## Attributes used by the builders {#ssec-go-common-attributes} + +Both `buildGoModule` and `buildGoPackage` can be tweaked to behave slightly differently, if the following attributes are used: + +### `buildFlagsArray` and `buildFlags`: {#ex-goBuildFlags-noarray} + +These attributes set build flags supported by `go build`. We recommend using `buildFlagsArray`. The most common use case of these attributes is to make the resulting executable aware of its own version. For example: + +```nix + buildFlagsArray = [ + # Note: single quotes are not needed. + "-ldflags=-X main.Version=${version} -X main.Commit=${version}" + ]; +``` + +```nix + buildFlagsArray = '' + -ldflags= + -X main.Version=${version} + -X main.Commit=${version} + ''; +``` + +### `deleteVendor` {#var-go-deleteVendor} + +Removes the pre-existing vendor directory. This should only be used if the dependencies included in the vendor folder are broken or incomplete. + +### `subPackages` {#var-go-subPackages} + +Limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built. diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/go.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/go.xml deleted file mode 100644 index ebdcf616054c..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/go.xml +++ /dev/null @@ -1,248 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-go"> - <title>Go</title> - - <section xml:id="ssec-go-modules"> - <title>Go modules</title> - - <para> - The function <varname> buildGoModule </varname> builds Go programs managed with Go modules. It builds a <link xlink:href="https://github.com/golang/go/wiki/Modules">Go modules</link> through a two phase build: - <itemizedlist> - <listitem> - <para> - An intermediate fetcher derivation. This derivation will be used to fetch all of the dependencies of the Go module. - </para> - </listitem> - <listitem> - <para> - A final derivation will use the output of the intermediate derivation to build the binaries and produce the final output. - </para> - </listitem> - </itemizedlist> - </para> - - <example xml:id='ex-buildGoModule'> - <title>buildGoModule</title> -<programlisting> -pet = buildGoModule rec { - pname = "pet"; - version = "0.3.4"; - - src = fetchFromGitHub { - owner = "knqyf263"; - repo = "pet"; - rev = "v${version}"; - sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s"; - }; - - vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; <co xml:id='ex-buildGoModule-1' /> - - runVend = true; <co xml:id='ex-buildGoModule-2' /> - - meta = with lib; { - description = "Simple command-line snippet manager, written in Go"; - homepage = "https://github.com/knqyf263/pet"; - license = licenses.mit; - maintainers = with maintainers; [ kalbasit ]; - platforms = platforms.linux ++ platforms.darwin; - }; -} -</programlisting> - </example> - - <para> - <xref linkend='ex-buildGoModule'/> is an example expression using buildGoModule, the following arguments are of special significance to the function: - <calloutlist> - <callout arearefs='ex-buildGoModule-1'> - <para> - <varname>vendorSha256</varname> is the hash of the output of the intermediate fetcher derivation. - </para> - </callout> - <callout arearefs='ex-buildGoModule-2'> - <para> - <varname>runVend</varname> runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build. - </para> - </callout> - </calloutlist> - </para> - - <para> - <varname>vendorSha256</varname> can also take <varname>null</varname> as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set 'vendorSha256 = null;' - </para> - </section> - - <section xml:id="ssec-go-legacy"> - <title>Go legacy</title> - - <para> - The function <varname> buildGoPackage </varname> builds legacy Go programs, not supporting Go modules. - </para> - - <example xml:id='ex-buildGoPackage'> - <title>buildGoPackage</title> -<programlisting> -deis = buildGoPackage rec { - pname = "deis"; - version = "1.13.0"; - - goPackagePath = "github.com/deis/deis"; <co xml:id='ex-buildGoPackage-1' /> - - src = fetchFromGitHub { - owner = "deis"; - repo = "deis"; - rev = "v${version}"; - sha256 = "1qv9lxqx7m18029lj8cw3k7jngvxs4iciwrypdy0gd2nnghc68sw"; - }; - - goDeps = ./deps.nix; <co xml:id='ex-buildGoPackage-2' /> -} -</programlisting> - </example> - - <para> - <xref linkend='ex-buildGoPackage'/> is an example expression using buildGoPackage, the following arguments are of special significance to the function: - <calloutlist> - <callout arearefs='ex-buildGoPackage-1'> - <para> - <varname>goPackagePath</varname> specifies the package's canonical Go import path. - </para> - </callout> - <callout arearefs='ex-buildGoPackage-2'> - <para> - <varname>goDeps</varname> is where the Go dependencies of a Go program are listed as a list of package source identified by Go import path. It could be imported as a separate <varname>deps.nix</varname> file for readability. The dependency data structure is described below. - </para> - </callout> - </calloutlist> - </para> - - <para> - The <varname>goDeps</varname> attribute can be imported from a separate <varname>nix</varname> file that defines which Go libraries are needed and should be included in <varname>GOPATH</varname> for <varname>buildPhase</varname>. - </para> - - <example xml:id='ex-goDeps'> - <title>deps.nix</title> -<programlisting> -[ <co xml:id='ex-goDeps-1' /> - { - goPackagePath = "gopkg.in/yaml.v2"; <co xml:id='ex-goDeps-2' /> - fetch = { - type = "git"; <co xml:id='ex-goDeps-3' /> - url = "https://gopkg.in/yaml.v2"; - rev = "a83829b6f1293c91addabc89d0571c246397bbf4"; - sha256 = "1m4dsmk90sbi17571h6pld44zxz7jc4lrnl4f27dpd1l8g5xvjhh"; - }; - } - { - goPackagePath = "github.com/docopt/docopt-go"; - fetch = { - type = "git"; - url = "https://github.com/docopt/docopt-go"; - rev = "784ddc588536785e7299f7272f39101f7faccc3f"; - sha256 = "0wwz48jl9fvl1iknvn9dqr4gfy1qs03gxaikrxxp9gry6773v3sj"; - }; - } -] -</programlisting> - </example> - - <para> - <calloutlist> - <callout arearefs='ex-goDeps-1'> - <para> - <varname>goDeps</varname> is a list of Go dependencies. - </para> - </callout> - <callout arearefs='ex-goDeps-2'> - <para> - <varname>goPackagePath</varname> specifies Go package import path. - </para> - </callout> - <callout arearefs='ex-goDeps-3'> - <para> - <varname>fetch type</varname> that needs to be used to get package source. If <varname>git</varname> is used there should be <varname>url</varname>, <varname>rev</varname> and <varname>sha256</varname> defined next to it. - </para> - </callout> - </calloutlist> - </para> - - <para> - To extract dependency information from a Go package in automated way use <link xlink:href="https://github.com/kamilchm/go2nix">go2nix</link>. It can produce complete derivation and <varname>goDeps</varname> file for Go programs. - </para> - - <para> - You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc: -<screen> -for p in $NIX_PROFILES; do - GOPATH="$p/share/go:$GOPATH" -done -</screen> - </para> - </section> - - <section xml:id="ssec-go-common-attributes"> - <title>Attributes used by the builders</title> - - <para> - Both <link xlink:href="#ssec-go-modules"><varname>buildGoModule</varname></link> and <link xlink:href="#ssec-go-modules"><varname>buildGoPackage</varname></link> can be tweaked to behave slightly differently, if the following attributes are used: - </para> - - <variablelist> - <varlistentry xml:id="var-go-buildFlagsArray"> - <term> - <varname>buildFlagsArray</varname> and <varname>buildFlags</varname> - </term> - <listitem> - <para> - These attributes set build flags supported by <varname>go build</varname>. We recommend using <varname>buildFlagsArray</varname>. The most common use case of these attributes is to make the resulting executable aware of its own version. For example: - </para> - <example xml:id='ex-goBuildFlags-nospaces'> - <title>buildFlagsArray</title> -<programlisting> - buildFlagsArray = [ - "-ldflags=-X main.Version=${version} -X main.Commit=${version}" <co xml:id='ex-goBuildFlags-1' /> - ]; -</programlisting> - </example> - <calloutlist> - <callout arearefs='ex-goBuildFlags-1'> - <para> - Note: single quotes are not needed. - </para> - </callout> - </calloutlist> - <example xml:id='ex-goBuildFlags-noarray'> - <title>buildFlagsArray</title> -<programlisting> - buildFlagsArray = '' - -ldflags= - -X main.Version=${version} - -X main.Commit=${version} - ''; -</programlisting> - </example> - </listitem> - </varlistentry> - <varlistentry xml:id="var-go-deleteVendor"> - <term> - <varname>deleteVendor</varname> - </term> - <listitem> - <para> - Removes the pre-existing vendor directory. This should only be used if the dependencies included in the vendor folder are broken or incomplete. - </para> - </listitem> - </varlistentry> - <varlistentry xml:id="var-go-subPackages"> - <term> - <varname>subPackages</varname> - </term> - <listitem> - <para> - Limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built. - </para> - </listitem> - </varlistentry> - </variablelist> - </section> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/haskell.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/haskell.section.md index c2838c733c72..67318a5b7467 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/haskell.section.md +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/haskell.section.md @@ -3,1125 +3,11 @@ title: User's Guide for Haskell in Nixpkgs author: Peter Simons date: 2015-06-01 --- -# Haskell - - -## How to install Haskell packages - -Nixpkgs distributes build instructions for all Haskell packages registered on -[Hackage](http://hackage.haskell.org/), but strangely enough normal Nix package -lookups don't seem to discover any of them, except for the default version of ghc, cabal-install, and stack: -``` -$ nix-env -i alex -error: selector ‘alex’ matches no derivations -$ nix-env -qa ghc -ghc-7.10.2 -``` - -The Haskell package set is not registered in the top-level namespace because it -is *huge*. If all Haskell packages were visible to these commands, then -name-based search/install operations would be much slower than they are now. We -avoided that by keeping all Haskell-related packages in a separate attribute -set called `haskellPackages`, which the following command will list: -``` -$ nix-env -f "<nixpkgs>" -qaP -A haskellPackages -haskellPackages.a50 a50-0.5 -haskellPackages.AAI AAI-0.2.0.1 -haskellPackages.abacate abacate-0.0.0.0 -haskellPackages.abc-puzzle abc-puzzle-0.2.1 -haskellPackages.abcBridge abcBridge-0.15 -haskellPackages.abcnotation abcnotation-1.9.0 -haskellPackages.abeson abeson-0.1.0.1 -[... some 14000 entries omitted ...] -``` - -To install any of those packages into your profile, refer to them by their -attribute path (first column): -```shell -nix-env -f "<nixpkgs>" -iA haskellPackages.Allure ... -``` - -The attribute path of any Haskell packages corresponds to the name of that -particular package on Hackage: the package `cabal-install` has the attribute -`haskellPackages.cabal-install`, and so on. (Actually, this convention causes -trouble with packages like `3dmodels` and `4Blocks`, because these names are -invalid identifiers in the Nix language. The issue of how to deal with these -rare corner cases is currently unresolved.) - -Haskell packages whose Nix name (second column) begins with a `haskell-` prefix -are packages that provide a library whereas packages without that prefix -provide just executables. Libraries may provide executables too, though: the -package `haskell-pandoc`, for example, installs both a library and an -application. You can install and use Haskell executables just like any other -program in Nixpkgs, but using Haskell libraries for development is a bit -trickier and we'll address that subject in great detail in section [How to -create a development environment](#how-to-create-a-development-environment). - -Attribute paths are deterministic inside of Nixpkgs, but the path necessary to -reach Nixpkgs varies from system to system. We dodged that problem by giving -`nix-env` an explicit `-f "<nixpkgs>"` parameter, but if you call `nix-env` -without that flag, then chances are the invocation fails: -``` -$ nix-env -iA haskellPackages.cabal-install -error: attribute ‘haskellPackages’ in selection path - ‘haskellPackages.cabal-install’ not found -``` - -On NixOS, for example, Nixpkgs does *not* exist in the top-level namespace by -default. To figure out the proper attribute path, it's easiest to query for the -path of a well-known Nixpkgs package, i.e.: -``` -$ nix-env -qaP coreutils -nixos.coreutils coreutils-8.23 -``` - -If your system responds like that (most NixOS installations will), then the -attribute path to `haskellPackages` is `nixos.haskellPackages`. Thus, if you -want to use `nix-env` without giving an explicit `-f` flag, then that's the way -to do it: -```shell -nix-env -qaP -A nixos.haskellPackages -nix-env -iA nixos.haskellPackages.cabal-install -``` - -Our current default compiler is GHC 8.8.x and the `haskellPackages` set -contains packages built with that particular version. Nixpkgs contains the last -three major releases of GHC and there is a whole family of package sets -available that defines Hackage packages built with each of those compilers, -too: -```shell -nix-env -f "<nixpkgs>" -qaP -A haskell.packages.ghc865 -nix-env -f "<nixpkgs>" -qaP -A haskell.packages.ghc8101 -``` - -The name `haskellPackages` is really just a synonym for -`haskell.packages.ghc882`, because we prefer that package set internally and -recommend it to our users as their default choice, but ultimately you are free -to compile your Haskell packages with any GHC version you please. The following -command displays the complete list of available compilers: -``` -$ nix-env -f "<nixpkgs>" -qaP -A haskell.compiler -haskell.compiler.ghc8101 ghc-8.10.1 -haskell.compiler.integer-simple.ghc8101 ghc-8.10.1 -haskell.compiler.ghcHEAD ghc-8.11.20200505 -haskell.compiler.integer-simple.ghcHEAD ghc-8.11.20200505 -haskell.compiler.ghc822Binary ghc-8.2.2-binary -haskell.compiler.ghc844 ghc-8.4.4 -haskell.compiler.ghc863Binary ghc-8.6.3-binary -haskell.compiler.ghc865 ghc-8.6.5 -haskell.compiler.integer-simple.ghc865 ghc-8.6.5 -haskell.compiler.ghc882 ghc-8.8.2 -haskell.compiler.integer-simple.ghc882 ghc-8.8.2 -haskell.compiler.ghc883 ghc-8.8.3 -haskell.compiler.integer-simple.ghc883 ghc-8.8.3 -haskell.compiler.ghcjs ghcjs-8.6.0.1 -``` - -We have no package sets for `jhc` or `uhc` yet, unfortunately, but for every -version of GHC listed above, there exists a package set based on that compiler. -Also, the attributes `haskell.compiler.ghcXYC` and -`haskell.packages.ghcXYC.ghc` are synonymous for the sake of convenience. - -## How to create a development environment - -### How to install a compiler - -A simple development environment consists of a Haskell compiler and one or both -of the tools `cabal-install` and `stack`. We saw in section -[How to install Haskell packages](#how-to-install-haskell-packages) how you can install those programs into your -user profile: -```shell -nix-env -f "<nixpkgs>" -iA haskellPackages.ghc haskellPackages.cabal-install -``` - -Instead of the default package set `haskellPackages`, you can also use the more -precise name `haskell.compiler.ghc7102`, which has the advantage that it refers -to the same GHC version regardless of what Nixpkgs considers "default" at any -given time. - -Once you've made those tools available in `$PATH`, it's possible to build -Hackage packages the same way people without access to Nix do it all the time: -```shell -cabal get lens-4.11 && cd lens-4.11 -cabal install -j --dependencies-only -cabal configure -cabal build -``` - -If you enjoy working with Cabal sandboxes, then that's entirely possible too: -just execute the command -```shell -cabal sandbox init -``` -before installing the required dependencies. - -The `nix-shell` utility makes it easy to switch to a different compiler -version; just enter the Nix shell environment with the command -```shell -nix-shell -p haskell.compiler.ghc784 -``` -to bring GHC 7.8.4 into `$PATH`. Alternatively, you can use Stack instead of -`nix-shell` directly to select compiler versions and other build tools -per-project. It uses `nix-shell` under the hood when Nix support is turned on. -See [How to build a Haskell project using Stack](#how-to-build-a-haskell-project-using-stack). - -If you're using `cabal-install`, re-running `cabal configure` inside the spawned -shell switches your build to use that compiler instead. If you're working on -a project that doesn't depend on any additional system libraries outside of GHC, -then it's even sufficient to just run the `cabal configure` command inside of -the shell: -```shell -nix-shell -p haskell.compiler.ghc784 --command "cabal configure" -``` - -Afterwards, all other commands like `cabal build` work just fine in any shell -environment, because the configure phase recorded the absolute paths to all -required tools like GHC in its build configuration inside of the `dist/` -directory. Please note, however, that `nix-collect-garbage` can break such an -environment because the Nix store paths created by `nix-shell` aren't "alive" -anymore once `nix-shell` has terminated. If you find that your Haskell builds -no longer work after garbage collection, then you'll have to re-run `cabal -configure` inside of a new `nix-shell` environment. - -### How to install a compiler with libraries - -GHC expects to find all installed libraries inside of its own `lib` directory. -This approach works fine on traditional Unix systems, but it doesn't work for -Nix, because GHC's store path is immutable once it's built. We cannot install -additional libraries into that location. As a consequence, our copies of GHC -don't know any packages except their own core libraries, like `base`, -`containers`, `Cabal`, etc. - -We can register additional libraries to GHC, however, using a special build -function called `ghcWithPackages`. That function expects one argument: a -function that maps from an attribute set of Haskell packages to a list of -packages, which determines the libraries known to that particular version of -GHC. For example, the Nix expression `ghcWithPackages (pkgs: [pkgs.mtl])` -generates a copy of GHC that has the `mtl` library registered in addition to -its normal core packages: -``` -$ nix-shell -p "haskellPackages.ghcWithPackages (pkgs: [pkgs.mtl])" - -[nix-shell:~]$ ghc-pkg list mtl -/nix/store/zy79...-ghc-7.10.2/lib/ghc-7.10.2/package.conf.d: - mtl-2.2.1 -``` - -This function allows users to define their own development environment by means -of an override. After adding the following snippet to `~/.config/nixpkgs/config.nix`, -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - myHaskellEnv = self.haskell.packages.ghc7102.ghcWithPackages - (haskellPackages: with haskellPackages; [ - # libraries - arrows async cgi criterion - # tools - cabal-install haskintex - ]); - }; -} -``` -it's possible to install that compiler with `nix-env -f "<nixpkgs>" -iA -myHaskellEnv`. If you'd like to switch that development environment to a -different version of GHC, just replace the `ghc7102` bit in the previous -definition with the appropriate name. Of course, it's also possible to define -any number of these development environments! (You can't install two of them -into the same profile at the same time, though, because that would result in -file conflicts.) - -The generated `ghc` program is a wrapper script that re-directs the real -GHC executable to use a new `lib` directory --- one that we specifically -constructed to contain all those packages the user requested: -``` -$ cat $(type -p ghc) -#! /nix/store/xlxj...-bash-4.3-p33/bin/bash -e -export NIX_GHC=/nix/store/19sm...-ghc-7.10.2/bin/ghc -export NIX_GHCPKG=/nix/store/19sm...-ghc-7.10.2/bin/ghc-pkg -export NIX_GHC_DOCDIR=/nix/store/19sm...-ghc-7.10.2/share/doc/ghc/html -export NIX_GHC_LIBDIR=/nix/store/19sm...-ghc-7.10.2/lib/ghc-7.10.2 -exec /nix/store/j50p...-ghc-7.10.2/bin/ghc "-B$NIX_GHC_LIBDIR" "$@" -``` - -The variables `$NIX_GHC`, `$NIX_GHCPKG`, etc. point to the *new* store path -`ghcWithPackages` constructed specifically for this environment. The last line -of the wrapper script then executes the real `ghc`, but passes the path to the -new `lib` directory using GHC's `-B` flag. - -The purpose of those environment variables is to work around an impurity in the -popular [ghc-paths](http://hackage.haskell.org/package/ghc-paths) library. That -library promises to give its users access to GHC's installation paths. Only, -the library can't possible know that path when it's compiled, because the path -GHC considers its own is determined only much later, when the user configures -it through `ghcWithPackages`. So we [patched -ghc-paths](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/patches/ghc-paths-nix.patch) -to return the paths found in those environment variables at run-time rather -than trying to guess them at compile-time. - -To make sure that mechanism works properly all the time, we recommend that you -set those variables to meaningful values in your shell environment, too, i.e. -by adding the following code to your `~/.bashrc`: -```bash -if type >/dev/null 2>&1 -p ghc; then - eval "$(egrep ^export "$(type -p ghc)")" -fi -``` - -If you are certain that you'll use only one GHC environment which is located in -your user profile, then you can use the following code, too, which has the -advantage that it doesn't contain any paths from the Nix store, i.e. those -settings always remain valid even if a `nix-env -u` operation updates the GHC -environment in your profile: -```bash -if [ -e ~/.nix-profile/bin/ghc ]; then - export NIX_GHC="$HOME/.nix-profile/bin/ghc" - export NIX_GHCPKG="$HOME/.nix-profile/bin/ghc-pkg" - export NIX_GHC_DOCDIR="$HOME/.nix-profile/share/doc/ghc/html" - export NIX_GHC_LIBDIR="$HOME/.nix-profile/lib/ghc-$($NIX_GHC --numeric-version)" -fi -``` - -### How to install a compiler with libraries, hoogle and documentation indexes - -If you plan to use your environment for interactive programming, not just -compiling random Haskell code, you might want to replace `ghcWithPackages` in -all the listings above with `ghcWithHoogle`. - -This environment generator not only produces an environment with GHC and all -the specified libraries, but also generates a `hoogle` and `haddock` indexes -for all the packages, and provides a wrapper script around `hoogle` binary that -uses all those things. A precise name for this thing would be -"`ghcWithPackagesAndHoogleAndDocumentationIndexes`", which is, regrettably, too -long and scary. - -For example, installing the following environment -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - myHaskellEnv = self.haskellPackages.ghcWithHoogle - (haskellPackages: with haskellPackages; [ - # libraries - arrows async cgi criterion - # tools - cabal-install haskintex - ]); - }; -} -``` -allows one to browse module documentation index [not too dissimilar to -this](https://downloads.haskell.org/~ghc/latest/docs/html/libraries/index.html) -for all the specified packages and their dependencies by directing a browser of -choice to `~/.nix-profile/share/doc/hoogle/index.html` (or -`/run/current-system/sw/share/doc/hoogle/index.html` in case you put it in -`environment.systemPackages` in NixOS). - -After you've marveled enough at that try adding the following to your -`~/.ghc/ghci.conf` -``` -:def hoogle \s -> return $ ":! hoogle search -cl --count=15 \"" ++ s ++ "\"" -:def doc \s -> return $ ":! hoogle search -cl --info \"" ++ s ++ "\"" -``` -and test it by typing into `ghci`: -``` -:hoogle a -> a -:doc a -> a -``` - -Be sure to note the links to `haddock` files in the output. With any modern and -properly configured terminal emulator you can just click those links to -navigate there. - -Finally, you can run -```shell -hoogle server --local -p 8080 -``` -and navigate to http://localhost:8080/ for your own local -[Hoogle](https://www.haskell.org/hoogle/). The `--local` flag makes the hoogle -server serve files from your nix store over http, without the flag it will use -`file://` URIs. Note, however, that Firefox and possibly other browsers -disallow navigation from `http://` to `file://` URIs for security reasons, -which might be quite an inconvenience. Versions before v5 did not have this -flag. See -[this page](http://kb.mozillazine.org/Links_to_local_pages_do_not_work) for -workarounds. - -For NixOS users there's a service which runs this exact command for you. -Specify the `packages` you want documentation for and the `haskellPackages` set -you want them to come from. Add the following to `configuration.nix`. - -```nix -services.hoogle = { - enable = true; - packages = (hpkgs: with hpkgs; [text cryptonite]); - haskellPackages = pkgs.haskellPackages; -}; -``` - -### How to install haskell-language-server - -In short: Install `pkgs.haskell-language-server` and use the -`haskell-language-server-wrapper` command to run it. See the [hls -README](https://github.com/haskell/haskell-language-server) on how to configure -your text editor to use hls and how to test your setup. - -Hls needs to be compiled with the ghc version of the project you use it on. - -`pkgs.haskell-language-server` provides `haskell-language-server-wrapper`, -`haskell-language-server`, `haskell-language-server-x.x` and -`haskell-language-server-x.x.x` binaries, where `x.x.x` is the ghc version for -which it is compiled. By default it includes binaries for all ghc versions -that are provided in the binary caches. You can override that list with e.g. - -```nix -pkgs.haskell-language-server.override { supportedGhcVersions = [ "884" "901" ]; } -``` - -When you run `haskell-language-server-wrapper` it will detect the ghc version -used by the project you are working on (by asking e.g. cabal or stack) and pick -the appropriate above mentioned binary from your path. - -Be careful when installing hls globally and using a pinned nixpkgs for a Haskell -project in a nix-shell. If the nixpkgs versions deviate to much (e.g. use -different `glibc` versions) hls might fail. It is recommended to then install hls -in the nix-shell from the nixpkgs version pinned in there. - -If you know, that you only use one ghc version, e.g. in a project specific -nix-shell You can either use an override as given above or simply install -`pkgs.haskellPackages.haskell-language-server` instead of the top-level -attribute `pkgs.haskell-language-server`. - -### How to build a Haskell project using Stack - -[Stack](http://haskellstack.org) is a popular build tool for Haskell projects. -It has first-class support for Nix. Stack can optionally use Nix to -automatically select the right version of GHC and other build tools to build, -test and execute apps in an existing project downloaded from somewhere on the -Internet. Pass the `--nix` flag to any `stack` command to do so, e.g. -```shell -git clone --recurse-submodules https://github.com/yesodweb/wai.git -cd wai -stack --nix build -``` - -If you want `stack` to use Nix by default, you can add a `nix` section to the -`stack.yaml` file, as explained in the [Stack documentation][stack-nix-doc]. For -example: -```yaml -nix: - enable: true - packages: [pkgconfig zeromq zlib] -``` - -The example configuration snippet above tells Stack to create an ad hoc -environment for `nix-shell` as in the below section, in which the `pkgconfig`, -`zeromq` and `zlib` packages from Nixpkgs are available. All `stack` commands -will implicitly be executed inside this ad hoc environment. - -Some projects have more sophisticated needs. For examples, some ad hoc -environments might need to expose Nixpkgs packages compiled in a certain way, or -with extra environment variables. In these cases, you'll need a `shell` field -instead of `packages`: -```yaml -nix: - enable: true - shell-file: shell.nix -``` - -For more on how to write a `shell.nix` file see the below section. You'll need -to express a derivation. Note that Nixpkgs ships with a convenience wrapper -function around `mkDerivation` called `haskell.lib.buildStackProject` to help you -create this derivation in exactly the way Stack expects. However for this to work -you need to disable the sandbox, which you can do by using `--option sandbox relaxed` -or `--option sandbox false` to the Nix command. All of the same inputs -as `mkDerivation` can be provided. For example, to build a Stack project that -including packages that link against a version of the R library compiled with -special options turned on: -```nix -with (import <nixpkgs> { }); - -let R = pkgs.R.override { enableStrictBarrier = true; }; -in -haskell.lib.buildStackProject { - name = "HaskellR"; - buildInputs = [ R zeromq zlib ]; -} -``` - -You can select a particular GHC version to compile with by setting the -`ghc` attribute as an argument to `buildStackProject`. Better yet, let -Stack choose what GHC version it wants based on the snapshot specified -in `stack.yaml` (only works with Stack >= 1.1.3): -```nix -{nixpkgs ? import <nixpkgs> { }, ghc ? nixpkgs.ghc}: - -with nixpkgs; - -let R = pkgs.R.override { enableStrictBarrier = true; }; -in -haskell.lib.buildStackProject { - name = "HaskellR"; - buildInputs = [ R zeromq zlib ]; - inherit ghc; -} -``` - -[stack-nix-doc]: http://docs.haskellstack.org/en/stable/nix_integration.html - -### How to create ad hoc environments for `nix-shell` - -The easiest way to create an ad hoc development environment is to run -`nix-shell` with the appropriate GHC environment given on the command-line: -```shell -nix-shell -p "haskellPackages.ghcWithPackages (pkgs: with pkgs; [mtl pandoc])" -``` - -For more sophisticated use-cases, however, it's more convenient to save the -desired configuration in a file called `shell.nix` that looks like this: -```nix -{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc7102" }: -let - inherit (nixpkgs) pkgs; - ghc = pkgs.haskell.packages.${compiler}.ghcWithPackages (ps: with ps; [ - monad-par mtl - ]); -in -pkgs.stdenv.mkDerivation { - name = "my-haskell-env-0"; - buildInputs = [ ghc ]; - shellHook = "eval $(egrep ^export ${ghc}/bin/ghc)"; -} -``` - -Now run `nix-shell` --- or even `nix-shell --pure` --- to enter a shell -environment that has the appropriate compiler in `$PATH`. If you use `--pure`, -then add all other packages that your development environment needs into the -`buildInputs` attribute. If you'd like to switch to a different compiler -version, then pass an appropriate `compiler` argument to the expression, i.e. -`nix-shell --argstr compiler ghc784`. - -If you need such an environment because you'd like to compile a Hackage package -outside of Nix --- i.e. because you're hacking on the latest version from Git ----, then the package set provides suitable nix-shell environments for you -already! Every Haskell package has an `env` attribute that provides a shell -environment suitable for compiling that particular package. If you'd like to -hack the `lens` library, for example, then you just have to check out the -source code and enter the appropriate environment: -``` -$ cabal get lens-4.11 && cd lens-4.11 -Downloading lens-4.11... -Unpacking to lens-4.11/ - -$ nix-shell "<nixpkgs>" -A haskellPackages.lens.env -[nix-shell:/tmp/lens-4.11]$ -``` - -At point, you can run `cabal configure`, `cabal build`, and all the other -development commands. Note that you need `cabal-install` installed in your -`$PATH` already to use it here --- the `nix-shell` environment does not provide -it. - -## How to create Nix builds for your own private Haskell packages - -If your own Haskell packages have build instructions for Cabal, then you can -convert those automatically into build instructions for Nix using the -`cabal2nix` utility, which you can install into your profile by running -`nix-env -i cabal2nix`. - -### How to build a stand-alone project - -For example, let's assume that you're working on a private project called -`foo`. To generate a Nix build expression for it, change into the project's -top-level directory and run the command: -```shell -cabal2nix . > foo.nix -``` -Then write the following snippet into a file called `default.nix`: -```nix -{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc7102" }: -nixpkgs.pkgs.haskell.packages.${compiler}.callPackage ./foo.nix { } -``` - -Finally, store the following code in a file called `shell.nix`: -```nix -{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc7102" }: -(import ./default.nix { inherit nixpkgs compiler; }).env -``` - -At this point, you can run `nix-build` to have Nix compile your project and -install it into a Nix store path. The local directory will contain a symlink -called `result` after `nix-build` returns that points into that location. Of -course, passing the flag `--argstr compiler ghc763` allows switching the build -to any version of GHC currently supported. - -Furthermore, you can call `nix-shell` to enter an interactive development -environment in which you can use `cabal configure` and `cabal build` to develop -your code. That environment will automatically contain a proper GHC derivation -with all the required libraries registered as well as all the system-level -libraries your package might need. - -If your package does not depend on any system-level libraries, then it's -sufficient to run -```shell -nix-shell --command "cabal configure" -``` -once to set up your build. `cabal-install` determines the absolute paths to all -resources required for the build and writes them into a config file in the -`dist/` directory. Once that's done, you can run `cabal build` and any other -command for that project even outside of the `nix-shell` environment. This -feature is particularly nice for those of us who like to edit their code with -an IDE, like Emacs' `haskell-mode`, because it's not necessary to start Emacs -inside of nix-shell just to make it find out the necessary settings for -building the project; `cabal-install` has already done that for us. - -If you want to do some quick-and-dirty hacking and don't want to bother setting -up a `default.nix` and `shell.nix` file manually, then you can use the -`--shell` flag offered by `cabal2nix` to have it generate a stand-alone -`nix-shell` environment for you. With that feature, running -```shell -cabal2nix --shell . > shell.nix -nix-shell --command "cabal configure" -``` -is usually enough to set up a build environment for any given Haskell package. -You can even use that generated file to run `nix-build`, too: -```shell -nix-build shell.nix -``` - -### How to build projects that depend on each other - -If you have multiple private Haskell packages that depend on each other, then -you'll have to register those packages in the Nixpkgs set to make them visible -for the dependency resolution performed by `callPackage`. First of all, change -into each of your projects top-level directories and generate a `default.nix` -file with `cabal2nix`: -```shell -cd ~/src/foo && cabal2nix . > default.nix -cd ~/src/bar && cabal2nix . > default.nix -``` -Then edit your `~/.config/nixpkgs/config.nix` file to register those builds in the -default Haskell package set: -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - haskellPackages = super.haskellPackages.override { - overrides = self: super: { - foo = self.callPackage ../src/foo {}; - bar = self.callPackage ../src/bar {}; - }; - }; - }; -} -``` -Once that's accomplished, `nix-env -f "<nixpkgs>" -qA haskellPackages` will -show your packages like any other package from Hackage, and you can build them -```shell -nix-build "<nixpkgs>" -A haskellPackages.foo -``` -or enter an interactive shell environment suitable for building them: -```shell -nix-shell "<nixpkgs>" -A haskellPackages.bar.env -``` -## Miscellaneous Topics - -### How to build with profiling enabled - -Every Haskell package set takes a function called `overrides` that you can use -to manipulate the package as much as you please. One useful application of this -feature is to replace the default `mkDerivation` function with one that enables -library profiling for all packages. To accomplish that add the following -snippet to your `~/.config/nixpkgs/config.nix` file: -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - profiledHaskellPackages = self.haskellPackages.override { - overrides = self: super: { - mkDerivation = args: super.mkDerivation (args // { - enableLibraryProfiling = true; - }); - }; - }; - }; -} -``` -Then, replace instances of `haskellPackages` in the `cabal2nix`-generated -`default.nix` or `shell.nix` files with `profiledHaskellPackages`. - -### How to override package versions in a compiler-specific package set - -Nixpkgs provides the latest version of -[`ghc-events`](http://hackage.haskell.org/package/ghc-events), which is 0.4.4.0 -at the time of this writing. This is fine for users of GHC 7.10.x, but GHC -7.8.4 cannot compile that binary. Now, one way to solve that problem is to -register an older version of `ghc-events` in the 7.8.x-specific package set. -The first step is to generate Nix build instructions with `cabal2nix`: -```shell -cabal2nix cabal://ghc-events-0.4.3.0 > ~/.nixpkgs/ghc-events-0.4.3.0.nix -``` -Then add the override in `~/.config/nixpkgs/config.nix`: -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - haskell = super.haskell // { - packages = super.haskell.packages // { - ghc784 = super.haskell.packages.ghc784.override { - overrides = self: super: { - ghc-events = self.callPackage ./ghc-events-0.4.3.0.nix {}; - }; - }; - }; - }; - }; -} -``` - -This code is a little crazy, no doubt, but it's necessary because the intuitive -version -```nix -{ # ... - - haskell.packages.ghc784 = super.haskell.packages.ghc784.override { - overrides = self: super: { - ghc-events = self.callPackage ./ghc-events-0.4.3.0.nix {}; - }; - }; -} -``` -doesn't do what we want it to: that code replaces the `haskell` package set in -Nixpkgs with one that contains only one entry,`packages`, which contains only -one entry `ghc784`. This override loses the `haskell.compiler` set, and it -loses the `haskell.packages.ghcXYZ` sets for all compilers but GHC 7.8.4. To -avoid that problem, we have to perform the convoluted little dance from above, -iterating over each step in hierarchy. - -Once it's accomplished, however, we can install a variant of `ghc-events` -that's compiled with GHC 7.8.4: -```shell -nix-env -f "<nixpkgs>" -iA haskell.packages.ghc784.ghc-events -``` -Unfortunately, it turns out that this build fails again while executing the -test suite! Apparently, the release archive on Hackage is missing some data -files that the test suite requires, so we cannot run it. We accomplish that by -re-generating the Nix expression with the `--no-check` flag: -```shell -cabal2nix --no-check cabal://ghc-events-0.4.3.0 > ~/.nixpkgs/ghc-events-0.4.3.0.nix -``` -Now the builds succeeds. - -Of course, in the concrete example of `ghc-events` this whole exercise is not -an ideal solution, because `ghc-events` can analyze the output emitted by any -version of GHC later than 6.12 regardless of the compiler version that was used -to build the `ghc-events` executable, so strictly speaking there's no reason to -prefer one built with GHC 7.8.x in the first place. However, for users who -cannot use GHC 7.10.x at all for some reason, the approach of downgrading to an -older version might be useful. - -### How to override packages in all compiler-specific package sets - -In the previous section we learned how to override a package in a single -compiler-specific package set. You may have some overrides defined that you want -to use across multiple package sets. To accomplish this you could use the -technique that we learned in the previous section by repeating the overrides for -all the compiler-specific package sets. For example: - -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - haskell = super.haskell // { - packages = super.haskell.packages // { - ghc784 = super.haskell.packages.ghc784.override { - overrides = self: super: { - my-package = ...; - my-other-package = ...; - }; - }; - ghc822 = super.haskell.packages.ghc784.override { - overrides = self: super: { - my-package = ...; - my-other-package = ...; - }; - }; - ... - }; - }; - }; -} -``` - -However there's a more convenient way to override all compiler-specific package -sets at once: - -```nix -{ - packageOverrides = super: let self = super.pkgs; in - { - haskell = super.haskell // { - packageOverrides = self: super: { - my-package = ...; - my-other-package = ...; - }; - }; - }; -} -``` - -### How to specify source overrides for your Haskell package - -When starting a Haskell project you can use `developPackage` -to define a derivation for your package at the `root` path -as well as source override versions for Hackage packages, like so: - -```nix -# default.nix -{ compilerVersion ? "ghc842" }: -let - # pinning nixpkgs using new Nix 2.0 builtin `fetchGit` - pkgs = import (fetchGit (import ./version.nix)) { }; - compiler = pkgs.haskell.packages."${compilerVersion}"; - pkg = compiler.developPackage { - root = ./.; - source-overrides = { - # Let's say the GHC 8.4.2 haskellPackages uses 1.6.0.0 and your test suite is incompatible with >= 1.6.0.0 - HUnit = "1.5.0.0"; - }; - }; -in pkg -``` - -This could be used in place of a simplified `stack.yaml` defining a Nix -derivation for your Haskell package. - -As you can see this allows you to specify only the source version found on -Hackage and nixpkgs will take care of the rest. - -You can also specify `buildInputs` for your Haskell derivation for packages -that directly depend on external libraries like so: - -```nix -# default.nix -{ compilerVersion ? "ghc842" }: -let - # pinning nixpkgs using new Nix 2.0 builtin `fetchGit` - pkgs = import (fetchGit (import ./version.nix)) { }; - compiler = pkgs.haskell.packages."${compilerVersion}"; - pkg = compiler.developPackage { - root = ./.; - source-overrides = { - HUnit = "1.5.0.0"; # Let's say the GHC 8.4.2 haskellPackages uses 1.6.0.0 and your test suite is incompatible with >= 1.6.0.0 - }; - }; - # in case your package source depends on any libraries directly, not just transitively. - buildInputs = [ zlib ]; -in pkg.overrideAttrs(attrs: { - buildInputs = attrs.buildInputs ++ buildInputs; -}) -``` - -Notice that you will need to override (via `overrideAttrs` or similar) the -derivation returned by the `developPackage` Nix lambda as there is no `buildInputs` -named argument you can pass directly into the `developPackage` lambda. - -### How to recover from GHC's infamous non-deterministic library ID bug - -GHC and distributed build farms don't get along well: - - - https://ghc.haskell.org/trac/ghc/ticket/4012 - -When you see an error like this one -``` -package foo-0.7.1.0 is broken due to missing package -text-1.2.0.4-98506efb1b9ada233bb5c2b2db516d91 -``` -then you have to download and re-install `foo` and all its dependents from -scratch: -```shell -nix-store -q --referrers /nix/store/*-haskell-text-1.2.0.4 \ - | xargs -L 1 nix-store --repair-path -``` - -If you're using additional Hydra servers other than `hydra.nixos.org`, then it -might be necessary to purge the local caches that store data from those -machines to disable these binary channels for the duration of the previous -command, i.e. by running: -```shell -rm ~/.cache/nix/binary-cache*.sqlite -``` - -### Builds on Darwin fail with `math.h` not found - -Users of GHC on Darwin have occasionally reported that builds fail, because the -compiler complains about a missing include file: -``` -fatal error: 'math.h' file not found -``` -The issue has been discussed at length in [ticket -6390](https://github.com/NixOS/nixpkgs/issues/6390), and so far no good -solution has been proposed. As a work-around, users who run into this problem -can configure the environment variables -```shell -export NIX_CFLAGS_COMPILE="-idirafter /usr/include" -export NIX_CFLAGS_LINK="-L/usr/lib" -``` -in their `~/.bashrc` file to avoid the compiler error. - -### Builds using Stack complain about missing system libraries - -``` --- While building package zlib-0.5.4.2 using: - runhaskell -package=Cabal-1.22.4.0 -clear-package-db [... lots of flags ...] -Process exited with code: ExitFailure 1 -Logs have been written to: /home/foo/src/stack-ide/.stack-work/logs/zlib-0.5.4.2.log - -Configuring zlib-0.5.4.2... -Setup.hs: Missing dependency on a foreign library: -* Missing (or bad) header file: zlib.h -This problem can usually be solved by installing the system package that -provides this library (you may need the "-dev" version). If the library is -already installed but in a non-standard location then you can use the flags ---extra-include-dirs= and --extra-lib-dirs= to specify where it is. -If the header file does exist, it may contain errors that are caught by the C -compiler at the preprocessing stage. In this case you can re-run configure -with the verbosity flag -v3 to see the error messages. -``` - -When you run the build inside of the nix-shell environment, the system -is configured to find `libz.so` without any special flags -- the compiler -and linker "just know" how to find it. Consequently, Cabal won't record -any search paths for `libz.so` in the package description, which means -that the package works fine inside of nix-shell, but once you leave the -shell the shared object can no longer be found. That issue is by no -means specific to Stack: you'll have that problem with any other -Haskell package that's built inside of nix-shell but run outside of that -environment. - -You can remedy this issue in several ways. The easiest is to add a `nix` section -to the `stack.yaml` like the following: -```yaml -nix: - enable: true - packages: [ zlib ] -``` - -Stack's Nix support knows to add `${zlib.out}/lib` and `${zlib.dev}/include` -as an `--extra-lib-dirs` and `extra-include-dirs`, respectively. -Alternatively, you can achieve the same effect by hand. First of all, run -``` -$ nix-build --no-out-link "<nixpkgs>" -A zlib -/nix/store/alsvwzkiw4b7ip38l4nlfjijdvg3fvzn-zlib-1.2.8 -``` -to find out the store path of the system's zlib library. Now, you can - - 1. add that path (plus a "/lib" suffix) to your `$LD_LIBRARY_PATH` - environment variable to make sure your system linker finds `libz.so` - automatically. It's no pretty solution, but it will work. - - 2. As a variant of (1), you can also install any number of system - libraries into your user's profile (or some other profile) and point - `$LD_LIBRARY_PATH` to that profile instead, so that you don't have to - list dozens of those store paths all over the place. - - 3. The solution I prefer is to call stack with an appropriate - --extra-lib-dirs flag like so: - ```shell - stack --extra-lib-dirs=/nix/store/alsvwzkiw4b7ip38l4nlfjijdvg3fvzn-zlib-1.2.8/lib build - ``` - -Typically, you'll need `--extra-include-dirs` as well. It's possible -to add those flag to the project's `stack.yaml` or your user's -global `~/.stack/global/stack.yaml` file so that you don't have to -specify them manually every time. But again, you're likely better off -using Stack's Nix support instead. - -The same thing applies to `cabal configure`, of course, if you're -building with `cabal-install` instead of Stack. - -### Creating statically linked binaries - -There are two levels of static linking. The first option is to configure the -build with the Cabal flag `--disable-executable-dynamic`. In Nix expressions, -this can be achieved by setting the attribute: -``` -enableSharedExecutables = false; -``` -That gives you a binary with statically linked Haskell libraries and -dynamically linked system libraries. - -To link both Haskell libraries and system libraries statically, the additional -flags `--ghc-option=-optl=-static --ghc-option=-optl=-pthread` need to be used. -In Nix, this is accomplished with: -``` -configureFlags = [ "--ghc-option=-optl=-static" "--ghc-option=-optl=-pthread" ]; -``` - -It's important to realize, however, that most system libraries in Nix are -built as shared libraries only, i.e. there is just no static library -available that Cabal could link! - -### Building GHC with integer-simple - -By default GHC implements the Integer type using the -[GNU Multiple Precision Arithmetic (GMP) library](https://gmplib.org/). -The implementation can be found in the -[integer-gmp](http://hackage.haskell.org/package/integer-gmp) package. - -A potential problem with this is that GMP is licensed under the -[GNU Lesser General Public License (LGPL)](https://www.gnu.org/copyleft/lesser.html), -a kind of "copyleft" license. According to the terms of the LGPL, paragraph 5, -you may distribute a program that is designed to be compiled and dynamically -linked with the library under the terms of your choice (i.e., commercially) but -if your program incorporates portions of the library, if it is linked -statically, then your program is a "derivative"--a "work based on the -library"--and according to paragraph 2, section c, you "must cause the whole of -the work to be licensed" under the terms of the LGPL (including for free). - -The LGPL licensing for GMP is a problem for the overall licensing of binary -programs compiled with GHC because most distributions (and builds) of GHC use -static libraries. (Dynamic libraries are currently distributed only for macOS.) -The LGPL licensing situation may be worse: even though -[The Glasgow Haskell Compiler License](https://www.haskell.org/ghc/license) -is essentially a "free software" license (BSD3), according to -paragraph 2 of the LGPL, GHC must be distributed under the terms of the LGPL! - -To work around these problems GHC can be build with a slower but LGPL-free -alternative implementation for Integer called -[integer-simple](http://hackage.haskell.org/package/integer-simple). - -To get a GHC compiler build with `integer-simple` instead of `integer-gmp` use -the attribute: `haskell.compiler.integer-simple."${ghcVersion}"`. -For example: -``` -$ nix-build -E '(import <nixpkgs> {}).haskell.compiler.integer-simple.ghc802' -... -$ result/bin/ghc-pkg list | grep integer - integer-simple-0.1.1.1 -``` -The following command displays the complete list of GHC compilers build with `integer-simple`: -``` -$ nix-env -f "<nixpkgs>" -qaP -A haskell.compiler.integer-simple -haskell.compiler.integer-simple.ghc7102 ghc-7.10.2 -haskell.compiler.integer-simple.ghc7103 ghc-7.10.3 -haskell.compiler.integer-simple.ghc722 ghc-7.2.2 -haskell.compiler.integer-simple.ghc742 ghc-7.4.2 -haskell.compiler.integer-simple.ghc783 ghc-7.8.3 -haskell.compiler.integer-simple.ghc784 ghc-7.8.4 -haskell.compiler.integer-simple.ghc801 ghc-8.0.1 -haskell.compiler.integer-simple.ghc802 ghc-8.0.2 -haskell.compiler.integer-simple.ghcHEAD ghc-8.1.20170106 -``` - -To get a package set supporting `integer-simple` use the attribute: -`haskell.packages.integer-simple."${ghcVersion}"`. For example -use the following to get the `scientific` package build with `integer-simple`: -```shell -nix-build -A haskell.packages.integer-simple.ghc802.scientific -``` - -### Quality assurance - -The `haskell.lib` library includes a number of functions for checking for -various imperfections in Haskell packages. It's useful to apply these functions -to your own Haskell packages and integrate that in a Continuous Integration -server like [hydra](https://nixos.org/hydra/) to assure your packages maintain a -minimum level of quality. This section discusses some of these functions. - -#### failOnAllWarnings - -Applying `haskell.lib.failOnAllWarnings` to a Haskell package enables the -`-Wall` and `-Werror` GHC options to turn all warnings into build failures. - -#### buildStrictly - -Applying `haskell.lib.buildStrictly` to a Haskell package calls -`failOnAllWarnings` on the given package to turn all warnings into build -failures. Additionally the source of your package is gotten from first invoking -`cabal sdist` to ensure all needed files are listed in the Cabal file. - -#### checkUnusedPackages - -Applying `haskell.lib.checkUnusedPackages` to a Haskell package invokes -the [packunused](http://hackage.haskell.org/package/packunused) tool on the -package. `packunused` complains when it finds packages listed as build-depends -in the Cabal file which are redundant. For example: - -``` -$ nix-build -E 'let pkgs = import <nixpkgs> {}; in pkgs.haskell.lib.checkUnusedPackages {} pkgs.haskellPackages.scientific' -these derivations will be built: - /nix/store/3lc51cxj2j57y3zfpq5i69qbzjpvyci1-scientific-0.3.5.1.drv -... -detected package components -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - - library - - testsuite(s): test-scientific - - benchmark(s): bench-scientific* - -(component names suffixed with '*' are not configured to be built) - -library -~~~~~~~ - -The following package dependencies seem redundant: - - - ghc-prim-0.5.0.0 - -testsuite(test-scientific) -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -no redundant packages dependencies found - -builder for ‘/nix/store/3lc51cxj2j57y3zfpq5i69qbzjpvyci1-scientific-0.3.5.1.drv’ failed with exit code 1 -error: build of ‘/nix/store/3lc51cxj2j57y3zfpq5i69qbzjpvyci1-scientific-0.3.5.1.drv’ failed -``` - -As you can see, `packunused` finds out that although the testsuite component has -no redundant dependencies the library component of `scientific-0.3.5.1` depends -on `ghc-prim` which is unused in the library. - -### Using hackage2nix with nixpkgs - -Hackage package derivations are found in the -[`hackage-packages.nix`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/hackage-packages.nix) -file within `nixpkgs` and are used as the initial package set for -`haskellPackages`. The `hackage-packages.nix` file is not meant to be edited -by hand, but rather autogenerated by [`hackage2nix`](https://github.com/NixOS/cabal2nix/tree/master/hackage2nix), -which by default uses the [`configuration-hackage2nix.yaml`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-hackage2nix.yaml) -file to generate all the derivations. - -To modify the contents `configuration-hackage2nix.yaml`, follow the -instructions on [`hackage2nix`](https://github.com/NixOS/cabal2nix/tree/master/hackage2nix). - -## Other resources - - - The Youtube video [Nix Loves Haskell](https://www.youtube.com/watch?v=BsBhi_r-OeE) - provides an introduction into Haskell NG aimed at beginners. The slides are - available at http://cryp.to/nixos-meetup-3-slides.pdf and also -- in a form - ready for cut & paste -- at - https://github.com/NixOS/cabal2nix/blob/master/doc/nixos-meetup-3-slides.md. - - - Another Youtube video is [Escaping Cabal Hell with Nix](https://www.youtube.com/watch?v=mQd3s57n_2Y), - which discusses the subject of Haskell development with Nix but also provides - a basic introduction to Nix as well, i.e. it's suitable for viewers with - almost no prior Nix experience. - - - Oliver Charles wrote a very nice [Tutorial how to develop Haskell packages with Nix](http://wiki.ocharles.org.uk/Nix). - - - The *Journey into the Haskell NG infrastructure* series of postings - describe the new Haskell infrastructure in great detail: - - - [Part 1](https://nixos.org/nix-dev/2015-January/015591.html) - explains the differences between the old and the new code and gives - instructions how to migrate to the new setup. - - - [Part 2](https://nixos.org/nix-dev/2015-January/015608.html) - looks in-depth at how to tweak and configure your setup by means of - overrides. +# Haskell - - [Part 3](https://nixos.org/nix-dev/2015-April/016912.html) - describes the infrastructure that keeps the Haskell package set in Nixpkgs - up-to-date. +The documentation for the Haskell infrastructure is published at +<https://haskell4nix.readthedocs.io/>. The source code for that +site lives in the `doc/` sub-directory of the +[`cabal2nix` Git repository](https://github.com/NixOS/cabal2nix) +and changes can be submitted there. diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/index.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/index.xml index 728a38c264a3..daa57cf1f865 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/index.xml +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/index.xml @@ -7,28 +7,29 @@ </para> <xi:include href="agda.section.xml" /> <xi:include href="android.section.xml" /> - <xi:include href="beam.xml" /> + <xi:include href="beam.section.xml" /> <xi:include href="bower.xml" /> - <xi:include href="coq.xml" /> + <xi:include href="coq.section.xml" /> <xi:include href="crystal.section.xml" /> <xi:include href="emscripten.section.xml" /> <xi:include href="gnome.xml" /> - <xi:include href="go.xml" /> + <xi:include href="go.section.xml" /> <xi:include href="haskell.section.xml" /> <xi:include href="idris.section.xml" /> <xi:include href="ios.section.xml" /> - <xi:include href="java.xml" /> + <xi:include href="java.section.xml" /> <xi:include href="lua.section.xml" /> + <xi:include href="maven.section.xml" /> <xi:include href="node.section.xml" /> - <xi:include href="ocaml.xml" /> + <xi:include href="ocaml.section.xml" /> <xi:include href="perl.xml" /> <xi:include href="php.section.xml" /> <xi:include href="python.section.xml" /> - <xi:include href="qt.xml" /> + <xi:include href="qt.section.xml" /> <xi:include href="r.section.xml" /> - <xi:include href="ruby.xml" /> + <xi:include href="ruby.section.xml" /> <xi:include href="rust.section.xml" /> - <xi:include href="texlive.xml" /> + <xi:include href="texlive.section.xml" /> <xi:include href="titanium.section.xml" /> <xi:include href="vim.section.xml" /> </chapter> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/java.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/java.section.md new file mode 100644 index 000000000000..77919d43f748 --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/java.section.md @@ -0,0 +1,91 @@ +# Java {#sec-language-java} + +Ant-based Java packages are typically built from source as follows: + +```nix +stdenv.mkDerivation { + name = "..."; + src = fetchurl { ... }; + + nativeBuildInputs = [ jdk ant ]; + + buildPhase = "ant"; +} +``` + +Note that `jdk` is an alias for the OpenJDK (self-built where available, +or pre-built via Zulu). Platforms with OpenJDK not (yet) in Nixpkgs +(`Aarch32`, `Aarch64`) point to the (unfree) `oraclejdk`. + +JAR files that are intended to be used by other packages should be +installed in `$out/share/java`. JDKs have a stdenv setup hook that add +any JARs in the `share/java` directories of the build inputs to the +`CLASSPATH` environment variable. For instance, if the package `libfoo` +installs a JAR named `foo.jar` in its `share/java` directory, and +another package declares the attribute + +```nix +buildInputs = [ libfoo ]; +nativeBuildInputs = [ jdk ]; +``` + +then `CLASSPATH` will be set to +`/nix/store/...-libfoo/share/java/foo.jar`. + +Private JARs should be installed in a location like +`$out/share/package-name`. + +If your Java package provides a program, you need to generate a wrapper +script to run it using a JRE. You can use `makeWrapper` for this: + +```nix +nativeBuildInputs = [ makeWrapper ]; + +installPhase = '' + mkdir -p $out/bin + makeWrapper ${jre}/bin/java $out/bin/foo \ + --add-flags "-cp $out/share/java/foo.jar org.foo.Main" +''; +``` + +Since the introduction of the Java Platform Module System in Java 9, +Java distributions typically no longer ship with a general-purpose JRE: +instead, they allow generating a JRE with only the modules required for +your application(s). Because we can't predict what modules will be +needed on a general-purpose system, the default jre package is the full +JDK. When building a minimal system/image, you can override the +`modules` parameter on `jre_minimal` to build a JRE with only the +modules relevant for you: + +```nix +let + my_jre = pkgs.jre_minimal.override { + modules = [ + # The modules used by 'something' and 'other' combined: + "java.base" + "java.logging" + ]; + }; + something = (pkgs.something.override { jre = my_jre; }); + other = (pkgs.other.override { jre = my_jre; }); +in + ... +``` + +Note all JDKs passthru `home`, so if your application requires +environment variables like `JAVA_HOME` being set, that can be done in a +generic fashion with the `--set` argument of `makeWrapper`: + +```bash +--set JAVA_HOME ${jdk.home} +``` + +It is possible to use a different Java compiler than `javac` from the +OpenJDK. For instance, to use the GNU Java Compiler: + +```nix +nativeBuildInputs = [ gcj ant ]; +``` + +Here, Ant will automatically use `gij` (the GNU Java Runtime) instead of +the OpenJRE. diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/java.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/java.xml deleted file mode 100644 index bf0fc4883922..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/java.xml +++ /dev/null @@ -1,63 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-java"> - <title>Java</title> - - <para> - Ant-based Java packages are typically built from source as follows: -<programlisting> -stdenv.mkDerivation { - name = "..."; - src = fetchurl { ... }; - - nativeBuildInputs = [ jdk ant ]; - - buildPhase = "ant"; -} -</programlisting> - Note that <varname>jdk</varname> is an alias for the OpenJDK (self-built where available, or pre-built via Zulu). Platforms with OpenJDK not (yet) in Nixpkgs (<literal>Aarch32</literal>, <literal>Aarch64</literal>) point to the (unfree) <literal>oraclejdk</literal>. - </para> - - <para> - JAR files that are intended to be used by other packages should be installed in <filename>$out/share/java</filename>. JDKs have a stdenv setup hook that add any JARs in the <filename>share/java</filename> directories of the build inputs to the <envar>CLASSPATH</envar> environment variable. For instance, if the package <literal>libfoo</literal> installs a JAR named <filename>foo.jar</filename> in its <filename>share/java</filename> directory, and another package declares the attribute -<programlisting> -buildInputs = [ libfoo ]; -nativeBuildInputs = [ jdk ]; -</programlisting> - then <envar>CLASSPATH</envar> will be set to <filename>/nix/store/...-libfoo/share/java/foo.jar</filename>. - </para> - - <para> - Private JARs should be installed in a location like <filename>$out/share/<replaceable>package-name</replaceable></filename>. - </para> - - <para> - If your Java package provides a program, you need to generate a wrapper script to run it using the OpenJRE. You can use <literal>makeWrapper</literal> for this: -<programlisting> -nativeBuildInputs = [ makeWrapper ]; - -installPhase = - '' - mkdir -p $out/bin - makeWrapper ${jre}/bin/java $out/bin/foo \ - --add-flags "-cp $out/share/java/foo.jar org.foo.Main" - ''; -</programlisting> - Note the use of <literal>jre</literal>, which is the part of the OpenJDK package that contains the Java Runtime Environment. By using <literal>${jre}/bin/java</literal> instead of <literal>${jdk}/bin/java</literal>, you prevent your package from depending on the JDK at runtime. - </para> - - <para> - Note all JDKs passthru <literal>home</literal>, so if your application requires environment variables like <envar>JAVA_HOME</envar> being set, that can be done in a generic fashion with the <literal>--set</literal> argument of <literal>makeWrapper</literal>: -<programlisting> ---set JAVA_HOME ${jdk.home} -</programlisting> - </para> - - <para> - It is possible to use a different Java compiler than <command>javac</command> from the OpenJDK. For instance, to use the GNU Java Compiler: -<programlisting> -nativeBuildInputs = [ gcj ant ]; -</programlisting> - Here, Ant will automatically use <command>gij</command> (the GNU Java Runtime) instead of the OpenJRE. - </para> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/maven.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/maven.section.md new file mode 100644 index 000000000000..fe183e7ba3cc --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/maven.section.md @@ -0,0 +1,354 @@ +--- +title: Maven +author: Farid Zakaria +date: 2020-10-15 +--- + +# Maven + +Maven is a well-known build tool for the Java ecosystem however it has some challenges when integrating into the Nix build system. + +The following provides a list of common patterns with how to package a Maven project (or any JVM language that can export to Maven) as a Nix package. + +For the purposes of this example let's consider a very basic Maven project with the following `pom.xml` with a single dependency on [emoji-java](https://github.com/vdurmont/emoji-java). + +```xml +<?xml version="1.0" encoding="UTF-8"?> +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> + <modelVersion>4.0.0</modelVersion> + <groupId>io.github.fzakaria</groupId> + <artifactId>maven-demo</artifactId> + <version>1.0</version> + <packaging>jar</packaging> + <name>NixOS Maven Demo</name> + + <dependencies> + <dependency> + <groupId>com.vdurmont</groupId> + <artifactId>emoji-java</artifactId> + <version>5.1.1</version> + </dependency> + </dependencies> +</project> +``` + +Our main class file will be very simple: + +```java +import com.vdurmont.emoji.EmojiParser; + +public class Main { + public static void main(String[] args) { + String str = "NixOS :grinning: is super cool :smiley:!"; + String result = EmojiParser.parseToUnicode(str); + System.out.println(result); + } +} +``` + +You find this demo project at https://github.com/fzakaria/nixos-maven-example + +## Solving for dependencies + +### buildMaven with NixOS/mvn2nix-maven-plugin + +> ⚠️ Although `buildMaven` is the "blessed" way within nixpkgs, as of 2020, it hasn't seen much activity in quite a while. + +`buildMaven` is an alternative method that tries to follow similar patterns of other programming languages by generating a lock file. It relies on the maven plugin [mvn2nix-maven-plugin](https://github.com/NixOS/mvn2nix-maven-plugin). + +First you generate a `project-info.json` file using the maven plugin. + +> This should be executed in the project's source repository or be told which `pom.xml` to execute with. + +```bash +# run this step within the project's source repository +❯ mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix + +❯ cat project-info.json | jq | head +{ + "project": { + "artifactId": "maven-demo", + "groupId": "org.nixos", + "version": "1.0", + "classifier": "", + "extension": "jar", + "dependencies": [ + { + "artifactId": "maven-resources-plugin", +``` + +This file is then given to the `buildMaven` function, and it returns 2 attributes. + +**`repo`**: + A Maven repository that is a symlink farm of all the dependencies found in the `project-info.json` + + +**`build`**: + A simple derivation that runs through `mvn compile` & `mvn package` to build the JAR. You may use this as inspiration for more complicated derivations. + +Here is an [example](https://github.com/fzakaria/nixos-maven-example/blob/main/build-maven-repository.nix) of building the Maven repository +```nix +{ pkgs ? import <nixpkgs> { } }: +with pkgs; +(buildMaven ./project-info.json).repo +``` + +The benefit over the _double invocation_ as we will see below, is that the _/nix/store_ entry is a _linkFarm_ of every package, so that changes to your dependency set doesn't involve downloading everything from scratch. + +```bash +❯ tree $(nix-build --no-out-link build-maven-repository.nix) | head +/nix/store/g87va52nkc8jzbmi1aqdcf2f109r4dvn-maven-repository +├── antlr +│ └── antlr +│ └── 2.7.2 +│ ├── antlr-2.7.2.jar -> /nix/store/d027c8f2cnmj5yrynpbq2s6wmc9cb559-antlr-2.7.2.jar +│ └── antlr-2.7.2.pom -> /nix/store/mv42fc5gizl8h5g5vpywz1nfiynmzgp2-antlr-2.7.2.pom +├── avalon-framework +│ └── avalon-framework +│ └── 4.1.3 +│ ├── avalon-framework-4.1.3.jar -> /nix/store/iv5fp3955w3nq28ff9xfz86wvxbiw6n9-avalon-framework-4.1.3.jar +``` +### Double Invocation + +> ⚠️ This pattern is the simplest but may cause unnecessary rebuilds due to the output hash changing. + +The double invocation is a _simple_ way to get around the problem that `nix-build` may be sandboxed and have no Internet connectivity. + +It treats the entire Maven repository as a single source to be downloaded, relying on Maven's dependency resolution to satisfy the output hash. This is similar to fetchers like `fetchgit`, except it has to run a Maven build to determine what to download. + +The first step will be to build the Maven project as a fixed-output derivation in order to collect the Maven repository -- below is an [example](https://github.com/fzakaria/nixos-maven-example/blob/main/double-invocation-repository.nix). + +> Traditionally the Maven repository is at `~/.m2/repository`. We will override this to be the `$out` directory. + +```nix +{ stdenv, maven }: +stdenv.mkDerivation { + name = "maven-repository"; + buildInputs = [ maven ]; + src = ./.; # or fetchFromGitHub, cleanSourceWith, etc + buildPhase = '' + mvn package -Dmaven.repo.local=$out + ''; + + # keep only *.{pom,jar,sha1,nbm} and delete all ephemeral files with lastModified timestamps inside + installPhase = '' + find $out -type f \ + -name \*.lastUpdated -or \ + -name resolver-status.properties -or \ + -name _remote.repositories \ + -delete + ''; + + # don't do any fixup + dontFixup = true; + outputHashAlgo = "sha256"; + outputHashMode = "recursive"; + # replace this with the correct SHA256 + outputHash = stdenv.lib.fakeSha256; +} +``` + +The build will fail, and tell you the expected `outputHash` to place. When you've set the hash, the build will return with a `/nix/store` entry whose contents are the full Maven repository. + +> Some additional files are deleted that would cause the output hash to change potentially on subsequent runs. + +```bash +❯ tree $(nix-build --no-out-link double-invocation-repository.nix) | head +/nix/store/8kicxzp98j68xyi9gl6jda67hp3c54fq-maven-repository +├── backport-util-concurrent +│ └── backport-util-concurrent +│ └── 3.1 +│ ├── backport-util-concurrent-3.1.pom +│ └── backport-util-concurrent-3.1.pom.sha1 +├── classworlds +│ └── classworlds +│ ├── 1.1 +│ │ ├── classworlds-1.1.jar +``` + +If your package uses _SNAPSHOT_ dependencies or _version ranges_; there is a strong likelihood that over-time your output hash will change since the resolved dependencies may change. Hence this method is less recommended then using `buildMaven`. + +## Building a JAR + +Regardless of which strategy is chosen above, the step to build the derivation is the same. + +```nix +{ stdenv, lib, maven, callPackage }: +# pick a repository derivation, here we will use buildMaven +let repository = callPackage ./build-maven-repository.nix { }; +in stdenv.mkDerivation rec { + pname = "maven-demo"; + version = "1.0"; + + src = builtins.fetchTarball "https://github.com/fzakaria/nixos-maven-example/archive/main.tar.gz"; + buildInputs = [ maven ]; + + buildPhase = '' + echo "Using repository ${repository}" + mvn --offline -Dmaven.repo.local=${repository} package; + ''; + + installPhase = '' + install -Dm644 target/${pname}-${version}.jar $out/share/java + ''; +} +``` + +> We place the library in `$out/share/java` since JDK package has a _stdenv setup hook_ that adds any JARs in the `share/java` directories of the build inputs to the CLASSPATH environment. + +```bash +❯ tree $(nix-build --no-out-link build-jar.nix) +/nix/store/7jw3xdfagkc2vw8wrsdv68qpsnrxgvky-maven-demo-1.0 +└── share + └── java + └── maven-demo-1.0.jar + +2 directories, 1 file +``` + +## Runnable JAR + +The previous example builds a `jar` file but that's not a file one can run. + +You need to use it with `java -jar $out/share/java/output.jar` and make sure to provide the required dependencies on the classpath. + +The following explains how to use `makeWrapper` in order to make the derivation produce an executable that will run the JAR file you created. + +We will use the same repository we built above (either _double invocation_ or _buildMaven_) to setup a CLASSPATH for our JAR. + +The following two methods are more suited to Nix then building an [UberJar](https://imagej.net/Uber-JAR) which may be the more traditional approach. + +### CLASSPATH + +> This is ideal if you are providing a derivation for _nixpkgs_ and don't want to patch the project's `pom.xml`. + +We will read the Maven repository and flatten it to a single list. This list will then be concatenated with the _CLASSPATH_ separator to create the full classpath. + +We make sure to provide this classpath to the `makeWrapper`. + +```nix +{ stdenv, lib, maven, callPackage, makeWrapper, jre }: +let + repository = callPackage ./build-maven-repository.nix { }; +in stdenv.mkDerivation rec { + pname = "maven-demo"; + version = "1.0"; + + src = builtins.fetchTarball + "https://github.com/fzakaria/nixos-maven-example/archive/main.tar.gz"; + buildInputs = [ maven makeWrapper ]; + + buildPhase = '' + echo "Using repository ${repository}" + mvn --offline -Dmaven.repo.local=${repository} package; + ''; + + installPhase = '' + mkdir -p $out/bin + + classpath=$(find ${repository} -name "*.jar" -printf ':%h/%f'); + install -Dm644 target/${pname}-${version}.jar $out/share/java + # create a wrapper that will automatically set the classpath + # this should be the paths from the dependency derivation + makeWrapper ${jre}/bin/java $out/bin/${pname} \ + --add-flags "-classpath $out/share/java/${pname}-${version}.jar:''${classpath#:}" \ + --add-flags "Main" + ''; +} +``` + +### MANIFEST file via Maven Plugin + +> This is ideal if you are the project owner and want to change your `pom.xml` to set the CLASSPATH within it. + +Augment the `pom.xml` to create a JAR with the following manifest: +```xml +<build> + <plugins> + <plugin> + <artifactId>maven-jar-plugin</artifactId> + <configuration> + <archive> + <manifest> + <addClasspath>true</addClasspath> + <classpathPrefix>../../repository/</classpathPrefix> + <classpathLayoutType>repository</classpathLayoutType> + <mainClass>Main</mainClass> + </manifest> + <manifestEntries> + <Class-Path>.</Class-Path> + </manifestEntries> + </archive> + </configuration> + </plugin> + </plugins> +</build> +``` + +The above plugin instructs the JAR to look for the necessary dependencies in the `lib/` relative folder. The layout of the folder is also in the _maven repository_ style. + +```bash +❯ unzip -q -c $(nix-build --no-out-link runnable-jar.nix)/share/java/maven-demo-1.0.jar META-INF/MANIFEST.MF + +Manifest-Version: 1.0 +Archiver-Version: Plexus Archiver +Built-By: nixbld +Class-Path: . ../../repository/com/vdurmont/emoji-java/5.1.1/emoji-jav + a-5.1.1.jar ../../repository/org/json/json/20170516/json-20170516.jar +Created-By: Apache Maven 3.6.3 +Build-Jdk: 1.8.0_265 +Main-Class: Main +``` + +We will modify the derivation above to add a symlink to our repository so that it's accessible to our JAR during the `installPhase`. + +```nix +{ stdenv, lib, maven, callPackage, makeWrapper, jre }: +# pick a repository derivation, here we will use buildMaven +let repository = callPackage ./build-maven-repository.nix { }; +in stdenv.mkDerivation rec { + pname = "maven-demo"; + version = "1.0"; + + src = builtins.fetchTarball + "https://github.com/fzakaria/nixos-maven-example/archive/main.tar.gz"; + buildInputs = [ maven makeWrapper ]; + + buildPhase = '' + echo "Using repository ${repository}" + mvn --offline -Dmaven.repo.local=${repository} package; + ''; + + installPhase = '' + mkdir -p $out/bin + + # create a symbolic link for the repository directory + ln -s ${repository} $out/repository + + install -Dm644 target/${pname}-${version}.jar $out/share/java + # create a wrapper that will automatically set the classpath + # this should be the paths from the dependency derivation + makeWrapper ${jre}/bin/java $out/bin/${pname} \ + --add-flags "-jar $out/share/java/${pname}-${version}.jar" + ''; +} +``` + +> Our script produces a dependency on `jre` rather than `jdk` to restrict the runtime closure necessary to run the application. + +This will give you an executable shell-script that launches your JAR with all the dependencies available. + +```bash +❯ tree $(nix-build --no-out-link runnable-jar.nix) +/nix/store/8d4c3ibw8ynsn01ibhyqmc1zhzz75s26-maven-demo-1.0 +├── bin +│ └── maven-demo +├── repository -> /nix/store/g87va52nkc8jzbmi1aqdcf2f109r4dvn-maven-repository +└── share + └── java + └── maven-demo-1.0.jar + +❯ $(nix-build --no-out-link --option tarball-ttl 1 runnable-jar.nix)/bin/maven-demo +NixOS 😀 is super cool 😃! +``` diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/ocaml.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/ocaml.section.md new file mode 100644 index 000000000000..1c5a5473a05e --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/ocaml.section.md @@ -0,0 +1,70 @@ +# OCaml {#sec-language-ocaml} + +OCaml libraries should be installed in `$(out)/lib/ocaml/${ocaml.version}/site-lib/`. Such directories are automatically added to the `$OCAMLPATH` environment variable when building another package that depends on them or when opening a `nix-shell`. + +Given that most of the OCaml ecosystem is now built with dune, nixpkgs includes a convenience build support function called `buildDunePackage` that will build an OCaml package using dune, OCaml and findlib and any additional dependencies provided as `buildInputs` or `propagatedBuildInputs`. + +Here is a simple package example. It defines an (optional) attribute `minimumOCamlVersion` that will be used to throw a descriptive evaluation error if building with an older OCaml is attempted. It uses the `fetchFromGitHub` fetcher to get its source. It sets the `doCheck` (optional) attribute to `true` which means that tests will be run with `dune runtest -p angstrom` after the build (`dune build -p angstrom`) is complete. It uses `alcotest` as a build input (because it is needed to run the tests) and `bigstringaf` and `result` as propagated build inputs (thus they will also be available to libraries depending on this library). The library will be installed using the `angstrom.install` file that dune generates. + +```nix +{ stdenv +, fetchFromGitHub +, buildDunePackage +, alcotest +, result +, bigstringaf +}: + +buildDunePackage rec { + pname = "angstrom"; + version = "0.10.0"; + + minimumOCamlVersion = "4.03"; + + src = fetchFromGitHub { + owner = "inhabitedtype"; + repo = pname; + rev = version; + sha256 = "0lh6024yf9ds0nh9i93r9m6p5psi8nvrqxl5x7jwl13zb0r9xfpw"; + }; + + buildInputs = [ alcotest ]; + propagatedBuildInputs = [ bigstringaf result ]; + doCheck = true; + + meta = { + homepage = "https://github.com/inhabitedtype/angstrom"; + description = "OCaml parser combinators built for speed and memory efficiency"; + license = stdenv.lib.licenses.bsd3; + maintainers = with stdenv.lib.maintainers; [ sternenseemann ]; + }; +} +``` + +Here is a second example, this time using a source archive generated with `dune-release`. It is a good idea to use this archive when it is available as it will usually contain substituted variables such as a `%%VERSION%%` field. This library does not depend on any other OCaml library and no tests are run after building it. + +```nix +{ stdenv +, fetchurl +, buildDunePackage +}: + +buildDunePackage rec { + pname = "wtf8"; + version = "1.0.1"; + + minimumOCamlVersion = "4.01"; + + src = fetchurl { + url = "https://github.com/flowtype/ocaml-${pname}/releases/download/v${version}/${pname}-${version}.tbz"; + sha256 = "1msg3vycd3k8qqj61sc23qks541cxpb97vrnrvrhjnqxsqnh6ygq"; + }; + + meta = with stdenv.lib; { + homepage = "https://github.com/flowtype/ocaml-wtf8"; + description = "WTF-8 is a superset of UTF-8 that allows unpaired surrogates."; + license = licenses.mit; + maintainers = [ maintainers.eqyiel ]; + }; +} +``` diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/ocaml.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/ocaml.xml deleted file mode 100644 index 3f72092ec150..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/ocaml.xml +++ /dev/null @@ -1,73 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-ocaml"> - <title>OCaml</title> - - <para> - OCaml libraries should be installed in <literal>$(out)/lib/ocaml/${ocaml.version}/site-lib/</literal>. Such directories are automatically added to the <literal>$OCAMLPATH</literal> environment variable when building another package that depends on them or when opening a <literal>nix-shell</literal>. - </para> - - <para> - Given that most of the OCaml ecosystem is now built with dune, nixpkgs includes a convenience build support function called <literal>buildDunePackage</literal> that will build an OCaml package using dune, OCaml and findlib and any additional dependencies provided as <literal>buildInputs</literal> or <literal>propagatedBuildInputs</literal>. - </para> - - <para> - Here is a simple package example. It defines an (optional) attribute <literal>minimumOCamlVersion</literal> that will be used to throw a descriptive evaluation error if building with an older OCaml is attempted. It uses the <literal>fetchFromGitHub</literal> fetcher to get its source. It sets the <literal>doCheck</literal> (optional) attribute to <literal>true</literal> which means that tests will be run with <literal>dune runtest -p angstrom</literal> after the build (<literal>dune build -p angstrom</literal>) is complete. It uses <literal>alcotest</literal> as a build input (because it is needed to run the tests) and <literal>bigstringaf</literal> and <literal>result</literal> as propagated build inputs (thus they will also be available to libraries depending on this library). The library will be installed using the <literal>angstrom.install</literal> file that dune generates. - </para> - -<programlisting> -{ stdenv, fetchFromGitHub, buildDunePackage, alcotest, result, bigstringaf }: - -buildDunePackage rec { - pname = "angstrom"; - version = "0.10.0"; - - minimumOCamlVersion = "4.03"; - - src = fetchFromGitHub { - owner = "inhabitedtype"; - repo = pname; - rev = version; - sha256 = "0lh6024yf9ds0nh9i93r9m6p5psi8nvrqxl5x7jwl13zb0r9xfpw"; - }; - - buildInputs = [ alcotest ]; - propagatedBuildInputs = [ bigstringaf result ]; - doCheck = true; - - meta = { - homepage = "https://github.com/inhabitedtype/angstrom"; - description = "OCaml parser combinators built for speed and memory efficiency"; - license = stdenv.lib.licenses.bsd3; - maintainers = with stdenv.lib.maintainers; [ sternenseemann ]; - }; -} -</programlisting> - - <para> - Here is a second example, this time using a source archive generated with <literal>dune-release</literal>. It is a good idea to use this archive when it is available as it will usually contain substituted variables such as a <literal>%%VERSION%%</literal> field. This library does not depend on any other OCaml library and no tests are run after building it. - </para> - -<programlisting> -{ stdenv, fetchurl, buildDunePackage }: - -buildDunePackage rec { - pname = "wtf8"; - version = "1.0.1"; - - minimumOCamlVersion = "4.01"; - - src = fetchurl { - url = "https://github.com/flowtype/ocaml-${pname}/releases/download/v${version}/${pname}-${version}.tbz"; - sha256 = "1msg3vycd3k8qqj61sc23qks541cxpb97vrnrvrhjnqxsqnh6ygq"; - }; - - meta = with stdenv.lib; { - homepage = "https://github.com/flowtype/ocaml-wtf8"; - description = "WTF-8 is a superset of UTF-8 that allows unpaired surrogates."; - license = licenses.mit; - maintainers = [ maintainers.eqyiel ]; - }; -} -</programlisting> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/python.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/python.section.md index 59f7389b9ad3..8a2fe2711c7a 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/python.section.md +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/python.section.md @@ -153,7 +153,7 @@ The dot product of [1 2] and [3 4] is: 11 But if we maintain the script ourselves, and if there are more dependencies, it may be nice to encode those dependencies in source to make the script re-usable without that bit of knowledge. That can be done by using `nix-shell` as a -[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix), like so: +[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)), like so: ```python #!/usr/bin/env nix-shell diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/qt.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/qt.section.md new file mode 100644 index 000000000000..4a37eb4ef7db --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/qt.section.md @@ -0,0 +1,124 @@ +# Qt {#sec-language-qt} + +This section describes the differences between Nix expressions for Qt libraries and applications and Nix expressions for other C++ software. Some knowledge of the latter is assumed. + +There are primarily two problems which the Qt infrastructure is designed to address: ensuring consistent versioning of all dependencies and finding dependencies at runtime. + +## Nix expression for a Qt package (default.nix) {#qt-default-nix} + +```{=docbook} +<programlisting> +{ mkDerivation, lib, qtbase }: <co xml:id='qt-default-nix-co-1' /> + +mkDerivation { <co xml:id='qt-default-nix-co-2' /> + pname = "myapp"; + version = "1.0"; + + buildInputs = [ qtbase ]; <co xml:id='qt-default-nix-co-3' /> +} +</programlisting> + + <calloutlist> + <callout arearefs='qt-default-nix-co-1'> + <para> + Import <literal>mkDerivation</literal> and Qt (such as <literal>qtbase</literal> modules directly. <emphasis>Do not</emphasis> import Qt package sets; the Qt versions of dependencies may not be coherent, causing build and runtime failures. + </para> + </callout> + <callout arearefs='qt-default-nix-co-2'> + <para> + Use <literal>mkDerivation</literal> instead of <literal>stdenv.mkDerivation</literal>. <literal>mkDerivation</literal> is a wrapper around <literal>stdenv.mkDerivation</literal> which applies some Qt-specific settings. This deriver accepts the same arguments as <literal>stdenv.mkDerivation</literal>; refer to <xref linkend='chap-stdenv' /> for details. + </para> + <para> + To use another deriver instead of <literal>stdenv.mkDerivation</literal>, use <literal>mkDerivationWith</literal>: +<programlisting> +mkDerivationWith myDeriver { + # ... +} +</programlisting> + If you cannot use <literal>mkDerivationWith</literal>, please refer to <xref linkend='qt-runtime-dependencies' />. + </para> + </callout> + <callout arearefs='qt-default-nix-co-3'> + <para> + <literal>mkDerivation</literal> accepts the same arguments as <literal>stdenv.mkDerivation</literal>, such as <literal>buildInputs</literal>. + </para> + </callout> + </calloutlist> +``` + +## Locating runtime dependencies {#qt-runtime-dependencies} +Qt applications need to be wrapped to find runtime dependencies. If you cannot use `mkDerivation` or `mkDerivationWith` above, include `wrapQtAppsHook` in `nativeBuildInputs`: + +```nix +stdenv.mkDerivation { + # ... + + nativeBuildInputs = [ wrapQtAppsHook ]; +} +``` +Entries added to `qtWrapperArgs` are used to modify the wrappers created by `wrapQtAppsHook`. The entries are passed as arguments to [wrapProgram executable makeWrapperArgs](#fun-wrapProgram). + +```nix +mkDerivation { + # ... + + qtWrapperArgs = [ ''--prefix PATH : /path/to/bin'' ]; +} +``` + +Set `dontWrapQtApps` to stop applications from being wrapped automatically. It is required to wrap applications manually with `wrapQtApp`, using the syntax of [wrapProgram executable makeWrapperArgs](#fun-wrapProgram): + +```nix +mkDerivation { + # ... + + dontWrapQtApps = true; + preFixup = '' + wrapQtApp "$out/bin/myapp" --prefix PATH : /path/to/bin + ''; +} +``` + +> Note: `wrapQtAppsHook` ignores files that are non-ELF executables. This means that scripts won't be automatically wrapped so you'll need to manually wrap them as previously mentioned. An example of when you'd always need to do this is with Python applications that use PyQT. + +Libraries are built with every available version of Qt. Use the `meta.broken` attribute to disable the package for unsupported Qt versions: + +```nix +mkDerivation { + # ... + + # Disable this library with Qt < 5.9.0 + meta.broken = builtins.compareVersions qtbase.version "5.9.0" < 0; +} +``` +## Adding a library to Nixpkgs + Add a Qt library to all-packages.nix by adding it to the collection inside `mkLibsForQt5`. This ensures that the library is built with every available version of Qt as needed. + +### Example Adding a Qt library to all-packages.nix {#qt-library-all-packages-nix} + +``` +{ + # ... + + mkLibsForQt5 = self: with self; { + # ... + + mylib = callPackage ../path/to/mylib {}; + }; + + # ... +} +``` +## Adding an application to Nixpkgs +Add a Qt application to *all-packages.nix* using `libsForQt5.callPackage` instead of the usual `callPackage`. The former ensures that all dependencies are built with the same version of Qt. + +### Example Adding a QT application to all-packages.nix {#qt-application-all-packages-nix} +```nix +{ + # ... + + myapp = libsForQt5.callPackage ../path/to/myapp/ {}; + + # ... +} +``` diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/qt.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/qt.xml deleted file mode 100644 index ec95621d8ff2..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/qt.xml +++ /dev/null @@ -1,149 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-qt"> - <title>Qt</title> - - <para> - This section describes the differences between Nix expressions for Qt libraries and applications and Nix expressions for other C++ software. Some knowledge of the latter is assumed. There are primarily two problems which the Qt infrastructure is designed to address: ensuring consistent versioning of all dependencies and finding dependencies at runtime. - </para> - - <example xml:id='qt-default-nix'> - <title>Nix expression for a Qt package (<filename>default.nix</filename>)</title> -<programlisting> -{ mkDerivation, lib, qtbase }: <co xml:id='qt-default-nix-co-1' /> - -mkDerivation { <co xml:id='qt-default-nix-co-2' /> - pname = "myapp"; - version = "1.0"; - - buildInputs = [ qtbase ]; <co xml:id='qt-default-nix-co-3' /> -} -</programlisting> - </example> - - <calloutlist> - <callout arearefs='qt-default-nix-co-1'> - <para> - Import <literal>mkDerivation</literal> and Qt (such as <literal>qtbase</literal> modules directly. <emphasis>Do not</emphasis> import Qt package sets; the Qt versions of dependencies may not be coherent, causing build and runtime failures. - </para> - </callout> - <callout arearefs='qt-default-nix-co-2'> - <para> - Use <literal>mkDerivation</literal> instead of <literal>stdenv.mkDerivation</literal>. <literal>mkDerivation</literal> is a wrapper around <literal>stdenv.mkDerivation</literal> which applies some Qt-specific settings. This deriver accepts the same arguments as <literal>stdenv.mkDerivation</literal>; refer to <xref linkend='chap-stdenv' /> for details. - </para> - <para> - To use another deriver instead of <literal>stdenv.mkDerivation</literal>, use <literal>mkDerivationWith</literal>: -<programlisting> -mkDerivationWith myDeriver { - # ... -} -</programlisting> - If you cannot use <literal>mkDerivationWith</literal>, please refer to <xref linkend='qt-runtime-dependencies' />. - </para> - </callout> - <callout arearefs='qt-default-nix-co-3'> - <para> - <literal>mkDerivation</literal> accepts the same arguments as <literal>stdenv.mkDerivation</literal>, such as <literal>buildInputs</literal>. - </para> - </callout> - </calloutlist> - - <formalpara xml:id='qt-runtime-dependencies'> - <title>Locating runtime dependencies</title> - <para> - Qt applications need to be wrapped to find runtime dependencies. If you cannot use <literal>mkDerivation</literal> or <literal>mkDerivationWith</literal> above, include <literal>wrapQtAppsHook</literal> in <literal>nativeBuildInputs</literal>: -<programlisting> -stdenv.mkDerivation { - # ... - - nativeBuildInputs = [ wrapQtAppsHook ]; -} -</programlisting> - </para> - </formalpara> - - <para> - Entries added to <literal>qtWrapperArgs</literal> are used to modify the wrappers created by <literal>wrapQtAppsHook</literal>. The entries are passed as arguments to <xref linkend='fun-wrapProgram' />. -<programlisting> -mkDerivation { - # ... - - qtWrapperArgs = [ ''--prefix PATH : /path/to/bin'' ]; -} -</programlisting> - </para> - - <para> - Set <literal>dontWrapQtApps</literal> to stop applications from being wrapped automatically. It is required to wrap applications manually with <literal>wrapQtApp</literal>, using the syntax of <xref linkend='fun-wrapProgram' />: -<programlisting> -mkDerivation { - # ... - - dontWrapQtApps = true; - preFixup = '' - wrapQtApp "$out/bin/myapp" --prefix PATH : /path/to/bin - ''; -} -</programlisting> - </para> - - <note> - <para> - <literal>wrapQtAppsHook</literal> ignores files that are non-ELF executables. This means that scripts won't be automatically wrapped so you'll need to manually wrap them as previously mentioned. An example of when you'd always need to do this is with Python applications that use PyQT. - </para> - </note> - - <para> - Libraries are built with every available version of Qt. Use the <literal>meta.broken</literal> attribute to disable the package for unsupported Qt versions: -<programlisting> -mkDerivation { - # ... - - # Disable this library with Qt < 5.9.0 - meta.broken = builtins.compareVersions qtbase.version "5.9.0" < 0; -} -</programlisting> - </para> - - <formalpara> - <title>Adding a library to Nixpkgs</title> - <para> - Add a Qt library to <filename>all-packages.nix</filename> by adding it to the collection inside <literal>mkLibsForQt5</literal>. This ensures that the library is built with every available version of Qt as needed. - <example xml:id='qt-library-all-packages-nix'> - <title>Adding a Qt library to <filename>all-packages.nix</filename></title> -<programlisting> -{ - # ... - - mkLibsForQt5 = self: with self; { - # ... - - mylib = callPackage ../path/to/mylib {}; - }; - - # ... -} -</programlisting> - </example> - </para> - </formalpara> - - <formalpara> - <title>Adding an application to Nixpkgs</title> - <para> - Add a Qt application to <filename>all-packages.nix</filename> using <literal>libsForQt5.callPackage</literal> instead of the usual <literal>callPackage</literal>. The former ensures that all dependencies are built with the same version of Qt. - <example xml:id='qt-application-all-packages-nix'> - <title>Adding a Qt application to <filename>all-packages.nix</filename></title> -<programlisting> -{ - # ... - - myapp = libsForQt5.callPackage ../path/to/myapp/ {}; - - # ... -} -</programlisting> - </example> - </para> - </formalpara> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.section.md index e4c4ffce0432..e292b3110ff4 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.section.md +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.section.md @@ -1,74 +1,38 @@ ---- -title: Ruby -author: Michael Fellinger -date: 2019-05-23 ---- +# Ruby {#sec-language-ruby} -# Ruby +## Using Ruby -## User Guide +Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. The attribute `ruby` refers to the default Ruby interpreter, which is currently MRI 2.6. It's also possible to refer to specific versions, e.g. `ruby_2_y`, `jruby`, or `mruby`. -### Using Ruby +In the Nixpkgs tree, Ruby packages can be found throughout, depending on what they do, and are called from the main package set. Ruby gems, however are separate sets, and there's one default set for each interpreter (currently MRI only). -#### Overview +There are two main approaches for using Ruby with gems. One is to use a specifically locked `Gemfile` for an application that has very strict dependencies. The other is to depend on the common gems, which we'll explain further down, and rely on them being updated regularly. -Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. -The attribute `ruby` refers to the default Ruby interpreter, which is currently -MRI 2.5. It's also possible to refer to specific versions, e.g. `ruby_2_6`, `jruby`, or `mruby`. +The interpreters have common attributes, namely `gems`, and `withPackages`. So you can refer to `ruby.gems.nokogiri`, or `ruby_2_6.gems.nokogiri` to get the Nokogiri gem already compiled and ready to use. -In the nixpkgs tree, Ruby packages can be found throughout, depending on what -they do, and are called from the main package set. Ruby gems, however are -separate sets, and there's one default set for each interpreter (currently MRI -only). +Since not all gems have executables like `nokogiri`, it's usually more convenient to use the `withPackages` function like this: `ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the Ruby in your environment will be able to find the gem and it can be used in your Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"` as usual. -There are two main approaches for using Ruby with gems. -One is to use a specifically locked `Gemfile` for an application that has very strict dependencies. -The other is to depend on the common gems, which we'll explain further down, and -rely on them being updated regularly. +### Temporary Ruby environment with `nix-shell` -The interpreters have common attributes, namely `gems`, and `withPackages`. So -you can refer to `ruby.gems.nokogiri`, or `ruby_2_5.gems.nokogiri` to get the -Nokogiri gem already compiled and ready to use. +Rather than having a single Ruby environment shared by all Ruby development projects on a system, Nix allows you to create separate environments per project. `nix-shell` gives you the possibility to temporarily load another environment akin to a combined `chruby` or `rvm` and `bundle exec`. -Since not all gems have executables like `nokogiri`, it's usually more -convenient to use the `withPackages` function like this: -`ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the -Ruby in your environment will be able to find the gem and it can be used in your -Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"` -as usual. +There are two methods for loading a shell with Ruby packages. The first and recommended method is to create an environment with `ruby.withPackages` and load that. -#### Temporary Ruby environment with `nix-shell` - -Rather than having a single Ruby environment shared by all Ruby -development projects on a system, Nix allows you to create separate -environments per project. `nix-shell` gives you the possibility to -temporarily load another environment akin to a combined `chruby` or -`rvm` and `bundle exec`. - -There are two methods for loading a shell with Ruby packages. The first and -recommended method is to create an environment with `ruby.withPackages` and load -that. - -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" ``` -The other method, which is not recommended, is to create an environment and list -all the packages directly. +The other method, which is not recommended, is to create an environment and list all the packages directly. -```shell -nix-shell -p ruby.gems.nokogiri ruby.gems.pry +```ShellSession +$ nix-shell -p ruby.gems.nokogiri ruby.gems.pry ``` -Again, it's possible to launch the interpreter from the shell. The Ruby -interpreter has the attribute `gems` which contains all Ruby gems for that -specific interpreter. +Again, it's possible to launch the interpreter from the shell. The Ruby interpreter has the attribute `gems` which contains all Ruby gems for that specific interpreter. -##### Load environment from `.nix` expression +#### Load Ruby environment from `.nix` expression -As explained in the Nix manual, `nix-shell` can also load an expression from a -`.nix` file. Say we want to have Ruby 2.5, `nokogori`, and `pry`. Consider a -`shell.nix` file with: +As explained in the Nix manual, `nix-shell` can also load an expression from a `.nix` file. Say we want to have Ruby 2.6, `nokogori`, and `pry`. Consider a `shell.nix` file with: ```nix with import <nixpkgs> {}; @@ -77,43 +41,33 @@ ruby.withPackages (ps: with ps; [ nokogiri pry ]) What's happening here? -1. We begin with importing the Nix Packages collections. `import <nixpkgs>` - imports the `<nixpkgs>` function, `{}` calls it and the `with` statement - brings all attributes of `nixpkgs` in the local scope. These attributes form - the main package set. +1. We begin with importing the Nix Packages collections. `import <nixpkgs>` imports the `<nixpkgs>` function, `{}` calls it and the `with` statement brings all attributes of `nixpkgs` in the local scope. These attributes form the main package set. 2. Then we create a Ruby environment with the `withPackages` function. -3. The `withPackages` function expects us to provide a function as an argument - that takes the set of all ruby gems and returns a list of packages to include - in the environment. Here, we select the packages `nokogiri` and `pry` from - the package set. +3. The `withPackages` function expects us to provide a function as an argument that takes the set of all ruby gems and returns a list of packages to include in the environment. Here, we select the packages `nokogiri` and `pry` from the package set. -##### Execute command with `--run` +#### Execute command with `--run` -A convenient flag for `nix-shell` is `--run`. It executes a command in the -`nix-shell`. We can e.g. directly open a `pry` REPL: +A convenient flag for `nix-shell` is `--run`. It executes a command in the `nix-shell`. We can e.g. directly open a `pry` REPL: -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry" ``` Or immediately require `nokogiri` in pry: -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry -rnokogiri" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry -rnokogiri" ``` Or run a script using this environment: -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "ruby example.rb" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "ruby example.rb" ``` -##### Using `nix-shell` as shebang +#### Using `nix-shell` as shebang -In fact, for the last case, there is a more convenient method. You can add a -[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script -specifying which dependencies `nix-shell` needs. With the following shebang, you -can just execute `./example.rb`, and it will run with all dependencies. +In fact, for the last case, there is a more convenient method. You can add a [shebang](<https://en.wikipedia.org/wiki/Shebang_(Unix)>) to your script specifying which dependencies `nix-shell` needs. With the following shebang, you can just execute `./example.rb`, and it will run with all dependencies. ```ruby #! /usr/bin/env nix-shell @@ -126,35 +80,24 @@ body = RestClient.get('http://example.com').body puts Nokogiri::HTML(body).at('h1').text ``` -### Developing with Ruby +## Developing with Ruby -#### Using an existing Gemfile +### Using an existing Gemfile -In most cases, you'll already have a `Gemfile.lock` listing all your dependencies. -This can be used to generate a `gemset.nix` which is used to fetch the gems and -combine them into a single environment. -The reason why you need to have a separate file for this, is that Nix requires -you to have a checksum for each input to your build. -Since the `Gemfile.lock` that `bundler` generates doesn't provide us with -checksums, we have to first download each gem, calculate its SHA256, and store -it in this separate file. +In most cases, you'll already have a `Gemfile.lock` listing all your dependencies. This can be used to generate a `gemset.nix` which is used to fetch the gems and combine them into a single environment. The reason why you need to have a separate file for this, is that Nix requires you to have a checksum for each input to your build. Since the `Gemfile.lock` that `bundler` generates doesn't provide us with checksums, we have to first download each gem, calculate its SHA256, and store it in this separate file. So the steps from having just a `Gemfile` to a `gemset.nix` are: -```shell -bundle lock -bundix +```ShellSession +$ bundle lock +$ bundix ``` -If you already have a `Gemfile.lock`, you can simply run `bundix` and it will -work the same. +If you already have a `Gemfile.lock`, you can simply run `bundix` and it will work the same. -To update the gems in your `Gemfile.lock`, you may use the `bundix -l` flag, -which will create a new `Gemfile.lock` in case the `Gemfile` has a more recent -time of modification. +To update the gems in your `Gemfile.lock`, you may use the `bundix -l` flag, which will create a new `Gemfile.lock` in case the `Gemfile` has a more recent time of modification. -Once the `gemset.nix` is generated, it can be used in a -`bundlerEnv` derivation. Here is an example you could use for your `shell.nix`: +Once the `gemset.nix` is generated, it can be used in a `bundlerEnv` derivation. Here is an example you could use for your `shell.nix`: ```nix # ... @@ -166,41 +109,26 @@ let in mkShell { buildInputs = [ gems gems.wrappedRuby ]; } ``` -With this file in your directory, you can run `nix-shell` to build and use the gems. -The important parts here are `bundlerEnv` and `wrappedRuby`. +With this file in your directory, you can run `nix-shell` to build and use the gems. The important parts here are `bundlerEnv` and `wrappedRuby`. -The `bundlerEnv` is a wrapper over all the gems in your gemset. This means that -all the `/lib` and `/bin` directories will be available, and the executables of -all gems (even of indirect dependencies) will end up in your `$PATH`. -The `wrappedRuby` provides you with all executables that come with Ruby itself, -but wrapped so they can easily find the gems in your gemset. +The `bundlerEnv` is a wrapper over all the gems in your gemset. This means that all the `/lib` and `/bin` directories will be available, and the executables of all gems (even of indirect dependencies) will end up in your `$PATH`. The `wrappedRuby` provides you with all executables that come with Ruby itself, but wrapped so they can easily find the gems in your gemset. -One common issue that you might have is that you have Ruby 2.6, but also -`bundler` in your gemset. That leads to a conflict for `/bin/bundle` and -`/bin/bundler`. You can resolve this by wrapping either your Ruby or your gems -in a `lowPrio` call. So in order to give the `bundler` from your gemset -priority, it would be used like this: +One common issue that you might have is that you have Ruby 2.6, but also `bundler` in your gemset. That leads to a conflict for `/bin/bundle` and `/bin/bundler`. You can resolve this by wrapping either your Ruby or your gems in a `lowPrio` call. So in order to give the `bundler` from your gemset priority, it would be used like this: ```nix # ... mkShell { buildInputs = [ gems (lowPrio gems.wrappedRuby) ]; } ``` +### Gem-specific configurations and workarounds -#### Gem-specific configurations and workarounds +In some cases, especially if the gem has native extensions, you might need to modify the way the gem is built. -In some cases, especially if the gem has native extensions, you might need to -modify the way the gem is built. +This is done via a common configuration file that includes all of the workarounds for each gem. -This is done via a common configuration file that includes all of the -workarounds for each gem. +This file lives at `/pkgs/development/ruby-modules/gem-config/default.nix`, since it already contains a lot of entries, it should be pretty easy to add the modifications you need for your needs. -This file lives at `/pkgs/development/ruby-modules/gem-config/default.nix`, -since it already contains a lot of entries, it should be pretty easy to add the -modifications you need for your needs. - -In the meanwhile, or if the modification is for a private gem, you can also add -the configuration to only your own environment. +In the meanwhile, or if the modification is for a private gem, you can also add the configuration to only your own environment. Two places that allow this modification are the `ruby` derivation, or `bundlerEnv`. @@ -261,10 +189,9 @@ let in pkgs.ruby.withPackages (ps: with ps; [ pg ]) ``` -Then we can get whichever postgresql version we desire and the `pg` gem will -always reference it correctly: +Then we can get whichever postgresql version we desire and the `pg` gem will always reference it correctly: -```shell +```ShellSession $ nix-shell --argstr pg_version 9_4 --run 'ruby -rpg -e "puts PG.library_version"' 90421 @@ -272,24 +199,15 @@ $ nix-shell --run 'ruby -rpg -e "puts PG.library_version"' 100007 ``` -Of course for this use-case one could also use overlays since the configuration -for `pg` depends on the `postgresql` alias, but for demonstration purposes this -has to suffice. +Of course for this use-case one could also use overlays since the configuration for `pg` depends on the `postgresql` alias, but for demonstration purposes this has to suffice. -#### Adding a gem to the default gemset +### Adding a gem to the default gemset -Now that you know how to get a working Ruby environment with Nix, it's time to -go forward and start actually developing with Ruby. -We will first have a look at how Ruby gems are packaged on Nix. Then, we will -look at how you can use development mode with your code. +Now that you know how to get a working Ruby environment with Nix, it's time to go forward and start actually developing with Ruby. We will first have a look at how Ruby gems are packaged on Nix. Then, we will look at how you can use development mode with your code. -All gems in the standard set are automatically generated from a single -`Gemfile`. The dependency resolution is done with `bundler` and makes it more -likely that all gems are compatible to each other. +All gems in the standard set are automatically generated from a single `Gemfile`. The dependency resolution is done with `bundler` and makes it more likely that all gems are compatible to each other. -In order to add a new gem to nixpkgs, you can put it into the -`/pkgs/development/ruby-modules/with-packages/Gemfile` and run -`./maintainers/scripts/update-ruby-packages`. +In order to add a new gem to nixpkgs, you can put it into the `/pkgs/development/ruby-modules/with-packages/Gemfile` and run `./maintainers/scripts/update-ruby-packages`. To test that it works, you can then try using the gem with: @@ -297,16 +215,11 @@ To test that it works, you can then try using the gem with: NIX_PATH=nixpkgs=$PWD nix-shell -p "ruby.withPackages (ps: with ps; [ name-of-your-gem ])" ``` -#### Packaging applications +### Packaging applications -A common task is to add a ruby executable to nixpkgs, popular examples would be -`chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp` -function, that allows you to make a package that only exposes the listed -executables, otherwise the package may cause conflicts through common paths like -`bin/rake` or `bin/bundler` that aren't meant to be used. +A common task is to add a ruby executable to nixpkgs, popular examples would be `chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp` function, that allows you to make a package that only exposes the listed executables, otherwise the package may cause conflicts through common paths like `bin/rake` or `bin/bundler` that aren't meant to be used. -The absolute easiest way to do that is to write a -`Gemfile` along these lines: +The absolute easiest way to do that is to write a `Gemfile` along these lines: ```ruby source 'https://rubygems.org' do @@ -314,10 +227,7 @@ source 'https://rubygems.org' do end ``` -If you want to package a specific version, you can use the standard Gemfile -syntax for that, e.g. `gem 'mdl', '0.5.0'`, but if you want the latest stable -version anyway, it's easier to update by simply running the `bundle lock` and -`bundix` steps again. +If you want to package a specific version, you can use the standard Gemfile syntax for that, e.g. `gem 'mdl', '0.5.0'`, but if you want the latest stable version anyway, it's easier to update by simply running the `bundle lock` and `bundix` steps again. Now you can also also make a `default.nix` that looks like this: @@ -331,20 +241,15 @@ bundlerApp { } ``` -All that's left to do is to generate the corresponding `Gemfile.lock` and -`gemset.nix` as described above in the `Using an existing Gemfile` section. +All that's left to do is to generate the corresponding `Gemfile.lock` and `gemset.nix` as described above in the `Using an existing Gemfile` section. -##### Packaging executables that require wrapping +#### Packaging executables that require wrapping -Sometimes your app will depend on other executables at runtime, and tries to -find it through the `PATH` environment variable. +Sometimes your app will depend on other executables at runtime, and tries to find it through the `PATH` environment variable. -In this case, you can provide a `postBuild` hook to `bundlerApp` that wraps the -gem in another script that prefixes the `PATH`. +In this case, you can provide a `postBuild` hook to `bundlerApp` that wraps the gem in another script that prefixes the `PATH`. -Of course you could also make a custom `gemConfig` if you know exactly how to -patch it, but it's usually much easier to maintain with a simple wrapper so the -patch doesn't have to be adjusted for each version. +Of course you could also make a custom `gemConfig` if you know exactly how to patch it, but it's usually much easier to maintain with a simple wrapper so the patch doesn't have to be adjusted for each version. Here's another example: diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.xml deleted file mode 100644 index 9b579d6804f4..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/ruby.xml +++ /dev/null @@ -1,107 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-ruby"> - <title>Ruby</title> - - <para> - There currently is support to bundle applications that are packaged as Ruby gems. The utility "bundix" allows you to write a <filename>Gemfile</filename>, let bundler create a <filename>Gemfile.lock</filename>, and then convert this into a nix expression that contains all Gem dependencies automatically. - </para> - - <para> - For example, to package sensu, we did: - </para> - -<screen> -<prompt>$ </prompt>cd pkgs/servers/monitoring -<prompt>$ </prompt>mkdir sensu -<prompt>$ </prompt>cd sensu -<prompt>$ </prompt>cat > Gemfile -source 'https://rubygems.org' -gem 'sensu' -<prompt>$ </prompt>$(nix-build '<nixpkgs>' -A bundix --no-out-link)/bin/bundix --magic -<prompt>$ </prompt>cat > default.nix -{ lib, bundlerEnv, ruby }: - -bundlerEnv rec { - name = "sensu-${version}"; - - version = (import gemset).sensu.version; - inherit ruby; - # expects Gemfile, Gemfile.lock and gemset.nix in the same directory - gemdir = ./.; - - meta = with lib; { - description = "A monitoring framework that aims to be simple, malleable, and scalable"; - homepage = "http://sensuapp.org/"; - license = with licenses; mit; - maintainers = with maintainers; [ theuni ]; - platforms = platforms.unix; - }; -} -</screen> - - <para> - Please check in the <filename>Gemfile</filename>, <filename>Gemfile.lock</filename> and the <filename>gemset.nix</filename> so future updates can be run easily. - </para> - - <para> - Updating Ruby packages can then be done like this: - </para> - -<screen> -<prompt>$ </prompt>cd pkgs/servers/monitoring/sensu -<prompt>$ </prompt>nix-shell -p bundler --run 'bundle lock --update' -<prompt>$ </prompt>nix-shell -p bundix --run 'bundix' -</screen> - - <para> - For tools written in Ruby - i.e. where the desire is to install a package and then execute e.g. <command>rake</command> at the command line, there is an alternative builder called <literal>bundlerApp</literal>. Set up the <filename>gemset.nix</filename> the same way, and then, for example: - </para> - -<programlisting> -<![CDATA[{ lib, bundlerApp }: - -bundlerApp { - pname = "corundum"; - gemdir = ./.; - exes = [ "corundum-skel" ]; - - meta = with lib; { - description = "Tool and libraries for maintaining Ruby gems."; - homepage = "https://github.com/nyarly/corundum"; - license = licenses.mit; - maintainers = [ maintainers.nyarly ]; - platforms = platforms.unix; - }; -}]]> -</programlisting> - - <para> - The chief advantage of <literal>bundlerApp</literal> over <literal>bundlerEnv</literal> is the executables introduced in the environment are precisely those selected in the <literal>exes</literal> list, as opposed to <literal>bundlerEnv</literal> which adds all the executables made available by gems in the gemset, which can mean e.g. <command>rspec</command> or <command>rake</command> in unpredictable versions available from various packages. - </para> - - <para> - Resulting derivations for both builders also have two helpful attributes, <literal>env</literal> and <literal>wrappedRuby</literal>. The first one allows one to quickly drop into <command>nix-shell</command> with the specified environment present. E.g. <command>nix-shell -A sensu.env</command> would give you an environment with Ruby preset so it has all the libraries necessary for <literal>sensu</literal> in its paths. The second one can be used to make derivations from custom Ruby scripts which have <filename>Gemfile</filename>s with their dependencies specified. It is a derivation with <command>ruby</command> wrapped so it can find all the needed dependencies. For example, to make a derivation <literal>my-script</literal> for a <filename>my-script.rb</filename> (which should be placed in <filename>bin</filename>) you should run <command>bundix</command> as specified above and then use <literal>bundlerEnv</literal> like this: - </para> - -<programlisting> -<![CDATA[let env = bundlerEnv { - name = "my-script-env"; - - inherit ruby; - gemfile = ./Gemfile; - lockfile = ./Gemfile.lock; - gemset = ./gemset.nix; -}; - -in stdenv.mkDerivation { - name = "my-script"; - buildInputs = [ env.wrappedRuby ]; - script = ./my-script.rb; - buildCommand = '' - install -D -m755 $script $out/bin/my-script - patchShebangs $out/bin/my-script - ''; -}]]> -</programlisting> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/rust.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/rust.section.md index 0e1d59e1a952..0230993d3f08 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/rust.section.md +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/rust.section.md @@ -16,9 +16,9 @@ cargo into the `environment.systemPackages` or bring them into scope with `nix-shell -p rustc cargo`. -For daily builds (beta and nightly) use either rustup from -nixpkgs or use the [Rust nightlies -overlay](#using-the-rust-nightlies-overlay). +For other versions such as daily builds (beta and nightly), +use either `rustup` from nixpkgs (which will manage the rust installation in your home directory), +or use Mozilla's [Rust nightlies overlay](#using-the-rust-nightlies-overlay). ## Compiling Rust applications with Cargo @@ -63,9 +63,52 @@ The fetcher will verify that the `Cargo.lock` file is in sync with the `src` attribute, and fail the build if not. It will also will compress the vendor directory into a tar.gz archive. -### Building a crate for a different target - -To build your crate with a different cargo `--target` simply specify the `target` attribute: +### Cross compilation + +By default, Rust packages are compiled for the host platform, just like any +other package is. The `--target` passed to rust tools is computed from this. +By default, it takes the `stdenv.hostPlatform.config` and replaces components +where they are known to differ. But there are ways to customize the argument: + + - To choose a different target by name, define + `stdenv.hostPlatform.rustc.config` as that name (a string), and that + name will be used instead. + + For example: + ```nix + import <nixpkgs> { + crossSystem = (import <nixpkgs/lib>).systems.examples.armhf-embedded // { + rustc.config = "thumbv7em-none-eabi"; + }; + } + ``` + will result in: + ```shell + --target thumbv7em-none-eabi + ``` + + - To pass a completely custom target, define + `stdenv.hostPlatform.rustc.config` with its name, and + `stdenv.hostPlatform.rustc.platform` with the value. The value will be + serialized to JSON in a file called + `${stdenv.hostPlatform.rustc.config}.json`, and the path of that file + will be used instead. + + For example: + ```nix + import <nixpkgs> { + crossSystem = (import <nixpkgs/lib>).systems.examples.armhf-embedded // { + rustc.config = "thumb-crazy"; + rustc.platform = { foo = ""; bar = ""; }; + }; + } + will result in: + ```shell + --target /nix/store/asdfasdfsadf-thumb-crazy.json # contains {"foo":"","bar":""} + ``` + +Finally, as an ad-hoc escape hatch, a computed target (string or JSON file +path) can be passed directly to `buildRustPackage`: ```nix pkgs.rustPlatform.buildRustPackage { @@ -74,6 +117,15 @@ pkgs.rustPlatform.buildRustPackage { } ``` +This is useful to avoid rebuilding Rust tools, since they are actually target +agnostic and don't need to be rebuilt. But in the future, we should always +build the Rust tools and standard library crates separately so there is no +reason not to take the `stdenv.hostPlatform.rustc`-modifying approach, and the +ad-hoc escape hatch to `buildRustPackage` can be removed. + +Note that currently custom targets aren't compiled with `std`, so `cargo test` +will fail. This can be ignored by adding `doCheck = false;` to your derivation. + ### Running package tests When using `buildRustPackage`, the `checkPhase` is enabled by default and runs @@ -478,8 +530,15 @@ Mozilla provides an overlay for nixpkgs to bring a nightly version of Rust into This overlay can _also_ be used to install recent unstable or stable versions of Rust, if desired. -To use this overlay, clone -[nixpkgs-mozilla](https://github.com/mozilla/nixpkgs-mozilla), +### Rust overlay installation + +You can use this overlay by either changing your local nixpkgs configuration, +or by adding the overlay declaratively in a nix expression, e.g. in `configuration.nix`. +For more information see [#sec-overlays-install](the manual on installing overlays). + +#### Imperative rust overlay installation + +Clone [nixpkgs-mozilla](https://github.com/mozilla/nixpkgs-mozilla), and create a symbolic link to the file [rust-overlay.nix](https://github.com/mozilla/nixpkgs-mozilla/blob/master/rust-overlay.nix) in the `~/.config/nixpkgs/overlays` directory. @@ -488,14 +547,42 @@ in the `~/.config/nixpkgs/overlays` directory. $ mkdir -p ~/.config/nixpkgs/overlays $ ln -s $(pwd)/nixpkgs-mozilla/rust-overlay.nix ~/.config/nixpkgs/overlays/rust-overlay.nix -The latest version can be installed with the following command: +### Declarative rust overlay installation + +Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.nix`, or similar: + +``` + nixpkgs = { + overlays = [ + (import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz)) + # Further overlays go here + ]; + }; +``` + +Note that this will fetch the latest overlay version when rebuilding your system. + +### Rust overlay usage + +The overlay contains attribute sets corresponding to different versions of the rust toolchain, such as: + +* `latest.rustChannels.stable` +* `latest.rustChannels.nightly` +* a function `rustChannelOf`, called as `(rustChannelOf { date = "2018-04-11"; channel = "nightly"; })`, or... +* `(nixpkgs.rustChannelOf { rustToolchain = ./rust-toolchain; })` if you have a local `rust-toolchain` file (see https://github.com/mozilla/nixpkgs-mozilla#using-in-nix-expressions for an example) + +Each of these contain packages such as `rust`, which contains your usual rust development tools with the respective toolchain chosen. +For example, you might want to add `latest.rustChannels.stable.rust` to the list of packages in your configuration. + +Imperatively, the latest stable version can be installed with the following command: - $ nix-env -Ai nixos.latest.rustChannels.stable.rust + $ nix-env -Ai nixpkgs.latest.rustChannels.stable.rust Or using the attribute with nix-shell: - $ nix-shell -p nixos.latest.rustChannels.stable.rust + $ nix-shell -p nixpkgs.latest.rustChannels.stable.rust +Substitute the `nixpkgs` prefix with `nixos` on NixOS. To install the beta or nightly channel, "stable" should be substituted by "nightly" or "beta", or use the function provided by this overlay to pull a version based on a diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/texlive.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/texlive.section.md new file mode 100644 index 000000000000..9584c56bb52f --- /dev/null +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/texlive.section.md @@ -0,0 +1,128 @@ + +# TeX Live {#sec-language-texlive} + +Since release 15.09 there is a new TeX Live packaging that lives entirely under attribute `texlive`. + +## User's guide {#sec-language-texlive-user-guide} + +- For basic usage just pull `texlive.combined.scheme-basic` for an environment with basic LaTeX support. +- It typically won't work to use separately installed packages together. Instead, you can build a custom set of packages like this: + + ```nix + texlive.combine { + inherit (texlive) scheme-small collection-langkorean algorithms cm-super; + } + ``` + +- There are all the schemes, collections and a few thousand packages, as defined upstream (perhaps with tiny differences). +- By default you only get executables and files needed during runtime, and a little documentation for the core packages. To change that, you need to add `pkgFilter` function to `combine`. + + ```nix + texlive.combine { + # inherit (texlive) whatever-you-want; + pkgFilter = pkg: + pkg.tlType == "run" || pkg.tlType == "bin" || pkg.pname == "cm-super"; + # elem tlType [ "run" "bin" "doc" "source" ] + # there are also other attributes: version, name + } + ``` + +- You can list packages e.g. by `nix repl`. + + ```ShellSession + $ nix repl + nix-repl> :l <nixpkgs> + nix-repl> texlive.collection-[TAB] + ``` + +- Note that the wrapper assumes that the result has a chance to be useful. For example, the core executables should be present, as well as some core data files. The supported way of ensuring this is by including some scheme, for example `scheme-basic`, into the combination. + +## Custom packages {#sec-language-texlive-custom-packages} + + +You may find that you need to use an external TeX package. A derivation for such package has to provide contents of the "texmf" directory in its output and provide the `tlType` attribute. Here is a (very verbose) example: + +```nix +with import <nixpkgs> {}; + +let + foiltex_run = stdenvNoCC.mkDerivation { + pname = "latex-foiltex"; + version = "2.1.4b"; + passthru.tlType = "run"; + + srcs = [ + (fetchurl { + url = "http://mirrors.ctan.org/macros/latex/contrib/foiltex/foiltex.dtx"; + sha256 = "07frz0krpz7kkcwlayrwrj2a2pixmv0icbngyw92srp9fp23cqpz"; + }) + (fetchurl { + url = "http://mirrors.ctan.org/macros/latex/contrib/foiltex/foiltex.ins"; + sha256 = "09wkyidxk3n3zvqxfs61wlypmbhi1pxmjdi1kns9n2ky8ykbff99"; + }) + ]; + + unpackPhase = '' + runHook preUnpack + + for _src in $srcs; do + cp "$_src" $(stripHash "$_src") + done + + runHook postUnpack + ''; + + nativeBuildInputs = [ texlive.combined.scheme-small ]; + + dontConfigure = true; + + buildPhase = '' + runHook preBuild + + # Generate the style files + latex foiltex.ins + + runHook postBuild + ''; + + installPhase = '' + runHook preInstall + + path="$out/tex/latex/foiltex" + mkdir -p "$path" + cp *.{cls,def,clo} "$path/" + + runHook postInstall + ''; + + meta = with lib; { + description = "A LaTeX2e class for overhead transparencies"; + license = licenses.unfreeRedistributable; + maintainers = with maintainers; [ veprbl ]; + platforms = platforms.all; + }; + }; + foiltex = { pkgs = [ foiltex_run ]; }; + + latex_with_foiltex = texlive.combine { + inherit (texlive) scheme-small; + inherit foiltex; + }; +in + runCommand "test.pdf" { + nativeBuildInputs = [ latex_with_foiltex ]; + } '' +cat >test.tex <<EOF +\documentclass{foils} + +\title{Presentation title} +\date{} + +\begin{document} +\maketitle +\end{document} +EOF + pdflatex test.tex + cp test.pdf $out +'' +``` diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/texlive.xml b/infra/libkookie/nixpkgs/doc/languages-frameworks/texlive.xml deleted file mode 100644 index 141c46e5a623..000000000000 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/texlive.xml +++ /dev/null @@ -1,152 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-texlive"> - <title>TeX Live</title> - - <para> - Since release 15.09 there is a new TeX Live packaging that lives entirely under attribute <varname>texlive</varname>. - </para> - - <section xml:id="sec-language-texlive-users-guide"> - <title>User's guide</title> - - <itemizedlist> - <listitem> - <para> - For basic usage just pull <varname>texlive.combined.scheme-basic</varname> for an environment with basic LaTeX support. - </para> - </listitem> - <listitem> - <para> - It typically won't work to use separately installed packages together. Instead, you can build a custom set of packages like this: -<programlisting> -texlive.combine { - inherit (texlive) scheme-small collection-langkorean algorithms cm-super; -} -</programlisting> - There are all the schemes, collections and a few thousand packages, as defined upstream (perhaps with tiny differences). - </para> - </listitem> - <listitem> - <para> - By default you only get executables and files needed during runtime, and a little documentation for the core packages. To change that, you need to add <varname>pkgFilter</varname> function to <varname>combine</varname>. -<programlisting> -texlive.combine { - # inherit (texlive) whatever-you-want; - pkgFilter = pkg: - pkg.tlType == "run" || pkg.tlType == "bin" || pkg.pname == "cm-super"; - # elem tlType [ "run" "bin" "doc" "source" ] - # there are also other attributes: version, name -} -</programlisting> - </para> - </listitem> - <listitem> - <para> - You can list packages e.g. by <command>nix repl</command>. -<programlisting> -<prompt>$ </prompt>nix repl -<prompt>nix-repl> </prompt>:l <nixpkgs> -<prompt>nix-repl> </prompt>texlive.collection-<keycap function="tab" /> -</programlisting> - </para> - </listitem> - <listitem> - <para> - Note that the wrapper assumes that the result has a chance to be useful. For example, the core executables should be present, as well as some core data files. The supported way of ensuring this is by including some scheme, for example <varname>scheme-basic</varname>, into the combination. - </para> - </listitem> - </itemizedlist> - </section> - - <section xml:id="sec-language-texlive-custom-packages"> - <title>Custom packages</title> - <para> - You may find that you need to use an external TeX package. A derivation for such package has to provide contents of the "texmf" directory in its output and provide the <varname>tlType</varname> attribute. Here is a (very verbose) example: -<programlisting><![CDATA[ -with import <nixpkgs> {}; - -let - foiltex_run = stdenvNoCC.mkDerivation { - pname = "latex-foiltex"; - version = "2.1.4b"; - passthru.tlType = "run"; - - srcs = [ - (fetchurl { - url = "http://mirrors.ctan.org/macros/latex/contrib/foiltex/foiltex.dtx"; - sha256 = "07frz0krpz7kkcwlayrwrj2a2pixmv0icbngyw92srp9fp23cqpz"; - }) - (fetchurl { - url = "http://mirrors.ctan.org/macros/latex/contrib/foiltex/foiltex.ins"; - sha256 = "09wkyidxk3n3zvqxfs61wlypmbhi1pxmjdi1kns9n2ky8ykbff99"; - }) - ]; - - unpackPhase = '' - runHook preUnpack - - for _src in $srcs; do - cp "$_src" $(stripHash "$_src") - done - - runHook postUnpack - ''; - - nativeBuildInputs = [ texlive.combined.scheme-small ]; - - dontConfigure = true; - - buildPhase = '' - runHook preBuild - - # Generate the style files - latex foiltex.ins - - runHook postBuild - ''; - - installPhase = '' - runHook preInstall - - path="$out/tex/latex/foiltex" - mkdir -p "$path" - cp *.{cls,def,clo} "$path/" - - runHook postInstall - ''; - - meta = with lib; { - description = "A LaTeX2e class for overhead transparencies"; - license = licenses.unfreeRedistributable; - maintainers = with maintainers; [ veprbl ]; - platforms = platforms.all; - }; - }; - foiltex = { pkgs = [ foiltex_run ]; }; - - latex_with_foiltex = texlive.combine { - inherit (texlive) scheme-small; - inherit foiltex; - }; -in - runCommand "test.pdf" { - nativeBuildInputs = [ latex_with_foiltex ]; - } '' -cat >test.tex <<EOF -\documentclass{foils} - -\title{Presentation title} -\date{} - -\begin{document} -\maketitle -\end{document} -EOF - pdflatex test.tex - cp test.pdf $out -'' -]]></programlisting> - </para> - </section> -</section> diff --git a/infra/libkookie/nixpkgs/doc/languages-frameworks/vim.section.md b/infra/libkookie/nixpkgs/doc/languages-frameworks/vim.section.md index 8e4826232e1a..84ad567e8c21 100644 --- a/infra/libkookie/nixpkgs/doc/languages-frameworks/vim.section.md +++ b/infra/libkookie/nixpkgs/doc/languages-frameworks/vim.section.md @@ -265,6 +265,20 @@ To add a new plugin, run `./update.py --add "[owner]/[name]"`. **NOTE**: This sc Finally, there are some plugins that are also packaged in nodePackages because they have Javascript-related build steps, such as running webpack. Those plugins are not listed in `vim-plugin-names` or managed by `update.py` at all, and are included separately in `overrides.nix`. Currently, all these plugins are related to the `coc.nvim` ecosystem of Language Server Protocol integration with vim/neovim. +## Updating plugins in nixpkgs + +Run the update script with a GitHub API token that has at least `public_repo` access. Running the script without the token is likely to result in rate-limiting (429 errors). For steps on creating an API token, please refer to [GitHub's token documentation](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token). + +```sh +GITHUB_API_TOKEN=my_token ./pkgs/misc/vim-plugins/update.py +``` + +Alternatively, set the number of processes to a lower count to avoid rate-limiting. + +```sh +./pkgs/misc/vim-plugins/update.py --proc 1 +``` + ## Important repositories - [vim-pi](https://bitbucket.org/vimcommunity/vim-pi) is a plugin repository |