13 files changed, 692 insertions, 208 deletions
diff --git a/nixpkgs/doc/languages-frameworks/agda.section.md b/nixpkgs/doc/languages-frameworks/agda.section.md
index 8cba6d9faa7..9ce046d05b6 100644
--- a/nixpkgs/doc/languages-frameworks/agda.section.md
+++ b/nixpkgs/doc/languages-frameworks/agda.section.md
@@ -67,7 +67,17 @@ A derivation can then be written using `agdaPackages.mkDerivation`. This has sim
 + `libraryName` should be the name that appears in the `*.agda-lib` file, defaulting to `pname`.
 + `libraryFile` should be the file name of the `*.agda-lib` file, defaulting to `${libraryName}.agda-lib`.
 
-The build phase for `agdaPackages.mkDerivation` simply runs `agda` on the `Everything.agda` file. If something else is needed to build the package (e.g. `make`) then the `buildPhase` should be overridden (or a `preBuild` or `configurePhase` can be used if there are steps that need to be done prior to checking the `Everything.agda` file). `agda` and the Agda libraries contained in `buildInputs` are made available during the build phase. The install phase simply copies all `.agda`, `.agdai` and `.agda-lib` files to the output directory. Again, this can be overridden.
+### Building Agda packages
+The default build phase for `agdaPackages.mkDerivation` simply runs `agda` on the `Everything.agda` file.
+If something else is needed to build the package (e.g. `make`) then the `buildPhase` should be overridden.
+Additionally, a `preBuild` or `configurePhase` can be used if there are steps that need to be done prior to checking the `Everything.agda` file.
+`agda` and the Agda libraries contained in `buildInputs` are made available during the build phase.
+
+### Installing Agda packages
+The default install phase copies agda source files, agda interface files (`*.agdai`) and `*.agda-lib` files to the output directory.
+This can be overridden.
+
+By default, agda sources are files ending on  `.agda`, or literate agda files ending on `.lagda`, `.lagda.tex`, `.lagda.org`, `.lagda.md`, `.lagda.rst`. The list of recognised agda source extensions can be extended by setting the `extraExtensions` config variable.
 
 To add an agda package to `nixpkgs`, the derivation should be written to `pkgs/development/libraries/agda/${library-name}/` and an entry should be added to `pkgs/top-level/agda-packages.nix`. Here it is called in a scope with access to all other agda libraries, so the top line of the `default.nix` can look like:
 ```
diff --git a/nixpkgs/doc/languages-frameworks/android.section.md b/nixpkgs/doc/languages-frameworks/android.section.md
index 1a8924082bf..f4f6c086a80 100644
--- a/nixpkgs/doc/languages-frameworks/android.section.md
+++ b/nixpkgs/doc/languages-frameworks/android.section.md
@@ -191,6 +191,8 @@ androidenv.emulateApp {
 }
 ```
 
+Additional flags may be applied to the Android SDK's emulator through the runtime environment variable `$NIX_ANDROID_EMULATOR_FLAGS`.
+
 It is also possible to specify an APK to deploy inside the emulator
 and the package and activity names to launch it:
 
diff --git a/nixpkgs/doc/languages-frameworks/beam.xml b/nixpkgs/doc/languages-frameworks/beam.xml
index 278535237c2..1d307e1d6dc 100644
--- a/nixpkgs/doc/languages-frameworks/beam.xml
+++ b/nixpkgs/doc/languages-frameworks/beam.xml
@@ -68,7 +68,7 @@
   <title>How to Install BEAM Packages</title>
 
   <para>
-   BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. 
+   BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users.
    To install any of those builders into your profile, refer to them by their attribute path <literal>beamPackages.rebar3</literal>:
   </para>
 
diff --git a/nixpkgs/doc/languages-frameworks/emscripten.section.md b/nixpkgs/doc/languages-frameworks/emscripten.section.md
index 80e1094809a..3663f962d5f 100644
--- a/nixpkgs/doc/languages-frameworks/emscripten.section.md
+++ b/nixpkgs/doc/languages-frameworks/emscripten.section.md
@@ -21,11 +21,11 @@ Modes of use of `emscripten`:
 * **Declarative usage**:
 
     This mode is far more power full since this makes use of `nix` for dependency management of emscripten libraries and targets by using the `mkDerivation` which is implemented by `pkgs.emscriptenStdenv` and `pkgs.buildEmscriptenPackage`. The source for the packages is in `pkgs/top-level/emscripten-packages.nix` and the abstraction behind it in `pkgs/development/em-modules/generic/default.nix`.
-    * build and install all packages: 
-        * `nix-env -iA emscriptenPackages` 
-          
-    * dev-shell for zlib implementation hacking: 
-        * `nix-shell -A emscriptenPackages.zlib` 
+    * build and install all packages:
+        * `nix-env -iA emscriptenPackages`
+
+    * dev-shell for zlib implementation hacking:
+        * `nix-shell -A emscriptenPackages.zlib`
 
 
 ## Imperative usage
@@ -90,7 +90,7 @@ See the `zlib` example:
         libz.so.${old.version} -I . -o example.js
 
         echo "Using node to execute the test"
-        ${pkgs.nodejs}/bin/node ./example.js 
+        ${pkgs.nodejs}/bin/node ./example.js
 
         set +x
         if [ $? -ne 0 ]; then
@@ -112,7 +112,7 @@ See the `zlib` example:
 
 ### Usage 2: pkgs.buildEmscriptenPackage
 
-This `xmlmirror` example features a emscriptenPackage which is defined completely from this context and no `pkgs.zlib.override` is used. 
+This `xmlmirror` example features a emscriptenPackage which is defined completely from this context and no `pkgs.zlib.override` is used.
 
     xmlmirror = pkgs.buildEmscriptenPackage rec {
       name = "xmlmirror";
@@ -163,7 +163,7 @@ This `xmlmirror` example features a emscriptenPackage which is defined completel
       checkPhase = ''
 
       '';
-    }; 
+    };
 
 ### Declarative debugging
 
@@ -182,4 +182,3 @@ Use `nix-shell -I nixpkgs=/some/dir/nixpkgs -A emscriptenPackages.libz` and from
 Using this toolchain makes it easy to leverage `nix` from NixOS, MacOSX or even Windows (WSL+ubuntu+nix). This toolchain is reproducible, behaves like the rest of the packages from nixpkgs and contains a set of well working examples to learn and adapt from.
 
 If in trouble, ask the maintainers.
-
diff --git a/nixpkgs/doc/languages-frameworks/go.xml b/nixpkgs/doc/languages-frameworks/go.xml
index ff39276f640..7cff7a85c62 100644
--- a/nixpkgs/doc/languages-frameworks/go.xml
+++ b/nixpkgs/doc/languages-frameworks/go.xml
@@ -40,6 +40,10 @@ pet = buildGoModule rec {
 
   subPackages = [ "." ]; <co xml:id='ex-buildGoModule-2' />
 
+  deleteVendor = true; <co xml:id='ex-buildGoModule-3' />
+
+  runVend = true; <co xml:id='ex-buildGoModule-4' />
+
   meta = with lib; {
     description = "Simple command-line snippet manager, written in Go";
     homepage = "https://github.com/knqyf263/pet";
@@ -64,6 +68,16 @@ pet = buildGoModule rec {
       <varname>subPackages</varname> limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built.
      </para>
     </callout>
+    <callout arearefs='ex-buildGoModule-3'>
+     <para>
+      <varname>deleteVendor</varname> removes the pre-existing vendor directory and fetches the dependencies. This should only be used if the dependencies included in the vendor folder are broken or incomplete.
+     </para>
+    </callout>
+    <callout arearefs='ex-buildGoModule-4'>
+     <para>
+      <varname>runVend</varname> runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build.
+     </para>
+    </callout>
    </calloutlist>
   </para>
 
@@ -72,7 +86,7 @@ pet = buildGoModule rec {
 
     When `null` is used as a value, rather than fetching the dependencies
     and vendoring them, we use the vendoring included within the source repo.
-    If you'd like to not have to update this field on dependency changes, 
+    If you'd like to not have to update this field on dependency changes,
     run `go mod vendor` in your source repo and set 'vendorSha256 = null;'
   </para>
  </section>
@@ -103,7 +117,9 @@ deis = buildGoPackage rec {
 
   goDeps = ./deps.nix; <co xml:id='ex-buildGoPackage-3' />
 
-  buildFlags = [ "--tags" "release" ]; <co xml:id='ex-buildGoPackage-4' />
+  deleteVendor = true; <co xml:id='ex-buildGoPackage-4' />
+
+  buildFlags = [ "--tags" "release" ]; <co xml:id='ex-buildGoPackage-5' />
 }
 </programlisting>
   </example>
@@ -131,6 +147,11 @@ deis = buildGoPackage rec {
     </callout>
     <callout arearefs='ex-buildGoPackage-4'>
      <para>
+      <varname>deleteVendor</varname> removes the pre-existing vendor directory. This should only be used if the dependencies included in the vendor folder are broken or incomplete.
+     </para>
+    </callout>
+    <callout arearefs='ex-buildGoPackage-5'>
+     <para>
       <varname>buildFlags</varname> is a list of flags passed to the go build command.
      </para>
     </callout>
diff --git a/nixpkgs/doc/languages-frameworks/index.xml b/nixpkgs/doc/languages-frameworks/index.xml
index 2414956995c..728a38c264a 100644
--- a/nixpkgs/doc/languages-frameworks/index.xml
+++ b/nixpkgs/doc/languages-frameworks/index.xml
@@ -18,7 +18,7 @@
  <xi:include href="idris.section.xml" />
  <xi:include href="ios.section.xml" />
  <xi:include href="java.xml" />
- <xi:include href="lua.xml" />
+ <xi:include href="lua.section.xml" />
  <xi:include href="node.section.xml" />
  <xi:include href="ocaml.xml" />
  <xi:include href="perl.xml" />
diff --git a/nixpkgs/doc/languages-frameworks/lua.section.md b/nixpkgs/doc/languages-frameworks/lua.section.md
new file mode 100644
index 00000000000..a0e9917b8ec
--- /dev/null
+++ b/nixpkgs/doc/languages-frameworks/lua.section.md
@@ -0,0 +1,252 @@
+---
+title: Lua
+author: Matthieu Coudron
+date: 2019-02-05
+---
+
+# User's Guide to Lua Infrastructure
+
+## Using Lua
+
+### Overview of Lua
+
+Several versions of the Lua interpreter are available: luajit, lua 5.1, 5.2, 5.3.
+The attribute `lua` refers to the default interpreter, it is also possible to refer to specific versions, e.g. `lua5_2` refers to Lua 5.2.
+
+Lua libraries are in separate sets, with one set per interpreter version.
+
+The interpreters have several common attributes. One of these attributes is
+`pkgs`, which is a package set of Lua libraries for this specific
+interpreter. E.g., the `busted` package corresponding to the default interpreter
+is `lua.pkgs.busted`, and the lua 5.2 version is `lua5_2.pkgs.busted`.
+The main package set contains aliases to these package sets, e.g.
+`luaPackages` refers to `lua5_1.pkgs` and `lua52Packages` to
+`lua5_2.pkgs`.
+
+### Installing Lua and packages
+
+#### Lua environment defined in separate `.nix` file
+
+Create a file, e.g. `build.nix`, with the following expression
+```nix
+with import <nixpkgs> {};
+
+lua5_2.withPackages (ps: with ps; [ busted luafilesystem ])
+```
+and install it in your profile with
+```shell
+nix-env -if build.nix
+```
+Now you can use the Lua interpreter, as well as the extra packages (`busted`,
+`luafilesystem`) that you added to the environment.
+
+#### Lua environment defined in `~/.config/nixpkgs/config.nix`
+
+If you prefer to, you could also add the environment as a package override to the Nixpkgs set, e.g.
+using `config.nix`,
+```nix
+{ # ...
+
+  packageOverrides = pkgs: with pkgs; {
+    myLuaEnv = lua5_2.withPackages (ps: with ps; [ busted luafilesystem ]);
+  };
+}
+```
+and install it in your profile with
+```shell
+nix-env -iA nixpkgs.myLuaEnv
+```
+The environment is is installed by referring to the attribute, and considering
+the `nixpkgs` channel was used.
+
+#### Lua environment defined in `/etc/nixos/configuration.nix`
+
+For the sake of completeness, here's another example how to install the environment system-wide.
+
+```nix
+{ # ...
+
+  environment.systemPackages = with pkgs; [
+    (lua.withPackages(ps: with ps; [ busted luafilesystem ]))
+  ];
+}
+```
+
+### How to override a Lua package using overlays?
+
+Use the following overlay template:
+
+```nix
+final: prev:
+{
+
+  lua = prev.lua.override {
+    packageOverrides = luaself: luaprev: {
+
+      luarocks-nix = luaprev.luarocks-nix.overrideAttrs(oa: {
+        pname = "luarocks-nix";
+        src = /home/my_luarocks/repository;
+      });
+  };
+
+  luaPackages = lua.pkgs;
+}
+```
+
+### Temporary Lua environment with `nix-shell`
+
+
+There are two methods for loading a shell with Lua packages. The first and recommended method
+is to create an environment with `lua.buildEnv` or `lua.withPackages` and load that. E.g.
+```sh
+$ nix-shell -p 'lua.withPackages(ps: with ps; [ busted luafilesystem ])'
+```
+opens a shell from which you can launch the interpreter
+```sh
+[nix-shell:~] lua
+```
+The other method, which is not recommended, does not create an environment and requires you to list the packages directly,
+
+```sh
+$ nix-shell -p lua.pkgs.busted lua.pkgs.luafilesystem
+```
+Again, it is possible to launch the interpreter from the shell.
+The Lua interpreter has the attribute `pkgs` which contains all Lua libraries for that specific interpreter.
+
+
+## Developing with Lua
+
+Now that you know how to get a working Lua environment with Nix, it is time
+to go forward and start actually developing with Lua. There are two ways to
+package lua software, either it is on luarocks and most of it can be taken care
+of by the luarocks2nix converter or the packaging has to be done manually.
+Let's present the luarocks way first and the manual one in a second time.
+
+### Packaging a library on luarocks
+
+[Luarocks.org](www.luarocks.org) is the main repository of lua packages.
+The site proposes two types of packages, the rockspec and the src.rock
+(equivalent of a [rockspec](https://github.com/luarocks/luarocks/wiki/Rockspec-format) but with the source).
+These packages can have different build types such as `cmake`, `builtin` etc .
+
+Luarocks-based packages are generated in pkgs/development/lua-modules/generated-packages.nix from
+the whitelist maintainers/scripts/luarocks-packages.csv and updated by running maintainers/scripts/update-luarocks-packages.
+
+[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock).
+The automation only goes so far though and some packages need to be customized.
+These customizations go in `pkgs/development/lua-modules/overrides.nix`.
+For instance if the rockspec defines `external_dependencies`, these need to be manually added in in its rockspec file then it won't work.
+
+You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`.
+Nix rely on luarocks to install lua packages, basically it runs:
+`luarocks make --deps-mode=none --tree $out`
+
+#### Packaging a library manually
+
+You can develop your package as you usually would, just don't forget to wrap it
+within a `toLuaModule` call, for instance
+```nix
+mynewlib = toLuaModule ( stdenv.mkDerivation { ... });
+```
+
+There is also the `buildLuaPackage` function that can be used when lua modules
+are not packaged for luarocks. You can see a few examples at `pkgs/top-level/lua-packages.nix`.
+
+## Lua Reference
+
+### Lua interpreters
+
+Versions 5.1, 5.2 and 5.3 of the lua interpreter are available as
+respectively `lua5_1`, `lua5_2` and `lua5_3`. Luajit is available too.
+The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`.
+
+
+#### Attributes on lua interpreters packages
+
+Each interpreter has the following attributes:
+
+- `interpreter`. Alias for `${pkgs.lua}/bin/lua`.
+- `buildEnv`. Function to build lua interpreter environments with extra packages bundled together. See section *lua.buildEnv function* for usage and documentation.
+- `withPackages`. Simpler interface to `buildEnv`.
+- `pkgs`. Set of Lua packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
+
+
+#### `buildLuarocksPackage` function
+
+The `buildLuarocksPackage` function is implemented in `pkgs/development/interpreters/lua-5/build-lua-package.nix`
+The following is an example:
+```nix
+luaposix = buildLuarocksPackage {
+  pname = "luaposix";
+  version = "34.0.4-1";
+
+  src = fetchurl {
+    url    = "https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/luaposix-34.0.4-1.src.rock";
+    sha256 = "0yrm5cn2iyd0zjd4liyj27srphvy0gjrjx572swar6zqr4dwjqp2";
+  };
+  disabled = (luaOlder "5.1") || (luaAtLeast "5.4");
+  propagatedBuildInputs = [ bit32 lua std_normalize ];
+
+  meta = with stdenv.lib; {
+    homepage = "https://github.com/luaposix/luaposix/";
+    description = "Lua bindings for POSIX";
+    maintainers = with maintainers; [ vyp lblasc ];
+    license.fullName = "MIT/X11";
+  };
+};
+```
+
+The `buildLuarocksPackage` delegates most tasks to luarocks:
+
+* it adds `luarocks` as an unpacker for `src.rock` files (zip files really).
+* configurePhase` writes a temporary luarocks configuration file which location
+is exported via the environment variable `LUAROCKS_CONFIG`.
+* the `buildPhase` does nothing.
+* `installPhase` calls `luarocks make --deps-mode=none --tree $out` to build and
+install the package
+* In the `postFixup` phase, the `wrapLuaPrograms` bash function is called to
+  wrap all programs in the `$out/bin/*` directory to include `$PATH`
+  environment variable and add dependent libraries to script's `LUA_PATH` and
+  `LUA_CPATH`.
+
+By default `meta.platforms` is set to the same value as the interpreter unless overridden otherwise.
+
+#### `buildLuaApplication` function
+
+The `buildLuaApplication` function is practically the same as `buildLuaPackage`.
+The difference is that `buildLuaPackage` by default prefixes the names of the packages with the version of the interpreter.
+Because with an application we're not interested in multiple version the prefix is dropped.
+
+#### lua.withPackages function
+
+The `lua.withPackages` takes a function as an argument that is passed the set of lua packages and returns the list of packages to be included in the environment.
+Using the `withPackages` function, the previous example for the luafilesystem environment can be written like this:
+```nix
+with import <nixpkgs> {};
+
+lua.withPackages (ps: [ps.luafilesystem])
+```
+
+`withPackages` passes the correct package set for the specific interpreter version as an argument to the function. In the above example, `ps` equals `luaPackages`.
+But you can also easily switch to using `lua5_2`:
+```nix
+with import <nixpkgs> {};
+
+lua5_2.withPackages (ps: [ps.lua])
+```
+
+Now, `ps` is set to `lua52Packages`, matching the version of the interpreter.
+
+
+### Possible Todos
+
+* export/use version specific variables such as `LUA_PATH_5_2`/`LUAROCKS_CONFIG_5_2`
+* let luarocks check for dependencies via exporting the different rocktrees in temporary config
+
+### Lua Contributing guidelines
+
+Following rules should be respected:
+
+* Make sure libraries build for all Lua interpreters.
+* Commit names of Lua libraries should reflect that they are Lua libraries, so write for example `luaPackages.luafilesystem: 1.11 -> 1.12`.
+
diff --git a/nixpkgs/doc/languages-frameworks/lua.xml b/nixpkgs/doc/languages-frameworks/lua.xml
deleted file mode 100644
index bcca6b73753..00000000000
--- a/nixpkgs/doc/languages-frameworks/lua.xml
+++ /dev/null
@@ -1,36 +0,0 @@
-<section xmlns="http://docbook.org/ns/docbook"
-         xmlns:xlink="http://www.w3.org/1999/xlink"
-         xml:id="sec-language-lua">
- <title>Lua</title>
-
- <para>
-  Lua packages are built by the <varname>buildLuaPackage</varname> function. This function is implemented in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules/generic/default.nix"> <filename>pkgs/development/lua-modules/generic/default.nix</filename></link> and works similarly to <varname>buildPerlPackage</varname>. (See <xref linkend="sec-language-perl"/> for details.)
- </para>
-
- <para>
-  Lua packages are defined in <link xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/lua-packages.nix"><filename>pkgs/top-level/lua-packages.nix</filename></link>. Most of them are simple. For example:
-<programlisting>
-fileSystem = buildLuaPackage {
-  name = "filesystem-1.6.2";
-  src = fetchurl {
-    url = "https://github.com/keplerproject/luafilesystem/archive/v1_6_2.tar.gz";
-    sha256 = "1n8qdwa20ypbrny99vhkmx8q04zd2jjycdb5196xdhgvqzk10abz";
-  };
-  meta = {
-    homepage = "https://github.com/keplerproject/luafilesystem";
-    hydraPlatforms = stdenv.lib.platforms.linux;
-    maintainers = with maintainers; [ flosse ];
-  };
-};
-</programlisting>
- </para>
-
- <para>
-  Though, more complicated package should be placed in a seperate file in <link
-  xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules"><filename>pkgs/development/lua-modules</filename></link>.
- </para>
-
- <para>
-  Lua packages accept additional parameter <varname>disabled</varname>, which defines the condition of disabling package from luaPackages. For example, if package has <varname>disabled</varname> assigned to <literal>lua.luaversion != "5.1"</literal>, it will not be included in any luaPackages except lua51Packages, making it only be built for lua 5.1.
- </para>
-</section>
diff --git a/nixpkgs/doc/languages-frameworks/node.section.md b/nixpkgs/doc/languages-frameworks/node.section.md
index c1f4294711a..2120adfc0b4 100644
--- a/nixpkgs/doc/languages-frameworks/node.section.md
+++ b/nixpkgs/doc/languages-frameworks/node.section.md
@@ -25,12 +25,13 @@ build system it uses. Here are some examples:
 
 After you have identified the correct system, you need to override your package
 expression while adding in build system as a build input. For example, `dat`
-requires `node-gyp-build`, so we override its expression in `default.nix`:
+requires `node-gyp-build`, so [we override](https://github.com/NixOS/nixpkgs/blob/32f5e5da4a1b3f0595527f5195ac3a91451e9b56/pkgs/development/node-packages/default.nix#L37-L40) its expression in [`default.nix`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/default.nix):
 
 ```nix
-dat = nodePackages.dat.override (oldAttrs: {
-  buildInputs = oldAttrs.buildInputs ++ [ nodePackages.node-gyp-build ];
-});
+    dat = super.dat.override {
+      buildInputs = [ self.node-gyp-build pkgs.libtool pkgs.autoconf pkgs.automake ];
+      meta.broken = since "12";
+    };
 ```
 
 To add a package from NPM to nixpkgs:
diff --git a/nixpkgs/doc/languages-frameworks/perl.xml b/nixpkgs/doc/languages-frameworks/perl.xml
index d9b6b2721c6..ff0f350e99c 100644
--- a/nixpkgs/doc/languages-frameworks/perl.xml
+++ b/nixpkgs/doc/languages-frameworks/perl.xml
@@ -3,159 +3,193 @@
          xml:id="sec-language-perl">
  <title>Perl</title>
 
- <para>
-  Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link
-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic"><filename>pkgs/development/perl-modules/generic</filename></link>.
- </para>
-
- <para>
-  Perl packages from CPAN are defined in <link
-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former:
-<programlisting>
-ClassC3 = buildPerlPackage rec {
-  name = "Class-C3-0.21";
-  src = fetchurl {
-    url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz";
-    sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz";
-  };
-};
-</programlisting>
-  Note the use of <literal>mirror://cpan/</literal>, and the <literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write
-<programlisting>
-foo = import ../path/to/foo.nix {
-  inherit stdenv fetchurl ...;
-  inherit (perlPackages) ClassC3;
-};
-</programlisting>
-  in <filename>all-packages.nix</filename>. You can test building a Perl package as follows:
-<screen>
-<prompt>$ </prompt>nix-build -A perlPackages.ClassC3
-</screen>
-  <varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
-<screen>
-<prompt>$ </prompt>nix-env -i perl-Class-C3
-</screen>
-  (Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.)
- </para>
-
- <para>
-  So what does <varname>buildPerlPackage</varname> do? It does the following:
-  <orderedlist>
-   <listitem>
-    <para>
-     In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename>
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.)
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies.
-    </para>
-   </listitem>
-  </orderedlist>
- </para>
-
- <para>
-  <varname>buildPerlPackage</varname> is built on top of <varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>:
-<programlisting>
-{ buildPerlPackage, fetchurl, db }:
-
-buildPerlPackage rec {
-  name = "BerkeleyDB-0.36";
-
-  src = fetchurl {
-    url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz";
-    sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1";
-  };
-
-  preConfigure = ''
-    echo "LIB = ${db.out}/lib" > config.in
-    echo "INCLUDE = ${db.dev}/include" >> config.in
-  '';
-}
-</programlisting>
- </para>
-
- <para>
-  Dependencies on other Perl packages can be specified in the <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules:
-<programlisting>
-ClassC3Componentised = buildPerlPackage rec {
-  name = "Class-C3-Componentised-1.0004";
-  src = fetchurl {
-    url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz";
-    sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1";
-  };
-  propagatedBuildInputs = [
-    ClassC3 ClassInspector TestException MROCompat
-  ];
-};
-</programlisting>
- </para>
-
- <para>
-  On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase:
-<programlisting>
-{ stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }:
-
-ImageExifTool = buildPerlPackage {
-  pname = "Image-ExifTool";
-  version = "11.50";
-
-  src = fetchurl {
-    url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz";
-    sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3";
-  };
-
-  buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang;
-  postInstall = stdenv.lib.optional stdenv.isDarwin ''
-    shortenPerlShebang $out/bin/exiftool
-  '';
-};
-</programlisting>
-  This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place.
- </para>
-
- <section xml:id="ssec-generation-from-CPAN">
-  <title>Generation from CPAN</title>
+ <section xml:id="ssec-perl-running">
+  <title>Running perl programs on the shell</title>
 
   <para>
-   Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows:
+   When executing a Perl script, it is possible you get an error such as <literal>./myscript.pl: bad interpreter: /usr/bin/perl: no such file or directory</literal>. This happens when the script expects Perl to be installed at <filename>/usr/bin/perl</filename>, which is not the case when using Perl from nixpkgs. You can fix the script by changing the first line to:
+  <programlisting>
+  #!/usr/bin/env perl
+  </programlisting>
+  to take the Perl installation from the <literal>PATH</literal> environment variable, or invoke Perl directly with:
+  <screen>
+  <prompt>$ </prompt>perl ./myscript.pl
+  </screen>
   </para>
 
-<screen>
-<prompt>$ </prompt>nix-env -i nix-generate-from-cpan
-</screen>
+  <para>
+   When the script is using a Perl library that is not installed globally, you might get an error such as <literal>Can't locate DB_File.pm in @INC (you may need to install the DB_File module)</literal>. In that case, you can use <command>nix-shell</command> to start an ad-hoc shell with that library installed, for instance:
+  <screen>
+  <prompt>$ </prompt>nix-shell -p perl perlPackages.DBFile --run ./myscript.pl
+  </screen>
+  </para>
 
   <para>
-   This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example:
-<screen>
-<prompt>$ </prompt>nix-generate-from-cpan XML::Simple
-  XMLSimple = buildPerlPackage rec {
-    name = "XML-Simple-2.22";
-    src = fetchurl {
-      url = "mirror://cpan/authors/id/G/GR/GRANTM/${name}.tar.gz";
-      sha256 = "b9450ef22ea9644ae5d6ada086dc4300fa105be050a2030ebd4efd28c198eb49";
-    };
-    propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ];
-    meta = {
-      description = "An API for simple XML files";
-      license = with stdenv.lib.licenses; [ artistic1 gpl1Plus ];
-    };
-  };
-</screen>
-   The output can be pasted into <filename>pkgs/top-level/perl-packages.nix</filename> or wherever else you need it.
+  If you are always using the script in places where <command>nix-shell</command> is available, you can embed the <command>nix-shell</command> invocation in the shebang like this:
+  <programlisting>
+  #!/usr/bin/env nix-shell
+  #! nix-shell -i perl -p perl perlPackages.DBFile
+  </programlisting>
   </para>
  </section>
 
- <section xml:id="ssec-perl-cross-compilation">
-  <title>Cross-compiling modules</title>
+ <section xml:id="ssec-perl-packaging">
+  <title>Packaging Perl programs</title>
+
+  <para>
+   Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link
+ xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/perl-modules/generic"><filename>pkgs/development/perl-modules/generic</filename></link>.
+  </para>
+
+  <para>
+   Perl packages from CPAN are defined in <link
+ xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former:
+ <programlisting>
+ ClassC3 = buildPerlPackage rec {
+   name = "Class-C3-0.21";
+   src = fetchurl {
+     url = "mirror://cpan/authors/id/F/FL/FLORA/${name}.tar.gz";
+     sha256 = "1bl8z095y4js66pwxnm7s853pi9czala4sqc743fdlnk27kq94gz";
+   };
+ };
+ </programlisting>
+   Note the use of <literal>mirror://cpan/</literal>, and the <literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write
+ <programlisting>
+ foo = import ../path/to/foo.nix {
+   inherit stdenv fetchurl ...;
+   inherit (perlPackages) ClassC3;
+ };
+ </programlisting>
+   in <filename>all-packages.nix</filename>. You can test building a Perl package as follows:
+ <screen>
+ <prompt>$ </prompt>nix-build -A perlPackages.ClassC3
+ </screen>
+   <varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
+ <screen>
+ <prompt>$ </prompt>nix-env -i perl-Class-C3
+ </screen>
+   (Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.)
+  </para>
+
+  <para>
+   So what does <varname>buildPerlPackage</varname> do? It does the following:
+   <orderedlist>
+    <listitem>
+     <para>
+      In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename>
+     </para>
+    </listitem>
+    <listitem>
+     <para>
+      It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.)
+     </para>
+    </listitem>
+    <listitem>
+     <para>
+      In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies.
+     </para>
+    </listitem>
+   </orderedlist>
+  </para>
+
+  <para>
+   <varname>buildPerlPackage</varname> is built on top of <varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>:
+ <programlisting>
+ { buildPerlPackage, fetchurl, db }:
+
+ buildPerlPackage rec {
+   name = "BerkeleyDB-0.36";
+
+   src = fetchurl {
+     url = "mirror://cpan/authors/id/P/PM/PMQS/${name}.tar.gz";
+     sha256 = "07xf50riarb60l1h6m2dqmql8q5dij619712fsgw7ach04d8g3z1";
+   };
+
+   preConfigure = ''
+     echo "LIB = ${db.out}/lib" > config.in
+     echo "INCLUDE = ${db.dev}/include" >> config.in
+   '';
+ }
+ </programlisting>
+  </para>
 
   <para>
-   Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example.
+   Dependencies on other Perl packages can be specified in the <varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules:
+ <programlisting>
+ ClassC3Componentised = buildPerlPackage rec {
+   name = "Class-C3-Componentised-1.0004";
+   src = fetchurl {
+     url = "mirror://cpan/authors/id/A/AS/ASH/${name}.tar.gz";
+     sha256 = "0xql73jkcdbq4q9m0b0rnca6nrlvf5hyzy8is0crdk65bynvs8q1";
+   };
+   propagatedBuildInputs = [
+     ClassC3 ClassInspector TestException MROCompat
+   ];
+ };
+ </programlisting>
   </para>
+
+  <para>
+   On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase:
+ <programlisting>
+ { stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }:
+
+ ImageExifTool = buildPerlPackage {
+   pname = "Image-ExifTool";
+   version = "11.50";
+
+   src = fetchurl {
+     url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz";
+     sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3";
+   };
+
+   buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang;
+   postInstall = stdenv.lib.optional stdenv.isDarwin ''
+     shortenPerlShebang $out/bin/exiftool
+   '';
+ };
+ </programlisting>
+   This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place.
+  </para>
+
+  <section xml:id="ssec-generation-from-CPAN">
+   <title>Generation from CPAN</title>
+
+   <para>
+    Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows:
+   </para>
+
+ <screen>
+ <prompt>$ </prompt>nix-env -i nix-generate-from-cpan
+ </screen>
+
+   <para>
+    This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example:
+ <screen>
+ <prompt>$ </prompt>nix-generate-from-cpan XML::Simple
+   XMLSimple = buildPerlPackage rec {
+     name = "XML-Simple-2.22";
+     src = fetchurl {
+       url = "mirror://cpan/authors/id/G/GR/GRANTM/${name}.tar.gz";
+       sha256 = "b9450ef22ea9644ae5d6ada086dc4300fa105be050a2030ebd4efd28c198eb49";
+     };
+     propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ];
+     meta = {
+       description = "An API for simple XML files";
+       license = with stdenv.lib.licenses; [ artistic1 gpl1Plus ];
+     };
+   };
+ </screen>
+    The output can be pasted into <filename>pkgs/top-level/perl-packages.nix</filename> or wherever else you need it.
+   </para>
+  </section>
+
+  <section xml:id="ssec-perl-cross-compilation">
+   <title>Cross-compiling modules</title>
+
+   <para>
+    Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example.
+   </para>
+  </section>
  </section>
 </section>
diff --git a/nixpkgs/doc/languages-frameworks/python.section.md b/nixpkgs/doc/languages-frameworks/python.section.md
index 838426afa04..f189ce31448 100644
--- a/nixpkgs/doc/languages-frameworks/python.section.md
+++ b/nixpkgs/doc/languages-frameworks/python.section.md
@@ -538,8 +538,123 @@ buildPythonPackage rec {
 ```
 Note also the line `doCheck = false;`, we explicitly disabled running the test-suite.
 
+#### Testing Python Packages
+
+It is highly encouraged to have testing as part of the package build. This
+helps to avoid situations where the package was able to build and install,
+but is not usable at runtime. Currently, all packages will use the `test`
+command provided by the setup.py (i.e. `python setup.py test`). However,
+this is currently deprecated https://github.com/pypa/setuptools/pull/1878
+and your package should provide its own checkPhase.
+
+*NOTE:* The `checkPhase` for python maps to the `installCheckPhase` on a
+normal derivation. This is due to many python packages not behaving well
+to the pre-installed version of the package. Version info, and natively
+compiled extensions generally only exist in the install directory, and
+thus can cause issues when a test suite asserts on that behavior.
+
+*NOTE:* Tests should only be disabled if they don't agree with nix
+(e.g. external dependencies, network access, flakey tests), however,
+as many tests should be enabled as possible. Failing tests can still be
+a good indication that the package is not in a valid state.
+
+#### Using pytest
+
+Pytest is the most common test runner for python repositories. A trivial
+test run would be:
+```
+  checkInputs = [ pytest ];
+  checkPhase = "pytest";
+```
+
+However, many repositories' test suites do not translate well to nix's build 
+sandbox, and will generally need many tests to be disabled.
+
+To filter tests using pytest, one can do the following:
+```
+  checkInputs = [ pytest ];
+  # avoid tests which need additional data or touch network
+  checkPhase = ''
+    pytest tests/ --ignore=tests/integration -k 'not download and not update'
+  '';
+```
+
+`--ignore` will tell pytest to ignore that file or directory from being
+collected as part of a test run. This is useful is a file uses a package
+which is not available in nixpkgs, thus skipping that test file is much
+easier than having to create a new package.
+
+`-k` is used to define a predicate for test names. In this example, we are
+filtering out tests which contain `download` or `update` in their test case name.
+Only one `-k` argument is allows, and thus a long predicate should be concatenated
+with "\" and wrapped to the next line.
+
+*NOTE:* In pytest==6.0.1, the use of "\" to continue a line (e.g. `-k 'not download \'`) has
+been removed, in this case, it's recommended to use `pytestCheckHook`.
+
+#### Using pytestCheckHook
+
+`pytestCheckHook` is a convenient hook which will substitute the setuptools
+`test` command for a checkPhase which runs `pytest`. This is also beneficial
+when a package may need many items disabled to run the test suite.
+
+Using the example above, the analagous pytestCheckHook usage would be:
+```
+  checkInputs = [ pytestCheckHook ];
+
+  # requires additional data
+  pytestFlagsArray = [ "tests/" "--ignore=tests/integration" ];
+
+  disabledTests = [
+    # touches network
+    "download"
+    "update"
+  ];
+```
+
+This is expecially useful when tests need to be conditionallydisabled,
+for example:
+
+```
+  disabledTests = [
+    # touches network
+    "download"
+    "update"
+  ] ++ lib.optionals (pythonAtLeast "3.8") [
+    # broken due to python3.8 async changes
+    "async"
+  ] ++ lib.optionals stdenv.isDarwin [
+    # can fail when building with other packages
+    "socket"
+  ];
+```
+Trying to concatenate the related strings to disable tests in a regular checkPhase
+would be much harder to read. This also enables us to comment on why specific tests
+are disabled.
+
+#### Using pythonImportsCheck
+
+Although unit tests are highly prefered to valid correctness of a package. Not
+all packages have test suites that can be ran easily, and some have none at all.
+To help ensure the package still works, `pythonImportsCheck` can attempt to import
+the listed modules.
 
-#### Develop local package
+```
+  pythonImportsCheck = [ "requests" "urllib" ];
+```
+roughly translates to:
+```
+  postCheck = ''
+    PYTHONPATH=$out/${python.sitePackages}:$PYTHONPATH
+    python -c "import requests; import urllib"
+  '';
+```
+However, this is done in it's own phase, and not dependent on whether `doCheck = true;`
+
+This can also be useful in verifying that the package doesn't assume commonly
+present packages (e.g. `setuptools`)
+
+### Develop local package
 
 As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode)
 (`python setup.py develop`); instead of installing the package this command
@@ -640,10 +755,10 @@ and in this case the `python38` interpreter is automatically used.
 
 ### Interpreters
 
-Versions 2.7, 3.5, 3.6, 3.7 and 3.8 of the CPython interpreter are available as
-respectively `python27`, `python35`, `python36`, `python37` and `python38`. The
+Versions 2.7, 3.6, 3.7 and 3.8 of the CPython interpreter are available as
+respectively `python27`, `python36`, `python37` and `python38`. The
 aliases `python2` and `python3` correspond to respectively `python27` and
-`python37`. The default interpreter, `python`, maps to `python2`. The PyPy
+`python38`. The default interpreter, `python`, maps to `python2`. The PyPy
 interpreters compatible with Python 2.7 and 3 are available as `pypy27` and
 `pypy3`, with aliases `pypy2` mapping to `pypy27` and `pypy` mapping to `pypy2`.
 The Nix expressions for the interpreters can be found in
@@ -689,15 +804,16 @@ attribute set is created for each available Python interpreter. The available
 sets are
 
 * `pkgs.python27Packages`
-* `pkgs.python35Packages`
 * `pkgs.python36Packages`
 * `pkgs.python37Packages`
+* `pkgs.python38Packages`
+* `pkgs.python39Packages`
 * `pkgs.pypyPackages`
 
 and the aliases
 
 * `pkgs.python2Packages` pointing to `pkgs.python27Packages`
-* `pkgs.python3Packages` pointing to `pkgs.python37Packages`
+* `pkgs.python3Packages` pointing to `pkgs.python38Packages`
 * `pkgs.pythonPackages` pointing to `pkgs.python2Packages`
 
 #### `buildPythonPackage` function
@@ -764,9 +880,6 @@ following are specific to `buildPythonPackage`:
 * `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
 * `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
   variable in wrapped programs.
-* `installFlags ? []`: A list of strings. Arguments to be passed to `pip
-  install`. To pass options to `python setup.py install`, use
-  `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"]`.
 * `format ? "setuptools"`: Format of the source. Valid options are
   `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`.
   `"setuptools"` is for when the source has a `setup.py` and `setuptools` is
@@ -782,6 +895,9 @@ following are specific to `buildPythonPackage`:
 * `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this
   defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications
   to `""`.
+* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip
+  install`. To pass options to `python setup.py install`, use
+  `--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`.
 * `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages
   in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`).
 * `preShellHook`: Hook to execute commands before `shellHook`.
@@ -1016,7 +1132,7 @@ are used in `buildPythonPackage`.
 - `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system
   (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
 - `pipInstallHook` to install wheels.
-- `pytestCheckHook` to run tests with `pytest`.
+- `pytestCheckHook` to run tests with `pytest`. See [example usage](#using-pytestcheckhook).
 - `pythonCatchConflictsHook` to check whether a Python package is not already existing.
 - `pythonImportsCheckHook` to check whether importing the listed modules works.
 - `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
diff --git a/nixpkgs/doc/languages-frameworks/rust.section.md b/nixpkgs/doc/languages-frameworks/rust.section.md
index cec3373cbee..7f9d93216ed 100644
--- a/nixpkgs/doc/languages-frameworks/rust.section.md
+++ b/nixpkgs/doc/languages-frameworks/rust.section.md
@@ -43,7 +43,6 @@ rustPlatform.buildRustPackage rec {
     homepage = "https://github.com/BurntSushi/ripgrep";
     license = licenses.unlicense;
     maintainers = [ maintainers.tailhook ];
-    platforms = platforms.all;
   };
 }
 ```
@@ -51,7 +50,7 @@ rustPlatform.buildRustPackage rec {
 `buildRustPackage` requires a `cargoSha256` attribute which is computed over
 all crate sources of this package. Currently it is obtained by inserting a
 fake checksum into the expression and building the package once. The correct
-checksum can be then take from the failed build.
+checksum can then be taken from the failed build.
 
 Per the instructions in the [Cargo Book](https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html)
 best practices guide, Rust applications should always commit the `Cargo.lock`
@@ -75,6 +74,90 @@ pkgs.rustPlatform.buildRustPackage {
 }
 ```
 
+### Running package tests
+
+When using `buildRustPackage`, the `checkPhase` is enabled by default and runs
+`cargo test` on the package to build. To make sure that we don't compile the
+sources twice and to actually test the artifacts that will be used at runtime,
+the tests will be ran in the `release` mode by default.
+
+However, in some cases the test-suite of a package doesn't work properly in the
+`release` mode. For these situations, the mode for `checkPhase` can be changed like
+so:
+
+```nix
+rustPlatform.buildRustPackage {
+  /* ... */
+  checkType = "debug";
+}
+```
+
+Please note that the code will be compiled twice here: once in `release` mode
+for the `buildPhase`, and again in `debug` mode for the `checkPhase`.
+
+#### Tests relying on the structure of the `target/` directory
+
+Some tests may rely on the structure of the `target/` directory. Those tests
+are likely to fail because we use `cargo --target` during the build. This means that
+the artifacts
+[are stored in `target/<architecture>/release/`](https://doc.rust-lang.org/cargo/guide/build-cache.html),
+rather than in `target/release/`.
+
+This can only be worked around by patching the affected tests accordingly.
+
+#### Disabling package-tests
+
+In some instances, it may be necessary to disable testing altogether (with `doCheck = false;`):
+
+* If no tests exist -- the `checkPhase` should be explicitly disabled to skip
+  unnecessary build steps to speed up the build.
+* If tests are highly impure (e.g. due to network usage).
+
+There will obviously be some corner-cases not listed above where it's sensible to disable tests.
+The above are just guidelines, and exceptions may be granted on a case-by-case basis.
+
+However, please check if it's possible to disable a problematic subset of the
+test suite and leave a comment explaining your reasoning.
+
+### Building a package in `debug` mode
+
+By default, `buildRustPackage` will use `release` mode for builds. If a package
+should be built in `debug` mode, it can be configured like so:
+
+```nix
+rustPlatform.buildRustPackage {
+  /* ... */
+  buildType = "debug";
+}
+```
+
+In this scenario, the `checkPhase` will be ran in `debug` mode as well.
+
+### Custom `build`/`install`-procedures
+
+Some packages may use custom scripts for building/installing, e.g. with a `Makefile`.
+In these cases, it's recommended to override the `buildPhase`/`installPhase`/`checkPhase`.
+
+Otherwise, some steps may fail because of the modified directory structure of `target/`.
+
+### Building a crate with an absent or out-of-date Cargo.lock file
+
+`buildRustPackage` needs a `Cargo.lock` file to get all dependencies in the
+source code in a reproducible way. If it is missing or out-of-date one can use
+the `cargoPatches` attribute to update or add it.
+
+```
+{ lib, rustPlatform, fetchFromGitHub }:
+
+rustPlatform.buildRustPackage rec {
+  (...)
+  cargoPatches = [
+    # a patch file to add/update Cargo.lock in the source code
+    ./add-Cargo.lock.patch
+  ];
+}
+```
+
 ## Compiling Rust crates using Nix instead of Cargo
 
 ### Simple operation
diff --git a/nixpkgs/doc/languages-frameworks/vim.section.md b/nixpkgs/doc/languages-frameworks/vim.section.md
index 4911509212e..8e4826232e1 100644
--- a/nixpkgs/doc/languages-frameworks/vim.section.md
+++ b/nixpkgs/doc/languages-frameworks/vim.section.md
@@ -263,6 +263,8 @@ Sometimes plugins require an override that must be changed when the plugin is up
 
 To add a new plugin, run `./update.py --add "[owner]/[name]"`. **NOTE**: This script automatically commits to your git repository. Be sure to check out a fresh branch before running.
 
+Finally, there are some plugins that are also packaged in nodePackages because they have Javascript-related build steps, such as running webpack. Those plugins are not listed in `vim-plugin-names` or managed by `update.py` at all, and are included separately in `overrides.nix`. Currently, all these plugins are related to the `coc.nvim` ecosystem of Language Server Protocol integration with vim/neovim.
+
 ## Important repositories
 
 - [vim-pi](https://bitbucket.org/vimcommunity/vim-pi) is a plugin repository