From 0d9c3299bc9db09cb69824f372794ee297953ced Mon Sep 17 00:00:00 2001 From: Katharina Fey Date: Mon, 9 Sep 2019 10:05:38 +0100 Subject: content/blog: refactoring naming/numbering scheme The idea here being that drafts are prefixed with `xxx_`, not knowing when and in what order they will be released. This fixes the problem where I start lots of articles, give them a number, then release them in non-linear sequences and get confused down the line. This commit _also_ releases the `nix-ocitools` article --- content/blog/105_allocation.md | 168 ++++++++++++++++++++++++++++++++++++ content/blog/105_sieve.md | 19 ----- content/blog/106_allocation.md | 168 ------------------------------------ content/blog/106_encrypted_zfs.md | 173 ++++++++++++++++++++++++++++++++++++++ content/blog/107_encrypted_zfs.md | 173 -------------------------------------- content/blog/107_usable_gpg.md | 92 ++++++++++++++++++++ content/blog/108_public_inbox.md | 42 +++++++++ content/blog/108_usable_gpg.md | 92 -------------------- content/blog/109_nix_ocitools.md | 115 +++++++++++++++++++++++++ content/blog/110_public_inbox.md | 42 --------- content/blog/xxx_no_google.md | 24 ++++++ content/blog/xxx_sieve.md | 19 +++++ 12 files changed, 633 insertions(+), 494 deletions(-) create mode 100644 content/blog/105_allocation.md delete mode 100644 content/blog/105_sieve.md delete mode 100644 content/blog/106_allocation.md create mode 100644 content/blog/106_encrypted_zfs.md delete mode 100644 content/blog/107_encrypted_zfs.md create mode 100644 content/blog/107_usable_gpg.md create mode 100644 content/blog/108_public_inbox.md delete mode 100644 content/blog/108_usable_gpg.md create mode 100644 content/blog/109_nix_ocitools.md delete mode 100644 content/blog/110_public_inbox.md create mode 100644 content/blog/xxx_no_google.md create mode 100644 content/blog/xxx_sieve.md diff --git a/content/blog/105_allocation.md b/content/blog/105_allocation.md new file mode 100644 index 0000000..4016718 --- /dev/null +++ b/content/blog/105_allocation.md @@ -0,0 +1,168 @@ +Title: Allocations are good, actually +Category: Blog +Tags: Rust, programming +Date: 2019-04-07 + +Something that you can often hear in the Rust community, +especially from people who were previously C or C++ developers, +is the adage "allocations are slow". + +The other day a friend asked me how to create a consecutive list of numbers. +I pointed her at `(0..).take(x).collect()` which can be made into a `Vec<_>`, +with a number of her choice. +It did made me think however about how this could be done +much nicer in a allocation-free manner. + +It lead me to come up with the following code which creates a `[_; _]` slice, +depending on which integer representation and length you choose. + +```rust +(0..) + .zip(0..x) + .fold([0; x], |mut acc, (step, i)| { + acc[i] = step; + acc + }) +``` + +So with this in mind, I wanted to run some comparisons. +I chose the numbers so that 32768 consecutive numbers would be generated. +I compiled the example with both `Debug` and `Release` mode. +(All of these measurements are done with `rustc 1.33.0 (2aa4c46cf 2019-02-28)`) + +Let's start with the non-allocating version. + +```console +$ time target/debug/playground +target/debug/playground 1.45s user 0.00s system 99% cpu 1.457 total +$ time target/release/playground +target/release/playground 0.27s user 0.00s system 99% cpu 0.270 total +``` + +Cool! So as you can see, the `Release` profile is over 500% faster. +And performance-wise this is quite reasonable. + +Let's see how an allocating implementation stacks up to it. +The code used here is the following. + +```rust +let vec: Vec = (0..) + .take(1024 * 32) + .collect(); +``` + +So how fast is this gonna be? + +```console +$ time target/debug/playground +target/debug/playground 0.01s user 0.00s system 93% cpu 0.010 total +$ time target/release/playground +target/release/playground 0.00s user 0.00s system 85% cpu 0.005 total +``` + +What? ...it's faster?! + +Well, I guess this does to show that it's not as simple as saying "allocations are bad". +Avoiding allocations at all cost can slow you down. + +Thanks for coming to my TED talk! + +*. . .* + +## Yes but *why*? + +Okay maybe you're more curious than that and want to understand what's going on here. +So come along, let's read some assembly! + +Let's focus mostly on the release profile here, +because `Debug` generates a lot of code that makes it harder to understand. +So we have two code snippets that we should throw into [godbolt] to see what rustc does. + +[godbolt]: https://rust.godbolt.org/ + +```rust +// This doesn't allocate +const length: usize = 1024 * 32; +pub fn slice() -> [u32; length] { + (0..) + .zip(0..length) + .fold([0; length], |mut acc, (num, idx)| { + acc[idx] = num; + acc + }) +} + +// This does +pub fn vec() -> Vec { + (0..).take(1024 * 32).collect() +} +``` + +Let's have a look at the assembly that the `vec()` function generates. + + + +```gas +.LCPI0_0: + .long 0 + .long 1 + .long 2 + .long 3 + +# ... snip ... + +example::vec: + push rbx + mov rbx, rdi + mov edi, 131072 + mov esi, 4 + call qword ptr [rip + __rust_alloc@GOTPCREL] + test rax, rax + je .LBB0_4 + movdqa xmm0, xmmword ptr [rip + .LCPI0_0] + mov ecx, 28 + movdqa xmm8, xmmword ptr [rip + .LCPI0_1] + movdqa xmm9, xmmword ptr [rip + .LCPI0_2] + movdqa xmm10, xmmword ptr [rip + .LCPI0_3] + movdqa xmm4, xmmword ptr [rip + .LCPI0_4] + movdqa xmm5, xmmword ptr [rip + .LCPI0_5] + movdqa xmm6, xmmword ptr [rip + .LCPI0_6] + movdqa xmm7, xmmword ptr [rip + .LCPI0_7] + movdqa xmm1, xmmword ptr [rip + .LCPI0_8] +.LBB0_2: + movdqa xmm2, xmm0 + paddd xmm2, xmm8 + # ... snip ... + ret +.LBB0_4: + mov edi, 131072 + mov esi, 4 + call qword ptr [rip + _ZN5alloc5alloc18...@GOTPCREL] + ud2 +``` + +(full code dump [here](https://pastebin.com/zDXi7qtt)) + + + +As you can see this uses the "Move Aligned Packed Integer Values" instructions in x86_64. +From some `x86` docs: + +> Moves 128, 256 or 512 bits of packed doubleword/quadword integer values from the source operand (the second operand) to the destination operand (the first operand). + +Basically the LLVM can figure out that our numbers are predictable +and can allocate them in a way that is batchable. + +We will already see how the non-alloc code is going to be slower: +because the code that assigns numbers is less unterstandable to a compiler +(i.e. assigning values to an array sequencially) this will not end up being batched. + +That's not to say that alloc code is going to be this fast on every platform +(RISC instruction sets lack many vectoring techniques) +and this doesn't even take embedded targets into account. + +But there you have it. + +LLVM is magic... + +... and saying "allocations are bad" really isn't telling the whole story. diff --git a/content/blog/105_sieve.md b/content/blog/105_sieve.md deleted file mode 100644 index 533ff72..0000000 --- a/content/blog/105_sieve.md +++ /dev/null @@ -1,19 +0,0 @@ -Title: Don't fear the sieve -Category: Blog -Tags: /dev/diary, e-mail, programming -Date: 2019-02-01 -Slug: understanding-sieve -Status: Draft - -If you don't already know, sieve (/siːv/) is an e-mail filtering language. -It's not touring complete (i.e. it doesn't allow recursion) -and has been defined through a series of RFCs for the base language -as well as several extentions. - -The RFCs aren't exactly nice to read. -But luckily, there are plenty of tutorials on the internet, -that try to explain sieve. -Unfortunately most of them are garbage. - -The main reason for this is, that the articles never deal -with a realistic set of constraints of requirements diff --git a/content/blog/106_allocation.md b/content/blog/106_allocation.md deleted file mode 100644 index 4016718..0000000 --- a/content/blog/106_allocation.md +++ /dev/null @@ -1,168 +0,0 @@ -Title: Allocations are good, actually -Category: Blog -Tags: Rust, programming -Date: 2019-04-07 - -Something that you can often hear in the Rust community, -especially from people who were previously C or C++ developers, -is the adage "allocations are slow". - -The other day a friend asked me how to create a consecutive list of numbers. -I pointed her at `(0..).take(x).collect()` which can be made into a `Vec<_>`, -with a number of her choice. -It did made me think however about how this could be done -much nicer in a allocation-free manner. - -It lead me to come up with the following code which creates a `[_; _]` slice, -depending on which integer representation and length you choose. - -```rust -(0..) - .zip(0..x) - .fold([0; x], |mut acc, (step, i)| { - acc[i] = step; - acc - }) -``` - -So with this in mind, I wanted to run some comparisons. -I chose the numbers so that 32768 consecutive numbers would be generated. -I compiled the example with both `Debug` and `Release` mode. -(All of these measurements are done with `rustc 1.33.0 (2aa4c46cf 2019-02-28)`) - -Let's start with the non-allocating version. - -```console -$ time target/debug/playground -target/debug/playground 1.45s user 0.00s system 99% cpu 1.457 total -$ time target/release/playground -target/release/playground 0.27s user 0.00s system 99% cpu 0.270 total -``` - -Cool! So as you can see, the `Release` profile is over 500% faster. -And performance-wise this is quite reasonable. - -Let's see how an allocating implementation stacks up to it. -The code used here is the following. - -```rust -let vec: Vec = (0..) - .take(1024 * 32) - .collect(); -``` - -So how fast is this gonna be? - -```console -$ time target/debug/playground -target/debug/playground 0.01s user 0.00s system 93% cpu 0.010 total -$ time target/release/playground -target/release/playground 0.00s user 0.00s system 85% cpu 0.005 total -``` - -What? ...it's faster?! - -Well, I guess this does to show that it's not as simple as saying "allocations are bad". -Avoiding allocations at all cost can slow you down. - -Thanks for coming to my TED talk! - -*. . .* - -## Yes but *why*? - -Okay maybe you're more curious than that and want to understand what's going on here. -So come along, let's read some assembly! - -Let's focus mostly on the release profile here, -because `Debug` generates a lot of code that makes it harder to understand. -So we have two code snippets that we should throw into [godbolt] to see what rustc does. - -[godbolt]: https://rust.godbolt.org/ - -```rust -// This doesn't allocate -const length: usize = 1024 * 32; -pub fn slice() -> [u32; length] { - (0..) - .zip(0..length) - .fold([0; length], |mut acc, (num, idx)| { - acc[idx] = num; - acc - }) -} - -// This does -pub fn vec() -> Vec { - (0..).take(1024 * 32).collect() -} -``` - -Let's have a look at the assembly that the `vec()` function generates. - - - -```gas -.LCPI0_0: - .long 0 - .long 1 - .long 2 - .long 3 - -# ... snip ... - -example::vec: - push rbx - mov rbx, rdi - mov edi, 131072 - mov esi, 4 - call qword ptr [rip + __rust_alloc@GOTPCREL] - test rax, rax - je .LBB0_4 - movdqa xmm0, xmmword ptr [rip + .LCPI0_0] - mov ecx, 28 - movdqa xmm8, xmmword ptr [rip + .LCPI0_1] - movdqa xmm9, xmmword ptr [rip + .LCPI0_2] - movdqa xmm10, xmmword ptr [rip + .LCPI0_3] - movdqa xmm4, xmmword ptr [rip + .LCPI0_4] - movdqa xmm5, xmmword ptr [rip + .LCPI0_5] - movdqa xmm6, xmmword ptr [rip + .LCPI0_6] - movdqa xmm7, xmmword ptr [rip + .LCPI0_7] - movdqa xmm1, xmmword ptr [rip + .LCPI0_8] -.LBB0_2: - movdqa xmm2, xmm0 - paddd xmm2, xmm8 - # ... snip ... - ret -.LBB0_4: - mov edi, 131072 - mov esi, 4 - call qword ptr [rip + _ZN5alloc5alloc18...@GOTPCREL] - ud2 -``` - -(full code dump [here](https://pastebin.com/zDXi7qtt)) - - - -As you can see this uses the "Move Aligned Packed Integer Values" instructions in x86_64. -From some `x86` docs: - -> Moves 128, 256 or 512 bits of packed doubleword/quadword integer values from the source operand (the second operand) to the destination operand (the first operand). - -Basically the LLVM can figure out that our numbers are predictable -and can allocate them in a way that is batchable. - -We will already see how the non-alloc code is going to be slower: -because the code that assigns numbers is less unterstandable to a compiler -(i.e. assigning values to an array sequencially) this will not end up being batched. - -That's not to say that alloc code is going to be this fast on every platform -(RISC instruction sets lack many vectoring techniques) -and this doesn't even take embedded targets into account. - -But there you have it. - -LLVM is magic... - -... and saying "allocations are bad" really isn't telling the whole story. diff --git a/content/blog/106_encrypted_zfs.md b/content/blog/106_encrypted_zfs.md new file mode 100644 index 0000000..7972466 --- /dev/null +++ b/content/blog/106_encrypted_zfs.md @@ -0,0 +1,173 @@ +Title: Bikeshedding disk partitioning +Category: Blog +Tags: linux, zfs, nixos +Date: 2019-07-11 + +I recently got a new Thinkpad. Well...new is a stretch. +It's an X230, featuring an i5 and 16GB of RAM. + +One of the first things I did with this laptop was to [flash coreboot on it][coreboot]. +This is something I've always wanted to be able to do, +but so far lacked hardware that was supported. +And generally, it felt like finally maybe I could have a laptop to tinker around with. + +[coreboot]: https://octodon.social/@spacekookie/102150706024564666 + +And that's where this post begins... + +## Encrypted disk + +So from the start I knew I wanted to have a fully encrypted disk. + +What that means is that your `/boot` partition +(whether it is it's own partition or not), is also encrypted. + +Secondly, I don't like (U)EFI... +What that means is that I'm installing GRUB +in the MBR (with a DOS partition table) instead. + +Now: GRUB stage 1 can handle the encryption for us, +but there's some limitations + +- Keyboard layout limited to `US` +- `/boot` can only be certain partition type +- `/` and `/boot` need be contained in an LVM + +That last one _might_ not be accurate if you only want +to have an `ext4` (or similar) rootfs. But because I +want to have a `zfs` root, I need to embed it into an LVM. +This is also the reason why `/boot` needs to be it's own partition. +After we've done all this, we will install a linux distribution of choice +(which we'll reveal later). + +Anyway, let's get started! + +## Preparing the disk + +(Feel free to skip this step) + +Something you might want to do is letting your disk look +otherwise uninitialised, or "securely erasing" any data +that is already on it. +But generating random data is a lot of work and `/dev/urandom` +is very slow. + +Instead you can create a crypto-disk (luks) on it, then fill it +with zero's. But because of the encryption it will seem random. + +(`/dev/sda` is my disk in this example because lolwat is nvme even?) + +```console +$ cryptsetup luksFormat /dev/sda1 +$ cryptsetup luksOpen /dev/sda sda_crypto +$ dd if=/dev/zero of=/dev/mapper/sda_crypto bs=512 status=progress +``` + +This might take a while, but considerably less time than filling +the disk with random data. After this is done, you might want to +actually wipe the first bunch of bytes. + +```console +$ cryptsetup luksClose sda_crypto +$ dd if=/dev/urandom of=/dev/sda bs=1M count=8 +``` + +## Basic partitioning + +So what we want to do is setup a single partition on `/dev/sda`, +the same way we did to prepare the disk. Then repeat the previous +command to setup a cryptodisk. + +What follows is the LVM setup: + +```console +$ pvcreate lvm +$ vgcreate vg0 /dev/mapper/lvm +$ lvcreate vg0-boot -l 1G +$ lvcreate vg0-swap -l 16G +$ lvcreate vg0-root -L +100%FREE +``` + +I included the `swap` partition in the LVM instead of as a ZFS subvolume +because those can sometimes deadlock and this just makes things easier. + +Now we want to create the filesystems. +For `/boot` we can just use `mkfs.ext4`, +but consider that I want to use `zfs` on `/`, +that will require some more work. + +```console +$ zpool create rtank /dev/mapper/vg0-root +``` + +Feel free to call your pool whatever! +At this point you could also create subvolumes to +split `/`, `/home`, ... if you wanted. + +## Mounting & Configuration + +So that's all good. How do we initialise this system now? +We need to mount the zfs pool first, then `/boot` and then install +our linux secret distribution of choice (spoilers: it's [NixOS]!) + +[NixOS]: https://nixos.org + +``` +$ mkdir -p /mnt/boot +$ zpool import rtank +$ mount -t zfs rtank /mnt +$ mount /dev/mapper/vg0-boot /mnt/boot +$ nixos-generate-config --root /mnt +``` + +That last line is obviously NixOS specific. +You now have a fully encrypted disk setup, without +using EFI. Wuuh! + +The rest of this post I want to talk about how to make this +all work with NixOS and reproducable configuration. + +Most of what we need to configure is in the `boot` option. +Let's go through the settings one by one: + +- `boot.loader.grub` + - `efiSupport = false` actually the default but I like being explicit + - `copyKernels = true` enable this to avoid problems with ZFS becoming unbootable + - `device = "/dev/sda"` replace this with the device that holds your GRUB + - `zfsSupport = true` to enable ZFS support 😅 + - `enableCryptodisk = true` to enable stage-1 encryption support +- `boot.zfs.devNodes = "/dev"` to point ZFS at the correct device tree (not 100% if required) +- `fileSystems."/".encrypted` + - `enable = true` + - `label = "lvm"` the label of your LVM + - `blkDev = "/dev/disk/by-uuid/f1440abd-99e3-46a8-aa36-7824972fee54"` the disk that + ZFS is installed to. You can find this out by looking at your symlinks in + `/dev/disk/by-uuid` and picking the correct one. +- `networking.hostId` needs to be set to some random 8 bytes + +Following is the complete config to make it easier to copy stuff from: + +``` +boot.loader.grub = { + efiSupport = false; + copyKernels = true; + device = "/dev/sda"; + zfsSupport = true; + enableCryptodisk = true; +}; +boot.zfs.devNodes = "/dev"; + +fileSystems."/" = { + encrypted = { + enable = true; + label = "lvm"; + blkDev = "/dev/disk/by-uuid/f1440abd-99e3-46a8-aa36-7824972fee54"; + }; +} + +networking.hostId = ""; +``` + +And that's it. +If you spot any errors in this article (or any for that matter), +feel free to e-mail me or send me a PR over on github. diff --git a/content/blog/107_encrypted_zfs.md b/content/blog/107_encrypted_zfs.md deleted file mode 100644 index 7972466..0000000 --- a/content/blog/107_encrypted_zfs.md +++ /dev/null @@ -1,173 +0,0 @@ -Title: Bikeshedding disk partitioning -Category: Blog -Tags: linux, zfs, nixos -Date: 2019-07-11 - -I recently got a new Thinkpad. Well...new is a stretch. -It's an X230, featuring an i5 and 16GB of RAM. - -One of the first things I did with this laptop was to [flash coreboot on it][coreboot]. -This is something I've always wanted to be able to do, -but so far lacked hardware that was supported. -And generally, it felt like finally maybe I could have a laptop to tinker around with. - -[coreboot]: https://octodon.social/@spacekookie/102150706024564666 - -And that's where this post begins... - -## Encrypted disk - -So from the start I knew I wanted to have a fully encrypted disk. - -What that means is that your `/boot` partition -(whether it is it's own partition or not), is also encrypted. - -Secondly, I don't like (U)EFI... -What that means is that I'm installing GRUB -in the MBR (with a DOS partition table) instead. - -Now: GRUB stage 1 can handle the encryption for us, -but there's some limitations - -- Keyboard layout limited to `US` -- `/boot` can only be certain partition type -- `/` and `/boot` need be contained in an LVM - -That last one _might_ not be accurate if you only want -to have an `ext4` (or similar) rootfs. But because I -want to have a `zfs` root, I need to embed it into an LVM. -This is also the reason why `/boot` needs to be it's own partition. -After we've done all this, we will install a linux distribution of choice -(which we'll reveal later). - -Anyway, let's get started! - -## Preparing the disk - -(Feel free to skip this step) - -Something you might want to do is letting your disk look -otherwise uninitialised, or "securely erasing" any data -that is already on it. -But generating random data is a lot of work and `/dev/urandom` -is very slow. - -Instead you can create a crypto-disk (luks) on it, then fill it -with zero's. But because of the encryption it will seem random. - -(`/dev/sda` is my disk in this example because lolwat is nvme even?) - -```console -$ cryptsetup luksFormat /dev/sda1 -$ cryptsetup luksOpen /dev/sda sda_crypto -$ dd if=/dev/zero of=/dev/mapper/sda_crypto bs=512 status=progress -``` - -This might take a while, but considerably less time than filling -the disk with random data. After this is done, you might want to -actually wipe the first bunch of bytes. - -```console -$ cryptsetup luksClose sda_crypto -$ dd if=/dev/urandom of=/dev/sda bs=1M count=8 -``` - -## Basic partitioning - -So what we want to do is setup a single partition on `/dev/sda`, -the same way we did to prepare the disk. Then repeat the previous -command to setup a cryptodisk. - -What follows is the LVM setup: - -```console -$ pvcreate lvm -$ vgcreate vg0 /dev/mapper/lvm -$ lvcreate vg0-boot -l 1G -$ lvcreate vg0-swap -l 16G -$ lvcreate vg0-root -L +100%FREE -``` - -I included the `swap` partition in the LVM instead of as a ZFS subvolume -because those can sometimes deadlock and this just makes things easier. - -Now we want to create the filesystems. -For `/boot` we can just use `mkfs.ext4`, -but consider that I want to use `zfs` on `/`, -that will require some more work. - -```console -$ zpool create rtank /dev/mapper/vg0-root -``` - -Feel free to call your pool whatever! -At this point you could also create subvolumes to -split `/`, `/home`, ... if you wanted. - -## Mounting & Configuration - -So that's all good. How do we initialise this system now? -We need to mount the zfs pool first, then `/boot` and then install -our linux secret distribution of choice (spoilers: it's [NixOS]!) - -[NixOS]: https://nixos.org - -``` -$ mkdir -p /mnt/boot -$ zpool import rtank -$ mount -t zfs rtank /mnt -$ mount /dev/mapper/vg0-boot /mnt/boot -$ nixos-generate-config --root /mnt -``` - -That last line is obviously NixOS specific. -You now have a fully encrypted disk setup, without -using EFI. Wuuh! - -The rest of this post I want to talk about how to make this -all work with NixOS and reproducable configuration. - -Most of what we need to configure is in the `boot` option. -Let's go through the settings one by one: - -- `boot.loader.grub` - - `efiSupport = false` actually the default but I like being explicit - - `copyKernels = true` enable this to avoid problems with ZFS becoming unbootable - - `device = "/dev/sda"` replace this with the device that holds your GRUB - - `zfsSupport = true` to enable ZFS support 😅 - - `enableCryptodisk = true` to enable stage-1 encryption support -- `boot.zfs.devNodes = "/dev"` to point ZFS at the correct device tree (not 100% if required) -- `fileSystems."/".encrypted` - - `enable = true` - - `label = "lvm"` the label of your LVM - - `blkDev = "/dev/disk/by-uuid/f1440abd-99e3-46a8-aa36-7824972fee54"` the disk that - ZFS is installed to. You can find this out by looking at your symlinks in - `/dev/disk/by-uuid` and picking the correct one. -- `networking.hostId` needs to be set to some random 8 bytes - -Following is the complete config to make it easier to copy stuff from: - -``` -boot.loader.grub = { - efiSupport = false; - copyKernels = true; - device = "/dev/sda"; - zfsSupport = true; - enableCryptodisk = true; -}; -boot.zfs.devNodes = "/dev"; - -fileSystems."/" = { - encrypted = { - enable = true; - label = "lvm"; - blkDev = "/dev/disk/by-uuid/f1440abd-99e3-46a8-aa36-7824972fee54"; - }; -} - -networking.hostId = ""; -``` - -And that's it. -If you spot any errors in this article (or any for that matter), -feel free to e-mail me or send me a PR over on github. diff --git a/content/blog/107_usable_gpg.md b/content/blog/107_usable_gpg.md new file mode 100644 index 0000000..350f352 --- /dev/null +++ b/content/blog/107_usable_gpg.md @@ -0,0 +1,92 @@ +Title: Usable GPG with WKD +Category: Blog +Tags: gpg, security, usability +Date: 2019-07-02 + +With the recent [SKS keyserver vulnerability][sks], +people have been arguing reasonably talking on the GnuPG mailing list +about how to proceed with keyservers, public key exchanges +and the GPG ecosystem as a whole. + +[sks]: https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f + +As part of this [WKD] was mentioned. +It stands for "Web Key Directory" and is a standard +for making a users public key available via their e-mail provider +or server with the domain that corresponds to their e-mail address. +There's several clients (such as [Enigmail] in Thunderbird) +that will use this standard to automatically fetch a user's public key, +when writing an e-mail to them. + +[WKD]: https://wiki.gnupg.org/WKD +[Enigmail]: https://www.enigmail.net/index.php/en/ + +As an example: my e-mails are hosted with [mailbox.org], +but I use my own website as an e-mail alias. +This means that I can make my public key available via my website, +and clients using WKS could then get it automatically. + +[mailbox.org]: https://mailbox.org + +If you don't have your own domain and use a webhoster instead, +you might still be able to use this. +There's a [list of supported hosters][list] that you should check out. + +[list]: https://wiki.gnupg.org/WKD#Mail_Service_Providers_offering_WKD + +## Setting this up + +(**Note:** in newer versions of `gpg` the tool `gpg-wks-client` is included, +which can handle setting up the folder structure for you automatically). + +There's two ways of making your public keys accessable this way: +the advanced and the direct way. +This post will only talk about the latter, because I find it easier. + +You need to create a `.well-known/openpgpkey` directory on your server. +In this directory, place a `policy` file. +This can be zero-length, but is used to check for WKD capability. +Next, create a `hu` folder inside it +(no idea what this stands for... +— as pointed out by an attentive reader, it stands for [hashed-userid]) + +[hashed-userid]: https://www.gnupg.org/blog/20160830-web-key-service.html + +Next, take the prefix of your e-mail address +(i.e. in `kookie@spacekookie.de`, this would be `kookie`), +hash it with SHA-1 and then encode the output with z-base-32. +You can use [this][cryptii] convenient encoding website. + +**Edit:** Also pointed out by a reader, you can actually use +`gpg --with-wkd -l ` to display your hashed User ID +instead of using an external resource for this. + +[cryptii]: https://cryptii.com/pipes/z-base-32 + +Export the **binary** version of your pubkey (so without `-a`) +and place it in the `hu` folder, under the name that you just computed. + +The resulting folder structure should look something like this: + +``` +$ tree .well-known/ +.well-known/ +└── openpgpkey + ├── hu + │   └── nzn5f4t6k15893omwk19pgzfztowwkhs + └── policy +``` + +You need to make sure that this folder is accessable through your webserver +(this either involves including it in a static site or configuring nginx correctly). +But fundamentally, that's it! + +You can test if it works by setting a new `GNUPGHOME` and running this: + +``` +$ env GNUPGHOME=$(mktemp -d) gpg --locate-keys +``` + +And that's it! Clients like Enigmail, KMail or GpgOL for Outlook +will now automatically fetch your public key for any message they send. + diff --git a/content/blog/108_public_inbox.md b/content/blog/108_public_inbox.md new file mode 100644 index 0000000..88a5e1e --- /dev/null +++ b/content/blog/108_public_inbox.md @@ -0,0 +1,42 @@ +Title: Starting a public inbox +Category: Blog +Tags: blogging, communication +Date: 2019-07-20 + +I've been a lot more active on this blog in the last year than I have +previously, and that makes me pretty happy. I'm trying to become less +obsessed about publishing only perfect diamonds made of words, but +instead also publishing articles that might still have some flaws or +that I haven't rewritten twelve times yet ;) + +As a result of this I have actually gotten more and more e-mails by +people saying that they read my blog, giving me feedback and sometimes +submitting patches to my [website] repository to fix typos or bad +formatting. And that's pretty cool. + +[website]: https://sr.ht/~spacekookie/website + +Recently I started thinking if the format of e-mail might not be well +suited for comments as well. Not just to me, but to allow other +readers to talk _about_ the stuff I post. I wasn't super sure if this +was such a great idea, after all...this is the internet we're talking +about. Someone will be an asshole and ruin it for everybody. + +Then I discovered Drew DeVaults `public-inbox` mailing list, +which is basically exactly what I thought about creating, +hosted on source hut. + +It might still be a terrible idea, but it's one I wanna try. I also +wanna automatically post new blog posts _to_ the mailing list, as +plain text, so people don't have to fuss around with my RSS feed if +they don't want to. I will host my `public-inbox` on source hut for +now too, especially considering that I've been trying it out for a lot +of smaller personal stuff, it makes sense. And I really quite like it +(might write about that in the future too) + +So, if you have comments, questions or want to fix typos, +feel free to check out my [public-inbox][inbox] + +[inbox]: https://lists.sr.ht/~spacekookie/public-inbox + +Let's see how this goes then 😀 diff --git a/content/blog/108_usable_gpg.md b/content/blog/108_usable_gpg.md deleted file mode 100644 index 350f352..0000000 --- a/content/blog/108_usable_gpg.md +++ /dev/null @@ -1,92 +0,0 @@ -Title: Usable GPG with WKD -Category: Blog -Tags: gpg, security, usability -Date: 2019-07-02 - -With the recent [SKS keyserver vulnerability][sks], -people have been arguing reasonably talking on the GnuPG mailing list -about how to proceed with keyservers, public key exchanges -and the GPG ecosystem as a whole. - -[sks]: https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f - -As part of this [WKD] was mentioned. -It stands for "Web Key Directory" and is a standard -for making a users public key available via their e-mail provider -or server with the domain that corresponds to their e-mail address. -There's several clients (such as [Enigmail] in Thunderbird) -that will use this standard to automatically fetch a user's public key, -when writing an e-mail to them. - -[WKD]: https://wiki.gnupg.org/WKD -[Enigmail]: https://www.enigmail.net/index.php/en/ - -As an example: my e-mails are hosted with [mailbox.org], -but I use my own website as an e-mail alias. -This means that I can make my public key available via my website, -and clients using WKS could then get it automatically. - -[mailbox.org]: https://mailbox.org - -If you don't have your own domain and use a webhoster instead, -you might still be able to use this. -There's a [list of supported hosters][list] that you should check out. - -[list]: https://wiki.gnupg.org/WKD#Mail_Service_Providers_offering_WKD - -## Setting this up - -(**Note:** in newer versions of `gpg` the tool `gpg-wks-client` is included, -which can handle setting up the folder structure for you automatically). - -There's two ways of making your public keys accessable this way: -the advanced and the direct way. -This post will only talk about the latter, because I find it easier. - -You need to create a `.well-known/openpgpkey` directory on your server. -In this directory, place a `policy` file. -This can be zero-length, but is used to check for WKD capability. -Next, create a `hu` folder inside it -(no idea what this stands for... -— as pointed out by an attentive reader, it stands for [hashed-userid]) - -[hashed-userid]: https://www.gnupg.org/blog/20160830-web-key-service.html - -Next, take the prefix of your e-mail address -(i.e. in `kookie@spacekookie.de`, this would be `kookie`), -hash it with SHA-1 and then encode the output with z-base-32. -You can use [this][cryptii] convenient encoding website. - -**Edit:** Also pointed out by a reader, you can actually use -`gpg --with-wkd -l ` to display your hashed User ID -instead of using an external resource for this. - -[cryptii]: https://cryptii.com/pipes/z-base-32 - -Export the **binary** version of your pubkey (so without `-a`) -and place it in the `hu` folder, under the name that you just computed. - -The resulting folder structure should look something like this: - -``` -$ tree .well-known/ -.well-known/ -└── openpgpkey - ├── hu - │   └── nzn5f4t6k15893omwk19pgzfztowwkhs - └── policy -``` - -You need to make sure that this folder is accessable through your webserver -(this either involves including it in a static site or configuring nginx correctly). -But fundamentally, that's it! - -You can test if it works by setting a new `GNUPGHOME` and running this: - -``` -$ env GNUPGHOME=$(mktemp -d) gpg --locate-keys -``` - -And that's it! Clients like Enigmail, KMail or GpgOL for Outlook -will now automatically fetch your public key for any message they send. - diff --git a/content/blog/109_nix_ocitools.md b/content/blog/109_nix_ocitools.md new file mode 100644 index 0000000..c92c358 --- /dev/null +++ b/content/blog/109_nix_ocitools.md @@ -0,0 +1,115 @@ +Title: `ociTools` in NixOS +Category: Blog +Date: 2019-09-09 10:00 +Tags: /dev/diary, NixOS, Containers + +With the release of NixOS 19.09, I thought I wanted to blog about +something that I've been working on, that [recently][0] made it into +`master`, and thus this new stable channel. So I thought, why not blog +about it a bunch. + +[0]: https://github.com/NixOS/nixpkgs/pull/56411 + +## What are OCI tools? + +[Open Container Initiative][1] (or OCI) is a spec that standardised what +format containers should use. It is implemented by a bunch of runners, +such as `runc` (the Docker/ standard Kubernetes backend) and `railcar` +(more to that later) and outlines in exactly what format a containers +metadata and filesystem are to be stored, so to achieve the largest +possible reusability. + +[1]: https://www.opencontainers.org/ + +The spec is pretty [long][3] and in some places not very +great. There's even a [blog post][4] from Oracle, talking about how +implementing an OCI runner in Rust made them find bugs in the +specification. + +[3]: https://github.com/opencontainers/runtime-spec +[4]: https://blogs.oracle.com/developers/building-a-container-runtime-in-rust + +## What are `ociTools`? + +So now the question is, what does that have to do with +NixOS/`nixpkgs`. The answer is simple: I wanted to be able to +containerise single applications on my server, without requiring a +container daemon (such as docker) or relying on externally built +"Docker containers" from a registry. + +So, `ociTools.buildContainer` was recently merged into `nixpkgs` +`master`, allowing you to do exactly that. It's usage is farely +straight forward: + +```nix +with pkgs; ociTools.buildContainer { + args = [ + (writeShellScript "run.sh" '' + ${hello}/bin/hello -g "Hello from OCI container!" + '').outPath + ]; +} +``` + +The `args` parameter refers to a list of paths and arguments that are +handed to a container runner to run as init. In this case it's +creating a shell script with some commands in it, then getting the +output derivation path. + +There's other options available, such as the `os`, `arch` and +`readonly` flags (which aren't very interesting and have sane +defaults). Additionally to that there's `mounts`. + +Simply specify any bind-mount you wish to setup at container init in a +similar way you would describe your filesystem with `nix` already: + +```nix +with pkgs; ociTools.buildContainer { + args = [ + (writeShellScript "run.sh" '' + ${hello}/bin/hello -g "Hello from OCI container!" + '').outPath + ]; + mounts."/data" = { + source = "/var/lib/mydata"; + }; +} +``` + +## Railcar + ociTools + +So that's all nice and good. But what about actually running these +containers. Well, as I previously said I didn't want to have a +dependency on a management daemon such as `docker`. Instead, I also +added a module for the afromentioned `railcar` container runner +(Oracle please merge my PR, thank you). + +It wraps very cleanly around `ociTools` and generates `systemd` units +to start containers, restarting them if they crash. This way you can +express applications purely in `nix`, give them access to only the +things they need, and be sure that their configuration is in line with +the rest of your system rebuild. + +```nix +services.railcar = { + enable = true; + containers = { + "hello" = { + cmd = '' + ${pkgs.hello}/bin/hello -g "Hello railcar!" + ''; + }; + }; +}; +``` + +The metadata interface for `mounts`, etc is the same for `railcar` as +for `ociTools`. + +Anyway, I hope you enjoy. There is definitely things to improve, +especially considering the vastness of the OCI spec. Plus, at the +moment `ociTools` does require a bunch of manual setup work for an +application to function, if it, say, runs a webserver. It would be +cool if some NixOS modules could be re-used to make this configuration +easier. But I'm sure someone else is gonna have fun figuring that out +x) diff --git a/content/blog/110_public_inbox.md b/content/blog/110_public_inbox.md deleted file mode 100644 index 88a5e1e..0000000 --- a/content/blog/110_public_inbox.md +++ /dev/null @@ -1,42 +0,0 @@ -Title: Starting a public inbox -Category: Blog -Tags: blogging, communication -Date: 2019-07-20 - -I've been a lot more active on this blog in the last year than I have -previously, and that makes me pretty happy. I'm trying to become less -obsessed about publishing only perfect diamonds made of words, but -instead also publishing articles that might still have some flaws or -that I haven't rewritten twelve times yet ;) - -As a result of this I have actually gotten more and more e-mails by -people saying that they read my blog, giving me feedback and sometimes -submitting patches to my [website] repository to fix typos or bad -formatting. And that's pretty cool. - -[website]: https://sr.ht/~spacekookie/website - -Recently I started thinking if the format of e-mail might not be well -suited for comments as well. Not just to me, but to allow other -readers to talk _about_ the stuff I post. I wasn't super sure if this -was such a great idea, after all...this is the internet we're talking -about. Someone will be an asshole and ruin it for everybody. - -Then I discovered Drew DeVaults `public-inbox` mailing list, -which is basically exactly what I thought about creating, -hosted on source hut. - -It might still be a terrible idea, but it's one I wanna try. I also -wanna automatically post new blog posts _to_ the mailing list, as -plain text, so people don't have to fuss around with my RSS feed if -they don't want to. I will host my `public-inbox` on source hut for -now too, especially considering that I've been trying it out for a lot -of smaller personal stuff, it makes sense. And I really quite like it -(might write about that in the future too) - -So, if you have comments, questions or want to fix typos, -feel free to check out my [public-inbox][inbox] - -[inbox]: https://lists.sr.ht/~spacekookie/public-inbox - -Let's see how this goes then 😀 diff --git a/content/blog/xxx_no_google.md b/content/blog/xxx_no_google.md new file mode 100644 index 0000000..c7c0506 --- /dev/null +++ b/content/blog/xxx_no_google.md @@ -0,0 +1,24 @@ +Title: No, I won't work at Google +Category: Blog +Tags: ethics +Date: 2019-07-28 + +Once in a while (about every 6-9 months or so), I get an e-mail like this in my inbox: + +> Hi Katherina, +> +> Hope you are doing well. I'm a tech sourcer with Google working on the +> expansion of our Europe software engineering teams. I came across your +> linkedin and github profiles and I'm interested in finding more about your +> experience. Would you be open having a quick formal chat about our current +> teams and projects we have across Europe to see if there's anything that +> may interest you? + +I always send the same polite decline, wishing them a nice day. +And a few months later, someone else will e-mail me the exact same question. + + +I know that a few years ago, when Google first reached out to me to ask me +if I was interested in working for them, I was quite excited about the idea. +Since then a lot of things have changed, including my attitudes towards +big companies, and Google in particular. diff --git a/content/blog/xxx_sieve.md b/content/blog/xxx_sieve.md new file mode 100644 index 0000000..533ff72 --- /dev/null +++ b/content/blog/xxx_sieve.md @@ -0,0 +1,19 @@ +Title: Don't fear the sieve +Category: Blog +Tags: /dev/diary, e-mail, programming +Date: 2019-02-01 +Slug: understanding-sieve +Status: Draft + +If you don't already know, sieve (/siːv/) is an e-mail filtering language. +It's not touring complete (i.e. it doesn't allow recursion) +and has been defined through a series of RFCs for the base language +as well as several extentions. + +The RFCs aren't exactly nice to read. +But luckily, there are plenty of tutorials on the internet, +that try to explain sieve. +Unfortunately most of them are garbage. + +The main reason for this is, that the articles never deal +with a realistic set of constraints of requirements -- cgit v1.2.3