path: root/blog/butter.md

+++
rss = "How I reinstalled NixOS on Btrfs with an amnesiac root
       and backed up my data"
date = Date(2021, 11, 14)
tags = ["backup", "btrfs", "fun", "nixos", "recipe"]
+++

# NixOS on Btrfs+tmpfs

In 2018, dad bought me a new laptop to replace the good ole Compaq nx7010
whose screen unfortunately got infected by some sort of microbe and dieded
shortly afterwards.  The new one, whilst having a considerably worse build
quality (like all other late-2010s ones when compared to mid-2000s models),
had a dozen times as much storage: a 250 GB M.2 SSD and a 500 GB SATA HDD.

My data hoarding habit has grown exponentially ever since.  Initially,
I used to back up the data from the SSD to the HDD but after a few years,
I ran out of space and decided to get some more storage.  Instead of buying
a portable hard disk like a normal person would, I went for an SATA SSD,
as it was rather difficult to find a 7200 rpm 2.5-inch[^metric] HDD
in the market at the time.

I then asked my father for a spare SATA-to-USB case (he switched to using
a dock a while ago, and like other dads, nothing is ever thrown away)
and prepared to swap the drives.  As cloning the data would have been too easy,
I decided to *spice things up* by reinstalling the OS.  Back then I was
dual-booting Debian and NixOS, but the former had hardly been ever booted
for months so it was time to let it go:

![Elsa rolling on the floor crying](/assets/let-it-go.png)

In addition, I wanted to hop on the new and shinny[^new] train of Btrfs.
It has compression, snapshots and subvolumes, what's not to love?  Let's
replace something I'd been using for nearly a decade with a file system
I had absolutely zero experience with, what could possibly go wrong, right?

\toc

## Reinstallation

I was going to reinstall NixOS with an ephemeral root,
which had been covered to death in the following brilliant resources:

* [Erase your darlings: immutable infrastructure for mutable systems][grahamc]
* [NixOS ❄: tmpfs as root][elis]
* [Nix community's impermanence modules][impermanence]
* [Paranoid NixOS Setup][christine]

The only twist here is that I was using Btrfs instead of ZFS or ext4 like in
other guides.  This choice would influence how to back up in the later section.

### Preparation

First of all, I temporarily copied data to the SATA SSD from the M.2,
including [my Nix configurations].  Using either `cp` or `rsync` didn't
seem to have any effect on the performance, and in the mean time I also
went ahead and grabbed a [NixOS unstable live image] and `dd`'ed it to
a flash drive.  As I'm tracking unstable, installing from the same version
would allowed me to skip switching the channel and a lot of downloading.

### Partitioning

After booting up the live image, I opened up a root shell with `sudo -i`.
As expected, `fdisk` reports the M.2 SSD as `/dev/nvme0n1`.  Paranoid as always,
I decided to give the EFI system partition a whole gibibyte, swap eight
to match memory[^memory] and the rest as a single chonky Btrfs partition:

```sh
parted /dev/nvme0n1 -- mklabel gpt
parted /dev/nvme0n1 -- mkpart ESP fat32 1MiB 1GiB
parted /dev/nvme0n1 -- set 1 boot on
mkfs.vfat /dev/nvme0n1p1

parted /dev/nvme0n1 -- mkpart Swap linux-swap 1GiB 9GiB
mkswap -L Swap /dev/nvme0n1p2
swapon /dev/nvme0n1p2

parted /dev/nvme0n1 -- mkpart primary 9GiB 100%
mkfs.btrfs -L Butter /dev/nvme0n1p3
```

As I typed this, I realized that I should have set up encryption
for the last partition so I would probably need to reinstall in the near future
to fix this mistake.  Anyway, with the target system's root mounted as tmpfs,
I would need to persist `/nix` (obviously), `/etc` (mostly for authentication
and other secret stuff not included in `configuration.nix` that I was too lazy
to opt in individually), `/var/log`, `/root` and `/home`:

```sh
mount /dev/nvme0n1p3 /mnt
btrfs subvolume create /mnt/nix
btrfs subvolume create /mnt/etc
btrfs subvolume create /mnt/log
btrfs subvolume create /mnt/root
btrfs subvolume create /mnt/home
umount /mnt
```

Most subvolumes can be mounted with `noatime`, except for `/home`
where I frequently need to sort files by modification time.  All of them
should have forced compression though:

```sh
mount -t tmpfs -o mode=755 none /mnt
mkdir -p /mnt/{boot,nix,etc,var/log,root,home}
mount /dev/nvme0n1p1 /mnt/boot
mount -o subvol=nix,compress-force=zstd,noatime /dev/nvme0n1p3 /mnt/nix
mount -o subvol=etc,compress-force=zstd,noatime /dev/nvme0n1p3 /mnt/etc
mount -o subvol=log,compress-force=zstd,noatime /dev/nvme0n1p3 /mnt/var/log
mount -o subvol=root,compress-force=zstd,noatime /dev/nvme0n1p3 /mnt/root
mount -o subvol=home,compress-force=zstd /dev/nvme0n1p3 /mnt/home
```

### Configuration

With everything mounted, `nixos-generate-config --root /mnt` could be run
to generate a basic configuration.  But wait, didn't I say something
about my dot files?  That's correct, but it's not easy to handcraft
the `hardware-configuration.nix`.  After making sure all are mounted
with the right options and `services.fstrim.enable` is `true`, I copied
other configuration files to `/etc/nixos` and finished this step.

### Installation

NixOS installation is as simple as running `nixos-install`.  But my job was
not done after setting the root password and rebooting into the new system.
It was working, but not functional.  There was nothing meaningful for me
to do on it, so I had to log in (as root), `passwd`'ed the user and copied
the home folder back from the temporary drive.

After freeing the new SATA SSD, I also filled it with butter.  Yes,
all the way, no GPT, no MBR, just Btrfs, whose subvolumes were used
in place of partitions:

```sh
mkfs.btrfs -f -L Fly /dev/sdb
mkdir -p /mnt
mount /dev/sdb /mnt
btrfs subvolume create /mnt/movies
```

At that time the only disposable data I had were my movies collection.
The HDD also contained other data but they were rebalanced at `/home`
(on the M.2).  After swapping the SATA SSD inside the laptop, I logged in
as the normal user and get the exact same environment before the reinstallation.

### Profits

Thanks to subvolumes and compression, the free spaces were no longer fragmented
and I think I gained like 100 GB (not counting the old Debian's root).  Backup
would also be less painful with Btrfs snapshots (instead of plain `rsync`
like I used to) as shown as follows.

## Backup

With all data migrated, the HDD could be used for backing up.  First,
some legacy data I no longer access were moved there, then I started to back up
my `/home` partition:

### Initialization

Having learned my lesson, I did not forget to set up [LUKS] this time:

```sh
cryptsetup luksFormat /dev/sdb
cryptsetup luksOpen /dev/sdb backup
```

To make use of snapshots, the backup drive gotta be Btrfs as well.
The compression level was turned up to 14 this time (default was 3):

```sh
mkfs.btrfs -L Backup /dev/mapper/backup
mkdir /backup
mount -o noatime,compress-force=zstd:14 /dev/mapper/backup /backup
```

Following [Btrfs Wiki], I made the first `/home` snapshot and sent it
to the backup drive:

```sh
btrfs subvolume create /backup/home
today=$(date --iso-8601)
btrfs subvolume snapshot -r /home /home/$today
sync
btrfs send /home/$today | btrfs receive /backup/home
sync
```

### Repetition

For next backups, I also mounted the drive and created a snapshot:

```sh
cryptsetup luksOpen /dev/sdb backup
mkdir -p /backup
mount -o noatime,compress-force=zstd:14 /dev/mapper/backup /backup
today=$(date --iso-8601)
btrfs subvolume snapshot -r /home /home/$today
sync
```

Say the latest snapshot was on the `$previous` day, I only needed to send
the difference between the old and new backup.  Afterwards, it is safe to delete the local `$previous` snapshot to save some space.

```sh
btrfs send -p /home/$previous /home/$today | btrfs receive /backup/home
btrfs subvolume delete /home/$previous
sync
```

Finally, unmount the drive and close the LUKS volume:

```sh
umount /backup
cryptsetup luksClose backup
```

Is this more complicated than good ole `rsync`?  Yes.  Is it safer?  Also yes,
thanks to copy-on-write.  Would I bother using one of the tools suggested
in the wiki?  Probably not, I've already documented everything in this article
in case I forget anything.

[^metric]: 63.5 mm for those outside of the land of guns and burgers
[^new]: OK, maybe not new, but certainly shinny
[^memory]: Slightly larger since some of the memory is dedicated to graphics

[grahamc]: https://grahamc.com/blog/erase-your-darlings
[elis]: https://elis.nu/blog/2020/05/nixos-tmpfs-as-root
[impermanence]: https://github.com/nix-community/impermanence
[christine]: https://christine.website/blog/paranoid-nixos-2021-07-18
[my Nix configurations]: https://git.sr.ht/~cnx/dotfiles/tree/master/item/nix
[NixOS unstable live image]: https://channels.nixos.org/nixos-unstable
[LUKS]: https://gitlab.com/cryptsetup/cryptsetup
[Btrfs Wiki]: https://btrfs.wiki.kernel.org/index.php/Incremental_Backup