Impermanence support #9

New issue

Closed

opened 2023-09-26 19:15:38 +00:00 by nerf · 1 comment

nerf commented

2023-09-26 19:15:38 +00:00

Owner

We should set up Impermanence, this will keep the machines clean,
and not let accidental state fly around.

Implementation might be non trivial though. What do we need permanent?

/boot/
/nix/
/etc/machine-id
/etc/ssh/*._key
/etc/ssh/*._key.pub
/var/log
/var/lib/nixos

/etc/machine-id is tricky, as it needs to be available before systemd, but the impermanence service uses a systemd unit for setup. As this file doesn't need to be edited after it was generated, we might link it with environment.etc, but that would make setup annoying, as
install and final machine config would differ. And we need to manually copy a file after first run.

for the mail server

config.mailserver.dkimKeyDirectory
config.mailserver.mailDirectory

but if these are already on a seperate disk everything is fine.

Implementing this basically means resetup of the disk layout. And even if this isn't a complete reinstall it can break things easily and is
not completely handled by nix. Also rollback is not an option, so this is more like heart surgery.

Boot setup might get a bit more tricky. If we put /boot/ and /nix/ on its own partition that would be easy again but would require kernel copys from the nix store to boot and more important all these devices have their own seperate size constraints.

A solution to all of this would making the persistent drive btrfs (or any other filesystem that supports subvolumes (afaik btrfs is the only one by default included in the kernel)). Then we would setup /boot/ and /nix/ as subvolumes. And another subvolume for the rest of persistent data. Then we don't need to think about which one needs exactly how much space. They would just share the same pool.
And if the subvolumes for /boot/ and /nix/ have the same relation to each other as on we don't even need to copy the kernels.

~~Maybe this can also archived with bind mounts, but then the order of mounting things during boot is important~~
The bootloader config generator thinks it is a separate drive and outputs the wrong paths

We should set up [Impermanence](https://github.com/nix-community/impermanence), this will keep the machines clean, and not let accidental state fly around. Implementation might be non trivial though. What do we need permanent? - `/boot/` - `/nix/` - `/etc/machine-id` - `/etc/ssh/*._key` - `/etc/ssh/*._key.pub` - `/var/log` - `/var/lib/nixos` `/etc/machine-id` is tricky, as it needs to be available before `systemd`, but the impermanence service uses a systemd unit for setup. As this file doesn't need to be edited after it was generated, we might link it with `environment.etc`, but that would make setup annoying, as install and final machine config would differ. And we need to manually copy a file after first run. for the mail server - config.mailserver.dkimKeyDirectory - config.mailserver.mailDirectory but if these are already on a seperate disk everything is fine. Implementing this basically means resetup of the disk layout. And even if this isn't a complete reinstall it can break things easily and is not completely handled by nix. Also rollback is not an option, so this is more like heart surgery. Boot setup might get a bit more tricky. If we put `/boot/` and `/nix/` on its own partition that would be easy again but would require kernel copys from the nix store to boot and more important all these devices have their own seperate size constraints. A solution to all of this would making the persistent drive btrfs (or any other filesystem that supports subvolumes (afaik btrfs is the only one by default included in the kernel)). Then we would setup `/boot/` and `/nix/` as subvolumes. And another subvolume for the rest of persistent data. Then we don't need to think about which one needs exactly how much space. They would just share the same pool. And if the subvolumes for `/boot/` and `/nix/` have the same relation to each other as on we don't even need to copy the kernels. ~~Maybe this can also archived with bind mounts, but then the order of mounting things during boot is important~~ The bootloader config generator thinks it is a separate drive and outputs the wrong paths