nixConfig

This repository contains the configuration of all our machines running NixOS.

Build a machine

There are multiple ways to build and deploy a machine configuration. Which is the most appropriate depends on the context and scenario. So first there will be a general explanation how this works and afterwards we will talk about some scenarios.

If you run nix flake show you should get an output similar to this

$ nix flake show
git+file:///home/nerf/git/nixConfig?ref=refs%2fheads%2fnyarlathtop&rev=9d0eb749287d1e9e793811759dfa29469ab706dc
├───apps
│   └───x86_64-linux
├───checks
│   └───x86_64-linux
├───devShells
│   └───x86_64-linux
├───formatter
├───legacyPackages
│   └───x86_64-linux omitted (use '--legacy' to show)
├───nixosConfigurations
│   └───nyarlathotep: NixOS configuration
├───nixosModules
├───overlays
└───packages
    └───x86_64-linux

we can see there is an output called nixosConfigurations.nyarlathotep. Which contains the configuration of the machine called nyarlathotep. nixosConfigurations is special in that sense, that nixos-rebuild will automatically look for this key and assume how it is structured. The interesting part for us is the derivation config.system.build.toplevel. Its closure contains the whole system and the resulting derivation a script that changes the current system to that derivation. (called /bin/switch-to-configuration).

So what we want to archive is populate the nix store of the target machine with the closure of the derivation .#nixosConfigurations.<name>.config.system.build.toplevel and run the the resulting script on the target machine.

Local

It has multiple benefits to build the system configuration on the local computer and push it to the target server. For example one doesn't stress the server with the load of evaluating the expression and building the closure. Also the server doesn't need to fetch the build dependencies this way. One has a local check if at least the nix syntax was correct. And so on...

Build

If you have this repository local in your current directory you can just run:

$ nix build .#nixosConfigurations.<name>.config.system.build.toplevel

But you don't need to clone this repository, for more see the nix flake --help documentation about flake urls.

Copy

After we build the derivation we need to get the closure onto the target system. Luckily nix has tools to do that via ssh. We could just run:

$ nix copy -s --to <however you setup your ssh stuff> .#nixosConfigurations.<name>.config.system.build.toplevel

This will evaluate the flake again to get the store path of the given derivation. If we want to avoid this we might supply the corresponding store path directly.

The -s is important it makes the target machine substitute all derivations it can (by default from chache.nixos.org). So you only upload configuration files and self build things.

To be able to copy things to a machine they need to be signed by someone trusted. Additional trusted nix keys are handled in ./nixos/roles/nix_keys.nix. So to get yourself trusted you either need to install one derivation from the machine itself, or find someone who is already trusted, to push your key.

For more information on signing and key creation see nix store sign --help and nix key --help.

Activate

Log into the remote machine and execute (with root privileges)

# /nix/store/<storepath>/bin/switch-to-configuration boot

That will setup a configuration switch at reboot. You can also switch the configuration live. For more details consider the --help output of that script. The storepath (or at least the hash of the derivation) is exactly the same it was on your machine.

If you have a nixos-rebuild available on your system it can automatize these things with the --flake and --target-host parameters. But there are some pitfalls so look at the nixos-rebuild documentation beforehand.

On the machine

clone this repository to /etc/nixos/ and nixos-rebuild boot or nixos-rebuild switch that will select the appropriate machine based on hostname.

If the hostname is not correct, or you don't want to clone this flake you can also use the --flake parameter.

In any case, to switch the system configuration you will need to have root privileges on the target machine.

How this flake is organized

This flake uses flake-parts see flake.parts for more details. It makes handling system and some other modules related things more convenient. For the general layout of nixos system configuration and modules, please see the corresponding documentation.

The toplevel flake.nix contains the flake inputs as usual and only calls a file flake-module.nix. This toplevel flake-module.nix imports further more specialized flake-modules.nix files from sub-directories. Right now the only one is nixos/flake-module.nix. But if we start to ship our own software (or software versions, with specific build flags), this might get more.

nixos

The nixos folder contains all machine configurations. It separates in two folders nixos/machines and nixos/roles. The corresponding flake-module.nix file automatically searches for machines/<name>/configuration.nix, and evalutes those as nixos configurations, and populates the flake.

machines

nixos/machines contains all machine specific configuration (in a sub-folder per machine). Like hardware configuration, specific network configuration. And service configuration that are too closely interwoven with the rest of that machine (for example mailserver configuration depends heavily on network settings). It also contains the root configuration for that machine called configuration.nix. This file usually only includes other modules. These configuration.nix files are almost usual nix configurations. The only difference is, that they take as an extra argument the flake inputs. This allows them to load modules from these flakes. For example nyarlathotep loads the simple-nixos-mailserver module that way.

roles

nixos/roles contains configuration that is potentially shared by some machines. It is expected that nixos/roles/default.nix is imported as (../../roles) in every machine. Notable are the files nixos/roles/admins.nix which contains common admin accounts for these machines and nixos/roles/nix_keys.nix which contains the additional trusted keys for the nix store.

sops

We are sharing secrets using sops and sops-nix As of right now we use only age keys. The machine keys are derived from their server ssh keys, that they generate at first boot. To read out a machines public key run the following command on the corresponding machine.

$ nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'

User keys are generated by the users. New keys and machines need entries into the .sops.yaml file within the root directory of this repository.

To make a secret available on a given machine you need to configure the following:

sops.secrets.example-key = {
  sopsFile = "relative path to file in the repo containing the secrets (optional else the sops.defaultSopsFile is used)";
  path = "optinal path where the secret gets symlinked to, practical if some program expects a specific path";
  owner = user that owns the secret file: config.users.users.nerf.name (for example);
  group = same as user just with groups: config.users.users.nerf.group;
  mode = "permission in usual octet: 0400 (for example)";
};

Afterwards the secret should be available in /run/secrets/example-key. If the accessing process is not root it must be member of the group config.users.groups.keys for systemd services this can be archived by setting serviceConfig.SupplementaryGroups = [ config.users.groups.keys.name ]; it the service configuration.