Merge pull request 'nyarlathtop' (#14) from dennis/nixConfig:nyarlathtop into nyarlathtop

Reviewed-on: #14

README.md

@@ -1,19 +1,115 @@
 # nixConfig
 
 ## Build a machine
+There are multiple ways to build and deploy a machine configuration. Which is the
+most appropriate depends on the context and scenario. So first there will be a general
+explanation how this works and afterwards we will talk about some scenarios.
+
+If you run `nix flake show` you should get an output similiar to this
+```
+$ nix flake show
+git+file:///home/nerf/git/nixConfig?ref=refs%2fheads%2fnyarlathtop&rev=9d0eb749287d1e9e793811759dfa29469ab706dc
+├───apps
+│   └───x86_64-linux
+├───checks
+│   └───x86_64-linux
+├───devShells
+│   └───x86_64-linux
+├───formatter
+├───legacyPackages
+│   └───x86_64-linux omitted (use '--legacy' to show)
+├───nixosConfigurations
+│   └───nyarlathotep: NixOS configuration
+├───nixosModules
+├───overlays
+└───packages
+    └───x86_64-linux
+```
+we can see there is an output callled `nixosConfigurations.nyarlathotep`. Which contains the config of the machine
+called nyarlathotep. `nixosConfigurations` is special in that sense, that `nixos-rebuild` will automatically look
+for this key and assume how it is structured. The interesting part for us is the derivation `config.system.build.toplevel`.
+Its closure contains the whole system and the resulting derivation a script that changes the current system to
+that derivation. (called `/bin/switch-to-configuration`).
+
+So what we want to archive is populate the nix store of the target machine with the closure of the derivation
+`.#nixosConfigurations.<name>.config.system.build.toplevel` and run the the resulting script on the target machine.
+
+
 ### Local
-If you want to build the machineconfiguration for machine <name>
+It has multiple benefits to build the system config on the local computer and push it to the target server.
-run
+For example one doesn't stress the server with the load coming with evaluating the expression. Also the server
+doesn't need to fetch the build dependencies this way. One has a local check if at least the nix syntax was correct.
+And so on...
+
+#### Build
+If you have this repository local in your current directory you can just run:
 ```
-nix build .#nixosConfiguration.<name>.config.system.build.toplevel
+$ nix build .#nixosConfigurations.<name>.config.system.build.toplevel
 ```
 
+But you don't need to clone this repository for more on flake urls see the `nix flake --help` documentation.
+
+#### Copy
+After we build the derivation we need to get the closure onto the target system. Luckily nix has tools to do that
+via ssh. We could just run:
+```
+$ nix copy -s --to <however you setup your ssh stuff> .#nixosConfigurations.<name>.config.system.build.toplevel
+```
+we do not need the flake anymore, instead of specifying the derivation name we could also give the store path
+directly.
+
+The `-s` is important it makes the target machine substitute all derivations it can (by default from chache.nixos.org).
+So you only upload config files and self build things.
+
+To be able to copy things to a machine they need to be signed by someone trusted. Additional trusted nix keys are handled
+in `./nixos/roles/nix_keys.nix`. So to get yourself trusted you either need to install one derivation from the machine itself,
+or find someone who is already trusted.
+
+For more information on signing and key creation see `nix store sign --help` and `nix key --help`.
+
+#### Activate
+Log into the remote machine and execute
+```
+# /nix/store/<storepath>/bin/switch-to-configuration boot
+```
+That will setup a configuration switch at reboot. You can also switch the configuration live. For more
+details consider the `--help` output of that script.
+
+
+If you have a `nixos-rebuild` available on your system it can automatize these things with the `--flake` and
+`--target-host` parameters. But there are some pitfalls so look at the `nixos-rebuild` documentation beforehand.
+
 ### On the machine
-clone this repo to `/etc/nixos/` and `nixos-rebuild` that will select
+clone this repo to `/etc/nixos/` and `nixos-rebuild boot` or `nixos-rebuild switch` that will select
-the appropriate machine based on hostname
+the appropriate machine based on hostname.
+
+If the hostname is not correct, or you don't want to clone this flake you can also use the `--flake` parameter.
+
+In any case, to switch the system configuration you will need to have root priviledges on the target machine.
 
 
-### sops
+## How this flake is organized
+
+This flake uses `flake-parts` see [flake.parts](https://flake.parts) for more details. It makes handling
+`system` and some other moudles related things more convenient.
+For the general layout of nixos system config and modules, please see the corresponding documentation.
+
+The toplevel `flake.nix` contains the flake inputs as usual and only calls a file `flake-module.nix`
+this toplevel `flake-module.nix` imports further more specialiesed `flake-modules.nix` files from subdirectories.
+Right now the only one is `nixos/flake-module.nix`.
+
+the `nixos` folder contains all machine configurations. It sepreates in two folders `nixos/machines` and `nixos/roles`.
+
+`nixos/machines` contains all machine specific configuration (in a subfolder per machine). Like hardware configuration, specific
+network configuration. And service configuration that are too closely intervowen with the rest of that machine. It also
+contains the root config for that machine called `configuration.nix`. This file usually only includes other modules.
+
+`nixos/roles` contains config that is pontentially shared by some machines. It is expected that `nixos/roles/default.nix`
+is imported as (`../../roles`) in every machine. Notable are the files `nixos/roles/admins.nix` which contains
+common admin accounts for these machines and `nixos/roles/nix_keys.nix` which contains the additional trusted
+keys for the nix store.
+
+## sops
 
 We are sharing secrets using [`sops`](https://github.com/getsops/sops) and [`sops-nix`](https://github.com/Mic92/sops-nix)
 As of right now we use only `age` keys.
@@ -35,4 +131,3 @@ afterwards the secret should be available in `/run/secrets/example-key`.
 If the accessing process is not root it must be member of the group `config.users.groups.keys`
 for systemd services this can be archived by setting `serviceConfig.SupplementaryGroups = [ config.users.groups.keys.name ];`
 it the service config.
-

flake.lock

@@ -21,11 +21,11 @@
         "nixpkgs-lib": "nixpkgs-lib"
       },
       "locked": {
-        "lastModified": 1685662779,
+        "lastModified": 1693611461,
-        "narHash": "sha256-cKDDciXGpMEjP1n6HlzKinN0H+oLmNpgeCTzYnsA2po=",
+        "narHash": "sha256-aPODl8vAgGQ0ZYFIRisxYG5MOGSkIczvu2Cd8Gb9+1Y=",
         "owner": "hercules-ci",
         "repo": "flake-parts",
-        "rev": "71fb97f0d875fd4de4994dfb849f2c75e17eb6c3",
+        "rev": "7f53fdb7bdc5bb237da7fefef12d099e4fd611ca",
         "type": "github"
       },
       "original": {
@@ -33,20 +33,36 @@
         "type": "indirect"
       }
     },
+    "impermanence": {
+      "locked": {
+        "lastModified": 1694622745,
+        "narHash": "sha256-z397+eDhKx9c2qNafL1xv75lC0Q4nOaFlhaU1TINqb8=",
+        "owner": "nix-community",
+        "repo": "impermanence",
+        "rev": "e9643d08d0d193a2e074a19d4d90c67a874d932e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "impermanence",
+        "type": "github"
+      }
+    },
     "nixos-mailserver": {
       "inputs": {
         "blobs": "blobs",
         "flake-compat": [],
         "nixpkgs": [],
         "nixpkgs-22_11": "nixpkgs-22_11",
+        "nixpkgs-23_05": "nixpkgs-23_05",
         "utils": "utils"
       },
       "locked": {
-        "lastModified": 1686468558,
+        "lastModified": 1689976554,
-        "narHash": "sha256-K69Ojlx3N8I6tRTZsrKFMIqK4yrnJ6/PjfKZi3wchYg=",
+        "narHash": "sha256-uWJq3sIhkqfzPmfB2RWd5XFVooGFfSuJH9ER/r302xQ=",
         "ref": "refs/heads/master",
-        "rev": "290d00f6db4e80467013728819ad73dd4a394d9a",
+        "rev": "c63f6e7b053c18325194ff0e274dba44e8d2271e",
-        "revCount": 554,
+        "revCount": 570,
         "type": "git",
         "url": "https://gitlab.com/simple-nixos-mailserver/nixos-mailserver.git"
       },
@@ -57,11 +73,11 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1686412476,
+        "lastModified": 1695145219,
-        "narHash": "sha256-inl9SVk6o5h75XKC79qrDCAobTD1Jxh6kVYTZKHzewA=",
+        "narHash": "sha256-Eoe9IHbvmo5wEDeJXKFOpKUwxYJIOxKUesounVccNYk=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "21951114383770f96ae528d0ae68824557768e81",
+        "rev": "5ba549eafcf3e33405e5f66decd1a72356632b96",
         "type": "github"
       },
       "original": {
@@ -86,14 +102,29 @@
         "type": "indirect"
       }
     },
+    "nixpkgs-23_05": {
+      "locked": {
+        "lastModified": 1684782344,
+        "narHash": "sha256-SHN8hPYYSX0thDrMLMWPWYulK3YFgASOrCsIL3AJ78g=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "8966c43feba2c701ed624302b6a935f97bcbdf88",
+        "type": "github"
+      },
+      "original": {
+        "id": "nixpkgs",
+        "ref": "nixos-23.05",
+        "type": "indirect"
+      }
+    },
     "nixpkgs-lib": {
       "locked": {
         "dir": "lib",
-        "lastModified": 1685564631,
+        "lastModified": 1693471703,
-        "narHash": "sha256-8ywr3AkblY4++3lIVxmrWZFzac7+f32ZEhH/A8pNscI=",
+        "narHash": "sha256-0l03ZBL8P1P6z8MaSDS/MvuU8E75rVxe5eE1N6gxeTo=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "4f53efe34b3a8877ac923b9350c874e3dcd5dc0a",
+        "rev": "3e52e76b70d5508f3cec70b882a29199f4d1ee85",
         "type": "github"
       },
       "original": {
@@ -123,6 +154,7 @@
     "root": {
       "inputs": {
         "flake-parts": "flake-parts",
+        "impermanence": "impermanence",
         "nixos-mailserver": "nixos-mailserver",
         "nixpkgs": "nixpkgs",
         "sops-nix": "sops-nix"

flake.nix

@@ -14,6 +14,9 @@
       url = "github:Mic92/sops-nix";
       inputs.nixpkgs.follows = "nixpkgs";
     };
+    impermanence = {
+      url = "github:nix-community/impermanence";
+    };
   };
 
   outputs = inputs@{ flake-parts, ... }:

nixos/flake-module.nix

@@ -2,6 +2,15 @@
 
 # This automatically searches for nixos configs in ./machines/${name}/configuration.nix
 # and exposes them as outputs.nixosConfigurations.${name}
+#
+
+# a comment regarding  pkgs.nixos vs lib.nixosSystem
+# while lib.nixosSystem is the usual enduser way to evaluate nixos configurations
+# in flakes, pkgs.nixos sets the package set to the packages it comes from.
+# This spares us tracking our potentiell overlays and own package additions, but just
+# using the right package set to begin with. Using lib.nixosSystem from the flake we would
+# need to specify that again.
+
 { withSystem, lib, inputs, ... }: {
   flake = {
     nixosConfigurations = withSystem "x86_64-linux" ({ pkgs, ... }:
@@ -12,6 +21,7 @@
           imports = [
             (import (./. + "/machines/${name}/configuration.nix") inputs)
             inputs.sops-nix.nixosModules.sops
+            inputs.impermanence.nixosModules.impermanence
           ];
         };
     in lib.genAttrs machines makeSystem);

nixos/machines/nyarlathotep/configuration.nix

@@ -0,0 +1,16 @@
+flake-inputs:
+{config, pkgs, lib, ... }: {
+
+imports = [
+  ./hardware-configuration.nix
+  (import ./mail.nix flake-inputs)
+  ../../roles
+  ../../roles/xen_guest.nix
+  ./network.nix
+];
+
+# System configuration here
+
+  networking.hostName = "nyarlathotep";
+  system.stateVersion = "23.11";
+}

nixos/machines/nyarlathotep/hardware-configuration.nix

@@ -0,0 +1,35 @@
+{config, lib, pkgs, modulesPath, ...}: {
+  imports = [ ];
+
+  fileSystems."/" = {
+    device = "nya-root";
+    fsType = "tmpfs";
+    options = [ "size=1G" "mode=755" ];
+  };
+  fileSystems."/persist" = {
+    device = "/dev/disk/by-uuid/a72da670-f631-49b1-bcb3-6d378cc1f2d0";
+    fsType = "btrfs";
+    options = [ "subvol=persist" ];
+    neededForBoot = true;
+  };
+  fileSystems."/boot" = {
+    device = "/dev/disk/by-uuid/75b01f48-e159-4d72-b049-54b7af072076";
+    fsType = "ext4";
+  };
+  fileSystems."/nix" = {
+    device = "/dev/disk/by-uuid/a72da670-f631-49b1-bcb3-6d378cc1f2d0";
+    fsType = "btrfs";
+    options = [ "subvol=nix" ];
+  };
+  fileSystems."/var/vmail" = {
+    device = "/dev/disk/by-uuid/23c44c93-5035-4e29-9e46-75c1c08f4cea";
+    fsType = "ext4";
+  };
+
+  swapDevices =
+    [{ device = "/dev/disk/by-uuid/8bc30d17-3c08-4648-ab18-8c723523be1a"; }];
+
+  nix.settings.max-jobs = lib.mkDefault 4;
+
+  nixpkgs.hostPlatform = lib.mkDefault "x86_64-linux";
+}

nixos/machines/nyarlathotep/mail.nix

@@ -0,0 +1,47 @@
+flake-inputs:
+{pkgs, config, lib, ...}: {
+  imports = [flake-inputs.nixos-mailserver.nixosModule];
+
+  mailserver = {
+    enable = true;
+    debug = false; # TODO disable
+    fqdn = "mathebau.de";
+    sendingFqdn = "fb04184.mathematik.tu-darmstadt.de";
+    domains = [
+      "mathebau.de"
+      "lists.mathebau.de"
+    ];
+    # forwards = #TODO
+    # loginAccounts = #TODO
+    # extraVirtualAliases = # TODO # only for local things (maybe don't use?)
+    certificateDomains = ["imap.mathebau.de"];
+    # certificateScheme = "manual"; # Do we need CERTS? We don't want to run a webmailer YES IMAP!!
+    #	certificateFile = #TODO
+    # keyFile = #TODO
+
+    enableSubmission = false; # no starttls smtp
+
+    # Fun dovecot stuff :
+
+    mailDirectory = "/var/vmail/vmail/"; # directory to store mail it was /var/mail/vmail but
+                                         # /var/mail ist special
+     
+    hierarchySeparator = "/"; # seperator for imap mailboxes from client view
+
+    # Caching of search indices
+    indexDir = "/var/vmail/lib/dovecot/indices";
+    fullTextSearch = {
+      enforced = "body"; # only brute force headers if no search index is available
+    };
+    lmtpSaveToDetailMailbox = "no";
+    # no starttls
+    enableImap = false;
+
+
+    # TODO checkout redis `config.services.redis.servers.rspamd.`
+
+    # TODO
+    # borgbackup = {
+    # };
+  };
+}

nixos/machines/nyarlathotep/network.nix

@@ -0,0 +1,15 @@
+# We sohuld put that config somewhere in roles and give it a parameter or something,
+# everyone gets the same nameserver and the same prefixLength and address vs defaultGateway alsways
+# depend on the same thing 
+{
+  imports = [ ];
+  networking = {
+    interfaces.enX0.ipv4.addresses = [ {
+      address = "192.168.0.28";
+      prefixLength = 16;
+    } ];
+    defaultGateway = "192.168.0.155";
+    nameservers = ["130.83.2.22" "130.83.56.60" "130.83.22.60" "130.82.22.63"];
+  };
+}
+

nixos/modules/impermanence.nix

@@ -0,0 +1,47 @@
+{lib, config, ...} :
+
+let 
+  inherit (lib)
+    mkEnableOption
+    mkIf
+    mkOption
+    types
+    ;
+  cfg = config.impermanence;
+in
+
+{
+imports = [ ];
+
+options.impermanence = {
+  enable = mkEnableOption "impermanence";
+  storagePath = mkOption {
+    type = types.path;
+    default = "/persist";
+    description = "The path where persistent data is stored";
+  };
+  name = mkOption {
+    type = types.str;
+    default = "persist";
+    description = "the name of the persistent data store";
+  };
+};
+
+config = mkIf cfg.enable {
+  environment.persistence.${cfg.name} = {
+    persistentStoragePath = cfg.storagePath;
+    directories = [
+      "/var/log"
+      "/var/lib/nixos"
+    ];
+    files = [
+      "/etc/ssh/ssh_host_ed25519_key"
+      "/etc/ssh/ssh_host_ed25519_key.pub"
+      "/etc/ssh/ssh_host_rsa_key"
+      "/etc/ssh/ssh_host_rsa_key.pub"
+    ];
+  };
+  environment.etc.machine-id.source = "${cfg.storagePath}/machine-id";
+};
+
+}

nixos/roles/admins.nix

@@ -0,0 +1,30 @@
+{lib, ...} :
+with lib;
+
+let
+  admins = {
+    nerf = {
+      hashedPassword =
+        "$y$j9T$SJcjUIcs3JYuM5oyxfEQa/$tUBQT07FK4cb9xm.A6ZKVnFIPNOYMOKC6Dt6hadCuJ7";
+        keys = [
+          "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEdA4LpEGUUmN8esFyrNZXFb2GiBID9/S6zzhcnofQuP nerf@nerflap2"
+        ];
+    };
+  };
+
+  mkAdmin = name :
+  {hashedPassword, keys}: {
+    "${name}" = {
+      isNormalUser = true;
+      createHome = true;
+      extraGroups = [ "wheel" ];
+      group = "users";
+      home = "/home/${name}";
+      openssh.authorizedKeys = { inherit keys; };
+      inherit hashedPassword;
+    };
+  };
+
+in {
+  users.users = mkMerge (mapAttrsToList mkAdmin admins);
+}

nixos/roles/default.nix

@@ -1,4 +1,58 @@
-{ ... } : {
+{pkgs, config, lib, ...} : {
+
+imports = [
+  ./admins.nix
+  ./nix_keys.nix
+  ../modules/impermanence.nix
+  ];
+nix = {
+  extraOptions = ''
+    experimental-features = nix-command flakes
+    builders-use-substitutes = true
+  '';
+};
+
+networking = {
+  firewall = { # these shoud be default, but better make sure!
+    enable = true;
+    allowPing = true;
+  };
+  nftables.enable = true;
+  useDHCP = false; # We don't speak DHCP and even if we would, we should enable it per interface
+  # hosts = # TODO write something to autogenerate ip adresses!
+};
+
+users = {
+  mutableUsers = false;
+  users.root.hashedPassword = "!";
+};
+
+impermanence.enable = true;
 
 sops.age.sshKeyPaths = [ "/etc/ssh/ssh_host_ed25519_key" ];
+
+environment = {
+  systemPackages = builtins.attrValues {
+    inherit (pkgs)
+      htop lsof tmux btop;
+  };
+};
+
+services = {
+  journald.extraConfig = "SystemMaxUse=5G";
+
+  nginx = {
+    recommendedOptimisation = true;
+    recommendedGzipSettings = true;
+    recommendedTlsSettings = true;
+  };
+
+  openssh = {
+    enable = true;
+    settings = {
+      PermitRootLogin = "no";
+      PasswordAuthentication = false;
+    };
+  };
+};
 }

nixos/roles/nix_keys.nix

@@ -0,0 +1,6 @@
+{
+  imports = [ ];
+  nix.settings.trusted-public-keys = [
+    "nerflap2-1:pDZCg0oo9PxNQxwVSQSvycw7WXTl53PGvVeZWvxuqJc="
+  ];
+}

nixos/roles/xen_guest.nix

@@ -0,0 +1,16 @@
+{...}: {
+  imports = [ ];
+  boot = {
+    loader.grub = {
+      device = "nodev";
+      enable = true;
+    };
+    initrd = {
+      availableKernelModules = [ "ata_piix" "sr_mod" "xen_blkfront" ];
+      kernelModules = [ ];
+    };
+    extraModulePackages = [ ];
+    tmp.useTmpfs = true;
+  };
+}
+