Mail machine #47
No reviewers
Labels
No labels
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Fachschaft/nixConfig#47
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "Gonne/nixConfig:nyarlathotep"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
A general replacement of Eihort with software that probably allows for easier DKIM and DMARC setup
TODO list:
Optional for deployment:
84e613f0b0
to044326ad38
044326ad38
to597f4d365c
597f4d365c
to85131d6f36
WIP: Mail machineto Mail machine85131d6f36
toe3006b8e03
e3006b8e03
tob3aac99ca7
b3aac99ca7
to3bb4dcb120
3bb4dcb120
to42102bb1a0
42102bb1a0
tod2ab4d8eea
@ -56,0 +56,4 @@
overlays = [
(_: _: {
alias-to-sieve = inputs.alias-to-sieve.packages.x86_64-linux.default; # add custom package to convert alias files to sieve scripts on the stalwart machine
The package selection should depend on system, and is there a reason we overlay like this and not
just use
flake-inputs.alias-to-sieve...
in the system config?The
flake-inputs
parameter is not available as a module parameter whilepkgs
is supplied by the nix module system.@ -0,0 +41,4 @@
sops.secrets = {
# Password for the HRZ API that gets a list of mailaddresses that we serve
allowlistPassMatheball = {
sopsFile = ./allowlistPassMatheball.yaml;
The passwords can be in one yaml file and still be exposed as different secrets by sops. This can
be managed by the yaml structure
@ -0,0 +1,301 @@
/*
* Building: For some reason, stalwart is not served by cache.nixos.org and thus needs to be built locally.
ufff
@ -0,0 +1,301 @@
/*
* Building: For some reason, stalwart is not served by cache.nixos.org and thus needs to be built locally.
* Be aware that this needs some hours, about 12Gb RAM and a few Gb free space in /tmp.
double uff
@ -0,0 +2,4 @@
* Building: For some reason, stalwart is not served by cache.nixos.org and thus needs to be built locally.
* Be aware that this needs some hours, about 12Gb RAM and a few Gb free space in /tmp.
* Forwarding mails: Update the Sops-secrets in the machine directory, rebuild and deploy.
* Everything else should happen automatically but new redirects might take up to two hours due HRZ infrastructure.
isn't really a problem, this means we have time to rebuild stalwart to deploy the new alias
@ -0,0 +5,4 @@
* Everything else should happen automatically but new redirects might take up to two hours due HRZ infrastructure.
* Using the web admin interface: Set your SSH to do portforwarding of some local port to port 80 of the VM and
* and use your personal admin account or create one using the fallback admin password.
* Create users with mail boxes: Go to the admin interface and create them.
if the mailboxes are data on the machine, maybe the aliases should be too. This would also spare us to rebuild stalwart to update an alias
If only aliases are changed rebuilds on the VM itself are fast and lean.
@ -0,0 +31,4 @@
type = listOf (lib.types.submodule {
options = {
domain = mkOption {
type = str;
did you think about a more specialized type, non empty string? string matching regex? something like this?
@ -0,0 +47,4 @@
};
config = mkIf cfg.enable {
environment.systemPackages = [pkgs.alias-to-sieve]; # install converter from alias files to sieve scripts
does this need to be a system wide package or can we just call it in the right places?
There is no need for system wide.
@ -0,0 +62,4 @@
protocol = "smtp";
};
"submissions" = {
# Enabling sending from these domains privately blocked on https://github.com/stalwartlabs/mail-server/issues/618
???
@ -0,0 +73,4 @@
tls.implicit = true;
};
"management" = {
bind = ["[::]:80"]; # This must also bind publically for ACME to work.
maybe a comment what cthulhu should forward to us, and on which connections it handles tls
@ -0,0 +87,4 @@
};
spam.header.is-spam = "Dummyheader"; # disable moving to spam which would conflict with forwarding
auth = {
# TODO check if HRZ conforms to these standards and we can validate them strictly
lol
can we do it?
@ -0,0 +110,4 @@
}
{"else" = "'hrz'";}
];
tls = {
this is fine because we only talk to vms and hrz which are "trusted" entities
@ -0,0 +120,4 @@
address = "mailout.hrz.tu-darmstadt.de";
port = 25;
protocol = "smtp";
tls.implicit = false; # somehow this is needed here
I HATE starttls can we burn down the hrz already?
@ -0,0 +132,4 @@
session.rcpt = {
# In order to accept mail that we only forward
# without having to generate an account.
# Invalid addresses are filtered by DFN beforehand.
I love how we kind of abuse being in a shielded environment
@ -0,0 +142,4 @@
{"else" = false;}
];
};
config.local-keys =
this needs a comment!
@ -0,0 +165,4 @@
authentication.fallback-admin = {
user = "admin";
secret = "$argon2i$v=19$m=4096,t=3,p=1$d0hYOTkzclpzSmFTZUplWnhVeWE$I7q9uB19RWL0oZKaPlMPSlGfFp6FQ/vrx80FFKCsalg"; # see machine secret for plaintext
comment is outdated
@ -0,0 +167,4 @@
user = "admin";
secret = "$argon2i$v=19$m=4096,t=3,p=1$d0hYOTkzclpzSmFTZUplWnhVeWE$I7q9uB19RWL0oZKaPlMPSlGfFp6FQ/vrx80FFKCsalg"; # see machine secret for plaintext
};
tracer.stdout.level = "debug";
in production?
@ -0,0 +202,4 @@
}: ''
echo "process ${domain}"
# Get the mail addresses' local-part
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(</run/secrets/stalwartAdmin)" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" | tee /tmp/addresses
This line needs explanation. A LOT OF IT
@ -0,0 +205,4 @@
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(</run/secrets/stalwartAdmin)" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" | tee /tmp/addresses
${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" /tmp/virt_aliases >> /tmp/addresses # This doesn't catch all RFC conform local parts. Improve if you need.
# Post local-parts to HRZ
${pkgs.curl}/bin/curl -s https://www-cgi.hrz.tu-darmstadt.de/mail/whitelist-update.php -F emaildomain=${domain} -F password=$(cat ${allowlistPass}) -F emailliste=@/tmp/addresses -F meldungen=voll
A linkt to hrz docu, so we can look at it if something breaks
d89b132fa1
to6dbd7d6a80
We may also want to manage mailboxes in config
6dbd7d6a80
tobbea28a6cf
bbea28a6cf
to1160081e82
a15b0ee5d6
to6fee31dcf1
Mail machineto WIP: Mail machine00c753debc
to5f2ded90f5
5f2ded90f5
to86e1841131
86e1841131
tof5374c3e80
f5374c3e80
tof6f3464582
WIP: Mail machineto Mail machinef6f3464582
to8f8ed254f9
8f8ed254f9
to70b3a694c4
d34eae7e13
to73441141d0
Take this more as request for comments than actual request for changes, but I don't have this button
@ -1,19 +1,25 @@
{
can we separate flake updates in the future, that makes reviewing less painful
@ -37,3 +61,2 @@
"locked": {
"lastModified": 1727649413,
"narHash": "sha256-FA53of86DjFdeQzRDVtvgWF9o52rWK70VHGx0Y8fElQ=",
"lastModified": 1737831083,
maybe we should read this: https://github.com/nix-community/impermanence/pull/242
does this change how we want to handle machine-id files?
Moved to #51
@ -38,2 +62,2 @@
"lastModified": 1727649413,
"narHash": "sha256-FA53of86DjFdeQzRDVtvgWF9o52rWK70VHGx0Y8fElQ=",
"lastModified": 1737831083,
"narHash": "sha256-LJggUHbpyeDvNagTUrdhe/pRVp4pnS6wVKALS782gRI=",
and does this fix our bind mount problems https://github.com/nix-community/impermanence/issues/237 ?
@ -10,2 +5,2 @@
nixpkgs.follows = "";
};
alias-to-sieve = {
url = "git+https://gitea.mathebau.de/fachschaft/alias_to_sieve";
not sure if this should be in its own repo or just in this one
Feels like its own project to me.
@ -0,0 +1,52 @@
allowlistPass:
Is there a reason why the secrets are in different files? If yes we probably want to mention the reason somewhere
They feel large enough to me that different files are easier to navigate. But I don't mind.
I understand this for the alias files, but why don't pool the rest?
@ -0,0 +6,4 @@
* Forwarding mails: Update the Sops-secrets in the machine directory, rebuild on the VM and deploy.
* Everything else should happen automatically but new redirects might take up to two hours due HRZ infrastructure.
* Using the web admin interface: Set your SSH to do portforwarding of some local port to port 80 of the VM and
* and use your personal admin account or create one using the fallback admin password.
This should either move somewhere closer to main documentation or should have a reference at least a reference.
This seems important and I probably wouldn't look here first to find this information.
I think its intuitive here and will wait for specific suggestions.
I think we should open an issue, to start a doc folder. I also want to write a document for people to get started. I kind of think the README has outgrown this purpose.
@ -0,0 +34,4 @@
description = "Path to a file that contains the stalwart fallback admin password encoded for HTTP Basic Auth";
};
stalwartAdminHash = mkOption {
type = str;
nonempty?
@ -0,0 +98,4 @@
domains = ["fb04184.mathematik.tu-darmstadt.de" "imap.mathebau.de" "smtp.mathebau.de"];
default = true;
};
spam.header.is-spam = "Dummyheader"; # disable moving to spam which would conflict with forwarding
maybe make a note here, so we find this again after we have DKIM and DMARC
@ -0,0 +100,4 @@
};
spam.header.is-spam = "Dummyheader"; # disable moving to spam which would conflict with forwarding
auth = {
# TODO check if HRZ conforms to these standards and we can validate them strictly
https://www.hrz.tu-darmstadt.de/hrz_aktuelles/news_details_177024.en.jsp
Maybe parts of this are now?
I would like to evaluate this in production based on logs.
Fine by me, that should be an easy fix
@ -0,0 +126,4 @@
tls = {
# we only talk to HRZ and our own VMs anyway
mta-sts = "disable";
dane = "disable";
I'm sad about the state of DANE. I like it.
@ -0,0 +127,4 @@
# we only talk to HRZ and our own VMs anyway
mta-sts = "disable";
dane = "disable";
starttls = "optional"; # e.g. Lobon does not offer starttls
Is there a domain specific option? then we could only disable this for mailman and else require it.
The documentation mentions no such thing.
sad, is this an issue for upstream?
Only if someone cares enough to write one. I don't.
@ -0,0 +150,4 @@
catch-all = true;
relay = [
{
"if" = "!is_empty(authenticated_as) || rcpt_domain == 'lists.mathebau.de' || starts_with(remote_ip, '192.168.0.')"; #TODO restrict trust by IP
does this only work for local ipv4 connections? That might change at some point, right?
Yes, but we probably want to give credentials to the VMs and remove the trust by IP anyways.
Fine by me, but we will need to document somewhere how this is done. Else people after us will be lost
@ -0,0 +157,4 @@
];
};
# Stalwart gets its configuration from two places: A TOML configuration file that we control in this module
Ok, so in the TOML file there is a list of options that defines which options are set in the remainder of the TOML file. I didn't read that from this comment. (And maybe my understanding is still wrong)
@ -0,0 +185,4 @@
authentication.fallback-admin = {
user = "admin";
# see passwd on azathoth for plaintext or machine secret in encoded format for HTTP Basic AUTH
I think we need a note of the places that we need to touch when we need to update this password
@ -0,0 +188,4 @@
# see passwd on azathoth for plaintext or machine secret in encoded format for HTTP Basic AUTH
secret = cfg.stalwartAdminHash;
};
store = {
should we run a centralized postgres?
This database will handle mailboxes, some DKIM keys and stuff.
Out of scope.
@ -0,0 +226,4 @@
}: ''
echo "process ${domain}"
# This line gets the available mailboxes from stalwart's Rest API, searches for their addresses and collects them to a file for submission.
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(<${cfg.stalwartAdmin})" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" | tee /tmp/addresses
can we have a comment for this regex?
@ -0,0 +228,4 @@
# This line gets the available mailboxes from stalwart's Rest API, searches for their addresses and collects them to a file for submission.
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(<${cfg.stalwartAdmin})" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" | tee /tmp/addresses
# This line searches for available redirects and adds them to the submission file.
${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" /tmp/virt_aliases >> /tmp/addresses # This doesn't catch all RFC conform local parts. Improve if you need.
and this one? is it the same?
@ -0,0 +239,4 @@
serviceConfig = {
Type = "oneshot";
User = "stalwart-mail";
NoNewPrivileges = true;
I put my trust in you, that these are good
@ -0,0 +260,4 @@
RestrictSUIDSGID = true;
};
};
"stalwart-mail" = {
Can we make a comment that this is only an extension of the existing service?
@ -0,0 +271,4 @@
serviceConfig = {
Type = "oneshot";
User = "stalwart-mail";
NoNewPrivileges = true;
again I just trust you here
@ -0,0 +307,4 @@
# We don't want this in order to not need to persist borg cache and simplify new deployments.
BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK = "yes";
};
repo = "borg@192.168.1.11:kaluut"; # TODO for https://gitea.mathebau.de/Fachschaft/nixConfig/issues/33
nyarlathotep
73441141d0
to361eed84af
I want to have an answer if we want to put in an upstream feature request. And I opened another documentation issue, but else I think this is ready to merge.
274412b604
tocc91339f80
New commits pushed, approval review dismissed automatically according to repository settings