nyarlathotep: cleanup after deployment #55
No reviewers
Labels
No labels
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Fachschaft/nixConfig#55
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "Gonne/nixConfig:nyarlathotep"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
590ea741d0
tod7f4598be3
d7f4598be3
tob3ac11ddc9
b3ac11ddc9
to9c26820b8f
9c26820b8f
toc5849b8695
First few comments
@ -187,0 +214,4 @@
++ ["sieve.trusted.*"]; #for macros to be able to include our redirection script
sieve.trusted = {
scripts.redirects.contents = "%{file:/tmp/virt_aliases}%"; # generated redirect script
trusted.from-addr = "sender"; # set the from-address to the original sender as specified in the MAIL FROM.
the
trusted
part is to much, also see the comment atfrom-name
below@ -187,0 +215,4 @@
sieve.trusted = {
scripts.redirects.contents = "%{file:/tmp/virt_aliases}%"; # generated redirect script
trusted.from-addr = "sender"; # set the from-address to the original sender as specified in the MAIL FROM.
from-name = "sender";
I couldn't figure out what
from-name
orfrom-addr
actually do. Reading the documentation I believe combined theyset the default value for the
From:
header of a generated mail. But I couldn't verify this in tests.Maybe it does something different that has to do with the
MAIL FROM:
see comment below@ -187,0 +216,4 @@
scripts.redirects.contents = "%{file:/tmp/virt_aliases}%"; # generated redirect script
trusted.from-addr = "sender"; # set the from-address to the original sender as specified in the MAIL FROM.
from-name = "sender";
return-path = "sender";
This seems to do what we want, even though the documentation reads like it sets the
Return-Path:
header. But it seems to set the reverse path (which is the argument to theMAIL FROM:
smtp command). Maybe it does both?This parameter seems to control the
MAIL FROM:
and with this all the headers we want to set.The documentation reads to me as if it sets the
Return-Path:
header. I'm unsure if it actually does it.So we want this?
4c946968e1
to5e0cb9ebcc
5e0cb9ebcc
tod27f4b942e
d27f4b942e
toc24757321e
c24757321e
to6271e04c10
nyarlathotep: cleanup after deploymentto WIP: nyarlathotep: cleanup after deployment6271e04c10
to19351ef316
19351ef316
tocb87976ce8
WIP: nyarlathotep: cleanup after deploymentto nyarlathotep: cleanup after deploymente05dde9ab3
to67a9179216
67a9179216
to23d85d5364
As the alias to sieve program is basically part of this config, it should end up under the same
moderation rules, right? So either integrate this as a package into this repo or at least do
branch protection on it. I'm fine with leaving it without until it is somewhat stable.
@ -74,2 +76,4 @@
mode = "0440";
};
"dkim_rsa" = {
sopsFile = ./dkim.keys.yaml;
see below
@ -76,0 +82,4 @@
mode = "0440";
};
"dkim_ed25519" = {
sopsFile = ./dkim.keys.yaml;
see below
@ -0,0 +1,40 @@
dkim_rsa: ENC[AES256_GCM,data:cVzKHs/1H/8UL2aQ6fiXLFn0Y0yTGUUss/G9NiXtJMwWpa1SDuONs6CaplWF/c1z8Ph4b4GgQQHQqXGKnZIacpUlv1C0y1W5rr4DNqsWQ9F1Ncx7NIZDHJ3nQ2KKXy+I7NgxwdIuqBtg9ZticYZjf1ArcWUGnt+UEDmgXw4fSo05YS+scg0o5hyrkrduZntBBlUu8hH0qMrE8usptGAmR+iwJ33U5Xan0G0eURVCQJ9xV7tUkZERmZi1TtEmuKa7TCTzNWTHWjuDFRdQ0u6EWajCVa8/UcswTKuKLh0h9OU6DPt8lHYgshiSF1SRRiDq5ytjAFMMpA0hfrqpDx2LQtnyZIv/E8ZGtt5QeikUUTgLMmrqIkMddGufPp8lvFCLh1dlCf1QuiQQmNyMsNPAuu5UzUNCel4ideJFYm3hEoPUQ8uHNmujCOi89NpTwFyp9p0By/4fGWFPezn9VxOKhID0/zKUHp7jUAbZT66XbyDmv6TG0AYGNWhWsrjcCyGKCybOjV7+Wm5viVDFY5chojHciQMG/nEu47vBNJwUhAD/r0T3hisfixuh3rtDvj6w/UXB6xkQi8TDyfjWpZF2ay/DwNcK0HAyOfAYyXVWU7Ck2D8NY3+YQrxaYhY/GAjBM/R0n/dpHBh9EInlyEFhvZhB5KwEuaVHSxtcudFxt5IZ8wzEC8PZIuFHnPJDXfjth5SjzVaQ6tBkvof/eMQmc2XDMofZoQODPOYL5RUifWDx7fQlgsKgLmhR6PgWigqZxis4V7XAT3BiqaYyxxdnYK08mR7dmm04o+TPWx6gQ7xTpW0zoufetBglwuxdEuzWoaTEs+vH5YCJfEdZ3ddk7IT3R8pTC3YrAIrD+IWkxolVk4nUvYWkaO+7pVSGO/QFI0ZaHDV4qK8cCD2p315LecL2bSnymXPKuHCGQHauwvgyGgja5+fs7VtteYPNLc71TONAWAV4Gh+LIejKDe6gnovEkHSKU1/q9qkELMTbnjYLM42CRGfg9K7Rf0ywwdv654yQr6wC/+wzDLcfmcqjiw1a3woEecAsqQ+RmpiFq80eCi6ZZCnLCa+kseV1+j48B1lwgQZg+9LwrV8YHG0ciW8IxhZ9O0wUMv/o2Udwo+NfA5iha+EcIBSr7VoV/PVIKZSpb3JeNbfZ/AwOr1y8/LyyoX7VtvIK8jOdulpOtwHAZ0GX5dYrH/gWgjdyfVbd7irehO15y1L5jbNulzouv69aLYwwQxUcmRK+O/krNDDp6Jy0Clz6+di2Lvm8W7ykk7NwMgTqlyUIi7jWTC5xEzY22bANqMuyE2s1sFdfxqLY7Tbb5PBJ9uzy45mwbM0760aOca1fAawwfwgsL4FkgHHQxn2SIMxmOB3+5kgCrelLKzk3Eu3Hq58rW53oVX+hSUd9YGLuCN0Re7+kybkHfWF/4r+A682Z5Zp5GLla/kCntZDPYODtz0Wl62AC21MAGv/RKWaUGWPaktx9M3w28YHa+mffuiCUSMdlN5TB12TVhsF3BSQ9rNztEfSuEtZzS8HbarsGg25wuv6gUQ36whBvgjmJJ/5/7Zc9a+l/mhKIblek+U+J5oKkQkiV3UuUdGzR7iYMXE9skt1b3JNYer6BaJQ+uaiJQsu4KVWj4H3G47owbtO9q7JMVnQ9SwbjuGf8tge1VV/ppD0t3Ay8S0bX+fd3dkDRR9zEG0UfKuWvpsLjyBqs+b/tsntMMB89BRrle4mZFhKlXVorQ7n1KV8o+2KC4y1Nkbg10HcPPQmsL+YGQG3OkWixpslMeIv8Y89RjBVxY/5A4BiO9FIe0Zt+rpAFUoFLvujkQc7Qau+b3kRFDk7agiETblUQxYMSPu4IqMxS5OM5mlahcsfEaYFn2AT9EBCGVi+ZKu+rufcsVkMf3TmOpMvXX+u7db8EvC1iosY5UUP6RziFd0WqUHbpRSrXXusPm038ddM5iifw5dW4s62cWfrcGZInD2mWwVXDtg3lDgAZZAK3flIMFnaTi1XTHJ5YrkrUm/DpYORsCXm2sLYPhUGdYT5OXYSjR6/3D6VyTHoxODLQSbc7t53LePFNw8cXK26vw6hDl/34ZE8NzE9RKBGI94FlX26VupYdcMdVWs5Ko+Q0ooFpYKGazDW+lLXWX/ntRODDcm+c0MI5Bq9zSt6b1WKoCrMZpDYEjMdjBdAdiK6Ia7zlOdOZwn97Xp1Lav0G7+eO4xwSTS/busXsOBSAKhk/Q3njkgBtnDuI71U28XP1BjGaTEQuXM0yJ0DX,iv:QbZVXp5FQhmYZvXxXNxWKrNm5GqM+2P3a5pPk499mlc=,tag:F+KNoPRnoLLhOpEj6Czj6Q==,type:str]
can this file be called
<something>.secrets.yaml
?@ -153,2 +158,3 @@
session = {
ehlo.require = [
{
"if" = "!is_empty(authenticated_as) || rcpt_domain == 'lists.mathebau.de' || starts_with(remote_ip, '192.168.0.')"; #TODO restrict trust by IP
the diff is very confusing here, this line should be compared to 182 in green
@ -1,6 +1,6 @@
/*
* Building: We patch our version of stalwart and thus need to built it locally.
* Be aware that this needs some hours, about 12Gb RAM and a few Gb free space in /tmp.
* Be aware that this needs some hours, about 20Gb RAM and a few Gb free space in /tmp.
how did this went up from 12 to 20?
I assume version updates from stalwart.
sigh, no local builds for me anymore (except nixpkgs, starts to cache stalwart)
@ -102,3 +102,3 @@
spam.header.is-spam = "Dummyheader"; # disable moving to spam which would conflict with forwarding
auth = {
# TODO check if HRZ conforms to these standards and we can validate them strictly
# TODO check if HRZ and our own VMs conform to these standards and we can validate them strictly
But Mailman shouldn't sign its mail, right? how do we solve that?
Possibly by moving all submission from our other vms to a separate listener.
I open an issue for later
@ -161,3 +191,3 @@
# Stalwart gets its configuration from two places: A TOML configuration file that we control in this module
# and from a database that can be configured from web management interface or via Rest API.
# We here define what comes from the TOML-file and especially add "sieve.trusted.scripts.*" to the default ones
# We here define what comes from the TOML-file and especially add "sieve.trusted.*" to the default ones
Is there still something left in the database that we could move here?
The list of domains served by stalwart and the existence of catch-all accounts remains in the database, but I have not found out how do set them from config.
I think we should move it to the config file, or do we need to dynamically change these?
Yes, at least while we don't provide mail boxes. But I don't know how.
That mailboxes need some state attached to them is clear to me (not necessarily, which mail boxes but user generated sieve scripts for example). Maybe I should open another question in the stalwart discussion thing.
@ -186,1 +214,3 @@
session.data.script = "'redirects'";
++ ["sieve.trusted.*"]; #for macros to be able to include our redirection script
sieve.trusted = {
scripts.redirects.contents = "%{file:/tmp/virt_aliases}%"; # generated redirect script
maybe we should move this file from
/tmp
into the/run
directory as this usually has properfile access rights set up.
There is also
JoinsNamespaceOf
forPrivateTmp
of systemd service units (https://www.man7.org/linux/man-pages/man5/systemd.exec.5.html) but I didn't succeed setting it.Reading https://lwn.net/Articles/436012/ I have no idea whether
/run
is a better place.We don't care about the early boot discussion. So the question is, is it runtime date or temporary data, wherever the boundary between these two is. Putting things in
/tmp
is often a buggy mess as this directory is usually somewhat of a global dumpster fire, where there are no guarantees that this file does not already exist with some important information for some other process (though unlikely).PrivateTmp
would save us here, but even the documentation says that one should not rely on this (and we would need to get it to work).If we already follow Lennart's advice then we also should use
mkstemp
to generate files in/tmp
which willbe very painful in a nix setting (as paths and filenames will only be available at runtime). Lennart also seems to think that
/run
is primarily for communication primitives (is this one?).See his blogpost.
So after reading all of this maybe the cleanest option is to get the
JoinNamespaceOf
option to work?I got the private
/tmp
withJoinsNamespaceOf
working.@ -187,0 +237,4 @@
private-key = "%{file:/run/secrets/dkim_rsa}%";
domain = "${domain}";
selector = "rsa-default";
headers = ["From" "To" "Cc" "Date" "Subject" "Message-ID" "Organization" "MIME-Version" "Content-Type" "In-Reply-To" "References" "List-Id" "User-Agent" "Thread-Topic" "Thread-Index"];
I will just trust you on the header selection
@ -33,4 +33,3 @@
transport_maps = ["hash:/var/lib/mailman/data/postfix_lmtp"];
local_recipient_maps = ["hash:/var/lib/mailman/data/postfix_lmtp"];
proxy_interfaces = "130.83.2.184";
smtputf8_enable = "no"; # HRZ does not know SMTPUTF8
but we talk to us and not hrz, right?
Does the commit message explain your question?
no, my comment was attached to the
smtputf8_enable
line not the deleted oneAh, right. Let's test that.
Works
@ -9,3 +9,3 @@
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "15ac54053752c25a0e545dd1953de716abcc80b12cfe0b6c2f2c1c73759d4f45"
+source = "git+https://gitea.mathebau.de/fachschaft/sieve-rs.git#71324550504c0f84fe3e814d05cbe22f90a3b228"
+source = "git+https://github.com/stalwartlabs/sieve.git#56450c6ccdf76f1de95931db24896599159efc53"
YaY, hopefully this can disappear soon
@ -186,1 +216,3 @@
session.data.script = "'redirects'";
++ ["sieve.trusted.*"]; #for macros to be able to include our redirection script
sieve.trusted = {
scripts.redirects.contents = "%{file:/tmp/virt_aliases}%"; # generated redirect script
where did the
/tmp
folder discussion disappeared to?ahh it got marked outdated
Apart from which code lives where, this config now has all features I can think of for now.
@ -15,2 +15,2 @@
"rev": "c37b6ec8654db4c6e3d79acaeeccb577a9fb66ce",
"revCount": 21,
"rev": "3570db39fda3627d60fbd4ef9c6326b3de074eb8",
"revCount": 23,
at some point someone should review that script...
@ -231,2 +302,3 @@
# The regex searches for alphanumerics combined with some special characters as local paths and the right domain.
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(<${cfg.stalwartAdmin})" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" | tee /tmp/addresses
# Exclude @domain.tld which is not a valid mail address but used for catch-all accounts.
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(<${cfg.stalwartAdmin})" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&*+-/=?^_{|}~]*@${domain}" | grep -v "@${domain}" | tee /tmp/addresses
The HRZ doesn't allow
'
either so we don't loose anything (see the mails from the cert team, or readour discussion in chat if you want to see the world burn)
@ -233,2 +304,3 @@
${pkgs.curl}/bin/curl -s --header "authorization: Basic $(<${cfg.stalwartAdmin})" http://localhost/api/principal | ${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&*+-/=?^_{|}~]*@${domain}" | grep -v "@${domain}" | tee /tmp/addresses
# This line searches for available redirects and adds them to the submission file.
${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&'*+-/=?^_{|}~]*@${domain}" /tmp/virt_aliases >> /tmp/addresses # This doesn't catch all RFC conform local parts. Improve if you need.
${pkgs.gnugrep}/bin/grep -o -e "[A-Za-z0-9.!#\$%&*+-/=?^_{|}~]*@${domain}" /tmp/virt_aliases >> /tmp/addresses # This doesn't catch all RFC conform local parts. Improve if you need.
See above. This parses a sieve script! That includes
'
in its syntax so this would break anyway if we allowed'
in E-Mail addresses. Maybe we should parse the alias file instead. That has its own problems though.We can discuss this in a separate issue.
@ -295,3 +344,1 @@
RestrictRealtime = true;
RestrictSUIDSGID = true;
};
#Generate a sieve script from the virtual alias file
smart