Mail: timeout per stap met fase en verstreken tijd in het log

De SMTP-verzender gaf alle stappen samen een tijdsbudget; nu krijgt elke stap (verbinden met STARTTLS,
inloggen, versturen) zijn eigen volledige timeout. Bij een timeout noemt de code de stap
(timeout_connect, timeout_auth, timeout_send) en toont het log per stap de verstreken milliseconden.
TimeoutSeconds blijft instelbaar (standaard 10), bereik nu 1 tot 120. Infra-doc met veiligheidsmodus,
timeout en leeswijzer voor het journal.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-09 01:30:16 +01:00
co-authored by Claude Sonnet 5.5
parent 892c548931
commit 190deb0da8
13 changed files with 168 additions and 22 deletions
@@ -61,3 +61,11 @@ The `From:` address of every mail is decided in this order: (1) the sender chose
**So for `mail.andreaslop.nl` the ASF host must set**, in `shared/env`: `Email__FromAddress=<an address on andreaslop.nl>` and `Email__AllowedFromAddresses__0=<the same address>` (plus `Email__FromName`), where that address is one the SMTP account is allowed to use as sender. With the default `no-reply@slpsoftware.nl` the server is likely to refuse the sender (error code `rejected_sender`, SMTP 5xx at MAIL FROM) or not relay it. A change of the committed ASF default to an `@andreaslop.nl` address is a decision for Bryan (not made).
**Diagnosis in the log** (Pi-Main journal): at start `mail_status enabled=true smtp_host=... smtp_port=... starttls=... default_from_domain=...`; per failed mail `identity_mail_failed code=... detail=... from_domain=... smtp_port=...`, where `detail` for a refused command is `SmtpCommandException phase=SenderNotAccepted|RecipientNotAccepted|MessageNotAccepted smtp_status=<name>(<number>) reply="<first line of the server reply, addresses replaced by [adres], at most 120 characters>"`. The codes shown to the administrator: `rejected_sender`, `rejected_recipient`, `rejected`, `auth_failed`, `tls_failed`, `connection_failed`, `timeout`, `not_configured`, `unknown`. Never logged: credentials, addresses, tokens, links, the rest of a server reply.
## SMTP connection, security mode and timeouts (update 2026-11-09, for the Pi-Main journal questions)
- **Security mode**: `Email__Smtp__EnableSsl=true` (the default) means MailKit `SecureSocketOptions.StartTls` (plain connection, then the STARTTLS upgrade; the normal setting for port 587). `false` means no encryption at all (`SecureSocketOptions.None`, only for a local catcher such as smtp4dev). Implicit TLS (port 465) is not supported.
- **Authentication**: only when `Email__Smtp__Username` is set (then the password is required); MailKit picks the strongest mechanism the server offers.
- **Timeout**: `Email__Smtp__TimeoutSeconds`, default **10**, allowed **1 to 120** (a value outside that switches mail off at start-up with an Error line naming the setting). It applies to EACH step separately (connect including the STARTTLS upgrade, login, send), so a slow login no longer uses up the time of the send; the worst case for one mail is three times the value. An administrator's request waits for it, so keep it modest; raise it (for example to 30) when the server is slow to log in.
- **Per environment**: every SMTP setting lives in that host's `shared/env` (host, port, EnableSsl, username, password, timeout, `App__PublicBaseUrl`); hosts can differ (SlpSoftware `mail.slpsoftware.nl`, ASF probably `mail.andreaslop.nl`).
- **Reading a failure in the journal**: `identity_mail_failed code=<code> detail=... phase=<connect|auth|send> timings="connect=120ms auth=4800ms(failed)" timeout_per_phase_s=10 from_domain=... smtp_port=...`. `timings` lists every step that ran and how long it took in milliseconds; the step marked `(failed)` is where it went wrong. MailKit does the STARTTLS upgrade inside the connect step, so "connect" covers the TCP connection, the greeting and the TLS upgrade together. Timeout codes: `timeout_connect`, `timeout_auth`, `timeout_send` (the screen text names the step). For a refused command the `detail` already holds the phase, the SMTP status code and the redacted first line of the server reply (since daaadcb). Nothing in the line is secret: no credentials, no addresses, no tokens.
- **Typical readings**: `auth_failed` with `phase=AUTH` = wrong username or password (and repeated failures can make a server throttle or block the client address, for example fail2ban); `timeout_auth` = the server accepted the connection but took too long to answer the login (slow authentication backend, or a throttle); `timeout_connect` = no greeting or STARTTLS in time (firewall, wrong port, server down); `timeout_send` = the server stalled while taking the message.
@@ -5,7 +5,7 @@ const t = (key: string) => key;
describe('mailErrorText', () => {
it.each([
'timeout', 'auth_failed', 'rejected', 'rejected_sender', 'rejected_recipient', 'tls_failed', 'connection_failed', 'not_configured', 'unknown',
'timeout', 'timeout_connect', 'timeout_auth', 'timeout_send', 'auth_failed', 'rejected', 'rejected_sender', 'rejected_recipient', 'tls_failed', 'connection_failed', 'not_configured', 'unknown',
])('has its own text for %s', (code) => {
expect(mailErrorText(t, code)).toBe(`users.invite.mailErrors.${code}`);
});
+1 -1
View File
@@ -1,7 +1,7 @@
/** A short, human reason for a failed mail (codes come from the server; unknown codes read as generic). */
export function mailErrorText(t: (key: string) => string, error: string | null | undefined): string {
const known = [
'timeout', 'auth_failed', 'rejected', 'rejected_sender', 'rejected_recipient',
'timeout', 'timeout_connect', 'timeout_auth', 'timeout_send', 'auth_failed', 'rejected', 'rejected_sender', 'rejected_recipient',
'tls_failed', 'connection_failed', 'not_configured', 'unknown',
];
return t(`users.invite.mailErrors.${known.includes(error ?? '') ? error : 'unknown'}`);
@@ -167,6 +167,9 @@
"mailFailed": "The e-mail could not be sent ({{reason}}). Copy the link and share it yourself, or try e-mailing again later.",
"mailErrors": {
"timeout": "the mail server did not respond in time",
"timeout_connect": "the mail server did not respond in time while connecting",
"timeout_auth": "the mail server did not respond in time while signing in",
"timeout_send": "the mail server did not respond in time while sending",
"auth_failed": "signing in to the mail server failed",
"rejected": "the mail server or the address refused the message",
"rejected_sender": "the mail server does not accept this sender address for this account",
@@ -168,6 +168,9 @@
"mailFailed": "De e-mail kon niet worden verstuurd ({{reason}}). Kopieer de link en deel die zelf, of probeer later opnieuw te mailen.",
"mailErrors": {
"timeout": "de mailserver reageerde niet op tijd",
"timeout_connect": "de mailserver reageerde niet op tijd tijdens het verbinden",
"timeout_auth": "de mailserver reageerde niet op tijd tijdens het inloggen",
"timeout_send": "de mailserver reageerde niet op tijd tijdens het versturen",
"auth_failed": "inloggen op de mailserver mislukt",
"rejected": "de mailserver of het adres weigerde het bericht",
"rejected_sender": "de mailserver accepteert dit afzenderadres niet voor dit account",
@@ -113,6 +113,7 @@ public class MailSoftFailureTests
[InlineData("Email:FromName", "", "Email:FromName")]
[InlineData("Email:Smtp:Port", "70000", "Email:Smtp:Port")]
[InlineData("Email:Smtp:TimeoutSeconds", "0", "Email:Smtp:TimeoutSeconds")]
[InlineData("Email:Smtp:TimeoutSeconds", "121", "Email:Smtp:TimeoutSeconds")]
[InlineData("Email:Smtp:Password", "", "Email:Smtp:Password")]
[InlineData("Email:LogoUrl", "javascript:x", "Email:LogoUrl")]
[InlineData("App:PublicBaseUrl", "", "App:PublicBaseUrl")]
@@ -25,8 +25,12 @@ internal sealed class StubSmtpServer : IDisposable
/// <param name="silent">Accept connections but never answer (to provoke the timeout).</param>
/// <param name="rejectMailFrom">Reply to MAIL FROM with this full line (for example "553 5.7.1 ...").</param>
/// <param name="rejectRcpt">Reply to RCPT TO with this full line.</param>
public StubSmtpServer(bool silent = false, string? rejectMailFrom = null, string? rejectRcpt = null)
private readonly string? _stallAt;
/// <param name="stallAt">"auth" advertises AUTH PLAIN and then never answers the login; "send" never answers MAIL FROM.</param>
public StubSmtpServer(bool silent = false, string? rejectMailFrom = null, string? rejectRcpt = null, string? stallAt = null)
{
_stallAt = stallAt;
_silent = silent;
_rejectMailFrom = rejectMailFrom;
_rejectRcpt = rejectRcpt;
@@ -47,7 +51,13 @@ internal sealed class StubSmtpServer : IDisposable
await writer.WriteLineAsync("220 stub ESMTP");
while (await reader.ReadLineAsync() is { } line)
{
if (line.StartsWith("EHLO", StringComparison.OrdinalIgnoreCase)) await writer.WriteLineAsync("250 stub");
if (line.StartsWith("EHLO", StringComparison.OrdinalIgnoreCase))
{
if (_stallAt == "auth") { await writer.WriteLineAsync("250-stub"); await writer.WriteLineAsync("250 AUTH PLAIN"); }
else await writer.WriteLineAsync("250 stub");
}
else if (line.StartsWith("AUTH", StringComparison.OrdinalIgnoreCase) && _stallAt == "auth") await Task.Delay(TimeSpan.FromSeconds(30));
else if (line.StartsWith("MAIL FROM", StringComparison.OrdinalIgnoreCase) && _stallAt == "send") await Task.Delay(TimeSpan.FromSeconds(30));
else if (line.StartsWith("MAIL FROM", StringComparison.OrdinalIgnoreCase)) { MailFrom = line; await writer.WriteLineAsync(_rejectMailFrom ?? "250 ok"); }
else if (line.StartsWith("RCPT TO", StringComparison.OrdinalIgnoreCase)) { Recipients.Add(line); await writer.WriteLineAsync(_rejectRcpt ?? "250 ok"); }
else if (line.StartsWith("DATA", StringComparison.OrdinalIgnoreCase))
@@ -137,7 +147,7 @@ public class SmtpEmailSenderTests
var started = DateTime.UtcNow;
var result = await Sender(server.Port, timeoutSeconds: 1).SendAsync(new OutgoingMail("to@example.org", "S", "<p>x</p>"));
result.Should().Be(MailSendResult.Fail(MailSendResult.Timeout));
result.Should().Be(MailSendResult.Fail(MailSendResult.TimeoutConnect));
(DateTime.UtcNow - started).Should().BeLessThan(TimeSpan.FromSeconds(10));
}
@@ -59,6 +59,9 @@ public class SmtpFailureTests(ITestOutputHelper output)
.Should().Be(MailSendResult.Rejected);
SmtpFailure.Classify(new AuthenticationException("bad pw-secret"), none).Should().Be(MailSendResult.AuthFailed);
SmtpFailure.Classify(new TimeoutException(), none).Should().Be(MailSendResult.Timeout);
SmtpFailure.Classify(new TimeoutException(), none, SmtpFailure.PhaseConnect).Should().Be(MailSendResult.TimeoutConnect);
SmtpFailure.Classify(new TimeoutException(), none, SmtpFailure.PhaseAuth).Should().Be(MailSendResult.TimeoutAuth);
SmtpFailure.Classify(new OperationCanceledException(), none, SmtpFailure.PhaseSend).Should().Be(MailSendResult.TimeoutSend);
SmtpFailure.Classify(new OperationCanceledException(), none).Should().Be(MailSendResult.Timeout);
SmtpFailure.Classify(new SocketException(), none).Should().Be(MailSendResult.ConnectionFailed);
SmtpFailure.Classify(new InvalidOperationException(), none).Should().Be(MailSendResult.Unknown);
@@ -96,6 +99,77 @@ public class SmtpFailureTests(ITestOutputHelper output)
public class SmtpSenderDiagnosticsTests
{
private static (SmtpEmailSender Sender, LogCapture Logs) SenderWith(StubSmtpServer server, int timeoutSeconds, string username = "")
{
var logs = new LogCapture();
var factory = LoggerFactory.Create(b => b.AddProvider(logs));
var sender = new SmtpEmailSender(
Options.Create(new EmailOptions
{
FromAddress = "no-reply@slpsoftware.nl", FromName = "Site",
Smtp = { Host = "127.0.0.1", Port = server.Port, EnableSsl = false, Username = username, Password = username.Length > 0 ? "pw-secret-9" : "", TimeoutSeconds = timeoutSeconds }
}),
factory.CreateLogger<SmtpEmailSender>());
return (sender, logs);
}
[Fact]
public async Task ATimeoutWhileConnecting_IsNamedAsSuch_WithTheElapsedTimeOfThatPhase()
{
using var server = new StubSmtpServer(silent: true);
var (sender, logs) = SenderWith(server, timeoutSeconds: 1);
var result = await sender.SendAsync(new OutgoingMail("a@example.org", "S", "<p>x</p>"));
result.Should().Be(MailSendResult.Fail(MailSendResult.TimeoutConnect));
var line = logs.Entries.Single(e => e.Level == LogLevel.Warning).Message;
line.Should().Contain("code=timeout_connect").And.Contain("phase=connect").And.Contain("timeout_per_phase_s=1");
System.Text.RegularExpressions.Regex.Match(line, @"connect=(\d+)ms\(failed\)").Groups[1].Value
.Should().NotBeEmpty();
long.Parse(System.Text.RegularExpressions.Regex.Match(line, @"connect=(\d+)ms").Groups[1].Value).Should().BeInRange(800, 5000);
}
[Fact]
public async Task ATimeoutDuringTheLogin_IsNamedAsSuch_AndShowsTheFinishedConnectPhase()
{
using var server = new StubSmtpServer(stallAt: "auth");
var (sender, logs) = SenderWith(server, timeoutSeconds: 1, username: "user-secret-9");
var result = await sender.SendAsync(new OutgoingMail("a@example.org", "S", "<p>x</p>"));
result.Should().Be(MailSendResult.Fail(MailSendResult.TimeoutAuth));
var line = logs.Entries.Single(e => e.Level == LogLevel.Warning).Message;
line.Should().Contain("code=timeout_auth").And.Contain("phase=auth");
line.Should().MatchRegex(@"connect=\d+ms ").And.MatchRegex(@"auth=\d+ms\(failed\)");
line.Should().NotContain("user-secret-9").And.NotContain("pw-secret-9");
}
[Fact]
public async Task ATimeoutWhileSending_IsNamedAsSuch()
{
using var server = new StubSmtpServer(stallAt: "send");
var (sender, logs) = SenderWith(server, timeoutSeconds: 1);
var result = await sender.SendAsync(new OutgoingMail("a@example.org", "S", "<p>x</p>"));
result.Should().Be(MailSendResult.Fail(MailSendResult.TimeoutSend));
logs.Entries.Single(e => e.Level == LogLevel.Warning).Message.Should().Contain("phase=send").And.MatchRegex(@"send=\d+ms\(failed\)");
}
[Fact]
public async Task EachPhaseGetsItsOwnFullTimeout_NotOneSharedBudget()
{
// The login stalls for one full timeout; the connect before it already used part of its own. With one shared
// budget the login would have been cut short: here it must run its whole second.
using var server = new StubSmtpServer(stallAt: "auth");
var (sender, logs) = SenderWith(server, timeoutSeconds: 1, username: "u");
await sender.SendAsync(new OutgoingMail("a@example.org", "S", "<p>x</p>"));
var line = logs.Entries.Single(e => e.Level == LogLevel.Warning).Message;
long.Parse(System.Text.RegularExpressions.Regex.Match(line, @"auth=(\d+)ms").Groups[1].Value).Should().BeGreaterThanOrEqualTo(900);
}
private static (SmtpEmailSender Sender, LogCapture Logs) Sender(StubSmtpServer server)
{
var logs = new LogCapture();
+4 -1
View File
@@ -48,7 +48,10 @@ public sealed class SmtpOptions
public string Username { get; set; } = "";
public string Password { get; set; } = "";
/// <summary>Connect and send are bounded by this, because mail is sent inside the request.</summary>
/// <summary>
/// Seconds allowed for EACH step (connect including STARTTLS, login, send), 1 to 120, default 10. Mail is sent
/// inside the administrator's request, so the worst case is three times this.
/// </summary>
public int TimeoutSeconds { get; set; } = 10;
}
@@ -32,8 +32,8 @@ public sealed class EmailOptionsValidator(IOptions<AppMailOptions> app) : IValid
failures.Add($"Email:FromName is required (1..{EmailAddressRules.MaxDisplayNameLength} characters, no control characters).");
if (options.Smtp.Port is < 1 or > 65535)
failures.Add("Email:Smtp:Port must be between 1 and 65535.");
if (options.Smtp.TimeoutSeconds is < 1 or > 60)
failures.Add("Email:Smtp:TimeoutSeconds must be between 1 and 60.");
if (options.Smtp.TimeoutSeconds is < 1 or > 120)
failures.Add("Email:Smtp:TimeoutSeconds must be between 1 and 120.");
if (!string.IsNullOrEmpty(options.Smtp.Username) && string.IsNullOrEmpty(options.Smtp.Password))
failures.Add("Email:Smtp:Password is required when Email:Smtp:Username is set.");
if (!string.IsNullOrEmpty(options.BrandColor) && !EmailAddressRules.IsValidColour(options.BrandColor))
@@ -6,6 +6,12 @@ public readonly record struct MailSendResult(bool Sent, string? ErrorCode)
public const string Disabled = "disabled";
public const string NotConfigured = "not_configured";
public const string Timeout = "timeout";
/// <summary>Timed out while connecting (TCP, greeting and the STARTTLS upgrade).</summary>
public const string TimeoutConnect = "timeout_connect";
/// <summary>Timed out during the login.</summary>
public const string TimeoutAuth = "timeout_auth";
/// <summary>Timed out while sending the message (MAIL FROM, RCPT TO, DATA).</summary>
public const string TimeoutSend = "timeout_send";
public const string AuthFailed = "auth_failed";
public const string Rejected = "rejected";
/// <summary>The server refused the sender address (MAIL FROM): the From address is not allowed for this account.</summary>
+36 -11
View File
@@ -41,32 +41,57 @@ public sealed class SmtpEmailSender(IOptions<EmailOptions> options, ILogger<Smtp
message.Body = new BodyBuilder { HtmlBody = mail.HtmlBody, TextBody = mail.TextBody }.ToMessageBody();
var timeout = TimeSpan.FromSeconds(_options.Smtp.TimeoutSeconds);
using var timeoutSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutSource.CancelAfter(timeout);
var phases = new List<string>(); // for the log: "connect=120ms auth=4800ms(timeout)"
var phase = SmtpFailure.PhaseConnect;
// Every phase gets its OWN full timeout (a slow login must not eat the time of the send), and the
// time each one took is kept, so a timeout says where it happened and how long that phase ran.
async Task Timed(string name, Func<CancellationToken, Task> action)
{
phase = name;
using var source = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
source.CancelAfter(timeout);
var clock = System.Diagnostics.Stopwatch.StartNew();
try
{
await action(source.Token);
phases.Add($"{name}={clock.ElapsedMilliseconds}ms");
}
catch
{
phases.Add($"{name}={clock.ElapsedMilliseconds}ms(failed)");
throw;
}
}
try
{
using var client = new SmtpClient { Timeout = (int)timeout.TotalMilliseconds };
await client.ConnectAsync(
// MailKit negotiates STARTTLS inside the connect step, so "connect" covers the TCP connection, the
// greeting and the TLS upgrade together.
await Timed(SmtpFailure.PhaseConnect, ct => client.ConnectAsync(
_options.Smtp.Host,
_options.Smtp.Port,
_options.Smtp.EnableSsl ? SecureSocketOptions.StartTls : SecureSocketOptions.None,
timeoutSource.Token);
ct));
if (!string.IsNullOrEmpty(_options.Smtp.Username))
await client.AuthenticateAsync(_options.Smtp.Username, _options.Smtp.Password, timeoutSource.Token);
await Timed(SmtpFailure.PhaseAuth, ct => client.AuthenticateAsync(_options.Smtp.Username, _options.Smtp.Password, ct));
await client.SendAsync(message, timeoutSource.Token);
await client.DisconnectAsync(true, timeoutSource.Token);
await Timed(SmtpFailure.PhaseSend, ct => client.SendAsync(message, ct));
await client.DisconnectAsync(true, CancellationToken.None);
return MailSendResult.Ok;
}
catch (Exception ex)
{
var code = SmtpFailure.Classify(ex, cancellationToken);
var code = SmtpFailure.Classify(ex, cancellationToken, phase);
// Never the whole exception text (it can quote addresses or credentials): the type, for a refused
// command its phase and status code, and a redacted, shortened first line of the server's reply.
logger.LogWarning("identity_mail_failed code={Code} detail={Detail} from_domain={FromDomain} smtp_port={Port}",
code, SmtpFailure.Describe(ex), DomainOf(fromAddress), _options.Smtp.Port);
// command its phase and status code, a redacted, shortened first line of the server's reply, and the
// time each phase took.
logger.LogWarning(
"identity_mail_failed code={Code} detail={Detail} phase={Phase} timings=\"{Timings}\" timeout_per_phase_s={TimeoutSeconds} from_domain={FromDomain} smtp_port={Port}",
code, SmtpFailure.Describe(ex), phase, string.Join(" ", phases), _options.Smtp.TimeoutSeconds, DomainOf(fromAddress), _options.Smtp.Port);
return MailSendResult.Fail(code);
}
}
+16 -3
View File
@@ -20,14 +20,18 @@ public static partial class SmtpFailure
{
public const int MaxReplyLength = 120;
public const string PhaseConnect = "connect";
public const string PhaseAuth = "auth";
public const string PhaseSend = "send";
[GeneratedRegex(@"[^\s@<>""'(),;:]+@[^\s@<>""'(),;:]+")]
private static partial Regex AddressPattern();
/// <summary>The short code shown to the administrator; the most specific one the failure allows.</summary>
public static string Classify(Exception ex, CancellationToken callerToken) => ex switch
public static string Classify(Exception ex, CancellationToken callerToken, string? phase = null) => ex switch
{
OperationCanceledException when !callerToken.IsCancellationRequested => MailSendResult.Timeout,
TimeoutException => MailSendResult.Timeout,
OperationCanceledException when !callerToken.IsCancellationRequested => TimeoutCode(phase),
TimeoutException => TimeoutCode(phase),
OperationCanceledException => MailSendResult.Unknown,
AuthenticationException => MailSendResult.AuthFailed,
SslHandshakeException => MailSendResult.TlsFailed,
@@ -43,6 +47,15 @@ public static partial class SmtpFailure
_ => MailSendResult.Unknown
};
/// <summary>"timeout_connect", "timeout_auth" or "timeout_send" when the phase is known, else plain "timeout".</summary>
public static string TimeoutCode(string? phase) => phase switch
{
PhaseConnect => MailSendResult.TimeoutConnect,
PhaseAuth => MailSendResult.TimeoutAuth,
PhaseSend => MailSendResult.TimeoutSend,
_ => MailSendResult.Timeout
};
/// <summary>The log description of a failure: the type, and for a refused command its phase, status code and a redacted first line.</summary>
public static string Describe(Exception ex)
{