Master-URL: ongeldige of placeholder-waarde wordt nooit opgeslagen of verstuurd (master slaat achtergrondchecks over, slave weigert met 400, poller meldt invalid_master_url); veilige diagnostiek bij een mislukte poll; releasenote Master__MasterUrl

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-09 21:08:47 +01:00
co-authored by Claude Sonnet 5.5
parent 64a467b41b
commit 195443d9a9
11 changed files with 230 additions and 4 deletions
@@ -20,6 +20,12 @@ The code ships through the existing Gitea Actions pipeline (`deploy_asf_test`, `
Common: `Email__Smtp__Username`, `Email__Smtp__Password`, `App__PublicBaseUrl` (https, no path, per environment), optional `Email__Smtp__TimeoutSeconds` (default 10 per step), `Email__FromName`. Full list: `construction/mail-foundation/infrastructure-design/infrastructure-design.md`.
## Master host: `Master__MasterUrl` is required (test and production)
On every host that runs the master module (SlpSoftware test, SlpSoftware production), set `Master__MasterUrl` to the public address of that master: **https, no path**, e.g. `https://master.example.com`. The shipped `appsettings.json` carries only a placeholder (`<public-url-of-this-master-instance>`).
- **Why:** the hourly integrity check (and the one at start-up) runs without a request, so it cannot derive the address from the request and uses this setting. With the placeholder it used to re-register the slaves with the placeholder, after which a slave could not poll the master any more (`master_poll_failed`). Now a missing or unusable value makes the master log `MasterModule:MasterUrl is not an absolute http(s) address` and skip those background checks; nothing is overwritten.
- **Slaves** (ASF test, later ASF production) need no such variable: they store the address the master registers. A slave refuses an unusable address (`400 invalid_master_url`) and the poller logs `invalid_master_url`.
- **Recovery** of a slave that holds a bad address: set the variable on the master and restart it; the start-up check re-registers the slave. A changed variable needs a restart.
## Open point: ASF production on IIS (decided 2026-11-09, not designed here)
Everything in this feature's and the andreaslop-site-hosting documents that talks about the host assumes the Raspberry Pi: a systemd user unit (`andreaslopfotografie-test`), an `EnvironmentFile` named `shared/env` (mode 600, readable only by the service user), `journalctl` for logs, `systemctl --user restart` after a change, `deploy-scp.yaml` (scp + symlink `current` + restart), a reverse-proxy Pi in front, and Linux paths. For an IIS host these assumptions need an answer in the later ASF-production feature (listed, NOT worked out now):
- **Environment variables**: the same variable names work (double underscore), but are set in `web.config` (`<aspNetCore><environmentVariables>`), as machine-level variables or per app pool, not in `shared/env`. Secrets: no file mode 600, so decide where the SMTP credentials live (ACL-protected `web.config` section or machine variables, never in the repository) and who may read them.
@@ -0,0 +1,28 @@
using FluentAssertions;
using SlpModularCms.Core.Availability;
namespace SlpModularCms.Core.Tests.Availability;
public class MasterUrlRulesTests
{
[Theory]
[InlineData("https://master.example.com")]
[InlineData("https://master.example.com/")]
[InlineData("http://localhost:7221")]
[InlineData(" https://master.example.com ")]
public void AnAbsoluteHttpOrHttpsAddress_IsValid(string url) => MasterUrlRules.IsValid(url).Should().BeTrue();
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
[InlineData("<public-url-of-this-master-instance>")]
[InlineData("master.example.com")]
[InlineData("/relative/path")]
[InlineData("ftp://master.example.com")]
[InlineData("javascript:alert(1)")]
[InlineData("https://user:pw@master.example.com")]
[InlineData("https://master.example.com/?x=1")]
[InlineData("https://master.example.com/#frag")]
public void Everything_Else_IsNot(string? url) => MasterUrlRules.IsValid(url).Should().BeFalse();
}
@@ -0,0 +1,21 @@
namespace SlpModularCms.Core.Availability;
/// <summary>
/// What counts as a usable master address. A slave stores whatever the master registers and later builds
/// "{url}/api/v1/SlaveStatus" from it, so an unusable value (the "&lt;public-url-...&gt;" placeholder of the shipped
/// appsettings, an empty string, a host without scheme) breaks every poll with an exception instead of a message.
/// </summary>
public static class MasterUrlRules
{
/// <summary>An absolute http or https URL with a host, without credentials, query or fragment.</summary>
public static bool IsValid(string? value)
{
if (string.IsNullOrWhiteSpace(value)) return false;
if (!Uri.TryCreate(value.Trim(), UriKind.Absolute, out var uri)) return false;
return (uri.Scheme == Uri.UriSchemeHttps || uri.Scheme == Uri.UriSchemeHttp)
&& !string.IsNullOrEmpty(uri.Host)
&& string.IsNullOrEmpty(uri.UserInfo)
&& string.IsNullOrEmpty(uri.Query)
&& string.IsNullOrEmpty(uri.Fragment);
}
}
@@ -63,6 +63,36 @@ public class MasterControllerTests
await _svc.DidNotReceive().RegisterAsync(Arg.Any<string>(), Arg.Any<string>());
}
[Theory]
[InlineData("<public-url-of-this-master-instance>")]
[InlineData("")]
[InlineData(" ")]
[InlineData("host-without-scheme.invalid")]
[InlineData("/relative")]
[InlineData("ftp://host.invalid")]
public async Task Register_Returns400_AndStoresNothing_WhenTheMasterUrlIsUnusable(string url)
{
SetApiKeyHeader("key123");
var result = await _controller.Register(new RegisterMasterRequest(url));
var bad = result.Should().BeOfType<BadRequestObjectResult>().Subject;
System.Text.Json.JsonSerializer.Serialize(bad.Value).Should().Contain("invalid_master_url");
if (url.Length > 3) System.Text.Json.JsonSerializer.Serialize(bad.Value).Should().NotContain(url); // the answer does not echo it
await _svc.DidNotReceive().RegisterAsync(Arg.Any<string>(), Arg.Any<string>());
}
[Fact]
public async Task Register_StoresAValidUrl_Trimmed()
{
SetApiKeyHeader("key123");
_svc.RegisterAsync("https://master.example.com", "key123").Returns(true);
var result = await _controller.Register(new RegisterMasterRequest(" https://master.example.com "));
result.Should().BeOfType<OkResult>();
}
// --- PushStatus ---
[Fact]
@@ -67,6 +67,59 @@ public class MasterStatusPollClientTests
ok.LastFailure.Should().BeNull();
}
[Theory]
[InlineData("<public-url-of-this-master-instance>")]
[InlineData("")]
[InlineData("master.example.com")]
[InlineData("/relative")]
public async Task AnUnusableMasterUrl_IsReportedAsInvalidMasterUrl_NotAsAnException_AndNothingIsSent(string url)
{
var sent = false;
var sut = CreateSut(new CapturingHttpMessageHandler(_ => sent = true));
var result = await sut.GetStatusAsync(url, ApiKey);
result.Should().BeNull();
sut.LastFailure.Should().Be("invalid_master_url");
sut.LastFailure.Should().NotContain(ApiKey);
sent.Should().BeFalse();
}
[Fact]
public async Task AFailure_ComesWithSafeDiagnostics_WhereItWasThrownAndFlagsAboutTheInput_NeverTheUrlKeyOrMessage()
{
var sut = CreateSut(new ThrowingHttpMessageHandler());
await sut.GetStatusAsync(MasterUrl, ApiKey);
sut.LastFailureDetail.Should().Contain("at=").And.Contain("inner=")
.And.Contain("url_absolute=true").And.Contain("scheme=https").And.Contain("key_present=true");
sut.LastFailureDetail.Should().NotContain(ApiKey).And.NotContain("master.test").And.NotContain("Connection refused");
}
[Fact]
public async Task AnInvalidAddress_ShowsInTheDetailFlags_AndAMissingKeyToo()
{
var sut = CreateSut(new FakeHttpMessageHandler(HttpStatusCode.OK));
await sut.GetStatusAsync("<public-url-of-this-master-instance>", "");
sut.LastFailure.Should().Be("invalid_master_url");
sut.LastFailureDetail.Should().Be("url_absolute=false scheme=none key_present=false");
}
[Fact]
public async Task ASuccess_ClearsTheDiagnostics()
{
var sut = CreateSut(new FakeHttpMessageHandler(HttpStatusCode.OK, JsonSerializer.Serialize(new { IsAvailable = true })));
await sut.GetStatusAsync("nonsense", ApiKey);
sut.LastFailureDetail.Should().NotBeNull();
await sut.GetStatusAsync(MasterUrl, ApiKey);
sut.LastFailureDetail.Should().BeNull();
}
[Fact]
public async Task GetStatusAsync_SetsApiKeyHeader()
{
@@ -72,8 +72,9 @@ public class MasterStatusPollingBackgroundService(
{
_failing = true;
logger.LogWarning(
"master_poll_failed reason={Reason}. Further failures are not logged until the master answers again.",
string.IsNullOrEmpty(pollClient.LastFailure) ? "unknown" : pollClient.LastFailure);
"master_poll_failed reason={Reason} detail={Detail}. Further failures are not logged until the master answers again.",
string.IsNullOrEmpty(pollClient.LastFailure) ? "unknown" : pollClient.LastFailure,
string.IsNullOrEmpty(pollClient.LastFailureDetail) ? "none" : pollClient.LastFailureDetail);
}
var failOpenAfter = TimeSpan.FromMinutes(options.Value.FailOpenAfterMinutes);
@@ -1,5 +1,6 @@
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
using SlpModularCms.Core.Availability;
using SlpModularCms.Core.Observability;
using SlpModularCms.Modules.Availability.Models;
using SlpModularCms.Modules.Availability.Services;
@@ -25,7 +26,14 @@ public class MasterController : ControllerBase
var apiKey = Request.Headers["X-Master-Api-Key"].FirstOrDefault();
if (string.IsNullOrEmpty(apiKey)) return RejectKey();
var success = await _svc.RegisterAsync(request.MasterUrl, apiKey);
// The address is stored and polled later: refuse one that cannot be used, and say so without echoing it.
if (!MasterUrlRules.IsValid(request.MasterUrl))
{
_logger.LogWarning("Master registration refused: the master URL is not an absolute http(s) address (invalid_master_url).");
return BadRequest(new { code = "invalid_master_url", detail = "The master URL must be an absolute http(s) address, for example https://master.example.com." });
}
var success = await _svc.RegisterAsync(request.MasterUrl.Trim(), apiKey);
return success ? Ok() : RejectKey();
}
@@ -9,6 +9,13 @@ public interface IMasterStatusPollClient
/// key or a message text. Null after a success. Read it on the same instance right after the call.
/// </summary>
string? LastFailure { get; }
/// <summary>
/// Safe extra context for a failure, for the log: where it was thrown (class.method of the first stack frame, the
/// inner exception type if any) and flags about the input (url_absolute, scheme, key_present). Never the URL, the
/// key or an exception message.
/// </summary>
string? LastFailureDetail { get; }
}
[System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverage]
@@ -1,5 +1,6 @@
using System.Net.Http.Json;
using System.Text.Json;
using SlpModularCms.Core.Availability;
namespace SlpModularCms.Modules.Availability.Services;
@@ -9,9 +10,21 @@ public class MasterStatusPollClient(HttpClient httpClient) : IMasterStatusPollCl
public string? LastFailure { get; private set; }
public string? LastFailureDetail { get; private set; }
public async Task<PolledMasterStatus?> GetStatusAsync(string masterUrl, string plainApiKey)
{
LastFailure = null;
LastFailureDetail = null;
if (!MasterUrlRules.IsValid(masterUrl))
{
// Not an exception and not the URL in the log: the stored address is unusable (empty, relative, a
// placeholder). It repairs itself at the next valid registration from the master.
LastFailure = "invalid_master_url";
LastFailureDetail = InputFlags(masterUrl, plainApiKey);
return null;
}
try
{
var request = new HttpRequestMessage(HttpMethod.Get, $"{masterUrl.TrimEnd('/')}/api/v1/SlaveStatus");
@@ -31,10 +44,25 @@ public class MasterStatusPollClient(HttpClient httpClient) : IMasterStatusPollCl
catch (Exception ex)
{
LastFailure = Describe(ex);
LastFailureDetail = $"at={ThrownAt(ex)} inner={ex.InnerException?.GetType().Name ?? "none"} {InputFlags(masterUrl, plainApiKey)}";
return null;
}
}
/// <summary>Class.method of the first stack frame: where it was thrown, without any message text.</summary>
private static string ThrownAt(Exception ex)
{
var method = new System.Diagnostics.StackTrace(ex, false).GetFrames().FirstOrDefault()?.GetMethod();
return method is null ? "unknown" : $"{method.DeclaringType?.Name ?? "?"}.{method.Name}";
}
/// <summary>Booleans and the scheme only: enough to tell a bad address or key from a bad call, nothing secret.</summary>
private static string InputFlags(string? masterUrl, string? plainApiKey)
{
var absolute = Uri.TryCreate(masterUrl, UriKind.Absolute, out var uri);
return $"url_absolute={absolute.ToString().ToLowerInvariant()} scheme={(absolute ? uri!.Scheme : "none")} key_present={(!string.IsNullOrEmpty(plainApiKey)).ToString().ToLowerInvariant()}";
}
/// <summary>Exception type, plus the HTTP-level or socket error when there is one. No message text: it can quote the URL.</summary>
private static string Describe(Exception ex)
{
@@ -252,6 +252,40 @@ public class CmsInstanceServiceTests
await _repo.DidNotReceive().GetActiveAsync();
}
[Theory]
[InlineData("<public-url-of-this-master-instance>")]
[InlineData("")]
[InlineData("master.example.com")]
public async Task VerifyIntegrityAsync_NeverReregistersASlave_WithAnUnusableConfiguredMasterUrl(string configured)
{
// The shipped appsettings hold a placeholder: it used to overwrite the slave's good master address.
_httpContextAccessor.HttpContext.Returns((HttpContext?)null);
var sut = new CmsInstanceService(new MasterServiceDependencies(
_repo, _slaveClient, _protector, Options.Create(new MasterModuleOptions { MasterUrl = configured }),
_httpContextAccessor, _logger));
await sut.VerifyIntegrityAsync();
await _repo.DidNotReceive().GetActiveAsync();
await _slaveClient.DidNotReceive().RegisterMasterAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<string>());
}
[Fact]
public async Task VerifyIntegrityAsync_RepairsAPlaceholderInTheSlave_AsSoonAsTheConfiguredUrlIsValid()
{
var instance = ActiveInstance();
_repo.GetActiveAsync().Returns([instance]);
_protector.Unprotect("encrypted-key").Returns("plain");
_slaveClient.GetRegisteredMasterUrlAsync(Arg.Any<string>(), Arg.Any<string>())
.Returns(RegisteredMasterUrlResult.Ok("<public-url-of-this-master-instance>"));
_slaveClient.RegisterMasterAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<string>()).Returns(true);
_httpContextAccessor.HttpContext.Returns((HttpContext?)null);
await CreateSut().VerifyIntegrityAsync(); // _options holds https://master.test
await _slaveClient.Received(1).RegisterMasterAsync(instance.Url, "plain", "https://master.test");
}
[Fact]
public async Task VerifyIntegrityAsync_SetsLastIntegrityCheckFailedAt_WhenSlaveUnreachable()
{
@@ -258,7 +258,17 @@ public class CmsInstanceService(MasterServiceDependencies deps) : ICmsInstanceSe
return $"{request.Scheme}://{request.Host}";
}
return deps.Options.Value.MasterUrl;
var configured = deps.Options.Value.MasterUrl;
if (!SlpModularCms.Core.Availability.MasterUrlRules.IsValid(configured))
{
// The shipped appsettings carry a placeholder here. Using it would overwrite a slave's good master
// address with garbage (the slave then fails every poll), so it counts as "not configured".
deps.Logger.LogWarning(
"MasterModule:MasterUrl is not an absolute http(s) address (set Master__MasterUrl, e.g. https://master.example.com); background checks are skipped.");
return null;
}
return configured!.Trim();
}
private static CmsInstanceDto ToDto(CmsInstance instance) => new(