Monitoring Setup: operations/plans/monitoring-setup-plan.md and
operations/monitoring/monitoring-instructions.md, covering Sentry alert
rules on the security_event tag, UptimeRobot's 6 liveness monitors, and
the two new Umami website entries for the admin SPA.
deployment-instructions.md gains a missing Observability__Environment
host var (without it, both environments would tag Sentry events as
"Production"), the nginx client_max_body_size fix for the 413 seen on
publish-test/production artifact uploads, and two troubleshooting notes
on Gitea Actions re-run behaviour: re-running deploy-test/production
alone loses the run's uploaded artifact, and re-running all jobs on an
existing (rather than a brand new) run can replay stale secrets.
deploy-scp.yaml: step names no longer show literal unresolved
${{ inputs.* }} text (Gitea doesn't interpolate that context in step
names), and a temporary debug step logs PI_MAIN_USERNAME/PASSWORD
length plus a username equality check to diagnose a persistent
Permission denied during the SSH steps, without ever logging the
secret values themselves.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015FffvxxJp5wG34Ru48GBig
nginx and certbot were designed assuming they lived on the same host
as the app. They don't - a dedicated proxy Pi terminates TLS and
forwards plain HTTP over the LAN. Kestrel now binds 0.0.0.0 instead
of localhost, the whole certbot procedure moved to the proxy Pi's
side, and pi-main gets a firewall rule restricting the backend ports
to just the proxy Pi's address - otherwise binding all interfaces
would let anything on the LAN skip the proxy's TLS entirely.
The target Pi only has MariaDB, and SQL Server has no ARM64 build at
all - not a config problem, a real gap discovered during deployment
setup. Swapped the EF Core provider, regenerated every migration,
updated connection strings and the backup script everywhere they
appear.
Took two tries to land on a provider that actually works: Pomelo
builds fine against this project's EF Core 10 packages but fails at
runtime (it's compiled against 9's internal API surface, which moved
in 10 wherever Identity/DataProtection force the newer packages).
Oracle's official provider builds and migrates fine but has a real
MariaDB bug in its own migration-lock code, reproduced against a live
database. Kept Oracle's provider and worked around just that one
broken method - everything else it does is correct - rather than
give up more of the stack to chase a workaround.
Verified against a real local MariaDB end to end: all three
migrations applied, both hosts start clean, full suite still green.
webadmin turned out to be FTP-only and belongs to whoever uploads
customer websites, not to this pipeline. Switched to a separate
deploy account with its own home-directory paths, well away from
where the other sites live, and made the persistent website symlink
cross into webadmin's territory instead of the other way around.
Host setup checklist, real domains and ports, the database backup
script the deploy workflow only ever invokes, and a rollback plan.
Also drafted the FTPS switch procedure for whenever production moves
off the Pi, with a note that shared hosting is likely IIS-based -
that's a bigger change than swapping the transport.
Unit suites already passed individually; this is the full solution
together plus what none of them could check alone: two real hosts
talking to each other, a slave API key encrypted by one process and
decrypted by the next one after a restart, and headers landing on
actual static files instead of a simulated response.
U7 was the last one. State doc corrected: it had U7 down for an
infrastructure design pass that never actually happened, because by
the time units were carved up U7 had shrunk to documentation only.
Records the stage as complete and logs the two defects that local testing
turned up after U3 and U4 were generated: ciphertext predating the key-ring
move, and an integrity check that could not tell an unreachable slave from
one that does not recognise the master.
Both were fixed in this branch rather than filed. The second is master/slave
domain behaviour rather than deployment work, so the state file says so
plainly — it sits here by decision, not because it belongs to the feature.
Verified at close: build 0 errors, 372 backend tests, 237 frontend tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHoJpxYXzHACSQguHrC5fw
U4. Console logging plus Sentry, a same-origin tunnel so ad blockers cannot
silence browser errors, Umami on the admin SPA, and six security events that
alert rules can actually be built on.
The correlation id is the W3C trace id from the ambient Activity, enabled by
one line of ActivityTrackingOptions so every entry from every category carries
it without touching a call site. It propagates across the master/slave
boundary via traceparent, which TraceIdentifier cannot do at all, and it is
the same value ProblemDetails already returns to the browser.
The security events use source-generated LoggerMessage with constant
templates. Sentry groups log events by message, so interpolating an email
address would give every address its own issue and "more than 20 failed
logins in five minutes" could never fire — the events would arrive, be
visible, be tagged, and the alerting would silently be impossible. A test
asserts the rendered message is identical across argument values.
Scrubbing happens in-process, before transmission, and covers Set-Cookie as
well as Cookie: the login response issues the refreshToken there, so
scrubbing only the request side would protect nothing. Transactions are
scrubbed too, because they carry request data and are the channel nobody
thinks of.
The tunnel derives its destination from the DSN once at startup and reads
nothing from the request, which is what separates a tunnel from a
server-side request forgery primitive. Size is capped by a bounded read
rather than by trusting Content-Length, and the endpoint is rate limited.
Two things found along the way. Zod 4's url() hands the value to the URL
constructor, which accepts any scheme — so the existing frontend validation
would have accepted the exact "htp://" typo BR-U4-24 names, and the SPA
would have called a nonexistent origin. Now constrained to http(s). And the
new appsettings comments are verified against the real configuration
provider, because the failure mode if it rejected them is both hosts
refusing to start after a release switch.
One deviation. IAdminTokenValidator was meant to gain a reason-reporting
overload; implemented that way, a substitute returning false by default
silently inverted the access decision while both methods compiled. Two
methods whose difference is invisible at the call site is the defect, so it
is now a single Validate returning AdminTokenResult.
Touches two files from already-committed units: DatabaseMigrationExtensions
(U2) gains a flush before the rethrow, or the one Critical event in the
system dies with the process; AdminTokenValidator (U1) classifies why a
bypass was refused.
Build 0 errors; 366 backend tests pass, up from 315, and 237 frontend tests,
up from 213. tsc clean, eslint clean on every changed file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHoJpxYXzHACSQguHrC5fw
NFR Design for U3 and U4. Two decisions the earlier stages had deliberately
left open, plus four risks the functional design did not name.
OPEN-01 closed: the correlation ID is the W3C trace ID from the ambient
Activity, with TraceIdentifier as the fallback. It propagates across the
master/slave boundary via traceparent, which TraceIdentifier cannot do at
all, and it is the same value ProblemDetails already returns to the client.
REF-U3-01 raised: BR-U3-22's Umami-origin startup warning cannot work. The
backend never sees VITE_UMAMI_WEBSITE_ID, so the check would either always
warn or never warn. Withdrawn from U3 and replaced by a blocking U5 CI gate
that compares the frontend build variable against that environment's CSP
origins, where both values are visible.
Four additions beyond the functional design:
- Set-Cookie added to the scrub list; the login response issues the
refreshToken there, so scrubbing only the request cookie protects nothing
- SetBeforeSendTransaction alongside SetBeforeSend; transactions carry
request data too
- OnRejected on the rate limiter; today a 429 leaves no trace anywhere
- FlushAsync before the migration-failure rethrow, or the one Critical
event in the system dies with the process
Two traps recorded with tests attached: Sentry groups log events by message
template, so interpolated messages make FR-19's rate-based alert rules
unimplementable while appearing to work; and DefaultHttpContext.Response
.OnStarting is a no-op, so the obvious middleware test asserts nothing.
Three values chosen rather than escalated, each one line to change and all
three listed for review at the end of U4's pattern document: JSON console
outside Development, TracesSampleRate 0.1, tunnel cap 200 KB.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHoJpxYXzHACSQguHrC5fw
Records the functional design for the two remaining application units,
before any of their code exists.
Security headers have to come from the application, because relying on
nginx or IIS configuration is exactly what this deployment model rules
out. Strict applies to /admin, /api/v1 and /health; a relaxed policy
applies to the public website, which this repository does not author.
The strict policy needs style-src 'unsafe-inline'. That is not a
shortcut: Radix positions dropdowns and dialogs with inline style
attributes recalculated per click and scroll position, and CSP nonces
apply only to style elements, never to style attributes. No nonce- or
hash-based variant leaves the admin UI working. The exception is bounded
to styles — script-src stays closed, which is where XSS actually lives.
The website's policy is enforcing rather than absent, so every
HTML-serving path carries a CSP and no exception has to be recorded. It
still blocks external script origins, so it remains a real boundary.
HSTS is skipped in development: browsers remember it per host and
localhost is shared with unrelated projects. Every other header applies
locally, so a CSP violation surfaces while developing.
For observability, browser error reports tunnel through the API rather
than going to Sentry directly. Ad blockers block Sentry domains, which
loses errors precisely for the users most likely to have browser
oddities. The tunnel forwards only to the host derived from the
configured DSN — a caller-supplied destination would turn an anonymous
endpoint into a request-forgery primitive.
Two consequences of the chosen options are recorded rather than left
implicit:
Enabling SendDefaultPii attaches request headers, and this application
carries two standing credentials in them. Besides the refreshToken
cookie, X-Master-Api-Key would have been sent to a third party on every
error raised during a master/slave call. The scrub list removes the
whole Cookie header, Authorization, X-Master-Api-Key and the request
body.
Console logging at Information plus structured logging to Sentry would,
taken literally, mean one Sentry event per request — exhausting the free
plan within hours and burying real errors in request noise. The
thresholds are split: console keeps Information, Sentry takes warnings
and above as events with Information as breadcrumbs, so every event
arrives carrying the trail that led to it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHoJpxYXzHACSQguHrC5fw
SluijsensandClaude Sonnet 5