# Deployment Instructions

**Target**: single Raspberry Pi (`linux-arm64`), test and production both on it, separated by
directory, `systemd --user` unit, and local port. Domains: `test.slpsoftware.nl` (test),
`slpsoftware.nl` (production).

**Naming note**: directories, systemd units and the website-upload path below are named
`slpsoftware` — the **customer/instance** running SlpModularCms (this repo's software), not the
software itself. SlpModularCms can host multiple customers; `slpsoftware` is this one — the first,
and the owner's own site. A future second customer on the same Pi would get its own instance name
throughout, following the same pattern.

**Who runs what, where** — three hosts and two accounts appear below; every command block is
labelled with one of these:
- 🖥️ **pi-main / root** — as yourself with `sudo`, on the Pi this application runs on
- 👤 **pi-main / `gitea-workflow`** — as the deploy account, in a **real login shell** (SSH in
  directly as `gitea-workflow`, not `su -`/`sudo -i -u` from your own session — § 1.3 explains why
  that distinction matters)
- 🌐 **proxy Pi / root** — as yourself with `sudo`, on the *separate* Pi that terminates TLS (§ 1.7)
- ☁️ **Gitea web UI** — not a host command at all; done in this repository's Actions settings

---

## 1. One-Time Host Setup

Do this once, before the first automated deploy. Everything here is host configuration the workflow
assumes already exists — `deploy-scp.yaml` (U6) never creates any of it.

### 1.1 Prerequisites
- .NET 10 runtime installed on the Pi (ASM-03) — the publish is framework-dependent
  (`infrastructure-design.md` § 1), so the Pi needs the runtime, not the full SDK
- `mariadb-client` (or `mariadb-dump`/`mysqldump` specifically) installed, for the backup script (§ 4)
  — already present on most Raspberry Pi OS images that also run `mariadb-server`; install
  `mariadb-client` explicitly if the dump tool isn't already there

### 1.2 Account Model (revised — `webadmin` is not the deploy account)

Clarified during Deployment Setup: `webadmin` is the FileZilla/SFTP account website-workspace
authors use to upload customer sites under `/mnt/storage1/www/html/` (`WEBSITE_WORKSPACE.md`'s
role) — it cannot SSH in, and should **stay** SFTP-only. `deploy-scp.yaml` needs a real SSH shell
(for `mkdir`, `ln -sfn`, `systemctl --user restart`, the backup script), which is a different kind
of access than FileZilla uses.

**Preferred**: a separate account for the deploy pipeline, but a **generic, host-wide** one — not
named after this project, since it may end up deploying other projects on this Pi too (the reference
`SlpSoftware` project's own pipeline may already have exactly this kind of account; check its
`PI_MAIN_USERNAME` secret first and reuse it directly if it already has SSH shell access, rather than
creating a second one). Example name: `gitea-workflow`.

🖥️ **pi-main / root:**
```bash
sudo useradd -m -s /bin/bash gitea-workflow
sudo passwd gitea-workflow
```
Each project deploying through this account gets its own subdirectory under its home (§ 1.4 already
namespaces by project: `~/apps/slpsoftware/<env>/`), so one generic account can serve multiple
projects without their release trees colliding.

**Fallback, only if no SSH-capable account exists at all and creating one genuinely isn't feasible**
(e.g. a hosting provider that doesn't allow arbitrary new system accounts): grant `webadmin` SSH
shell access instead (`sudo usermod -s /bin/bash webadmin`, plus enabling SSH password/key auth for
it if currently blocked at the `sshd_config` level). This merges the FTP and deploy roles onto one
account — acceptable as a fallback, but worth revisiting later, since it means a website-workspace
author's FTP credential would also be able to run shell commands on the Pi.

The rest of this document uses `gitea-workflow` as the example account name — rename consistently
if you pick something else.

### 1.3 Enable Lingering (INFRA-U6-01 — do this first, easy to forget)

🖥️ **pi-main / root:**
```bash
sudo loginctl enable-linger gitea-workflow
```
Without this, the `systemd --user` service manager is torn down when the deploy SSH session ends,
killing the just-restarted app a few seconds after every successful deploy. This also fixes the
common "Failed to connect to bus" error `systemctl --user` can throw when invoked from a
non-interactive SSH command — lingering keeps the user's systemd instance (and `XDG_RUNTIME_DIR`)
running independent of any login session.

**If `systemctl --user ...` still fails with `Failed to connect to bus: No medium found`** while
setting the rest of this up manually (before the deploy workflow ever runs): enabling lingering
does not retroactively start the user manager — that happens on the next real login, reboot, or
manually:

🖥️ **pi-main / root:**
```bash
sudo systemctl start user@$(id -u gitea-workflow).service
```
And run `systemctl --user` commands from a **real login shell** for that account — not
`sudo -u gitea-workflow systemctl --user ...`, `sudo -i -u gitea-workflow`, **or `su - gitea-workflow`**
from your own session. All three are common ways to reach this exact error even when the user
manager is already running and `/run/user/<uid>/bus` already exists: none of them reliably go
through `pam_systemd` (the PAM module that actually exports `XDG_RUNTIME_DIR`), because
`/etc/pam.d/su` and most `sudo` PAM configs don't include it, unlike `/etc/pam.d/sshd` or
`/etc/pam.d/login`. Confirmed in practice: `su - gitea-workflow` reproduces this exactly.

Two fixes, in order of preference:
- **SSH in directly as `gitea-workflow`** instead of logging in as yourself and switching user —
  a real SSH login does go through `sshd`'s PAM stack and sets `XDG_RUNTIME_DIR` correctly
- Or, after `su -`/`sudo -i -u`, just set it by hand once per shell:

  👤 **pi-main / `gitea-workflow`:**
  ```bash
  export XDG_RUNTIME_DIR=/run/user/$(id -u)
  ```

**This does not affect the actual deploy workflow** — `deploy-scp.yaml` always connects over a
genuine SSH session (`sshpass ssh ...`), which sets `XDG_RUNTIME_DIR` correctly on its own. This
whole gotcha is specific to poking around on the host by hand via `su`/`sudo -i`.

Verify with `loginctl show-user gitea-workflow | grep Linger` (expect `Linger=yes`) and
`ls /run/user/<uid>` (should exist once the user manager has actually started).

### 1.4 Directory Skeleton

Deliberately placed under `gitea-workflow`'s **own home directory**, not under
`/mnt/storage1/www/html/` — since it's now a separate account from `webadmin`, there is no reason
for the CMS's own release/current/shared structure to live anywhere near the other websites at all,
which directly avoids interfering with them (as you asked in Q2):

👤 **pi-main / `gitea-workflow`:**
```bash
mkdir -p ~/apps/slpsoftware/<env>/releases
```

`current` is created by the first deploy itself (`ln -sfn`) — don't pre-create it.

**`shared/wwwroot-web` is the one exception** — it must resolve to wherever `webadmin` actually
uploads *this* customer's website via FileZilla, e.g. `/mnt/storage1/www/html/slpsoftware/<env>/`
(adjust the exact folder name to whatever convention the other sites under `html/` already use, if
one exists). Rather than a plain directory, make it a symlink across accounts:

👤 **pi-main / `gitea-workflow`:**
```bash
mkdir -p ~/apps/slpsoftware/<env>
ln -s /mnt/storage1/www/html/slpsoftware/<env> ~/apps/slpsoftware/<env>/shared/wwwroot-web
```

`deploy-scp.yaml`'s existing logic (`releases/{ts}/wwwroot/web -> ../../../shared/wwwroot-web`)
needs no changes for this — it only ever resolves the symlink chain, it doesn't care how many hops
that chain has. What **does** need attention: `gitea-workflow` needs read + traverse permission on
`/mnt/storage1/www/html/slpsoftware/<env>/` and its parent directories, which `webadmin` owns.
Simplest fix: put both accounts in a shared group (e.g. `webshared`), then, 🖥️ **pi-main / root**:
```bash
sudo chgrp -R webshared /mnt/storage1/www/html/slpsoftware/<env>
```
and make sure webadmin's FTP server creates new uploads group-readable (`g+rx`, not just
owner-readable) — a one-time permission setup, not something either pipeline touches per deploy.

**`shared/modules` is the equivalent mechanism for optional plugin modules** (future modular
support) — but simpler, since it lives entirely under `gitea-workflow`'s own tree, with no
cross-account permission dance:

👤 **pi-main / `gitea-workflow`:**
```bash
mkdir -p ~/apps/slpsoftware/<env>/shared/modules
```

`deploy-scp.yaml`'s "Link persistent modules" step already runs this `mkdir -p` on every deploy, so
this line is only useful if you want the directory to exist before the very first deploy — it isn't
required. To add a module: build a `SlpModularCms.Modules.<Name>.dll` against the same version of
`SlpModularCms.Core` the running app was built against (mismatched contracts are caught per-module
by `ModuleOrchestrator`'s existing try/catch — logged and skipped, not a crash), `scp`/`sftp` it as
`gitea-workflow` into `~/apps/slpsoftware/<env>/shared/modules/`, then either wait for the next
deploy or restart the service by hand:
```bash
systemctl --user restart slpsoftware-<env>.service
```
Either way, `ModuleOrchestrator.DiscoverModules()` picks it up from the release directory the next
time the process starts — no CI run required to add a module this way.

### 1.5 Runtime Configuration File
One file per environment, **outside** the release directory so it survives every switch:

👤 **pi-main / `gitea-workflow`:**
```bash
touch ~/apps/slpsoftware/<env>/shared/env
chmod 600 ~/apps/slpsoftware/<env>/shared/env
```
Contents (fill in real values — this file is never read by the workflow, only by the systemd unit
below):
```ini
ASPNETCORE_ENVIRONMENT=Production
# 0.0.0.0, not localhost: the reverse proxy handling TLS for this domain runs on a SEPARATE Pi
# (§ 1.7), so it must reach Kestrel over the LAN, not loopback. See § 1.7's firewall note — binding
# to all interfaces means the port must be restricted to the proxy Pi's address, not left open to
# the whole LAN.
ASPNETCORE_URLS=http://0.0.0.0:<port>
ConnectionStrings__DefaultConnection=Server=127.0.0.1;Port=3306;Database=SlpSoftware<Env>;Uid=<user>;Pwd=<password>
JwtSettings__Secret=<secure-long-random-secret>
JwtSettings__Issuer=SlpModularCms
JwtSettings__Audience=SlpModularCmsPortal

# Optional — only meaningful if this instance manages slave CMS instances via the /cms page.
# MasterModuleOptions.MasterUrl is nullable and unvalidated at startup: if left unset, the Master
# module's background reconciliation just logs a warning and skips its work ("MasterUrl not
# configured; skipping integrity check") — it never crashes or blocks startup. Safe to leave out
# entirely if you don't plan to register any slave instances under this deployment; set it to this
# environment's own public URL (e.g. https://slpsoftware.nl) if you do.
MasterModule__MasterUrl=https://<this-environment-domain>

# Optional — empty/absent is a fully supported state: Sentry is simply skipped and console logging
# continues (see Observability section in README.md).
Observability__SentryDsn=<sentry-dsn-or-empty>

# REQUIRED if the DSN above is set, and must differ between this file and the other environment's.
# D-19 uses one Sentry project for both test and production, distinguished only by this tag.
# ASPNETCORE_ENVIRONMENT is deliberately "Production" for both environments (see note below), so
# without this explicit override, ObservabilityOptions.Environment falls back to
# ASPNETCORE_ENVIRONMENT and every event — test and production alike — would be tagged "Production",
# silently defeating D-19's whole point of telling them apart in Sentry.
Observability__Environment=<test-or-production>

# Optional — both only needed if this instance uses Umami analytics (VITE_UMAMI_SCRIPT_URL set for
# the frontend build). Leave both lines out entirely if you don't use Umami; there is no other
# origin either one needs by default.
#
# Set BOTH to the Umami script's origin (scheme + host only, no path — e.g. if
# VITE_UMAMI_SCRIPT_URL=https://analytics.slpsoftware.nl/script.js, use
# https://analytics.slpsoftware.nl for both lines below):
#   - AllowedScriptOrigins: lets the browser load Umami's tracking script (CSP script-src)
#   - AllowedConnectOrigins: lets that script send its analytics beacons back (CSP connect-src) —
#     loading a script and letting it phone home are two separate CSP directives
#
# Sentry does NOT need an entry here, on either line: U4 built a same-origin tunnel
# (Program.cs -> MapSentryTunnel()) specifically so browser error reports never leave this origin,
# precisely to avoid needing a connect-src exception (and to dodge ad blockers, which commonly
# block direct requests to Sentry's own domains).
SecurityHeaders__AllowedScriptOrigins__0=<umami-script-origin-or-omit-this-line>
SecurityHeaders__AllowedConnectOrigins__0=<same-umami-script-origin-or-omit-this-line>
```
Use ports **5100** (test) and **5101** (production) unless something else on the Pi already
occupies them. `ASPNETCORE_ENVIRONMENT=Production` is used for **both** environments deliberately —
`Development` disables HSTS and exposes the Scalar API explorer (`Program.cs`), neither of which
should be true for anything reachable at a real domain, including test.

**`SecurityHeaders__AllowedScriptOrigins__0` / `_AllowedConnectOrigins__0` must exactly match** the
Gitea variables `SECURITY_ALLOWED_SCRIPT_ORIGINS_TEST` / `_PRODUCTION` (see § 1.9) — REF-U5-01's CI
gate only catches drift between the frontend build and that Gitea variable; it cannot see this file,
so keeping the two in sync is a manual discipline, not something enforced automatically.

### 1.6 systemd User Units

👤 **pi-main / `gitea-workflow`** — create `~/.config/systemd/user/slpsoftware-test.service`:
```ini
[Unit]
Description=SlpModularCms API (test)
After=network.target

[Service]
WorkingDirectory=%h/apps/slpsoftware/test/current
ExecStart=/usr/bin/dotnet %h/apps/slpsoftware/test/current/SlpModularCms.Api.dll
EnvironmentFile=%h/apps/slpsoftware/test/shared/env
Restart=on-failure
RestartSec=5
KillSignal=SIGINT
TimeoutStopSec=20

[Install]
WantedBy=default.target
```
And `~/.config/systemd/user/slpsoftware-production.service` — identical, with `test` replaced by
`production` throughout (including the port inside `shared/env`).

Enable both (does not start them yet — nothing is deployed there until the first CI run):

👤 **pi-main / `gitea-workflow`** (real login shell — § 1.3):
```bash
systemctl --user daemon-reload
systemctl --user enable slpsoftware-test.service
systemctl --user enable slpsoftware-production.service
```

### 1.7 TLS-Terminating Reverse Proxy — On a Separate Pi, Not This One

**Revised from the original design**: the reverse proxy that terminates TLS for these domains runs
on a **different, dedicated Pi** ("the proxy Pi"), not on this one ("pi-main", where
`gitea-workflow` and the release directories from §§ 1.2–1.4 live). The proxy Pi already handles
SSL and forwards plain HTTP to pi-main. Two consequences that change earlier sections:

- **Kestrel must be reachable over the LAN, not just loopback** — hence `ASPNETCORE_URLS=http://0.0.0.0:<port>`
  in § 1.5, not `http://localhost:<port>`
- **pi-main runs no nginx and holds no certificates for these domains at all** — everything in this
  section happens **on the proxy Pi**, except the firewall step at the end, which is on pi-main

As in § 1.7.1–1.7.2 below: § 1.7.1 is the repeatable, from-scratch procedure for routing *any* new
domain through the proxy Pi to *any* backend (this deployment's domains included, the first time);
§ 1.7.2 is what that procedure produced for `slpsoftware.nl` — the worked example, not a separate
step.

> **Do not add a `/sentry-tunnel` proxy block on the proxy Pi.** The reference project (a static
> frontend with no backend of its own) had to tunnel through nginx directly to Sentry's ingest URL.
> This application does not need that: `SentryTunnelExtensions.cs` (built in U4) serves
> `/sentry-tunnel` itself, with its own rate limiter, a payload-size cap
> (`Observability:TunnelMaxPayloadBytes`), and a destination derived from `Observability__SentryDsn`
> rather than a hardcoded org/project ID. An nginx-level tunnel block at the same path intercepts
> the request before it ever reaches pi-main, silently bypassing all three of those protections —
> let `/sentry-tunnel` fall through the plain `location /` block like any other request instead.

#### 1.7.1 Adding a New Domain From Scratch (on the proxy Pi)

Starting from nothing — no existing server block, no certificate — for a new domain
`<new-domain>` that should route to `<backend-host>:<backend-port>` (a service on some other Pi
or container on the LAN — pi-main and its two ports, in this deployment's case):

**Step 1 — plain HTTP block, no TLS yet.** Certbot's nginx plugin (step 2) needs a working HTTP
server block for the domain to attach to and to answer the HTTP-01 validation challenge; asking
for a certificate before this exists will fail.

🌐 **proxy Pi / root:**
```nginx
server {
    listen 80;
    server_name <new-domain>;
    location / {
        proxy_pass http://<backend-host>:<backend-port>;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

Save as `/etc/nginx/sites-available/<new-domain>`, symlink it into `sites-enabled/`, then:
```bash
sudo nginx -t && sudo systemctl reload nginx
```

Confirm the domain's DNS `A`/`AAAA` record already points at **the proxy Pi** (not the backend)
before continuing — the HTTP-01 challenge in step 2 needs the domain to resolve to whichever host
is answering on port 80, which is the proxy Pi.

**Step 2 — request and install the certificate.** The nginx plugin edits the file from step 1 in
place: it adds the `listen 443 ssl` block, the certificate/key paths, and (by default) an
HTTP→HTTPS redirect for the port 80 block:

🌐 **proxy Pi / root:**
```bash
sudo certbot --nginx -d <new-domain>
```
Certbot's own systemd timer handles renewal automatically — nothing further to set up for that.

**Step 3 — verify.** Can be run from anywhere with network access to the domain (your own machine
is fine — this isn't tied to either Pi). After certbot finishes:
```bash
curl -I https://<new-domain>/health   # or whichever path this new site/API actually serves
```
Confirm it resolves over HTTPS with a valid certificate and reaches the expected backend.

Repeat steps 1–3 once per domain. This is the same procedure regardless of whether the new domain
is another environment for this feature, a completely different project's API, or a plain static
site — nginx and certbot don't know or care what's actually listening on the backend they proxy to.

**Step 4 — restrict the backend port on pi-main (do this once the domain works end to end).**
Since Kestrel now listens on `0.0.0.0:<backend-port>` (§ 1.5), anything on the LAN can reach it
directly, bypassing the proxy Pi's TLS entirely, unless pi-main's firewall says otherwise.

🖥️ **pi-main / root:**
```bash
sudo ufw allow from <proxy-pi-lan-ip> to any port <backend-port> proto tcp
sudo ufw deny <backend-port>/tcp
```
(Or the equivalent `nftables`/`iptables` rules if `ufw` isn't what this Pi uses — the point is:
only the proxy Pi's address may reach these ports, everything else is denied.)

#### 1.7.2 Current State for This Deployment

Running the procedure above (on the proxy Pi) for `test.slpsoftware.nl` and `slpsoftware.nl`,
pointing at pi-main's LAN address, produces server blocks equivalent to:

```nginx
server {
    listen 443 ssl;
    server_name test.slpsoftware.nl;
    location / {
        proxy_pass http://<pi-main-lan-ip>:5100;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    # certbot-managed ssl_certificate / ssl_certificate_key / include lines omitted here
}

server {
    listen 443 ssl;
    server_name slpsoftware.nl;
    location / {
        proxy_pass http://<pi-main-lan-ip>:5101;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    # certbot-managed ssl_certificate / ssl_certificate_key / include lines omitted here
}
```

#### 1.7.3 Artifact Upload Body Size (413 on `publish-test`/`publish-production`)

The Gitea instance itself (`gitea.slpsoftware.nl`) sits behind the same proxy Pi as every other
domain in § 1.7 — Actions' own web UI/API traffic is proxied through it exactly like
`test.slpsoftware.nl` and `slpsoftware.nl` are. Gitea Actions' artifact upload
(`actions/upload-artifact`) sends the build output in chunks; if a chunk exceeds nginx's
`client_max_body_size` (default 1m), nginx itself rejects it with `413 Request Entity Too Large`
before the request ever reaches Gitea — surfacing in the `publish-test`/`publish-production` job log
as repeated `A 413 status code has been received, will attempt to retry the upload` followed by
`Retry limit has been reached` on individual files once the action's retries are exhausted.

**pi-main is not involved** — § 1.7 already established it runs no nginx and holds no certificates
for any of these domains; this is purely a proxy Pi setting.

**Fix — set once, globally, not per server block.** `client_max_body_size` is inherited
(`http` → `server` → `location`); setting it in the top-level `http {}` block covers every current
and future domain's server blocks — both the `listen 80` and `listen 443` blocks certbot manages —
without needing to repeat it each time § 1.7.1's procedure is run for a new domain.

🌐 **proxy Pi / root:**
```nginx
# /etc/nginx/nginx.conf, inside the http { } block:
http {
    client_max_body_size 512m;
    ...
}
```
```bash
sudo nginx -t && sudo systemctl reload nginx
```

#### 1.7.4 Never Re-Run `deploy-test`/`deploy-production` Alone — Gitea Platform Limitation

`deploy-test` and `deploy-production` in `continuous_integration.yaml` are not ordinary jobs — each
one *calls* the reusable `deploy-scp.yaml` workflow (`uses: ./.gitea/workflows/deploy-scp.yaml`).
Re-running **only** one of these two jobs after a failure (Gitea's per-job "re-run" action) is a
known Gitea Actions limitation, not a bug in either workflow file here: Gitea does not cleanly resume
a `workflow_call` job inside its original run — it re-executes the call in a way that loses access to
that run's already-uploaded artifact. The `Download build artifact` step then fails immediately with:
```
List Artifacts - Error is not retryable
Status Code: 404
Error: List Artifacts failed: Artifact service responded with 404
```
even though `publish-test`/`publish-production` genuinely succeeded and uploaded the artifact
moments earlier in the same run (retention had not expired — this is not the same failure mode as
§ 1.7.3, and not a retention issue at all).

**Fix: re-run the entire workflow, not just this job** — but see § 1.7.5 immediately below before
doing that: Gitea's "re-run all jobs" on an *existing* run is not the same as triggering a genuinely
new run, and has its own, different failure mode.

#### 1.7.5 "Re-Run All Jobs" Can Silently Reuse Stale Secrets/Variables — Trigger a New Run Instead

Symptom: a secret (e.g. `PI_MAIN_PASSWORD`) was wrong, causing `sshpass`/`ssh` to fail with
`Permission denied, please try again.` (`sshpass` exit code 5 — the password itself was rejected,
not a connectivity or config problem). The secret is corrected in Gitea's UI and confirmed working
via a manual SSH test with the same value. **"Re-run all jobs" is used on the existing, already-failed
run — and it fails again, identically**, as if the fix never happened.

This matches a documented behaviour of Actions-style re-run implementations, seen concretely in
GitHub's own tooling ([cli/cli#13522](https://github.com/cli/cli/issues/13522)): re-running an
*existing* run can replay against secrets/variables as they were **when that run was first created**,
not their current values — particularly for a job that calls a reusable workflow with
`secrets: inherit` (exactly what `deploy-test`/`deploy-production` do here). The documented contract
("secrets are fetched at the time of the re-run") does not hold in practice for this case. A genuinely
**new** run (fresh push, or `workflow_dispatch`) always fetches current values correctly — only
re-running an existing run risks the stale snapshot.

**Net effect of § 1.7.4 + § 1.7.5 together**: neither of Gitea's two "re-run" options is fully safe
for `deploy-test`/`deploy-production` — "re-run this job alone" loses the artifact, "re-run all jobs"
can keep stale secrets. **The one reliable option is to trigger a brand new run** (an actual push, or
`workflow_dispatch`'s "Run workflow" button) rather than using either re-run action on a failed run,
whenever a secret or variable was just changed to fix that failure. Re-running an existing run is
only safe when nothing about its secrets/variables changed since it was created.

### 1.8 Database Backup Credentials (§ 4 depends on this)

👤 **pi-main / `gitea-workflow`:**
```bash
touch ~/.config/slpsoftware-db-backup.env
chmod 600 ~/.config/slpsoftware-db-backup.env
```
```ini
DB_HOST=127.0.0.1
DB_PORT=3306
DB_NAME=SlpSoftwareProduction
DB_USER=<a-login-with-just-SELECT-LOCK-TABLES-permission>
DB_PASSWORD=<password>
```
Kept **separate** from `shared/env` (§ 1.5) deliberately — the backup script needs its own
credential, ideally scoped to just read access (`SELECT`, `LOCK TABLES` — everything `mariadb-dump`
needs) rather than the application's own data-access login.

### 1.9 Gitea Actions Variables and Secrets

☁️ **Gitea web UI** — not a host command, no SSH involved. Set once, in this repository's Actions
settings. This is exactly where the real path lives —
`deploy-scp.yaml` never hardcodes it, it only reads `${{ vars.DEPLOY_PATH_TEST }}` /
`${{ vars.DEPLOY_PATH_PRODUCTION }}`, so changing the path later is a variable edit, not a workflow
change:

| Name | Kind | Value |
|---|---|---|
| `PI_MAIN_ADDRESS` | secret | the Pi's address |
| `PI_MAIN_PORT` | secret | SSH port |
| `PI_MAIN_USERNAME` | secret | `gitea-workflow` — the generic, host-wide deploy account (§ 1.2), **not** `webadmin`; reuse the reference project's if it already has one |
| `PI_MAIN_PASSWORD` | secret | `gitea-workflow`'s password |
| `DEPLOY_PATH_TEST` | variable | `/home/gitea-workflow/apps/slpsoftware/test` |
| `DEPLOY_PATH_PRODUCTION` | variable | `/home/gitea-workflow/apps/slpsoftware/production` |
| `SERVICE_NAME_TEST` | variable | `slpsoftware-test.service` |
| `SERVICE_NAME_PRODUCTION` | variable | `slpsoftware-production.service` |
| `HEALTH_CHECK_URL_TEST` | variable | `https://test.slpsoftware.nl/health` |
| `HEALTH_CHECK_URL_PRODUCTION` | variable | `https://slpsoftware.nl/health` |
| `VITE_SENTRY_DSN` | variable | Sentry DSN (shared, not sensitive — safe in the client bundle) |
| `VITE_UMAMI_SCRIPT_URL` | variable | Umami script host (shared) |
| `VITE_UMAMI_WEBSITE_ID_TEST` / `_PRODUCTION` | variable | per-environment Umami website ID |
| `SECURITY_ALLOWED_SCRIPT_ORIGINS_TEST` / `_PRODUCTION` | variable | **must match** § 1.5's `SecurityHeaders__AllowedScriptOrigins__0` for that environment |

---

## 2. Deploy Sequence (What Actually Happens on a Run)

Already built (U5/U6) — this is the read-only walkthrough for whoever operates it:

1. Push to `master`, or a manual `workflow_dispatch` → the six gates run
2. `publish-test` runs (and `publish-production`, only if `workflow_dispatch` with the flag)
3. `deploy-test` (always, if gates pass) calls `deploy-scp.yaml`, which uploads into a new
   `releases/{timestamp}/`, links `shared/wwwroot-web` and any `shared/modules/*.dll` in, switches
   `current`, restarts `slpsoftware-test.service`, verifies `https://test.slpsoftware.nl/health`,
   then prunes old releases (only test — `run_db_backup: false`)
4. `deploy-production` (only with the flag) does the same, plus a database backup first
   (`run_db_backup: true`) — see § 4

## 3. First-Ever Deploy Notes

- `wwwroot/web/` will be empty until a website workspace deploys into it — `/` serves the built-in
  placeholder until then (this is expected, not a failure)
- The very first run has no "previous release" to keep — pruning naturally has nothing to prune
- Verify manually after the first run (from anywhere — not tied to either Pi):
  `curl https://test.slpsoftware.nl/health` and `curl https://slpsoftware.nl/health` (after the
  first production run) both return `200`

## 4. Database Backup Script

👤 **pi-main / `gitea-workflow`** — create `~/scripts/backup-slpsoftware-db.sh` on the Pi (this
script is host-side by design — never part of this repository, so no DB credential ever reaches
Gitea):

```bash
#!/usr/bin/env bash
set -euo pipefail

ENVIRONMENT="${1:?Usage: backup-slpsoftware-db.sh <environment>}"
CREDENTIALS_FILE="$HOME/.config/slpsoftware-db-backup.env"

if [[ ! -f "$CREDENTIALS_FILE" ]]; then
  echo "Missing $CREDENTIALS_FILE — see deployment-instructions.md § 1.8" >&2
  exit 1
fi
# shellcheck source=/dev/null
source "$CREDENTIALS_FILE"
: "${DB_HOST:?}" "${DB_PORT:?}" "${DB_NAME:?}" "${DB_USER:?}" "${DB_PASSWORD:?}"

# A brand-new environment's very first production deploy runs this step before the app has ever
# started, so before EF Core's migrations have had a chance to create the database — there is
# nothing to back up yet, and that's fine, not a failure. Distinguish that specific case ("Unknown
# database", MariaDB error 1049) from every other failure (wrong credentials, network issue,
# permissions): only the former is safe to skip, since skipping any OTHER error would silently mask
# a real backup failure against a database that may hold real data.
USE_ERROR=$(mariadb -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" -p"$DB_PASSWORD" -e "USE \`$DB_NAME\`;" 2>&1 >/dev/null) || true
if echo "$USE_ERROR" | grep -q "Unknown database"; then
  echo "Database '$DB_NAME' does not exist yet — nothing to back up (expected on a first deploy). Skipping."
  exit 0
elif [[ -n "$USE_ERROR" ]]; then
  echo "Could not verify database '$DB_NAME' exists: $USE_ERROR" >&2
  exit 1
fi

BACKUP_DIR="$HOME/backups/slpsoftware/${ENVIRONMENT}"
mkdir -p "$BACKUP_DIR"
TIMESTAMP=$(date -u +%Y%m%d%H%M%S)
BACKUP_FILE="$BACKUP_DIR/${DB_NAME}-${TIMESTAMP}.sql.gz"

# --single-transaction: consistent snapshot without locking the tables for the whole dump duration
# (InnoDB only — every table here is, since that's EF Core's MySQL-provider default).
mariadb-dump \
  -h "$DB_HOST" -P "$DB_PORT" -u "$DB_USER" -p"$DB_PASSWORD" \
  --single-transaction --routines --triggers \
  "$DB_NAME" | gzip > "$BACKUP_FILE"

echo "Backup written to $BACKUP_FILE"

# Retention: keep the 7 most recent backups for this environment
ls -1t "$BACKUP_DIR"/*.sql.gz 2>/dev/null | tail -n +8 | xargs -r rm -f
```

`mariadb-dump` is MariaDB's own name for the tool (present since MariaDB 10.4-ish); if the host only
has the older `mysqldump` name, substitute it — same tool, same flags.

👤 **pi-main / `gitea-workflow`:**
```bash
chmod +x ~/scripts/backup-slpsoftware-db.sh
```

**Verify once, manually**, before relying on it in a real deploy — 👤 **pi-main / `gitea-workflow`**
(this matches exactly how `deploy-scp.yaml` itself invokes the script over SSH):
```bash
~/scripts/backup-slpsoftware-db.sh production
```
Confirm a `.sql.gz` file appears under `~/backups/slpsoftware/production/` and that the dump didn't
silently fail (the script uses `set -euo pipefail`, so a real error does propagate as a non-zero
exit — which fails the calling `deploy-scp.yaml` step, correctly blocking the deploy). Worth a
one-time restore rehearsal too — an untested backup is not a verified one:

👤 **pi-main / `gitea-workflow`** (or wherever you have a MariaDB client and network access to
`127.0.0.1`'s database — this one isn't identity-sensitive, just needs DB access):
```bash
gunzip -c ~/backups/slpsoftware/production/<file>.sql.gz | mariadb -h 127.0.0.1 -u root -p <a-scratch-database>
```

## 5. Future: Switching Production to FTPS (Shared Hosting)

D-02/NFR-09 required the workflow's *transport* to be swappable without restructuring — satisfied by
the `transport` input already on `deploy-scp.yaml` (currently only `scp` is implemented). This
section documents what actually changes when that day comes (OPEN-04 — not scheduled, drafted now
per Q5 = B of the deployment setup plan).

### 5.1 What carries over unchanged
- The CI workflow's gates, the two-build split, the `config` job pattern
- The overall shape of the interface: `artifact_name`, `environment`, `deploy_path`

### 5.2 What does not carry over — read this before assuming it's a drop-in swap
Shared .NET hosting is almost always **Windows/IIS-based**, not Linux/systemd. That changes more
than the transport:
- **No `systemctl --user` restart** — IIS picks up a new deployment via an app-pool recycle, usually
  triggered by touching `web.config` or the app-pool's own recycle mechanism, not a service restart
  command
- **The atomic release-switch pattern may not be available at all** — many shared hosts expose only
  a single web root over FTPS, with no ability to create sibling directories and swap a symlink.
  `wwwroot/web/` persistence (FR-08, ASM-01) would need a **different** mechanism on such a host —
  e.g. never touching a specific subfolder during upload, rather than linking a persistent directory
  outside a swapped release tree, since "outside the release tree" may not be an available concept
- **Database backup** — shared hosting frequently does not expose direct `mariadb-dump`/SSH access
  at all; the backup step in `deploy-scp.yaml` (§ 4 script) would need to become either a
  provider-specific API call (e.g. a hosting-panel database backup feature) or a documented manual
  pre-production step (the FR-20 fallback U6 already designed for)

### 5.3 What building `deploy-ftps.yaml` would actually require
1. A new reusable workflow implementing the **same five required inputs**
   (`artifact_name`, `environment`, `deploy_path`, `service_name`, `health_check_url`) so
   `continuous_integration.yaml` calls it identically via the `transport` input
2. An FTPS upload step (e.g. `lftp` mirror or `curl --ftp-ssl`) replacing the `scp` step
3. A **new decision, not yet made**: what "atomic switch" and "restart" even mean on the target
   host — this cannot be answered generically; it depends on which shared host is actually chosen
4. Re-running this feature's Infrastructure Design step for U6, scoped to the new host, once a
   specific shared-hosting provider is selected — not a checklist item that can be pre-answered today

**Recommendation**: treat this section as a starting brief for that future Infrastructure Design
pass, not as a ready-to-execute procedure — the concrete host was unknown at the time this was
written (OPEN-04), so several of the decisions above are necessarily provider-specific.