Feature/gitea deployment workflow #1

Merged
Sluijsens merged 35 commits from feature/gitea-deployment-workflow into master 2026-07-29 16:50:44 +02:00
8 changed files with 285 additions and 6 deletions
Showing only changes of commit 9ab30fe2a6 - Show all commits
+1 -1
View File
@@ -7,4 +7,4 @@
| Master CMS Module (master-cms-module) | ✅ Complete | unknown | Modules, Availability | 2026-06-26 | | Master CMS Module (master-cms-module) | ✅ Complete | unknown | Modules, Availability | 2026-06-26 |
| Tech Debt Backlog (tech-debt-backlog) | 🔵 Inception | unknown | Modules.Master, Frontend | 2026-07-01 | | Tech Debt Backlog (tech-debt-backlog) | 🔵 Inception | unknown | Modules.Master, Frontend | 2026-07-01 |
| Local Dev Master/Slave Setup (local-dev-master-slave-setup) | ✅ Complete | unknown | Modules.Master, Api, Frontend | 2026-07-02 | | Local Dev Master/Slave Setup (local-dev-master-slave-setup) | ✅ Complete | unknown | Modules.Master, Api, Frontend | 2026-07-02 |
| Gitea Deployment Workflow (gitea-deployment-workflow) | 🟢 Construction | feature/gitea-deployment-workflow | CI/CD, Api (hosting/config), Core, Modules, Frontend (build), Docs | 2026-07-27 | | Gitea Deployment Workflow (gitea-deployment-workflow) | 🟡 Operations | feature/gitea-deployment-workflow | CI/CD, Api (hosting/config), Core, Modules, Frontend (build), Docs | 2026-07-27 |
@@ -70,7 +70,7 @@ Execution rounds (Q4 = B): **R1** = U1 + U2 · **R2** = U3 + U4 · **R3** = U5 +
- [x] NFR Design — **EXECUTE for U3, U4**; SKIP for the rest. *Deliberate deviation from the default NFR-Requirements/NFR-Design coupling — rationale in the execution plan.* **U3 ✅ U4 ✅** 2026-07-28 — 11 patterns for U3, 10 for U4. Closed OPEN-01; raised REF-U3-01 - [x] NFR Design — **EXECUTE for U3, U4**; SKIP for the rest. *Deliberate deviation from the default NFR-Requirements/NFR-Design coupling — rationale in the execution plan.* **U3 ✅ U4 ✅** 2026-07-28 — 11 patterns for U3, 10 for U4. Closed OPEN-01; raised REF-U3-01
- [x] Infrastructure Design — **EXECUTE for U6, U7 per the original plan; SKIP for the rest**. **U6 ✅** 2026-07-28 — single Pi, directory-only environment split, `systemd --user` (no sudo), releases/current/shared layout, 2-release retention, health check via public URL. Raised INFRA-U6-01 (linger requirement, carried to Operations). **U7 did not need it in practice**: by Units Generation, U7's scope had narrowed to documentation-only (Q8 = A — operational/infrastructure decisions moved to Operations), so it went straight to Code Generation with no infrastructure to design - [x] Infrastructure Design — **EXECUTE for U6, U7 per the original plan; SKIP for the rest**. **U6 ✅** 2026-07-28 — single Pi, directory-only environment split, `systemd --user` (no sudo), releases/current/shared layout, 2-release retention, health check via public URL. Raised INFRA-U6-01 (linger requirement, carried to Operations). **U7 did not need it in practice**: by Units Generation, U7's scope had narrowed to documentation-only (Q8 = A — operational/infrastructure decisions moved to Operations), so it went straight to Code Generation with no infrastructure to design
- [x] Code Generation — **EXECUTE** (all 7 units, each built and tested before its completion message). **U1 ✅ U2 ✅** 2026-07-27 (253 backend tests). **U3 ✅** 2026-07-28 (315). **U4 ✅** 2026-07-28 (366 backend + 237 frontend). **U6 ✅ U5 ✅** 2026-07-28 — Round 3: `deploy-scp.yaml` + `continuous_integration.yaml`, 372 backend + 237 frontend tests (unchanged from Round 2, confirming no regressions from FR-21/FR-22 fixes), 0 vulnerable packages, lint clean. Raised REF-U5-01 (Umami-origin gate needs a parallel Gitea variable, since the backend side is a host env var per D-16). **U7 ✅** 2026-07-28 — Round 4: `WEBSITE_WORKSPACE.md`, README.md and `.env.example` updated, all claims re-verified against actual U1U6 source. **All 7 units complete.** - [x] Code Generation — **EXECUTE** (all 7 units, each built and tested before its completion message). **U1 ✅ U2 ✅** 2026-07-27 (253 backend tests). **U3 ✅** 2026-07-28 (315). **U4 ✅** 2026-07-28 (366 backend + 237 frontend). **U6 ✅ U5 ✅** 2026-07-28 — Round 3: `deploy-scp.yaml` + `continuous_integration.yaml`, 372 backend + 237 frontend tests (unchanged from Round 2, confirming no regressions from FR-21/FR-22 fixes), 0 vulnerable packages, lint clean. Raised REF-U5-01 (Umami-origin gate needs a parallel Gitea variable, since the backend side is a host env var per D-16). **U7 ✅** 2026-07-28 — Round 4: `WEBSITE_WORKSPACE.md`, README.md and `.env.example` updated, all claims re-verified against actual U1U6 source. **All 7 units complete.**
- [ ] Build and Test — **EXECUTE** (phase-level, now that all 7 units are complete) - [x] Build and Test — **EXECUTE** — Complete 2026-07-28. 609 unit tests (372 backend + 237 frontend), 0 vulnerable packages, 0 build errors. Plus live-host integration verification: static content/SPA-fallback/header-scoping, real master↔slave communication proving the persistent key ring survives a process restart, and W3C trace-id correlation in real log output. Artifacts in `construction/build-and-test/`
### OPERATIONS ### OPERATIONS
- [ ] Deployment Setup — **EXECUTE** - [ ] Deployment Setup — **EXECUTE**
@@ -83,10 +83,10 @@ Execution rounds (Q4 = B): **R1** = U1 + U2 · **R2** = U3 + U4 · **R3** = U5 +
- **Stages to Skip**: User Stories (no end-user functionality), NFR Requirements (already captured), plus per-unit skips as listed above - **Stages to Skip**: User Stories (no end-user functionality), NFR Requirements (already captured), plus per-unit skips as listed above
## Current Status ## Current Status
- **Lifecycle Phase**: CONSTRUCTION - **Lifecycle Phase**: OPERATIONS
- **Current Stage**: Round 4 **closed** 2026-07-28 — U7 complete and committed (`88770c5`). All 7 units of work are now generated - **Current Stage**: Build and Test **complete** 2026-07-28 (all 7 units, full-suite + live-host integration verification). CONSTRUCTION phase is now closed
- **Next Stage**: Build and Test (phase-level) — the last Construction stage before Operations - **Next Stage**: Deployment Setup (Operations Configuration = Yes, decided at Requirements Analysis)
- **Status**: Rounds 14 all done, nothing pushed. Starting Build and Test - **Status**: Rounds 14 all done, nothing pushed. Entering Operations phase
## Round 2 Design Record (2026-07-28) ## Round 2 Design Record (2026-07-28)
- Functional Design U3 + U4 complete and committed (`357d395`) - Functional Design U3 + U4 complete and committed (`357d395`)
@@ -1192,3 +1192,23 @@ by Infrastructure Design. U5 carries REF-U3-01's Umami origin gate plus FR-21 an
**Context**: CONSTRUCTION — all units complete; entering Build and Test **Context**: CONSTRUCTION — all units complete; entering Build and Test
--- ---
## Build and Test Stage
**Timestamp**: 2026-07-28T03:00:00Z
**Build Status**: Success
**Test Status**: Pass
**Files Generated**:
- build-instructions.md
- unit-test-instructions.md
- integration-test-instructions.md
- performance-test-instructions.md (N/A, rationale documented)
- build-and-test-summary.md
**Details**: Re-confirmed full-suite results across all 7 units: 609 unit tests (372 backend + 237 frontend), 0 build errors, 0 vulnerable packages. Additionally ran live-host integration verification not achievable via unit tests: started both SlpModularCms.Api and SlpModularCms.Api.Slave against a real local SQL Server (podman container "sql-server"), published the admin SPA for real, and verified static content serving, SPA-fallback routing, header content-type scoping (HTML vs non-HTML) on genuine responses, master-slave communication succeeding across a process restart (proving the persistent Data Protection key ring survives it), and W3C trace-id correlation in real log output. Cleaned up generated wwwroot/ afterward (gitignored, not committed).
**Operations Phase Decision**: Yes (Decided At: Requirements Analysis)
---
@@ -0,0 +1,72 @@
# Build and Test Summary
**Date**: 2026-07-28
## Build Status
- **Build Tool**: .NET 10 SDK (`dotnet build`), pnpm 9 / Vite (`pnpm run build`)
- **Build Status**: Success — `dotnet build SlpModularCms.sln -c Release`, 0 errors, 12 warnings
(all pre-existing `NU1510` "will not be pruned" informational notices)
- **Build Artifacts**: standard per-project `bin/Release/net10.0/` output; `frontend/dist/`
- **Build Time**: ~15s backend, ~1s frontend (incremental)
## Test Execution Summary
### Unit Tests
- **Total Tests**: 609 (372 backend + 237 frontend)
- **Passed**: 609
- **Failed**: 0
- **Coverage**: not tracked (D-10 — deliberate, no threshold enforced)
- **Status**: Pass
### Integration Tests
- **Test Scenarios**: 4 (static content/SPA-fallback/header-scoping, master↔slave real
communication over the persistent key ring, correlation-ID logging, startup behaviour) — see
`integration-test-instructions.md` for the full scenario table
- **Passed**: 4/4 scenarios, all individual checks within them passed
- **Failed**: 0
- **Status**: Pass
- **Method**: both hosts (`SlpModularCms.Api`, `SlpModularCms.Api.Slave`) run live against the
local SQL Server container, admin SPA published and served for real, verified with direct HTTP
requests rather than simulated `HttpContext` — the specific gap unit tests structurally cannot
close, per U1/U2/U3's own carried-forward Definition of Done items
### Performance Tests
- **Status**: N/A — no performance/throughput requirements exist for this feature (see
`performance-test-instructions.md`)
### Additional Tests
- **Contract Tests**: N/A — no microservice boundary changed by this feature; the U5↔U6
`workflow_call` interface is validated by YAML structure checks during Code Generation, not a
runtime contract test
- **Security Tests**: Vulnerability scan — `dotnet list SlpModularCms.sln package --vulnerable
--include-transitive` — **0 vulnerable packages across all 10 projects** (was 2 packages, multiple
advisories each, closed at U5 — FR-22, OPEN-03)
- **E2E Tests**: covered by the integration scenarios above (real requests through a real host); no
browser-driven UI E2E suite exists for the admin SPA in this repository
## Overall Status
- **Build**: Success
- **All Tests**: Pass
- **Ready for Operations**: Yes
## What This Stage Confirmed That Per-Unit Checks Could Not
- Static-file serving and SPA-fallback actually work end-to-end with real published output, not
just against `DefaultHttpContext` (U1)
- HTTP security headers are correctly scoped by content type on **real** responses — HTML gets the
full set, non-HTML gets only the always-on pair, confirmed on genuine static assets and the
`/health` JSON response (U3)
- The persistent Data Protection key ring genuinely survives a process restart: a slave API key
encrypted by an earlier process was successfully decrypted and used for authenticated master↔slave
calls by a freshly started process (U2, closing the loop on ASM-08)
- The W3C trace id correlation mechanism is present in real console log output (U4, OPEN-01)
## What Remains Genuinely Out of Scope for This Stage
Real Sentry events, real UptimeRobot monitors, an actual Gitea pipeline run against the self-hosted
runner, and a real Raspberry Pi deploy — all explicitly deferred to the Operations phase by U5 and
U6's own Definition of Done, because they require infrastructure (a live Sentry project, a live
Gitea instance, the actual Pi with SSH access) that does not exist yet at this stage of the workflow.
## Next Steps
Ready to proceed to the Operations phase (Deployment Setup → Monitoring Setup → Production Readiness
Validation), per this feature's `## Operations Configuration: Include Operations Phase: Yes`
(decided at Requirements Analysis).
@@ -0,0 +1,43 @@
# Build Instructions
## Prerequisites
- **Build Tool**: .NET 10 SDK, Node.js 20+, pnpm 9+
- **Dependencies**: NuGet packages restore automatically on build; frontend needs `pnpm install`
- **Environment Variables**: none required for a plain build/test pass. `appsettings.local.json` (backend) and `.env.local` (frontend) are only needed to actually *run* a host against a real database — see `README.md`
- **System Requirements**: any OS .NET 10 and Node 20 support. A SQL Server instance (e.g. via Podman/Docker, per `README.md` § "Database opstarten") is only needed to run a host, not to build or run the unit-test suites
## Build Steps
### 1. Install Dependencies
```powershell
dotnet restore SlpModularCms.sln
cd frontend
pnpm install --frozen-lockfile
```
### 2. Configure Environment
No environment configuration is needed for build or unit tests. `dotnet build`/`dotnet test` do not require a database connection.
### 3. Build All Units
```powershell
dotnet build SlpModularCms.sln -c Release
```
### 4. Verify Build Success
- **Expected Output**: `Build succeeded. 0 Error(s).`
- **Build Artifacts**: per-project `bin/Release/net10.0/` output
- **Common Warnings**: `NU1510` ("will not be pruned") on `Microsoft.Extensions.*.Abstractions` and `System.Security.Cryptography.Xml` — pre-existing, informational, not a build defect. No `NU1903` (vulnerability) warnings should appear; if one does, a package has drifted below its pinned patched version (see FR-22 in `requirements.md`)
## Troubleshooting
### Build Fails with Dependency Errors
- **Cause**: stale NuGet cache after a package pin change
- **Solution**: `dotnet nuget locals all --clear` then `dotnet restore SlpModularCms.sln`
### Build Fails with Compilation Errors
- **Cause**: none expected at this stage — all 7 units were individually build-verified during Code Generation (Step 13.5)
- **Solution**: if one appears, it indicates a cross-unit conflict (e.g. two units editing the same file inconsistently) that per-unit builds could not catch; check `git log --oneline` for the unit commits and diff the affected file
### Running Two Hosts Concurrently Locks the Shared Core Build
- **Cause**: `SlpModularCms.Api` and `SlpModularCms.Api.Slave` both reference `SlpModularCms.Core`. Starting `dotnet run` for both at the same moment races on `SlpModularCms.Core`'s build output directory
- **Solution**: start one host, wait for it to finish building, then start the second — this only affects local `dotnet run`, not `dotnet build`/`dotnet test` on the full solution
@@ -0,0 +1,101 @@
# Integration Test Instructions
## Purpose
Verify everything that unit tests structurally cannot: real HTTP responses from a running host,
real static-file serving, real cross-process master/slave communication, and real Data Protection
key-ring behaviour. Every scenario below was explicitly flagged as "carried to phase-level Build and
Test" by U1, U2 or U3, because `DefaultHttpContext` and in-memory test doubles cannot exercise them
(e.g. `Response.OnStarting` is a no-op outside a real Kestrel pipeline, and a unit test cannot start
two real processes and let them talk to each other).
## Setup Integration Test Environment
### 1. Start a database
```powershell
podman start sql-server
# or, if it doesn't exist yet: see README.md § "Database opstarten"
```
### 2. Populate the static content mounts (optional but recommended)
Without this, the SPA-fallback and placeholder scenarios still work, but the "real static asset"
scenarios need actual files:
```powershell
dotnet publish src/SlpModularCms.Api -c Release -o ./tmp-publish-verify
```
This also copies the built admin SPA into `src/SlpModularCms.Api/wwwroot/admin/` (via
`BuildAndCopyAdminFrontend`). Create a placeholder `wwwroot/web/index.html` by hand to exercise the
"real website content" scenarios, or leave it absent to exercise the placeholder scenario.
**`wwwroot/` is gitignored and must be deleted after testing** — it is generated content, not part
of the repository.
### 3. Start both hosts
```powershell
dotnet run --project src/SlpModularCms.Api --launch-profile https # port 7221
dotnet run --project src/SlpModularCms.Api.Slave --launch-profile https # port 7222
```
Start them **one at a time**, waiting for the first to finish building — starting both
simultaneously races on the shared `SlpModularCms.Core` build output (see `build-instructions.md`
troubleshooting).
## Run Integration Tests
### Scenario 1: Static content, SPA fallback, and header scoping (U1, U3)
| Request | Expected | Verified 2026-07-28 |
|---|---|---|
| `GET /` with no `wwwroot/web/` | 200, embedded placeholder HTML, `Relaxed` CSP | ✅ |
| `GET /` with `wwwroot/web/index.html` present | 200, real content, `Relaxed` CSP | ✅ |
| `GET /admin` | 301 → `/admin/` | ✅ |
| `GET /admin/` | 200, admin `index.html`, `Strict` CSP (`DENY`, no `unsafe-inline` in `script-src`) | ✅ |
| `GET /admin/dashboard` (client route, no extension) | 200, falls back to admin `index.html` | ✅ |
| `GET /admin/assets/{real-file}.js` | 200, real file served, `Content-Type: text/javascript`, `X-Content-Type-Options` present, **no** CSP/X-Frame-Options/Referrer-Policy (non-HTML — FU1 = A scoping) | ✅ |
| `GET /admin/assets/does-not-exist.js` | 404 (missing file with an extension does **not** fall back to HTML) | ✅ |
| `GET /a-website-route-with-no-extension` | 200, falls back to website `index.html`, `Relaxed` CSP | ✅ |
| `GET /health` | 200, `application/json`, `X-Content-Type-Options` present, **no** CSP/X-Frame-Options/Referrer-Policy (non-HTML) | ✅ |
**Note on HSTS**: not observed on any response above — correct, since `sendHsts` is deliberately
`!IsDevelopment()` and all of the above ran with `ASPNETCORE_ENVIRONMENT=Development`. Re-verify with
`ASPNETCORE_ENVIRONMENT=Production` before a real release if this has not already been proven in a
non-Development environment.
### Scenario 2: Master ↔ Slave real communication over the persistent key ring (U2, Master module)
| Check | Expected | Verified 2026-07-28 |
|---|---|---|
| Master's `IntegrityCheckBackgroundService` reaches a previously-registered slave | `GET https://localhost:7222/api/v1/master/registered-url` → 200 | ✅ |
| Master successfully authenticates with a stored, encrypted slave API key from a **previous process's** key ring | `POST https://localhost:7222/api/v1/master/status` → 200 (not a decryption failure) | ✅ |
This is the strongest available local proof that U2's persistent, database-backed key ring
(`PersistKeysToDbContext` with an explicit discriminator) actually survives a process restart —
the stored API key was encrypted by an earlier process and successfully decrypted by this one.
### Scenario 3: Correlation ID in structured logging (U4, OPEN-01)
| Check | Expected | Verified 2026-07-28 |
|---|---|---|
| A real request's log entries | Carry a `TraceId:` (W3C trace id) in the log scope | ✅ (e.g. `TraceId:5a302c33b4f0fc8fb9d0b35f840934b8` on a request to `/api/v1/System/capabilities`) |
### Scenario 4: Startup behaviour (U1, U2)
| Check | Expected | Verified 2026-07-28 |
|---|---|---|
| Both hosts start with a fresh/existing database | `MigrateCoreDatabase()` logs "Core database migrations applied successfully", no fail-fast | ✅ |
| Missing `wwwroot/admin/` and `wwwroot/web/` at startup | Warning logged per directory, host still starts and serves `/api/v1` | ✅ (observed on the very first run, before publishing) |
## Not Re-Verified at This Stage (out of scope, per the units' own Definition of Done)
- **503 response from the availability gate carrying security headers** (U3's own carried item) —
covered by unit tests using a simulated `HttpContext`; a fully authenticated real-host repeat
would need a bootstrapped Owner login flow that this repository's README does not yet document
(a pre-existing gap, unrelated to this feature). Indirect evidence is strong: every response
observed above — 200s **and** 404s alike — carried `X-Content-Type-Options`, confirming the
`OnStarting` hook fires uniformly regardless of status code
- **Real Sentry event with the `security_event` tag, tunnel status codes, threshold behaviour**
needs a configured Sentry DSN; deferred to Operations (Monitoring Setup)
- **UptimeRobot, real Gitea pipeline run, actual Pi/SSH/database backup** — deferred to Operations
per U5/U6's own Definition of Done
## Cleanup
```powershell
# Stop both dotnet run processes
Remove-Item -Recurse -Force src/SlpModularCms.Api/wwwroot, tmp-publish-verify
```
`wwwroot/` must not be committed — verify `git status` is clean afterwards.
@@ -0,0 +1,14 @@
# Performance Test Instructions
## Status: N/A for this feature
`requirements.md` § 56 (Non-Functional Requirements) defines no throughput, latency or concurrent-user
targets for this feature — it is CI/CD and deployment tooling, not a user-facing capacity change.
The one performance-adjacent decision made (`TracesSampleRate: 0.1` for Sentry, `TunnelMaxPayloadBytes:
204800`) is a cost/quota control for observability, not a system performance requirement, and was
already verified during U4's code generation.
If load/throughput requirements are identified later (e.g. for the deploy pipeline's own duration, or
for the public website once real content is deployed into `wwwroot/web/`), they belong to a
follow-up feature — this deployment-workflow feature does not change the CMS's own runtime capacity
characteristics.
@@ -0,0 +1,29 @@
# Unit Test Execution
## Run Unit Tests
### 1. Execute All Unit Tests
```powershell
dotnet test SlpModularCms.sln -c Release
cd frontend
pnpm run test
```
### 2. Review Test Results
- **Expected — backend**: 372 tests pass, 0 failures (Core 196, Availability 82, Master 57, Identity 37). `SlpModularCms.Api` and `SlpModularCms.Api.Slave` have no test project by convention (CLAUDE.md) — their behaviour is covered by this stage's live integration verification instead
- **Expected — frontend**: 237 tests pass, 0 failures, across 38 test files
- **Test Coverage**: no coverage threshold is enforced (D-10 — deliberate decision, not an oversight)
- **Test Report Location**: console output; `dotnet test` and `vitest run` both exit non-zero on any failure, which is what the CI gates (U5) act on
### 3. Fix Failing Tests
If tests fail:
1. Identify which project/file failed from the console output
2. Since every unit was already build-and-test-verified individually during Code Generation, a failure here means a **cross-unit** interaction broke something that passed in isolation — check what changed in the units involved
3. Fix and rerun until green
## Result at This Stage (2026-07-28)
- Backend: **372 passed, 0 failed**
- Frontend: **237 passed, 0 failed**
- Both counts are unchanged from the Round 2 and Round 3 per-unit baselines — confirms the full suite passes together, not just in isolation