Makes a redeploy safe for the key ring and the schema
Nothing here is visible in normal operation. Its whole purpose is that swapping the release directory on deploy cannot silently destroy state. Data Protection secures the API keys that authenticate master/slave communication. Two separate defaults would each have destroyed them: keys are held on the filesystem, which a release swap discards, and the application discriminator is derived from the content root path, which changes with every release directory — so even keys stored in a database would have stopped being derivable. Keys now live in ApplicationDbContext and the discriminator is a fixed constant. Losing them produces no error. It produces stored keys that no longer decrypt, which presents as an apparent network fault between a Master and its slaves and is easily misdiagnosed. That is also why the tests assert the resulting configuration rather than the registration: the XmlRepository must be the EF one and the discriminator must be the constant, plus a round-trip proving a value encrypted before a deploy is readable after one. A test that only checked "Data Protection is registered" would have passed in the broken case too. Both modules previously called AddDataProtection() themselves. Module registration runs after the host's, so those calls re-registered the configuration chain and would have overridden the persistent store while IDataProtector still resolved. They are removed, with a comment at each site — the deletion otherwise looks like a regression. Each module's own test project now guards against it being reintroduced. ApplicationDbContext also migrates itself at startup. Deploy targets offer no CLI, so migrations cannot be a manual step on the server. Failures are classified rather than treated alike: a connection failure means the database is not up yet, normal when the app and the database start together after a reboot, and is retried with backoff; a migration failure means something is broken and fails at once. Either way the process does not start, which is what makes the liveness health check trustworthy — an application that cannot reach its schema never answers /health, so monitoring goes red instead of reporting a healthy instance that cannot serve a request. The cost of migrating without a human gate is that migrations must stay forward-compatible and non-destructive, since rollback is "redeploy the previous release". The new migration is purely additive. Also wires this and the preceding hosting commit into both hosts, as they touch the same lines of Program.cs. Two constraints are enforced by documentation rather than code, and belong in the deployment instructions: the key table must never be pruned, and only one instance may migrate a given database at a time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHoJpxYXzHACSQguHrC5fw
This commit is contained in:
Claude Opus 5
+145
@@ -0,0 +1,145 @@
|
||||
# Code Generation Plan — U2 Data Durability
|
||||
|
||||
**This plan is the single source of truth for Code Generation of U2.** Generation executes exactly these steps in order; no step is added or skipped during execution.
|
||||
|
||||
---
|
||||
|
||||
## Unit Context
|
||||
|
||||
| Aspect | Detail |
|
||||
|---|---|
|
||||
| **Unit** | U2 Data Durability |
|
||||
| **Round** | R1 (with U1 Hosting & Serving) |
|
||||
| **Workspace root** | `K:\Development\Projects\SlpModularCms` |
|
||||
| **Project type** | Brownfield — existing structure retained, files modified in place |
|
||||
| **Requirements** | FR-11, FR-12 |
|
||||
| **Components** | C-05 Data Protection, C-06 keys table, C-07 migration runner, U2 portion of C-16 |
|
||||
| **Business rules** | BR-U2-01 … BR-U2-18 |
|
||||
| **Depends on** | Nothing. U1 and U2 are mutually independent |
|
||||
| **Depended on by** | U6 — durability must land before the first automated deploy |
|
||||
| **New database entities** | One: the Data Protection keys table, owned by `ApplicationDbContext` |
|
||||
|
||||
### Requirement traceability
|
||||
|
||||
| Requirement | Implemented by steps |
|
||||
|---|---|
|
||||
| FR-11 — automatic `ApplicationDbContext` migration at startup | 4, 5, 6 |
|
||||
| FR-12 — persistent Data Protection key ring | 1, 2, 3, 5, 6 |
|
||||
|
||||
### Why this unit exists
|
||||
|
||||
Nothing here is user-visible. Its whole purpose is that U6's atomic release switch **cannot** silently destroy schema state or Master↔slave trust. Two failure modes are being closed, both of which would otherwise appear only after the first production deploy and present as something else entirely.
|
||||
|
||||
---
|
||||
|
||||
## Generation Steps
|
||||
|
||||
### Step 1: Package reference
|
||||
- [x] Modify `src/SlpModularCms.Core/SlpModularCms.Core.csproj` to add `Microsoft.AspNetCore.DataProtection.EntityFrameworkCore` version **10.0.9**, matching the existing 10.0.x line
|
||||
|
||||
### Step 2: Keys table on the Core context
|
||||
- [x] Modify `src/SlpModularCms.Core/Data/ApplicationDbContext.cs`:
|
||||
- [x] Implement `IDataProtectionKeyContext`
|
||||
- [x] Add `DbSet<DataProtectionKey> DataProtectionKeys`
|
||||
- [x] Leave every existing entity configuration untouched
|
||||
|
||||
### Step 3: Data Protection registration
|
||||
- [x] Create `src/SlpModularCms.Core/Hosting/DataProtectionExtensions.cs` with `AddCmsDataProtection()`
|
||||
- [x] Configure `PersistKeysToDbContext<ApplicationDbContext>()` (BR-U2-01)
|
||||
- [x] Set the application discriminator to a **fixed constant in code** (BR-U2-02) — not configurable, not derived from any path
|
||||
- [x] Leave key lifetime at the framework default of 90 days (BR-U2-05)
|
||||
- [x] Document in code why the discriminator is a constant: the default derives from the content root path, which changes on every atomic release switch
|
||||
|
||||
### Step 4: Startup migration runner
|
||||
- [x] Create `src/SlpModularCms.Core/Hosting/DatabaseMigrationExtensions.cs` with `MigrateCoreDatabase()`
|
||||
- [x] Apply `ApplicationDbContext` migrations before the application accepts traffic (BR-U2-09)
|
||||
- [x] Classify failures (BR-U2-11, BR-U2-12): retry **connection** failures with increasing delay up to a bounded number of attempts; fail **migration** failures immediately with no retry
|
||||
- [x] Log the reason before failing, with diagnostic context but **no** connection string, credentials or secrets (BR-U2-14)
|
||||
- [x] Propagate the exception when retries are exhausted or the failure is a migration failure, so the process does not start (BR-U2-13)
|
||||
|
||||
### Step 5: Remove the conflicting module registrations
|
||||
- [x] Modify `src/SlpModularCms.Modules.Availability/AvailabilityModule.cs` — remove `services.AddDataProtection()`
|
||||
- [x] Modify `src/SlpModularCms.Modules.Master/MasterModule.cs` — remove `services.AddDataProtection()`
|
||||
- [x] Leave every other registration in both modules unchanged; they continue consuming `IDataProtector` (BR-U2-04)
|
||||
|
||||
*This is the § 5.1 conflict. Module registration runs **after** the host's, so these bare calls would override the persistent key store. `IDataProtector` resolves either way, so the defect would surface only after the first release switch as slave API keys that no longer decrypt — presenting as a network fault between Master and slave.*
|
||||
|
||||
### Step 6: Host composition
|
||||
- [x] Modify `src/SlpModularCms.Api/Program.cs` — call `AddCmsDataProtection()` **before** `orchestrator.RegisterModuleServices(...)` (BR-U2-03), and `MigrateCoreDatabase()` after `builder.Build()` and before `orchestrator.UseModules(app)` (BR-U2-10)
|
||||
- [x] Modify `src/SlpModularCms.Api.Slave/Program.cs` — the same two calls in the same positions
|
||||
|
||||
### Step 7: Core migration
|
||||
- [x] Generate the EF Core migration for the keys table into `src/SlpModularCms.Core/Migrations/`
|
||||
- [x] Verify the migration is purely additive — no dropped or narrowed columns, so redeploying an earlier release stays safe (BR-U2-16)
|
||||
|
||||
### Step 8: Data Protection unit tests
|
||||
- [x] Create `src/SlpModularCms.Core.Tests/Hosting/DataProtectionExtensionsTests.cs`:
|
||||
- [x] The persistent key store **survives module registration** — the highest-value assertion in this unit, since registration alone passes in both the broken and fixed cases
|
||||
- [x] The application discriminator is the fixed constant, not a path-derived value
|
||||
- [x] A protected value round-trips across a simulated content-root change
|
||||
- [x] Neither module registers Data Protection, so the conflict cannot be reintroduced by a future change
|
||||
|
||||
### Step 9: Migration runner unit tests
|
||||
- [x] Create `src/SlpModularCms.Core.Tests/Hosting/DatabaseMigrationExtensionsTests.cs`:
|
||||
- [x] A connection failure is retried
|
||||
- [x] A migration failure is **not** retried and fails immediately
|
||||
- [x] Retry exhaustion propagates
|
||||
- [x] Failure logging contains no connection string or credentials
|
||||
|
||||
### Step 10: Documentation
|
||||
- [x] Create `aidlc-docs/features/gitea-deployment-workflow/construction/u2-data-durability/code/generation-summary.md` — files created and modified, decisions taken, and any deviation from this plan
|
||||
- [x] Record the two operational constraints that are enforced by documentation rather than code, for later inclusion in the Operations deployment instructions: the keys table must **never** be pruned (BR-U2-06), and only one instance may migrate a given database at a time (BR-U2-17)
|
||||
|
||||
### Step 11: Build and test verification (automatic)
|
||||
- [x] `dotnet build SlpModularCms.sln -c Release`
|
||||
- [x] `dotnet test` for `SlpModularCms.Core.Tests`, `SlpModularCms.Modules.Availability.Tests` and `SlpModularCms.Modules.Master.Tests`
|
||||
- [x] Fix any failure directly and re-run until green
|
||||
- [x] Record the outcome for the completion message
|
||||
|
||||
---
|
||||
|
||||
## Files Touched
|
||||
|
||||
### Created
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `src/SlpModularCms.Core/Hosting/DataProtectionExtensions.cs` | Persistent key ring registration |
|
||||
| `src/SlpModularCms.Core/Hosting/DatabaseMigrationExtensions.cs` | Startup migration with failure classification |
|
||||
| `src/SlpModularCms.Core/Migrations/*_AddDataProtectionKeys.cs` | Keys table migration |
|
||||
| `src/SlpModularCms.Core.Tests/Hosting/DataProtectionExtensionsTests.cs` | Tests |
|
||||
| `src/SlpModularCms.Core.Tests/Hosting/DatabaseMigrationExtensionsTests.cs` | Tests |
|
||||
|
||||
### Modified
|
||||
| Path | Change |
|
||||
|---|---|
|
||||
| `src/SlpModularCms.Core/SlpModularCms.Core.csproj` | Add the Data Protection EF Core package |
|
||||
| `src/SlpModularCms.Core/Data/ApplicationDbContext.cs` | Implement `IDataProtectionKeyContext`, add the keys set |
|
||||
| `src/SlpModularCms.Modules.Availability/AvailabilityModule.cs` | Remove `AddDataProtection()` |
|
||||
| `src/SlpModularCms.Modules.Master/MasterModule.cs` | Remove `AddDataProtection()` |
|
||||
| `src/SlpModularCms.Api/Program.cs` | Data Protection registration and startup migration |
|
||||
| `src/SlpModularCms.Api.Slave/Program.cs` | Same |
|
||||
|
||||
**Brownfield rule**: every file above that exists is modified in place. No parallel copies.
|
||||
|
||||
---
|
||||
|
||||
## Risk Notes for the Executor
|
||||
|
||||
| Risk | Mitigation in this plan |
|
||||
|---|---|
|
||||
| A test that merely asserts "Data Protection is registered" passes in both the broken and fixed cases | Step 8 asserts the **resulting configuration**, not the registration |
|
||||
| The discriminator silently reverting to the path-derived default | Step 8 asserts the constant explicitly |
|
||||
| The new migration being non-additive and breaking rollback | Step 7 verifies additivity |
|
||||
| Startup migration masking a genuine migration fault by retrying it | Step 4 classifies failures; Step 9 asserts the classification |
|
||||
| Both hosts must still start | Step 11 builds the whole solution; the composed startup is verified at the phase-level Build and Test stage, where the Slave — which has no test project — is started |
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope for U2
|
||||
|
||||
- Static content, health endpoint, availability-gate changes — U1
|
||||
- Security headers — U3
|
||||
- Sentry, Umami, frontend configuration — U4
|
||||
- Anything under `.gitea/` — U5 and U6
|
||||
- Certificate-based key encryption — deferred as DEV-05 follow-up
|
||||
- A distributed migration lock — Q4 = C, handled by documented operational constraint
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# Functional Design Questions — U2 Data Durability
|
||||
|
||||
Vul je keuze in achter elke `[Answer]:`-tag. Kies de laatste optie (`Anders`) als niets past.
|
||||
|
||||
---
|
||||
|
||||
## Question 1 — Waar komt de application discriminator vandaan?
|
||||
|
||||
**Context**: de application discriminator bepaalt of twee processen dezelfde Data Protection-sleutels kunnen gebruiken. Standaard leidt ASP.NET Core hem af uit het content root-pad — en dat verandert bij élke atomaire release-switch. Zonder expliciete waarde is de key ring dus alsnog effectief weg na een deploy, ondanks dat hij in de database staat.
|
||||
|
||||
Hij moet dus vast staan. De vraag is waar die waarde vandaan komt.
|
||||
|
||||
A) Een vaste constante in de code (bijv. `"SlpModularCms"`) — kan niet per ongeluk verkeerd gezet worden, en is voor alle instanties gelijk
|
||||
B) Uit configuratie, met een vaste standaardwaarde — dan kun je per klant/instantie een eigen waarde zetten als dat ooit nodig is
|
||||
C) Uit configuratie, verplicht in te vullen — dwingt een bewuste keuze af per omgeving
|
||||
X) Anders (beschrijf hieronder na de [Answer]:-tag)
|
||||
|
||||
[Answer]:A
|
||||
|
||||
---
|
||||
|
||||
## Question 2 — Moeten de sleutels versleuteld in de database staan?
|
||||
|
||||
**Context**: `PersistKeysToDbContext` slaat de sleutels standaard **onversleuteld** op als XML in de tabel. Wie de database kan lezen, kan daarmee de opgeslagen slave-API-keys ontsleutelen.
|
||||
|
||||
Op Windows lost DPAPI dit normaal op, maar dat werkt niet op Linux (de Pi), dus dat is hier geen optie. Het alternatief is versleutelen met een X.509-certificaat — maar dan moet dat certificaat mee gedeployed worden en beschikbaar blijven, wat een nieuwe versie van hetzelfde probleem introduceert: raak je het certificaat kwijt, dan zijn de sleutels alsnog onleesbaar.
|
||||
|
||||
SECURITY-01 vraagt om versleuteling at rest.
|
||||
|
||||
A) Onversleuteld in de database, en de encryptie-at-rest van de database zelf is de maatregel — vastleggen als bewuste onderbouwde keuze, met de eis dat de databaseverbinding TLS gebruikt en de database niet publiek benaderbaar is
|
||||
B) Versleutelen met een X.509-certificaat — sterker, maar verplaatst het bewaarprobleem naar het certificaat en voegt een deploystap toe
|
||||
C) Onversleuteld nu, en certificaat-encryptie als apart vervolgpunt vastleggen
|
||||
X) Anders (beschrijf hieronder na de [Answer]:-tag)
|
||||
|
||||
[Answer]:C
|
||||
|
||||
---
|
||||
|
||||
## Question 3 — Wat gebeurt er als de database bij het opstarten net niet bereikbaar is?
|
||||
|
||||
**Context**: je koos fail fast bij een migratiefout. Maar er is een verschil tussen "de migratie klopt niet" (echt fout) en "de database is er nog even niet" (tijdelijk) — bijvoorbeeld als de app en de SQL Server-container tegelijk opstarten na een herstart van de Pi.
|
||||
|
||||
Bij strikte fail-fast start de app dan niet, en moet iets anders hem opnieuw starten.
|
||||
|
||||
A) Strikt fail fast, geen retry — de procesmanager (systemd) herstart de app toch al automatisch, dus dat lost het vanzelf op
|
||||
B) Een korte retry met toenemende wachttijd (bijv. 5 pogingen over ~30 seconden) en dán pas falen — vangt het opstartvenster af zonder een echte fout te verbergen
|
||||
C) Retry alleen bij verbindingsfouten, direct falen bij een migratiefout — onderscheid tussen "nog niet bereikbaar" en "kapot"
|
||||
X) Anders (beschrijf hieronder na de [Answer]:-tag)
|
||||
|
||||
[Answer]:C
|
||||
|
||||
---
|
||||
|
||||
## Question 4 — Wat als twee instanties tegelijk opstarten en migreren?
|
||||
|
||||
**Context**: `Database.Migrate()` is niet ontworpen om veilig gelijktijdig te draaien. In jouw huidige opzet draait er één instantie per omgeving, dus dit speelt nu niet. Maar de master/slave-opzet betekent dat er meerdere instanties naar **verschillende** databases wijzen, en een herstart kan ze wel gelijktijdig laten opstarten.
|
||||
|
||||
A) Negeren — één instantie per database, dus dit kan niet voorkomen. Wel als aanname vastleggen
|
||||
B) Een migratielock in de database gebruiken zodat gelijktijdig migreren veilig is — robuuster, maar meer complexiteit voor een situatie die zich nu niet voordoet
|
||||
C) Alleen documenteren in de deployment-instructies dat instanties niet gelijktijdig gemigreerd moeten worden
|
||||
X) Anders (beschrijf hieronder na de [Answer]:-tag)
|
||||
|
||||
[Answer]:C
|
||||
|
||||
---
|
||||
|
||||
## Question 5 — Sleutellevensduur en rotatie
|
||||
|
||||
**Context**: standaard maakt Data Protection elke 90 dagen een nieuwe sleutel aan en houdt oude sleutels beschikbaar om bestaande waarden te kunnen blijven ontsleutelen. Voor de versleutelde slave-API-keys betekent dat: die blijven leesbaar, ook na rotatie, zolang de oude sleutels in de tabel blijven staan.
|
||||
|
||||
A) De standaard van 90 dagen aanhouden en oude sleutels nooit opruimen — bestaande waarden blijven altijd leesbaar
|
||||
B) Een langere levensduur instellen zodat er minder sleutels ontstaan
|
||||
C) De standaard aanhouden, plus expliciet vastleggen in de documentatie dat de sleuteltabel nooit opgeschoond mag worden — want dat zou de opgeslagen API-keys onleesbaar maken
|
||||
X) Anders (beschrijf hieronder na de [Answer]:-tag)
|
||||
|
||||
[Answer]:C
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Functional Design Plan — U2 Data Durability
|
||||
|
||||
**Unit**: U2 Data Durability
|
||||
**Round**: R1 (with U1 Hosting & Serving)
|
||||
**Requirements**: FR-11, FR-12
|
||||
**Components**: C-05, C-06, C-07, U2 portion of C-16
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Analyze unit context
|
||||
- [x] Read the U2 definition from `unit-of-work.md`
|
||||
- [x] Read the requirement assignment from `unit-of-work-story-map.md`
|
||||
- [x] Read the carried-in design items — § 5.1 duplicate registration conflict, explicit application discriminator
|
||||
|
||||
## Step 2: Design the Data Protection key ring
|
||||
- [x] Define the key-storage entity and its owning context
|
||||
- [x] Define the application-discriminator source and stability guarantee
|
||||
- [x] Define key encryption at rest
|
||||
- [x] Define key lifetime and rotation behaviour
|
||||
- [x] Define the registration-order rule that resolves the duplicate-registration conflict
|
||||
|
||||
## Step 3: Design startup migration behaviour
|
||||
- [x] Define which contexts migrate and in what order
|
||||
- [x] Define failure behaviour and what is logged before failing
|
||||
- [x] Define behaviour when the database is temporarily unreachable at startup
|
||||
- [x] Define behaviour when a migration is applied concurrently by two starting instances
|
||||
|
||||
## Step 4: Define business rules
|
||||
- [x] Enumerate key-ring durability rules
|
||||
- [x] Enumerate migration rules
|
||||
- [x] Identify error and edge-case scenarios
|
||||
|
||||
## Step 5: Design verification approach
|
||||
- [x] Define how "the persistent key store survives module registration" is asserted
|
||||
- [x] Define how discriminator stability across a content-root change is asserted
|
||||
|
||||
## Step 6: Generate artifacts
|
||||
- [x] Generate `business-logic-model.md`
|
||||
- [x] Generate `business-rules.md`
|
||||
- [x] Generate `domain-entities.md`
|
||||
- [x] Validate all diagrams against the Mermaid standards
|
||||
- [x] Verify Security Baseline compliance for this unit's design
|
||||
Reference in New Issue
Block a user