Files
slp-modular-cms/aidlc-docs/features/media-library/inception/application-design/migration-plan.md
T
SluijsensandClaude Sonnet 5.5 150c156fd7
Continuous Integration / config-sls (pull_request) Successful in 10s
Continuous Integration / config-asf (pull_request) Successful in 7s
Continuous Integration / changes (pull_request) Successful in 20s
Continuous Integration / compile-sites (pull_request) Canceled after 0s
Continuous Integration / backend-test (pull_request) Canceled after 0s
Continuous Integration / vulnerability-scan (pull_request) Canceled after 0s
Continuous Integration / frontend-build (pull_request) Canceled after 0s
Continuous Integration / frontend-test (pull_request) Canceled after 0s
Continuous Integration / frontend-prepare (pull_request) Canceled after 27s
Continuous Integration / frontend-lint (pull_request) Canceled after 0s
Continuous Integration / publish-sls-test (pull_request) Canceled after 0s
Continuous Integration / publish-sls-production (pull_request) Canceled after 0s
Continuous Integration / deploy-sls-test (pull_request) Canceled after 0s
Continuous Integration / deploy-sls-production (pull_request) Canceled after 0s
Continuous Integration / publish-asf-test (pull_request) Canceled after 0s
Continuous Integration / deploy-asf-test (pull_request) Canceled after 0s
Continuous Integration / smoke-sls-test (pull_request) Canceled after 0s
Continuous Integration / smoke-asf-test (pull_request) Canceled after 0s
Continuous Integration / backend-build (pull_request) Canceled after 54s
Mediabibliotheek: module Media met eigen tabellen en geverifieerde kopie van de Hero-foto's (fase 1 van de migratie, standaard uit)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-17 15:38:40 +01:00

90 lines
6.8 KiB
Markdown

# Migration plan: HeroImages to the library's own tables (decision 13)
Status: plan, not built, not run anywhere. Decided by Bryan 2026-11-17 (via SLP Software - SLS - Main): it may be built in the next step, **only after this plan with its back-up and rollback exists**, **rolled out to TEST only (never production)**, and **SLS-Main is told before anything is deployed to test**.
## Goal
Give every photo one library record that does not depend on the Hero: `MediaAsset` (the photo) and `MediaUse` (where it is used), with **the same ids, hashes, set hashes and public addresses** as today. Nothing in `/media/hero/<setHash>/...` changes, so ASF, SLS and the golden files stay as they are.
## Why expand then contract
| Phase | What | Reversible? |
|---|---|---|
| 0 Back-up | Back-up of the database and a listing of the media directory (below). | n/a |
| 1 Expand | New tables `MediaAssets` and `MediaUses`, filled by copying `HeroImages` and `HeroSlides`. The Hero keeps reading and writing its own tables; the library reads the new tables behind a setting. | Yes: drop the two new tables, nothing else changed. |
| 2 Switch | Hero reads photos from the library tables (setting on). New uploads write both (dual write) so the old tables stay complete. | Yes: switch the setting off; the old tables are complete. |
| 3 Contract (a later release, after acceptance) | The Hero stops writing its own image columns; they are dropped in a separate migration. | Only from the back-up of that moment. Not part of the first rollout. |
Files are never moved or renamed: the migration changes rows only, so the media directory is the same before and after.
## Mapping
| `HeroImages` column | `MediaAssets` column | Note |
|---|---|---|
| `Id` | `Id` | Same id, so references keep working. |
| `OriginalSha256`, `OriginalKey`, `SourceFormat`, `Width`, `Height`, `Bytes` | same | |
| `FocusX`, `FocusY`, `Zoom` | `DefaultFocusX`, `DefaultFocusY`, `DefaultZoom` | The photo's default focus. |
| `WideFocus*`, `TallFocus*` | `MediaUses` rows (one per slide use, ratios `wide` and `tall`) | The per-use crop; for a photo without a slide the default focus is enough. |
| `SetHash`, `Variants` | `MediaUses.SetHash` (per use) | Unchanged values; computed by the existing formula. |
| `CreatedAt`, `CreatedByUserId` | same | |
| `OriginalFileName`, `LastUsedAt`, `DeletedAt` | same | Added in the media-library step (additive columns of the Hero). |
| `HeroSlides` | `MediaUses` |
|---|---|
| `Id`, `ImageId` | `Id`, `AssetId` (module `hero`, place kind `slide`, place id = slide id) |
## Back-up (before phase 1, every environment)
1. **Database**: a full dump of the Hero database (and of the CMS database if it is one), written to a dated file outside the deploy directory, for example `mysqldump --single-transaction <database> > hero-before-media-migration-YYYYMMDD.sql`. Keep it until the contract phase is accepted.
2. **Media directory**: nothing is moved, but a listing with sizes and hashes is saved (`find <Media__Directory> -type f -exec sha256sum {} +` to a text file) so the directory can be compared afterwards. On the Raspberry Pi a copy of the directory is also a quick option (`cp -a`), if there is room.
3. **Row counts** noted: `HeroImages`, `HeroSlides`, `HeroRetiredFiles`.
## Verification (right after phase 1, every environment)
The migration job compares and **aborts** (rolls its transaction back) on any difference:
- number of `MediaAssets` = number of `HeroImages` (all rows, also those in the trash);
- every `HeroImages.Id`, `OriginalSha256` and `SetHash` has an identical `MediaAssets`/`MediaUses` counterpart;
- number of `MediaUses` of module `hero` = number of `HeroSlides`;
- a sample of ten public addresses (`/media/hero/<setHash>/<file>`) is requested before and after and gives the same bytes (checked by the smoke test).
## Rollback
| When | How |
|---|---|
| The verification fails | The migration runs in one transaction and rolls back by itself; nothing is left. |
| After phase 1, before the switch | Drop `MediaUses` and `MediaAssets` (the EF `Down` migration). The Hero never read them. |
| After the switch | Switch the setting back (`Media:Library:ReadFromLibrary=false`); the old tables are complete because of the dual write. Then drop the new tables if wanted. |
| After the contract phase (later) | Restore the database dump taken before that migration and the media listing check. |
## Roll-out rules (from Bryan)
1. **Test only.** No production variables, no production deploy, no release branch.
2. **Tell SLS-Main before deploying to test** (they run the deploy and the smoke test).
3. Back-up first (above), on the test host too.
4. Deploy with the setting **off**, check the verification result in the log, then switch it on, then the smoke test (hero markers, `/theme-init.js`, a sample of media addresses), then Bryan checks the admin.
## Status (2026-11-17): phase 1 (expand) is built, behind a setting that is OFF
Built on `feature/media-library-screen`, not yet deployed anywhere:
- New module `SlpModularCms.Modules.Media` (own `MediaDbContext`, tables `MediaAssets` and `MediaUses`, additive migration `InitialCreate`; no existing table is touched). It is referenced by the three hosts, so a deploy of this code creates the two empty tables at start-up and nothing else.
- `IMediaMigrationSource` (Core) and `HeroMediaMigrationSource` (Hero, read-only): the hero's photos (also those in the trash) and slides with their own ids, hashes and set hashes.
- `MediaLibraryCopier`: copies and **verifies** (counts, ids, hash, key, size, set hash) in one transaction; any difference or error rolls the whole copy back and nothing is left. It MIRRORS the source, so running it again gives the same result; rows of other modules are not touched.
- The copy runs only when `Media__Library__CopyOnStartup=true` (default **false**) and a failure is logged, never fatal.
- Tested in memory and against a real MariaDB (migration, copy twice, rollback on a failing copy).
Nothing reads from the new tables yet (phase 2, the switch, is not built). The library screen still reads the hero through the usage provider.
### Roll-out on TEST (needs SLS-Main to be told first)
1. Back-up (database dump and media listing) as above.
2. Deploy the branch with `Media__Library__CopyOnStartup` unset: the two empty tables appear; check the log for the Media migration.
3. Set `Media__Library__CopyOnStartup=true` and restart: the log says `Media library copy verified: N photos and M uses from hero` (or the error and the rollback).
4. Rollback at any point: set it back to false (the tables stay, unused) or drop `MediaUses` and `MediaAssets`.
5. Never on production.
### Still to build (phases 2 and 3)
The switch (`Media:Library:ReadFromLibrary`, dual write of new uploads, library reading from the new tables) and the contract phase. They need Bryan's acceptance of the copy on test first.