Files
slp-modular-cms/aidlc-docs/features/media-library/inception/application-design/migration-plan.md
T
SluijsensandClaude Sonnet 5.5 9bc4153c70
Continuous Integration / config-sls (pull_request) Successful in 8s
Continuous Integration / config-asf (pull_request) Successful in 6s
Continuous Integration / changes (pull_request) Successful in 13s
Continuous Integration / backend-build (pull_request) Successful in 1m53s
Continuous Integration / compile-sites (pull_request) Successful in 1m54s
Continuous Integration / vulnerability-scan (pull_request) Successful in 1m56s
Continuous Integration / frontend-prepare (pull_request) Successful in 2m24s
Continuous Integration / frontend-build (pull_request) Successful in 35s
Continuous Integration / frontend-lint (pull_request) Successful in 38s
Continuous Integration / backend-test (pull_request) Successful in 4m18s
Continuous Integration / frontend-test (pull_request) Successful in 2m35s
Continuous Integration / publish-sls-test (pull_request) Skipped
Continuous Integration / publish-sls-production (pull_request) Skipped
Continuous Integration / deploy-sls-test (pull_request) Skipped
Continuous Integration / deploy-sls-production (pull_request) Skipped
Continuous Integration / publish-asf-test (pull_request) Skipped
Continuous Integration / deploy-asf-test (pull_request) Skipped
Continuous Integration / smoke-sls-test (pull_request) Skipped
Continuous Integration / smoke-asf-test (pull_request) Skipped
Mediabibliotheek en Hero: kopie-schakelaar in de deploy-workflow (test), kiezer enkel en meervoud, hint bij gedeelde uitsnede, plan uitsnede per gebruik
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-17 17:20:09 +01:00

9.9 KiB

Migration plan: HeroImages to the library's own tables (decision 13)

Status: plan, not built, not run anywhere. Decided by Bryan 2026-11-17 (via SLP Software - SLS - Main): it may be built in the next step, only after this plan with its back-up and rollback exists, rolled out to TEST only (never production), and SLS-Main is told before anything is deployed to test.

Goal

Give every photo one library record that does not depend on the Hero: MediaAsset (the photo) and MediaUse (where it is used), with the same ids, hashes, set hashes and public addresses as today. Nothing in /media/hero/<setHash>/... changes, so ASF, SLS and the golden files stay as they are.

Why expand then contract

Phase What Reversible?
0 Back-up Back-up of the database and a listing of the media directory (below). n/a
1 Expand New tables MediaAssets and MediaUses, filled by copying HeroImages and HeroSlides. The Hero keeps reading and writing its own tables; the library reads the new tables behind a setting. Yes: drop the two new tables, nothing else changed.
2 Switch Hero reads photos from the library tables (setting on). New uploads write both (dual write) so the old tables stay complete. Yes: switch the setting off; the old tables are complete.
3 Contract (a later release, after acceptance) The Hero stops writing its own image columns; they are dropped in a separate migration. Only from the back-up of that moment. Not part of the first rollout.

Files are never moved or renamed: the migration changes rows only, so the media directory is the same before and after.

Mapping

HeroImages column MediaAssets column Note
Id Id Same id, so references keep working.
OriginalSha256, OriginalKey, SourceFormat, Width, Height, Bytes same
FocusX, FocusY, Zoom DefaultFocusX, DefaultFocusY, DefaultZoom The photo's default focus.
WideFocus*, TallFocus* MediaUses rows (one per slide use, ratios wide and tall) The per-use crop; for a photo without a slide the default focus is enough.
SetHash, Variants MediaUses.SetHash (per use) Unchanged values; computed by the existing formula.
CreatedAt, CreatedByUserId same
OriginalFileName, LastUsedAt, DeletedAt same Added in the media-library step (additive columns of the Hero).
HeroSlides MediaUses
Id, ImageId Id, AssetId (module hero, place kind slide, place id = slide id)

Back-up (before phase 1, every environment)

  1. Database: a full dump of the Hero database (and of the CMS database if it is one), written to a dated file outside the deploy directory, for example mysqldump --single-transaction <database> > hero-before-media-migration-YYYYMMDD.sql. Keep it until the contract phase is accepted.
  2. Media directory: nothing is moved, but a listing with sizes and hashes is saved (find <Media__Directory> -type f -exec sha256sum {} + to a text file) so the directory can be compared afterwards. On the Raspberry Pi a copy of the directory is also a quick option (cp -a), if there is room.
  3. Row counts noted: HeroImages, HeroSlides, HeroRetiredFiles.

Verification (right after phase 1, every environment)

The migration job compares and aborts (rolls its transaction back) on any difference:

  • number of MediaAssets = number of HeroImages (all rows, also those in the trash);
  • every HeroImages.Id, OriginalSha256 and SetHash has an identical MediaAssets/MediaUses counterpart;
  • number of MediaUses of module hero = number of HeroSlides;
  • a sample of ten public addresses (/media/hero/<setHash>/<file>) is requested before and after and gives the same bytes (checked by the smoke test).

Rollback

When How
The verification fails The migration runs in one transaction and rolls back by itself; nothing is left.
After phase 1, before the switch Drop MediaUses and MediaAssets (the EF Down migration). The Hero never read them.
After the switch Switch the setting back (Media:Library:ReadFromLibrary=false); the old tables are complete because of the dual write. Then drop the new tables if wanted.
After the contract phase (later) Restore the database dump taken before that migration and the media listing check.

Roll-out rules (from Bryan)

  1. Test only. No production variables, no production deploy, no release branch.
  2. Tell SLS-Main before deploying to test (they run the deploy and the smoke test).
  3. Back-up first (above), on the test host too.
  4. Deploy with the setting off, check the verification result in the log, then switch it on, then the smoke test (hero markers, /theme-init.js, a sample of media addresses), then Bryan checks the admin.

Status (2026-11-17): phase 1 (expand) is built, behind a setting that is OFF

Built on feature/media-library-screen, not yet deployed anywhere:

  • New module SlpModularCms.Modules.Media (own MediaDbContext, tables MediaAssets and MediaUses, additive migration InitialCreate; no existing table is touched). It is referenced by the three hosts, so a deploy of this code creates the two empty tables at start-up and nothing else.
  • IMediaMigrationSource (Core) and HeroMediaMigrationSource (Hero, read-only): the hero's photos (also those in the trash) and slides with their own ids, hashes and set hashes.
  • MediaLibraryCopier: copies and verifies (counts, ids, hash, key, size, set hash) in one transaction; any difference or error rolls the whole copy back and nothing is left. It MIRRORS the source, so running it again gives the same result; rows of other modules are not touched.
  • The copy runs only when Media__Library__CopyOnStartup=true (default false) and a failure is logged, never fatal.
  • Tested in memory and against a real MariaDB (migration, copy twice, rollback on a failing copy).

Nothing reads from the new tables yet (phase 2, the switch, is not built). The library screen still reads the hero through the usage provider.

Roll-out on TEST (needs SLS-Main to be told first)

  1. Back-up (database dump and media listing) as above.
  2. Deploy the branch with Media__Library__CopyOnStartup unset: the two empty tables appear; check the log for the Media migration.
  3. Set Media__Library__CopyOnStartup=true and restart: the log says Media library copy verified: N photos and M uses from hero (or the error and the rollback).
  4. Rollback at any point: set it back to false (the tables stay, unused) or drop MediaUses and MediaAssets.
  5. Never on production.

Still to build (phases 2 and 3)

The switch (Media:Library:ReadFromLibrary, dual write of new uploads, library reading from the new tables) and the contract phase. They need Bryan's acceptance of the copy on test first.

Test host: what to check around the copy (prepared 2026-11-17; the copy is NOT switched on yet)

Deploy without the copy (decided on assumption by SLS-Main, Bryan away): the test deploy of the branch carries two additive database changes, the Hero migration AddMediaLibraryColumns (three nullable columns) and the empty tables of the Media module. No separate database dump is taken on the Raspberry Pi (SLS-Main cannot make one); the assumption is that the test data can be rebuilt with the seed scripts and that both changes are additive. Bryan checks this assumption later.

After that deploy, check in the log:

  • Applying migration '..._AddMediaLibraryColumns' and Applying migration '..._InitialCreate' (Media) appear once, no error;
  • the Hero admin still opens, a photo can be uploaded and a slide saved (smoke test), and /media/hero/<setHash>/... addresses still answer;
  • Media__Library__CopyOnStartup is NOT set: no line Media library copy verified appears.

To switch the copy on (only after Bryan's agreement):

  1. Make a database dump first if it can be done, and note the number of rows: SELECT COUNT(*) FROM HeroImages; SELECT COUNT(*) FROM HeroSlides;.
  2. Set the environment variable Media__Library__CopyOnStartup=true on the test host and restart the application.
  3. The log must say Media library copy verified: N photos and M uses from hero. with N equal to the HeroImages count and M equal to the HeroSlides count. An error line The media library copy failed and was rolled back means nothing was copied.
  4. Check: SELECT COUNT(*) FROM MediaAssets; SELECT COUNT(*) FROM MediaUses; equals the same counts; the library screen still shows the same photos (it still reads the hero).
  5. Switch it off again (remove the variable, restart) when done: the copy can be repeated at any time; it is a mirror.

Rollback: remove the variable (the tables stay, unused), or drop the tables with the EF Down migration of the Media module; the Hero's own tables are never changed by the copy.

Switching the copy on at the test hosts (no shell on the Pi needed)

The application reads Media__Library__CopyOnStartup from the host's shared/env (the file the service starts with). The deploy workflow can set exactly this one key, so nobody needs a shell on the Raspberry Pi:

  1. Dispatch the CI workflow (Continuous Integration) on the branch with deploy_sls_test and/or deploy_asf_test checked and media_library_copy = on. Each test deploy then sets Media__Library__CopyOnStartup=true in its own shared/env before restarting the service, and the run log shows the line Media library copy verified: N photos and M uses from hero (or the error line that says the copy was rolled back).
  2. Check N and M against the hero (photos and slides) and the library screen.
  3. Dispatch once more with media_library_copy = off (and the deploy flags again): the key is removed and the next start does not copy. (unchanged, the default, never touches the file.)

The input is refused for any environment other than test, and only the three values are accepted; the workflow never prints the file (it holds secrets). Production is not reachable.