Files

12 KiB

Phase 2: the Hero works through the library, and a crop per use

Status (2026-11-17): steps B (crop per slide), A (write-through) and C (read switch) are built on this branch, switches off by default; phase 3 is not built. Branch feature/media-library-phase2 (no PR until #22 is merged). Test only; every switch is OFF by default; SLS-Main is told before every test deploy; phase 3 (dropping the old tables) only after phase 2 is verified on test and never on production without a separate go from Bryan.

What Bryan wants

  1. A photo on two slides can have its own crop per slide (per-use-crop.md).
  2. The Hero reads and writes through the library instead of owning the photo data (migration-plan.md, phase 2).
  3. The public addresses (/media/hero/<setHash>/<ratio>-<width>.<ext>), ids and hashes do not change for anything that exists today; the golden files stay as they are.

The idea in one paragraph

Until phase 3 the Hero's own tables (HeroImages, HeroSlides) stay the source of truth and the safety net. Phase 2 adds, in three independent steps, (A) a write-through so every change the Hero makes to a photo or a slide lands in the library tables too, (B) a crop per slide with its own set, and (C) a read switch so the Hero and the library screen can read the photos from the library tables. Each step has its own switch, can be rolled back by switching it off, and none of them removes or rewrites data of the old tables. Phase 3 is then only: flip the authority and drop the old columns/tables.

flowchart LR
    subgraph Hero module
      HS[HeroService / HeroImageService]
      HT[(HeroImages, HeroSlides)]
    end
    subgraph Media module
      W[IMediaLibraryWriter]
      MT[(MediaAssets, MediaUses)]
    end
    HS -->|always| HT
    HS -->|"A: DualWrite on"| W --> MT
    MT -->|"C: ReadFromLibrary on"| HS
    classDef hero fill:#9ae6b4,stroke:#2f855a,color:#000;
    classDef media fill:#63b3ed,stroke:#2b6cb0,color:#000;
    class HS,HT hero;
    class W,MT media;

Text alternative: the Hero always writes its own tables; with step A on it also writes the library tables; with step C on it reads photos back from the library tables.

Step A: write-through (Media:Library:DualWrite, default off)

  • New Core interface IMediaLibraryWriter (Upsert asset, Upsert/replace the uses of a module, Trash, Restore, Purge). Implemented in the Media module on its own MediaDbContext; the Hero module depends only on the Core interface (as it does for IMediaUsageProvider). With the switch off or no implementation registered it is a no-op.
  • The Hero calls it after each successful change: upload (new or restored photo), focus change, trash, restore, purge, saving the hero (the slides become the photo's uses, with position, active flag and set hash), the renderer finishing a set (new set hash).
  • Everything a Hero change writes is also exactly what the verified copy writes: the copier is reused as the backfill and the check (CopyOnStartup stays; running it after dual write was on must report the same counts). A write to the library that fails is logged and never fails the Hero request (the Hero tables are the truth); the next copy repairs the difference.
  • Tests: in-memory and MariaDB; after each Hero operation the library tables equal what the copier would produce.

Step B: a crop per slide (always on once built; additive and nullable)

Data (additive migration of the Hero, all nullable, no existing column changes):

Table New columns Meaning
HeroSlides FocusX/Y/Zoom, WideFocus*, TallFocus* (all nullable) The slide's own crop. Null = no override: the slide uses its photo's crop and set exactly as today.
HeroSlides SetHash (nullable), Variants (JSON, nullable) The slide's own set, only when it has an override and the set is complete.

Behaviour:

  • Set per (photo, crop). The set hash uses the existing formula MediaPaths.SetHash(originalSha256, crops), so a slide with an override whose crop equals the photo's crop gets the same hash and shares the photo's files; two slides with the same crop share one set. A slide without override keeps the photo's set and hash: no existing address changes.
  • Renderer. RenderNextPendingAsync also looks at slides with an override whose SetHash differs from the hash of their crop (or have no variants), and cuts that set the same way it cuts a photo's set (same progress, retries and "old set stays live until the new one is complete"). A set that nothing references any more goes to HeroRetiredFiles like today; a set shared by photo and slide is only retired when neither uses it (HeroRepository already has the shared-files check, extended to slides).
  • Public mapping. HeroMapping picks, per slide, the slide's own set when it has a complete one, else the photo's set. Slides without override map byte-for-byte as before (golden files unchanged).
  • Admin. The focus dialog is opened per slide and edits that slide's crop (PUT hero/slides/{slideId}/focus, applies at once, "Wordt verwerkt…" progress like now). "Reset to the photo's crop" clears the override. The "crop is shared" hint disappears; the photo's own crop stays as the default for new slides. The existing PUT hero/images/{id}/focus keeps working (sets the photo's default).
  • Library. Step A mirrors the slide's crop and set into MediaUses.Crops and MediaUses.SetHash.
  • Tests: two slides, one photo, different crops give two sets and two addresses; same crop gives one set; no override gives today's hash; the old set is live until the new one is complete; trash/purge with shared sets; golden files unchanged.

Step C: read switch (Media:Library:ReadFromLibrary, default off)

  • The library screen (IMediaUsageProvider data) and the Hero's photo lookups (GetImage, the admin DTOs, the trash/purge checks) read the photo rows from MediaAssets and MediaUses when the switch is on; the Hero's own tables are still written (step A) so switching off is instant and loses nothing.
  • Compare mode first: with ReadFromLibrary=compare the Hero still answers from its own tables but logs a warning for every photo or slide where the library row differs. Bryan/SLS-Main look at the log on test before on is used.
  • Tests: the same Hero tests run twice (switch off and on) and give the same answers.

Back-up and rollback plan (test hosts)

Before the first deploy of each step (A, B, C) on a test host:

  1. Restore point of the test sites (made by Bryan, 2026-11-17 21:37, confirmed by Pi-Main): /mnt/storage1/backups/test-herstelpunt/2026-11-17-213757/ (readable by root only) with db-SlpSoftwareTest.sql.gz, db-AndreaSlopFotografieTest.sql.gz, media.tar.gz (both media directories) and SHA256SUMS. Made with sudo bash /home/claude/server-ops/drafts/2026-11-17-test-herstelpunt/test-herstelpunt.sh. Restoring is step 22b in drafts/2026-11-17-test-herstelpunt/README.md on the Pi, done by Bryan (not tested yet). Take a new one before the next step if a lot changed.
  2. Row counts noted: HeroImages, HeroSlides, HeroRetiredFiles, MediaAssets, MediaUses.
  3. Sample of public addresses (ten /media/hero/<setHash>/... URLs and the hero markers) requested before and after; same bytes.
Step What can go wrong Rollback
A write-through Library rows differ from the Hero's Switch DualWrite off; run the copy once (mirror) to repair, or ignore: the Hero never read them.
B crop per slide A wrong crop or set for a slide Clear the override of the slide in the admin ("reset"), or set the slide's new columns to null (UPDATE HeroSlides SET FocusX=NULL ...); the photo's own set is untouched. The migration is additive; the EF Down drops only the new columns.
C read switch The library answers differ Switch ReadFromLibrary off (instant; the Hero tables are complete because of A).
After all steps Anything Restore the dump of the step (if one was taken). The media directory is never changed destructively: new sets are new directories, old sets stay until the retention time passes.

Switches are set through the existing deploy workflow input path (the same way as media_library_copy; test only) or shared/env, never on production.

Order of work and what is reported

  1. Now: this design and the rollback plan pushed for Bryan to read (this file).
  2. B first (it is what Bryan sees; independent of A): backend (migration, entity, renderer, mapping, endpoint, tests) then the frontend (per-slide focus dialog), reported with a test-deploy request, no switch needed because it is additive and null = unchanged.
  3. A, then the copy check on test.
  4. C in compare mode, then on.
  5. Phase 3 only afterwards and only on test, with its own plan and notice.

Assumptions made here (also in assumptions-and-open-questions.md)

  • Test data is no longer assumed to be rebuildable: there is a recorded restore point (above). The Hero tables stay authoritative until phase 3; "reads and writes through the library" is reached in steps A and C behind switches, not by replacing the tables in one go (a one-go replacement cannot be rolled back by a switch).
  • The crop per slide lives in the slide (nullable) in the Hero until phase 3 and is mirrored to MediaUses.Crops; moving the authority to MediaUses is part of phase 3.

Step A as built (2026-11-17)

  • IMediaLibrarySync (Core; default NoMediaLibrarySync) is called by the Hero after every saved change: upload (also the same photo again), focus of a photo, focus and reset of a slide, trash, restore, purge, delete, saving the hero, the renderer finishing a set (photo or slide) and the clean-up. The Media module replaces it with MediaLibrarySync, which runs the verified copier for the hero module only, one at a time, only when Media__Library__DualWrite=true, and never throws (a failure is logged; the next change or the start-up copy repairs it). The Hero's own tables are never touched by it.
  • The slide as it is shown is what lands in the library: for a slide with its own crop the use gets the slide's set hash and crops (MediaUses.SetHash, MediaUses.Crops).
  • Switch through the deploy workflow like the copy: dispatch input media_library_dual_write (unchanged, on, off; test only).
  • What Bryan can check on test with the switch on: make any change in the Hero or the library (upload, crop a slide, save, trash, restore); then in the database SELECT COUNT(*) FROM MediaAssets / MediaUses equals HeroImages / HeroSlides, and for a slide with its own crop MediaUses.SetHash equals HeroSlides.SetHash. The log has no line "The write-through of hero to the media library failed". With the switch off nothing changes at all.

Step C as built (2026-11-17)

  • Switch Media__Library__ReadFromLibrary = off (default) | compare | on; deploy workflow input media_library_read (unchanged, off, compare, on; test only).
  • What is read from the library tables: what the library screen shows of the hero (photos, trash, where each photo is used): identity, size, name, dates and the uses with position and on/off. The thumbnail address still comes from the hero (the variants are not in the library tables until phase 3), and the hero's own admin and public endpoints keep reading their own tables: they need the variants. Writing (trash, restore, purge, crops) always goes to the hero's own tables, and with the write-through on, on to the library.
  • compare: the answer stays the hero's own, but the library tables are read too; every difference is a warning in the log (Media library read check (hero): ...) when the library screen is opened, and GET media-library/read-check (administrators) lists them for all modules in words (matches, differences), whatever the switch says, so it can be used before on.
  • on: the library tables answer; when they cannot be read the hero's own tables answer and the error is logged. on needs DualWrite=on, otherwise the library screen shows stale data.
  • What Bryan can check on test: (1) with DualWrite=on and ReadFromLibrary=compare, open the library screen and call /api/v1/media-library/read-check (or look at the log): matches: true for the hero means the library tables are a faithful mirror; (2) then ReadFromLibrary=on: the library screen looks exactly the same (same photos, trash, uses, thumbnails); change something (upload, trash, restore, crop a slide) and the screen follows; (3) switch back to off at any moment, nothing is lost.