diff --git a/aidlc-docs/features/prerendering/aidlc-state.md b/aidlc-docs/features/prerendering/aidlc-state.md index 905447b..a81a10d 100644 --- a/aidlc-docs/features/prerendering/aidlc-state.md +++ b/aidlc-docs/features/prerendering/aidlc-state.md @@ -5,7 +5,7 @@ - **Feature Slug**: prerendering - **Project Type**: Brownfield - **Start Date**: 2026-10-31T14:15:01Z -- **Current Stage**: CONSTRUCTION - Code Generation (unit 3: hosting-and-targets): B0 to B5 done; B6 running +- **Current Stage**: CONSTRUCTION - Code Generation (unit 3: hosting-and-targets): B0 to B6 done; CI proof of parity running - **Branch**: feature/prerendering (from master c4c2f83); commits and pushes allowed for this feature (owner, 2026-10-31); workflow dispatch without deploy allowed; deploy to test after all units (owner, 2026-11-01), after the nginx config is placed - **Worktree**: K:\Development\SlpSoftware\Projects\AndreaSlopFotografie-prerendering diff --git a/aidlc-docs/features/prerendering/construction/hosting-and-targets/code/verification-log.md b/aidlc-docs/features/prerendering/construction/hosting-and-targets/code/verification-log.md index a21a1c9..417aad8 100644 --- a/aidlc-docs/features/prerendering/construction/hosting-and-targets/code/verification-log.md +++ b/aidlc-docs/features/prerendering/construction/hosting-and-targets/code/verification-log.md @@ -13,10 +13,10 @@ Status values: OK, BLOCKED, DEVIATION, WITHDRAWN, OPEN. | V5 | A HTTP/1.0 request with `Accept-Encoding: gzip` to the generated block gets `Content-Encoding: gzip` for `.html`, `.js`, `.css`, `.svg`, `.json`, `.xml` (all sizes above 256 bytes) and not for `.woff2`/images | 6, 16 | OK: HTTP/1.1 and raw HTTP/1.0 requests get gzip for html, js, css, svg, json, xml, txt in both mime variants, none for woff2 and png; Vary: Accept-Encoding present. Not proven: nginx 1.30.6 on the Pi (CI has 1.24.0); the real check is nginx -t plus the curl matrix during the Pi rollout (H2, H3). | CI run 593 (commit 8cb9185), job lighthouse, step Browsertests, nginx/1.24.0 (Ubuntu) on the Pi runner (container, root); e2e/serverRules.e2e.ts: 134 tests, all passed, none skipped (2088 ms), 67 per mime variant (.js as text/javascript and as application/javascript); the run totals 348 tests with 341 passed and 7 failed, all 7 in images.e2e.ts and reveal.e2e.ts (timeouts, unrelated); the test prints "[serverRules] nginx -v: nginx version: nginx/1.24.0 (Ubuntu)" once per variant. Earlier cycles found and fixed three defects through this proof: workers as www-data could not read the mkdtemp fixture (harness, run 592: 403), map plain keys ignore case so /Over-Mij was not redirected (generator), and /%5Cevil.example/ produced Location /evil.example (generator, BR-HST-06). | OK | | V6 | Withdrawn for this unit (IIS needs a host). The checks move to the "Before go-live" list, U1 to U6 | WITHDRAWN (no host) | | | WITHDRAWN | | V7 | `previewServer.ts`: the new `resolveRequest` contract (redirect result, query handling, `--audit-route`); unit 1 tests for BR-SRV-07 are updated (the "interim" rules become final); `pageFiles.ts` stays the single path to file mapping | 7, 8 | OK: `resolveRequest` returns `FileResult` or `RedirectResult` (301) and takes `{ auditRoute }`; `handle()` sends the 301 with a relative `Location` plus the raw query, `Content-Length: 0`, the security headers and no `Cache-Control`, also for HEAD; `/404` and `/404.html` are 404; `/__audit/404` is 200 only with `--audit-route`; `//x` and `/%5Cevil.example/` give the 404 page. The BR-SRV-07 tests are rewritten into the final rules. `pageFiles.ts` is still the only path-to-file mapping. | `pnpm exec vitest run config/previewServer.test.ts`: 89 passed (two runs, seeds 943967193 and 1045251215); `tsc --noEmit` and eslint clean | OK | -| V8 | The 301 is followed by Chrome with the query string intact for `?pakket=` and `?shoot=` links, and the hydrated page has 1 title (extend the existing "URL die de server niet kent en de router wel" e2e) | 17 | | | open | +| V8 | The 301 is followed by Chrome with the query string intact for `?pakket=` and `?shoot=` links, and the hydrated page has 1 title (extend the existing "URL die de server niet kent en de router wel" e2e) | 17 | OK in Chrome (urls.e2e.ts, hydration.e2e.ts): /Over-Mij, /Over-Mij?pakket=gezin, /over-mij/, /over-mij/?x=1, /soorten-fotoshoots/, /index.html and /over-mij.html each give exactly one 301 and then 200 on the canonical URL with the query kept; hydrated page has 1 title, the title of its own page, no robots tag, no console message or CSP violation. Unknown URL, /404 and /404.html: 404, 1 title, 1 noindex. | Local run, pnpm build then pnpm e2e: 12 files passed, 248 tests passed, 204 skipped (nginx), seed 1174228318. The ?shoot= links are not in the list: the redirect keeps the raw query byte for byte (P-HST-05) | OK | | V9 | One Lighthouse run of the 404 page on the preview audit route, and (as comparison) `/404.html` with `ignoreStatusCode`; record best-practices for both, then keep the simpler passing option (O2) | 23 (measurement-final.md) | | | open | | V10 | lftp: dry run against a `file:` target proves that pass 1 skips all nested `*.html` and `web.config`, pass 2 uploads them, pass 3 deletes only stale files | 14 | Not proven: lftp is not installed on this PC (and may not be installed). Script upload-production.sh uses `--exclude-glob '*.html' --exclude-glob web.config` (pass 1), `put -O` of web.config plus `--include-glob '*.html'` (pass 2) and `mirror --reverse --delete` (pass 3); the generated lftp command was checked with a fake lftp, not with lftp. The test in e2e/deployScripts.e2e.ts (file: target, UPLOAD_PASS=1|2|3, upload order from the verbose output) is written and skipped locally with a visible line; in CI it fails when lftp is missing, and the lighthouse job installs lftp. Unverified until the first CI run: whether `--include-glob` alone includes only HTML and still descends into subdirectories, and the `file:///` target form. | e2e/deployScripts.e2e.ts: 5 skipped locally ("lftp niet gevonden"); deploy/upload-production.sh passes bash -n | OPEN | | V11 | `verify-deployment.sh` fails against a copy of the old nginx config (home page for `/over-mij`) and passes against the new one | 13 | OK against stubs: verify-deployment.sh exits 1 against a stub that imitates the old nginx config (index.html with 200 for every path), with the messages "/over-mij: antwoord is niet gelijk aan over-mij.html", "status 200, verwacht 404" and "/over-mij/: status 200, verwacht 301"; it exits 0 against a stub with the new rules (also with a trailing slash on the URL) and against the preview server on the fixture site. Also fails per missing security header, with --test without X-Robots-Tag or gzip; gzip missing only warns without --test. Not run against the real old or new nginx file on the Pi (nothing is contacted); the first deploy-test run shows it against the new config. | pnpm exec vitest run --config vitest.e2e.config.ts e2e/deployScripts.e2e.ts: 22 passed, 5 skipped (lftp) | OK (stubs) | | V12 | CI: the `lighthouse` job (or a new `server-rules` job) installs nginx, `pnpm e2e` includes `serverRules.e2e.ts`, and the job fails if nginx is missing | 5, 16 | OK: the lighthouse job installs nginx and lftp, Browsertests runs with CI=true, serverRules ran with nginx present. The missing-nginx failure path was proven locally (CI=true without nginx fails with "nginx ontbreekt in CI (BR-HST-24)"). | CI run 593 (commit 8cb9185), job lighthouse, step Browsertests, nginx/1.24.0 (Ubuntu) on the Pi runner (container, root); e2e/serverRules.e2e.ts: 134 tests, all passed, none skipped (2088 ms), 67 per mime variant (.js as text/javascript and as application/javascript); the run totals 348 tests with 341 passed and 7 failed, all 7 in images.e2e.ts and reveal.e2e.ts (timeouts, unrelated); the test prints "[serverRules] nginx -v: nginx version: nginx/1.24.0 (Ubuntu)" once per variant. Earlier cycles found and fixed three defects through this proof: workers as www-data could not read the mkdtemp fixture (harness, run 592: 403), map plain keys ignore case so /Over-Mij was not redirected (generator), and /%5Cevil.example/ produced Location /evil.example (generator, BR-HST-06). The install step in run 593 uses policy-rc.d to keep the package from starting a service. | OK | -| V13 | `lighthouserc.cjs`: the exception table is empty on delivery unless an owner decision exists; P-HST-11 passes; `pnpm lighthouse` still starts the preview with `--audit-route` and `Local` appears for `startServerReadyPattern` | 11, 12 | Exception table holds the 8 carried-over entries (targets unchanged by plan; decision text "tijdelijk, tot de eindmeting van prerendering" until the owner decides); effective per-URL assertions identical before/after for mobile and desktop, performance level error and warn; `config/lighthouserc.test.ts` + `src/app.test.tsx` 55 tests green; start command has `--audit-route` | Open: `Local` in the `--audit-route` preview output is Step 7 (B2) and not checked here | +| V13 | `lighthouserc.cjs`: the exception table is empty on delivery unless an owner decision exists; P-HST-11 passes; `pnpm lighthouse` still starts the preview with `--audit-route` and `Local` appears for `startServerReadyPattern` | 11, 12 | Exception table holds the 8 carried-over entries (targets unchanged by plan; decision text "tijdelijk, tot de eindmeting van prerendering" until the owner decides); effective per-URL assertions identical before/after for mobile and desktop, performance level error and warn; `config/lighthouserc.test.ts` + `src/app.test.tsx` 55 tests green; start command has `--audit-route` | OK: `pnpm exec lhci collect --url=http://localhost:4173/__audit/404 --numberOfRuns=1` started `pnpm preview --port 4173 --strictPort --audit-route`, the `Local` line was seen, Lighthouse audited the page with status 200 and no run warning (mobile: performance 0.96, accessibility 1, best-practices 1, seo 0.58 because the page is noindex; one run, indicative only). Port 4173 was free before and after | | V14 | The generated header comment fits the Pi: install path `/etc/nginx/conf.d/andreaslop-test.conf` (included from `nginx.conf:38`), no `sites-enabled`, no global `nginx.conf` change; the file works for a first install and for an update of an existing file | 3 | Header comment rewritten: install path /etc/nginx/conf.d/andreaslop-test.conf, included in http {} from nginx.conf, no sites-enabled, no global nginx.conf change, own gzip block because the Pi has none, works behind an HTTP/1.0 proxy, 444 from 00-default.conf for an unknown host (old WordPress remark removed), server_name and root adaptable, no default_server. The same file serves a first install and an update (it replaces the existing file in place). | Generated deploy/nginx/andreaslop-test.conf (lines 1-12); equals nginxTestConf() (serverConfig.test.ts sync test green) | OK | diff --git a/aidlc-docs/features/prerendering/construction/plans/hosting-and-targets-code-generation-plan.md b/aidlc-docs/features/prerendering/construction/plans/hosting-and-targets-code-generation-plan.md index 915e98a..7cba099 100644 --- a/aidlc-docs/features/prerendering/construction/plans/hosting-and-targets-code-generation-plan.md +++ b/aidlc-docs/features/prerendering/construction/plans/hosting-and-targets-code-generation-plan.md @@ -380,7 +380,7 @@ The riskiest mechanism is the nginx block (a `map`, a server-level `if` and `rew ## Part G: Server and browser tests (batch B6, after CP2) -### Step 16: Parity and full contract against real nginx (P-HST-09; V2 to V5, V12) [ ] +### Step 16: Parity and full contract against real nginx (P-HST-09; V2 to V5, V12) [x] - Executor: Sonnet. - Implements: P-HST-09; BR-HST-24; V2 to V5, V12. @@ -392,8 +392,9 @@ The riskiest mechanism is the nginx block (a `map`, a server-level `if` and `rew 4. Print `nginx -v` once in the test output. - Done when: `pnpm build && pnpm exec vitest run --config vitest.e2e.config.ts e2e/serverRules.e2e.ts` green with a local nginx, or skipped with the visible line without one. With no local nginx, this step is written and type-checked only; its first real run is the CI cycle that closes CP1/CP3. - Record: whether it ran, with which nginx. + - Written and type-checked; it did not run: no nginx on this PC, so the three new suites skip locally (the existing gate prints the skip line, and in CI a missing nginx fails the suite). First real run is the CI cycle. New in `e2e/serverRules.e2e.ts`: the parity suite (the rows of § 3.1, the fixture files, and a 100-run property over `pageVariant` and `unknownPath` with and without slash and query), a block "lege segmenten, buiten de pariteit", and a suite on the real `dist/client` (every page byte for byte, unknown URL, the directory next to `soorten-fotoshoots.html`, `/assets`). `get()` got a `raw` option because the URL parser rewrites a backslash to a slash. `nginx -v` is printed once. `vitest.e2e.config.ts` now also runs `config/testSeed.ts`, so the property test uses the global seed. Decisions: `//` and `///` (preview 301 to `/`, nginx 200) and every path with an empty segment are outside parity, pinned on both sides as safety checks; Cache-Control is compared only on HTML responses and not on a 301 (preview sends none, nginx sends no-cache), and the preview sets none on non-HTML files; `/%2e%2e/` and NUL are outside parity (nginx answers 400). -### Step 17: Browser tests for the new URL behaviour and adjusted tests (V8) [ ] +### Step 17: Browser tests for the new URL behaviour and adjusted tests (V8) [x] - Executor: Sonnet. - Implements: BR-HST-02, 05, 08, 22; FR-PR-14, 15, 16; V8. @@ -404,13 +405,15 @@ The riskiest mechanism is the nginx block (a `map`, a server-level `if` and `rew 3. Any other e2e expectation that relied on `/404.html` returning 200 is adjusted to 404 (same strictness) and listed in the Record. - Done when: `pnpm exec tsc --noEmit`, `pnpm lint` clean; the files are written and each runs green on a current build in Step 18. - Record: the list of e2e files and tests that changed. + - Created `e2e/urls.e2e.ts` (7 redirect rows, 3 404 rows, 2 audit-route tests). `e2e/support/server.ts`: `startServer({ auditRoute })`. `e2e/hydration.e2e.ts`: `expectedStatus` no longer gives 200 for `/404.html`; the late-chunk test on `/404.html` passes `expectNotFound: true`; the observation test `URL die de server niet kent en de router wel (BR-404-07)` is replaced by `URL die alleen de router kent wordt doorverwezen (BR-HST-08)` (`/Over-Mij` and `/over-mij/`: 301 to `/over-mij`, hydrated, no messages). `e2e/rawHtml.e2e.ts`: the test `heeft één titel, één noindex en geen beschrijving of canonical` fetches `/404.html` with status 404. `e2e/displayHtml.e2e.ts`: `fetchDocument` expects 404 for `/404.html` (it expected 200 for all). No other e2e file or `src/*.test.tsx` depended on 200 for `/404.html` (grep of `404` in `e2e` and `src`; `deployScripts.e2e.ts` already follows the new contract). -### Step 18: Run all server and browser tests on a current build (CP3) [ ] +### Step 18: Run all server and browser tests on a current build (CP3) [x] - Executor: Haiku. - Do: from `web/`, in one command line `pnpm build && pnpm e2e`; no other job running. Wait for completion (rule 11). Fix nothing; report failures with output to Sonnet's steps (16, 17) or STOP. - Done when: **CP3**: all e2e files green (the nginx suite either ran green or printed the skip line; if CP1 is still open the report says so). Record counts in `code-summary.md` later. - Record: file and test counts, the nginx line. + - Run on a current build (`pnpm build`, then `pnpm e2e`, seed 1174228318): 12 files passed, 248 tests passed, 204 skipped, 0 failed. The 204 skipped are the nginx suites (no nginx here). `pnpm test`: 132 files, 1390 tests passed, coverage thresholds met. CP3: nginx suites skipped locally; CP1 is closed by CI run 593, but the new parity and real-build suites have not yet run on a real nginx. --- diff --git a/web/e2e/displayHtml.e2e.ts b/web/e2e/displayHtml.e2e.ts index 72e4e39..e0f58d5 100644 --- a/web/e2e/displayHtml.e2e.ts +++ b/web/e2e/displayHtml.e2e.ts @@ -18,7 +18,8 @@ afterAll(() => server.close()); async function fetchDocument(path: string) { const response = await fetch(server.origin + path); - expect(response.status).toBe(200); + // /404.html is een echte 404 (BR-HST-05) + expect(response.status).toBe(path === '/404.html' ? 404 : 200); const html = await response.text(); return { html, diff --git a/web/e2e/hydration.e2e.ts b/web/e2e/hydration.e2e.ts index 7250cbe..590f397 100644 --- a/web/e2e/hydration.e2e.ts +++ b/web/e2e/hydration.e2e.ts @@ -25,9 +25,9 @@ const notFoundPaths = [ '/soorten-fotoshoots/onbekend', ]; -// Alleen een document dat de server bewust met 404 beantwoordt (BR-SRV-05) -const expectedStatus = (path: string) => - path === '/404.html' || pages.includes(path) ? 200 : 404; +// Alleen een document dat de server bewust met 404 beantwoordt (BR-SRV-05); +// /404 en /404.html zijn echte 404's (BR-HST-05) +const expectedStatus = (path: string) => (pages.includes(path) ? 200 : 404); describe.each(widths)('hydratatie op %i px', (width) => { it.each([...pages, ...notFoundPaths])( @@ -74,6 +74,7 @@ describe('de 404-pagina, gemaakt voor /404 en getoond op een andere URL', () => server.origin + '/404.html', { width, + expectNotFound: true, prepare: async (page) => { await page.route( /\/assets\/_-[^/]+\.js$/, @@ -282,47 +283,26 @@ describe('contactformulier op /contact (V8)', () => { }); }); -// BR-404-07: een URL die de server niet kent maar de router wel. Dit is een -// waarneming, geen controle die de suite laat falen. De -// preview-server draait op Windows en Linux anders (hoofdletters in -// bestandsnamen), dus de Linux-situatie wordt nagespeeld: 404.html met 404. -describe('URL die de server niet kent en de router wel (BR-404-07)', () => { - it.each(['/Over-Mij', '/over-mij/'])('observeert %s', async (path) => { - const real = await fetch(server.origin + path, { redirect: 'manual' }); - const observe = async ( - served: string, - prepare?: (page: Page) => Promise, - ) => { - const { page, ...opened } = await openPage(server.origin + path, { - prepare, +// BR-404-07, BR-HST-08: een URL die de router wel kent maar de server niet als +// pagina bedient kan geen verschil tussen HTML en eerste render meer geven, +// want de server verwijst hem eerst door naar het canonieke pad. Het volledige +// gedrag in Chrome staat in urls.e2e.ts. +describe('URL die alleen de router kent wordt doorverwezen (BR-HST-08)', () => { + it.each(['/Over-Mij', '/over-mij/'])( + '%s gaat met één 301 naar /over-mij', + async (path) => { + const real = await fetch(server.origin + path, { + redirect: 'manual', }); - await page.waitForTimeout(1500); - const state = await page.evaluate(() => ({ - url: location.pathname, - h1: document.querySelector('h1')?.textContent, - titles: [...document.querySelectorAll('title')].map( - (t) => t.textContent, - ), - })); - console.info( - `BR-404-07 ${path} (${served}): ${JSON.stringify({ ...state, status: opened.status, messages: opened.messages })}`, - ); - }; - console.info( - `BR-404-07 ${path}: preview-server geeft ${real.status} ${real.headers.get('content-type')}`, - ); - await observe('preview-server van deze machine'); - await observe( - '404.html met 404, zoals een hoofdlettergevoelige host', - async (page) => { - await page.route(server.origin + path, async (route) => { - const notFound = await route.fetch({ - url: server.origin + '/404.html', - }); - await route.fulfill({ response: notFound, status: 404 }); - }); - }, - ); - expect(real.status).toBeGreaterThan(0); - }); + expect(real.status).toBe(301); + expect(real.headers.get('location')).toBe('/over-mij'); + + const { page, ...opened } = await openPage(server.origin + path); + await waitForHydration(page); + expect(new URL(page.url()).pathname).toBe('/over-mij'); + expect(opened.status).toBe(200); + expect(opened.messages).toEqual([]); + expect(opened.violations).toEqual([]); + }, + ); }); diff --git a/web/e2e/rawHtml.e2e.ts b/web/e2e/rawHtml.e2e.ts index 9cdbf09..b84438f 100644 --- a/web/e2e/rawHtml.e2e.ts +++ b/web/e2e/rawHtml.e2e.ts @@ -120,8 +120,8 @@ describe.each(paths)('rauwe HTML van %s', (path) => { describe('rauwe HTML van 404.html', () => { it('heeft één titel, één noindex en geen beschrijving of canonical', async () => { - // 404.html als bestand: status 200 (BR-SRV-04) - const { html, document } = await fetchDocument('/404.html'); + // 404.html als URL is een echte 404 (BR-HST-05) + const { html, document } = await fetchDocument('/404.html', 404); const titles = document.querySelectorAll('title'); expect(titles).toHaveLength(1); expect(titles[0].textContent).toBe( diff --git a/web/e2e/serverRules.e2e.ts b/web/e2e/serverRules.e2e.ts index 2f9ce7d..4291b23 100644 --- a/web/e2e/serverRules.e2e.ts +++ b/web/e2e/serverRules.e2e.ts @@ -1,8 +1,19 @@ -import { request } from 'node:http'; -import { connect } from 'node:net'; +import fc from 'fast-check'; +import { existsSync, readFileSync, statSync } from 'node:fs'; +import { request, type IncomingMessage, type Server } from 'node:http'; +import { connect, type AddressInfo } from 'node:net'; +import { join } from 'node:path'; import { gunzipSync } from 'node:zlib'; -import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import { afterAll, beforeAll, describe, expect, inject, it } from 'vitest'; +import { pagePaths } from '../config/canonicalUrls.ts'; +import { NOT_FOUND_FILE, pageFile } from '../config/pageFiles.ts'; +import { startPreviewServer } from '../config/previewServer.ts'; import { securityHeaders } from '../config/securityHeaders.ts'; +import { + pageVariant, + queryString, + unknownPath, +} from '../src/test/urlArbitraries.ts'; import { createFixtureSite, type FixtureSite } from './support/fixtureSite.ts'; import { findNginx, @@ -10,11 +21,15 @@ import { type MimeVariant, type RunningNginx, } from './support/nginx.ts'; +import { clientDir } from './support/server.ts'; // BR-HST-24: de nginx-regels worden bewezen door een echte nginx, niet door de // tekst te lezen. De site is een kleine fixture (geen build nodig); dezelfde // regels voor de previewserver staan in previewServer.test.ts. +// Seed van deze run, gekozen en gelogd in config/testSeed.ts (PBT-08) +fc.configureGlobal({ seed: inject('fcSeed') }); + const nginxBin = findNginx(); const inCi = Boolean(process.env.CI); @@ -43,34 +58,44 @@ interface Reply { body: Buffer; } -// node:http en niet fetch: fetch voegt dubbele headers samen en pakt gzip uit +// node:http en niet fetch: fetch voegt dubbele headers samen en pakt gzip uit. +// Met raw gaat het pad ongewijzigd over de lijn; anders maakt de URL-parser er +// zijn eigen vorm van (een backslash wordt een slash). function get( origin: string, path: string, - options: { method?: string; headers?: Record } = {}, + options: { + method?: string; + headers?: Record; + raw?: boolean; + } = {}, ): Promise { return new Promise((resolve, reject) => { - const req = request( - origin + path, - { method: options.method ?? 'GET', headers: options.headers }, - (res) => { - const chunks: Buffer[] = []; - res.on('data', (chunk: Buffer) => chunks.push(chunk)); - res.on('end', () => - resolve({ - status: res.statusCode ?? 0, - headers: Object.fromEntries( - Object.entries(res.headers).map(([k, v]) => [ - k, - String(v), - ]), - ), - rawHeaders: res.rawHeaders, - body: Buffer.concat(chunks), - }), - ); - }, - ); + const init = { + method: options.method ?? 'GET', + headers: options.headers, + }; + const onReply = (res: IncomingMessage) => { + const chunks: Buffer[] = []; + res.on('data', (chunk: Buffer) => chunks.push(chunk)); + res.on('end', () => + resolve({ + status: res.statusCode ?? 0, + headers: Object.fromEntries( + Object.entries(res.headers).map(([k, v]) => [ + k, + String(v), + ]), + ), + rawHeaders: res.rawHeaders, + body: Buffer.concat(chunks), + }), + ); + }; + const { hostname, port } = new URL(origin); + const req = options.raw + ? request({ hostname, port, path, ...init }, onReply) + : request(origin + path, init, onReply); req.on('error', reject); req.end(); }); @@ -123,6 +148,14 @@ function getHttp10(origin: string, path: string, headers: string[]) { }); } +// De versie van nginx één keer in de uitvoer, hoeveel instanties er ook starten +let versionAnnounced = false; +function announce(nginx: RunningNginx) { + if (versionAnnounced) return; + versionAnnounced = true; + console.log(`[serverRules] nginx -v: ${nginx.version}`); +} + const mimeVariants: MimeVariant[] = ['new', 'old']; const rulesSuite = nginxBin ? describe : describe.skip; @@ -139,9 +172,7 @@ rulesSuite.each(mimeVariants)( siteDir: site.dir, mimeVariant: variant, }); - console.log( - `[serverRules] nginx -v: ${nginx.version} (${nginx.origin})`, - ); + announce(nginx); }); afterAll(async () => { await nginx?.stop(); @@ -374,3 +405,298 @@ rulesSuite.each(mimeVariants)( }); }, ); + +// P-HST-09 (oracle): dezelfde fixturesite achter de echte nginx en achter de +// previewserver. Voor elk pad moeten ze gelijk antwoorden op status, Location +// (zonder schema en host), het bestand dat geserveerd is en gzip. Bewust +// anders, en dus niet vergeleken: +// - 405-tekst en de Server-header: nginx heeft eigen teksten; +// - het MIME-type van .js: hangt van de nginx-versie af (beide tabellen staan +// hierboven in de suite per mime-variant); +// - /robots.txt: nginx vervangt hem door Disallow, de previewserver serveert +// het bestand (de fixture heeft er een); +// - X-Robots-Tag: alleen nginx (testsite), de previewserver stuurt hem nooit; +// - Cache-Control op een 301: nginx zet no-cache via add_header op +// server-niveau, de previewserver stuurt er geen (BR-HST-12); op bestanden +// buiten HTML zet de previewserver er geen; +// - paden met een leeg segment (//), zie het blok onderaan de suite; +// - /%2e%2e/ en een NUL in het pad: nginx weigert die met 400, de +// previewserver geeft 404. +const parityRules = nginxBin ? describe : describe.skip; +parityRules('pariteit: previewserver tegen echte nginx (P-HST-09)', () => { + let site: FixtureSite; + let nginx: RunningNginx; + let preview: Server; + let previewOrigin: string; + + beforeAll(async () => { + site = await createFixtureSite(); + nginx = await startNginx({ siteDir: site.dir }); + announce(nginx); + preview = await startPreviewServer(site.dir, 0); + previewOrigin = `http://127.0.0.1:${(preview.address() as AddressInfo).port}`; + }); + afterAll(async () => { + preview?.closeAllConnections(); + await new Promise((resolve) => + preview ? preview.close(() => resolve()) : resolve(), + ); + await nginx?.stop(); + await site?.remove(); + }); + + const acceptGzip = { 'Accept-Encoding': 'gzip' }; + const search = (query: string) => (query ? `?${query}` : ''); + const decoded = (reply: Reply) => + reply.headers['content-encoding'] === 'gzip' + ? gunzipSync(reply.body) + : reply.body; + const servedFile = (reply: Reply) => + /fixture:[\w./-]+/.exec(decoded(reply).toString('latin1'))?.[0]; + const locationOf = (reply: Reply) => + reply.headers.location?.replace(/^https?:\/\/[^/]+/, ''); + + // Wat aan de headers van één antwoord niet klopt + function headerFindings( + server: 'nginx' | 'preview', + reply: Reply, + checkCache: boolean, + ) { + const found: string[] = []; + for (const [name, value] of Object.entries( + securityHeaders('production'), + )) { + const count = countHeader(reply, name); + const actual = reply.headers[name.toLowerCase()]; + if (count !== 1 || actual !== value) { + found.push(`${server}: ${name} ${count}x, waarde ${actual}`); + } + } + const robots = countHeader(reply, 'X-Robots-Tag'); + if (robots !== (server === 'nginx' ? 1 : 0)) { + found.push(`${server}: X-Robots-Tag ${robots}x`); + } + if (checkCache) { + const count = countHeader(reply, 'Cache-Control'); + const expected = + server === 'preview' && reply.status === 301 ? 0 : 1; + if (count !== expected) { + found.push( + `${server}: Cache-Control ${count}x, verwacht ${expected}x`, + ); + } + } + return found; + } + + // Alle verschillen voor één pad, leesbaar met het pad erbij + async function differences(path: string, checkCache = true) { + const [fromNginx, fromPreview] = await Promise.all([ + get(nginx.origin, path, { headers: acceptGzip, raw: true }), + get(previewOrigin, path, { headers: acceptGzip, raw: true }), + ]); + const found: string[] = []; + const same = (what: string, a: unknown, b: unknown) => { + if (a !== b) found.push(`${what}: nginx ${a}, preview ${b}`); + }; + same('status', fromNginx.status, fromPreview.status); + same('Location', locationOf(fromNginx), locationOf(fromPreview)); + same('bestand', servedFile(fromNginx), servedFile(fromPreview)); + same( + 'Content-Encoding', + fromNginx.headers['content-encoding'], + fromPreview.headers['content-encoding'], + ); + if ( + fromNginx.status !== 301 && + !decoded(fromNginx).equals(decoded(fromPreview)) + ) { + found.push('inhoud: de bytes verschillen'); + } + if (checkCache && fromNginx.status !== 301) { + same( + 'Cache-Control', + fromNginx.headers['cache-control'], + fromPreview.headers['cache-control'], + ); + } + found.push( + ...headerFindings('nginx', fromNginx, checkCache), + ...headerFindings('preview', fromPreview, checkCache), + ); + return found.map((finding) => `${path} ${finding}`); + } + + // De tabel van het URL-contract, rij voor rij + it.each([ + '/', + '/over-mij', + '/soorten-fotoshoots', + '/soorten-fotoshoots/newborn', + '/over-mij?x=1', + '/?x=1', + '/over-mij/', + '/over-mij///', + '/soorten-fotoshoots/', + '/soorten-fotoshoots/?x=1', + '/over-mij.html', + '/soorten-fotoshoots/newborn.html', + '/index', + '/index.html', + '/index/', + '/Over-Mij', + '/OVER-MIJ.HTML', + '/Over-Mij/', + '/Over-Mij?x=1&y=%20z', + '/404', + '/404.html', + '/onbekend', + '/Onbekend', + '/x.html', + '/onbekend/', + '/onbekend/?a=b', + '/assets', + '/assets/', + '/assets/missing.js', + '/%5Cevil.example', + '/%5Cevil.example/', + ])('%s: nginx en previewserver antwoorden gelijk', async (path) => { + expect(await differences(path)).toEqual([]); + }); + + // Bestanden: zelfde status, bytes en gzip-keuze (Cache-Control: zie boven) + it.each([ + '/sitemap.xml', + '/favicon.png', + '/assets/index-AbC123xYz.js', + '/assets/style-Qw9.css', + '/assets/icon-Zz1.svg', + '/assets/data.json', + '/assets/readme.txt', + '/assets/font-Ab.woff2', + ])('%s: gelijke status, bytes en gzip', async (path) => { + expect(await differences(path, false)).toEqual([]); + }); + + // Bekende pagina's in elke vorm en onbekende paden met en zonder slash, + // met en zonder query + const parityPath = fc.oneof( + fc + .tuple(pageVariant, queryString) + .map(([path, query]) => path + search(query)), + fc + .tuple(unknownPath, fc.boolean(), queryString) + .map( + ([path, slash, query]) => + path + (slash ? '/' : '') + search(query), + ), + ); + it('nginx en previewserver antwoorden gelijk op gegenereerde paden (oracle)', async () => { + await fc.assert( + fc.asyncProperty(parityPath, async (path) => { + expect(await differences(path)).toEqual([]); + }), + { numRuns: 100 }, + ); + }); + + // Buiten de pariteit: een leeg segment. nginx voegt de slashes samen + // (merge_slashes) en bedient of verwijst het samengevoegde pad; de + // previewserver kent een leeg segment niet (BR-SRV-07) en geeft 404, op + // slashes alleen na (// en /// gaan naar /). Een verschil in uitkomst kan + // geen verkeerde eerste render geven, want browser en router maken zulke + // paden niet, en geen doorverwijzing begint met // (BR-HST-06). + describe('lege segmenten, buiten de pariteit', () => { + it.each([ + ['//', 'index.html'], + ['///', 'index.html'], + ['//over-mij', 'over-mij.html'], + ])( + 'nginx voegt de slashes van %s samen en bedient %s', + async (path, file) => { + const reply = await get(nginx.origin, path, { raw: true }); + expect(reply.status).toBe(200); + expect(reply.body.toString()).toContain( + site.markers.get(file) ?? file, + ); + }, + ); + + it.each([ + ['/a//b/', '/a/b'], + ['//evil.example/', '/evil.example'], + ])('nginx verwijst %s naar %s', async (path, target) => { + const reply = await get(nginx.origin, path, { raw: true }); + expect(reply.status).toBe(301); + expect(reply.headers.location).toBe(target); + }); + + it.each(['//', '///', '//over-mij', '/a//b/', '//evil.example/'])( + 'de previewserver geeft voor %s 404 of een doorverwijzing naar /', + async (path) => { + const reply = await get(previewOrigin, path, { raw: true }); + if (reply.status === 301) { + expect(reply.headers.location).toBe('/'); + } else { + expect(reply.status).toBe(404); + } + }, + ); + }); +}); + +// De echte build achter de echte nginx: elke pagina, de 404 en de map naast +// het paginabestand. Heeft een actuele dist nodig (pnpm build). +const buildSuite = nginxBin ? describe : describe.skip; +buildSuite('nginx op de echte build (dist/client)', () => { + let nginx: RunningNginx; + const bytesOf = (file: string) => readFileSync(join(clientDir, file)); + + beforeAll(async () => { + nginx = await startNginx({ siteDir: clientDir }); + announce(nginx); + }); + afterAll(async () => { + await nginx?.stop(); + }); + + it.each([...pagePaths])( + '%s geeft 200 met de bytes van het paginabestand', + async (path) => { + const reply = await get(nginx.origin, path, { raw: true }); + expect(reply.status).toBe(200); + expect(reply.body.equals(bytesOf(pageFile(path)))).toBe(true); + }, + ); + + it('een onbekende URL geeft 404 met de bytes van 404.html', async () => { + const reply = await get(nginx.origin, '/bestaat-niet', { raw: true }); + expect(reply.status).toBe(404); + expect(reply.body.equals(bytesOf(NOT_FOUND_FILE))).toBe(true); + }); + + it('de map soorten-fotoshoots/ staat naast soorten-fotoshoots.html zonder dat /soorten-fotoshoots omleidt', async () => { + expect( + statSync(join(clientDir, 'soorten-fotoshoots')).isDirectory(), + ).toBe(true); + expect(existsSync(join(clientDir, 'soorten-fotoshoots.html'))).toBe( + true, + ); + const page = await get(nginx.origin, '/soorten-fotoshoots', { + raw: true, + }); + expect(page.status).toBe(200); + expect(page.body.equals(bytesOf('soorten-fotoshoots.html'))).toBe(true); + const dir = await get(nginx.origin, '/soorten-fotoshoots/', { + raw: true, + }); + expect(dir.status).toBe(301); + expect(dir.headers.location).toBe('/soorten-fotoshoots'); + }); + + it('toont geen maplijst voor /assets', async () => { + const reply = await get(nginx.origin, '/assets', { raw: true }); + expect(reply.status).toBe(404); + expect(reply.body.equals(bytesOf(NOT_FOUND_FILE))).toBe(true); + }); +}); diff --git a/web/e2e/support/server.ts b/web/e2e/support/server.ts index 3c305fe..06543c5 100644 --- a/web/e2e/support/server.ts +++ b/web/e2e/support/server.ts @@ -8,9 +8,10 @@ import { startPreviewServer } from '../../config/previewServer.ts'; export const clientDir = resolve('dist/client'); // Dezelfde server als `pnpm preview`, met de productie-CSP en echte 404's, -// op een poort die het systeem kiest zodat tests elkaar niet in de weg zitten -export async function startServer() { - const server = await startPreviewServer(clientDir, 0); +// op een poort die het systeem kiest zodat tests elkaar niet in de weg zitten. +// auditRoute zet de opt-in route /__audit/404 aan, zoals Lighthouse hem start. +export async function startServer(options: { auditRoute?: boolean } = {}) { + const server = await startPreviewServer(clientDir, 0, options); const { port } = server.address() as AddressInfo; return { origin: `http://127.0.0.1:${port}`, diff --git a/web/e2e/urls.e2e.ts b/web/e2e/urls.e2e.ts new file mode 100644 index 0000000..422c30a --- /dev/null +++ b/web/e2e/urls.e2e.ts @@ -0,0 +1,139 @@ +import type { Page } from 'playwright-core'; +import { afterAll, beforeAll, describe, expect, it } from 'vitest'; +import type { RoutePath } from '../src/navigation.ts'; +import { pageHead, siteName } from '../src/seo.ts'; +import { openPage, waitForHydration } from './support/browser.ts'; +import { startServer } from './support/server.ts'; + +// BR-HST-02, 05, 08, 22; V8: Chrome volgt de 301 van de previewserver, komt op +// het canonieke adres uit met de query intact en hydrateert zonder melding. +// Een onbekende URL en /404.html tonen de 404-pagina met status 404. +let server: Awaited>; +beforeAll(async () => { + server = await startServer(); +}); +afterAll(() => server.close()); + +// De documentverzoeken van de hoofdpagina in volgorde, ook de doorverwijzing: +// page.goto geeft alleen het eindantwoord +function recordNavigations() { + const log: { status: number; url: string }[] = []; + const prepare = async (page: Page) => { + page.on('response', (response) => { + if ( + response.request().resourceType() === 'document' && + response.frame() === page.mainFrame() + ) { + log.push({ status: response.status(), url: response.url() }); + } + }); + }; + return { log, prepare }; +} + +const countTags = (page: Page) => + page.evaluate(() => ({ + titles: document.querySelectorAll('title').length, + robots: [...document.querySelectorAll('meta[name="robots"]')].map( + (meta) => meta.getAttribute('content'), + ), + })); + +describe('doorverwijzingen naar het canonieke adres (BR-HST-03, 04, 08)', () => { + it.each([ + ['/Over-Mij', '/over-mij'], + ['/Over-Mij?pakket=gezin', '/over-mij?pakket=gezin'], + ['/over-mij/', '/over-mij'], + ['/over-mij/?x=1', '/over-mij?x=1'], + ['/soorten-fotoshoots/', '/soorten-fotoshoots'], + ['/index.html', '/'], + ['/over-mij.html', '/over-mij'], + ])('%s gaat met één 301 naar %s', async (path, canonical) => { + const { log, prepare } = recordNavigations(); + const { page, ...opened } = await openPage(server.origin + path, { + prepare, + }); + // Eén 301, dan de pagina zelf, met de query van het verzoek erbij + expect(log).toEqual([ + { status: 301, url: server.origin + path }, + { status: 200, url: server.origin + canonical }, + ]); + expect(opened.status).toBe(200); + + await waitForHydration(page); + const canonicalPath = new URL(server.origin + canonical).pathname; + expect(new URL(page.url()).pathname).toBe(canonicalPath); + // De titel is die van de pagina zelf, niet van de 404-pagina + await expect(page.title()).resolves.toBe( + pageHead(canonicalPath as RoutePath).meta[0].title, + ); + const tags = await countTags(page); + expect(tags.titles).toBe(1); + expect(tags.robots).toEqual( + canonicalPath === '/privacyverklaring' ? ['noindex'] : [], + ); + expect(opened.messages).toEqual([]); + expect(opened.violations).toEqual([]); + }); +}); + +describe('de 404-pagina met status 404 (BR-HST-02, 05)', () => { + it.each(['/onbekend', '/404.html', '/404'])( + '%s toont de 404-pagina en blijft op dat adres', + async (path) => { + const { log, prepare } = recordNavigations(); + const { page, ...opened } = await openPage(server.origin + path, { + expectNotFound: true, + prepare, + }); + // Geen doorverwijzing: het document zelf is de 404 + expect(log).toEqual([{ status: 404, url: server.origin + path }]); + expect(opened.status).toBe(404); + + await waitForHydration(page); + expect(new URL(page.url()).pathname).toBe(path); + await expect(page.title()).resolves.toBe( + `Pagina niet gevonden | ${siteName}`, + ); + const tags = await countTags(page); + expect(tags.titles).toBe(1); + expect(tags.robots).toEqual(['noindex']); + expect(opened.messages).toEqual([]); + expect(opened.violations).toEqual([]); + }, + ); +}); + +describe('de auditroute /__audit/404 (BR-HST-22)', () => { + it('is zonder de optie een gewone onbekende URL met status 404', async () => { + const { status } = await openPage(server.origin + '/__audit/404', { + expectNotFound: true, + }); + expect(status).toBe(404); + }); + + it('geeft met de optie de 404-pagina met status 200 en hydrateert zonder melding', async () => { + const audit = await startServer({ auditRoute: true }); + try { + const { page, ...opened } = await openPage( + audit.origin + '/__audit/404', + ); + expect(opened.status).toBe(200); + await waitForHydration(page); + await expect(page.title()).resolves.toBe( + `Pagina niet gevonden | ${siteName}`, + ); + const tags = await countTags(page); + expect(tags.titles).toBe(1); + expect(tags.robots).toEqual(['noindex']); + expect(opened.messages).toEqual([]); + expect(opened.violations).toEqual([]); + + // /404.html blijft ook met de optie een echte 404 + const real = await fetch(audit.origin + '/404.html'); + expect(real.status).toBe(404); + } finally { + await audit.close(); + } + }); +}); diff --git a/web/vitest.e2e.config.ts b/web/vitest.e2e.config.ts index 788db48..a5b1d29 100644 --- a/web/vitest.e2e.config.ts +++ b/web/vitest.e2e.config.ts @@ -17,7 +17,8 @@ export default defineConfig({ // moet opgelost worden en niet langer mogen duren. testTimeout: process.env.CI ? 180_000 : 60_000, hookTimeout: 60_000, - // Bouwt niets; controleert alleen dat dist/client er is - globalSetup: ['./e2e/support/globalSetup.ts'], + // De seed voor de property-tests (zoals vite.config.ts); bouwt niets, + // controleert alleen dat dist/client er is + globalSetup: ['./config/testSeed.ts', './e2e/support/globalSetup.ts'], }, });