Adds the step that moves inline scripts to files
TanStack Start writes inline scripts into each page, which the strict CSP blocks. This step moves them to hashed files and checks the output: every page is present, no executable inline script remains and the e-mail address appears nowhere. It is not part of the build yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+18
-4
@@ -27,8 +27,8 @@ Notes:
|
||||
|---|---|---|---|---|---|
|
||||
| V1 | Custom `src/client.tsx` hydrates with Start; `startSentry()` before hydration | 4 | Start picks `src/client.tsx` by name (default entry `client` in `srcDirectory`); it replaces the default entry, whose body is `startTransition(() => hydrateRoot(document, <StrictMode><StartClient /></StrictMode>))`. `startSentry()` runs at module level before that call. | Build output `dist/client/assets/index-*.js` contains `hydrateRoot` and the Sentry loader (`sentryClient-*.js` dynamic import). Chrome (playwright-core, system Chrome, static server, no CSP yet): `/`, `/over-mij`, `/contact`, `/contact?pakket=gezin` hydrate with no console error or warning. Not proven here: that Sentry initialises (only on the test and production hostnames, so a no-op on 127.0.0.1); unit tests of `monitoring/` cover that. | OK |
|
||||
| V2 | `RouterProvider` in jsdom renders `shellComponent` | 5 | Yes, Router calls the root `shellComponent` (`Match.js`: `ShellComponent = route.isRoot && route.options.shellComponent`). React 19 treats `html`/`head`/`body` as singletons, so no `<html>` lands in the RTL container (container has 0 html/head/body), but it sets `lang="nl"` on `document.documentElement` and wraps `ChakraProvider` a second time. Variant used: `renderApp` replaces the root route's `shellComponent` by a test shell (`HeadContent` + children) on its router instance; production code untouched. | Probe test (removed) printed `shellComponent: function`, `html=0 head=0 body=0`, `document.documentElement lang=nl`. All 888 tests pass with the test shell (with the real shell, 45 of 46 in `app.test.tsx` passed; the one failure was the `/$` register test). | OK |
|
||||
| V3 | Inline scripts per real page | 4 (first look), 13 (final) | Per page (12 files) the inline executable scripts are: (a) router scroll restoration, 613 chars, no attributes, same body everywhere; (b) Start bootstrap `data-tsr-stream-part=""`, 916 to 1349 chars, differs per page; (c) boundary `document.currentScript.remove();/*$tsr-stream-boundary*/`, 56 chars, same everywhere. `/contact` has two more (lazy form Suspense): `<script id="_R_">requestAnimationFrame(function(){$RT=performance.now()});</script>` (57 chars) and the React completion runtime `$RB=[];$RV=function(a){...` (845 chars). No `$RC` seen on other pages. External: `<script type="module" async src="/assets/index-*.js">` only (not touched by the runner). JSON-LD blocks (`application/ld+json`) are data and stay. BR-SCR-05: no inline body contains `<!--`, no classic inline script has `async`/`defer`, no inline `module` script exists. | `scratchpad/scripts.mjs` over `dist/client` (first look, to be repeated in Step 13 with the real runner). | OK (first look) |
|
||||
| V4 | Minimum moved scripts | 13 | open (first look: every page has at least 3 inline executable scripts, so a minimum of one is far below what Start emits) | | open |
|
||||
| V3 | Inline scripts per real page | 4 (first look), 13 (final) | Final list from the real runner on a scratch copy of the Step 4 output (12 HTML files): every page has 3 moved scripts, `/contact` has 5, 16 distinct files in total. (1) router scroll restoration, 613 chars, no attributes, same file on all pages; (2) Start bootstrap `data-tsr-stream-part=""`, 916 to 1349 chars, one file per page; (3) boundary `document.currentScript.remove();/*$tsr-stream-boundary*/`, 56 chars, same file everywhere; only `/contact`: (4) `<script id="_R_">requestAnimationFrame(...)` 57 chars and (5) React completion runtime `$RB=[];$RV=...;$RC("B:0","S:0")` 845 chars. No `<!--`, no `async`/`defer` on an inline script, no inline module, so no UNSUPPORTED_SCRIPT or UNTERMINATED_SCRIPT. Positions (see Script positions below): scroll restoration is the last child of `<main>` (before the footer); bootstrap, the external module entry and boundary sit after the footer at the end of body; on `/contact` `_R_` is followed by the streamed form (`<div hidden id="S:0">`) and the completion runtime is the last element before `</body>`. | `node config/externalizeInlineScripts.ts <scratch copy>` printed only EMAIL_IN_OUTPUT (12 files) and wrote nothing; a throwaway script in the scratch folder (`b2-scratch/tools/run.mjs`) ran `planExternalization` with `address: null`: no violations, 16 script files plus 12 changed HTML files planned, applied, and a second plan had 0 writes and 0 violations. | OK |
|
||||
| V4 | Minimum moved scripts | 13 | Keep the minimum of one (BR-SCR-17 unchanged). Every real page has at least 3 moved scripts (5 on `/contact`), so the minimum of one is far below what Start emits and cannot fail on today's output; its job is to catch a step that silently moves nothing, which one reference also catches. Demanding more than one adds nothing on current output and would break on a Start version that bundles scripts differently. A stricter check (for example requiring the bootstrap with `data-tsr-stream-part`) would be a design change; not made. | Counts per page in V3. | OK (recommendation: keep one) |
|
||||
| V5 | `index.html` written, crawling off | 4 | `/` is written as `index.html` with `autoSubfolderIndex: false`; `/x` as `x.html`, `/soorten-fotoshoots/x` as `soorten-fotoshoots/x.html`. Option names (from the `start-plugin-core` schema): `prerender.crawlLinks: false` and `prerender.autoStaticPathsDiscovery: false` (the latter stops Start adding static routes by itself), page list from `Object.keys(sitePages)` plus `{ path: '/404', prerender: { outputPath: '/404.html' } }`. | Prerender log: "Prerendered 12 pages". `find dist/client -name '*.html'` gives exactly 404, index, over-mij, portfolio, privacyverklaring, contact, soorten-fotoshoots and five `soorten-fotoshoots/<shoot>.html` (12 files, no others). | OK |
|
||||
| V6 | Code splitting; router plugin unused | 4 (30 later) | Start's plugin includes `tanstackRouterGenerator` and `tanStackRouterCodeSplitter` itself (client and server environments), so route code splitting is on without an option and `@tanstack/router-plugin` is not needed as a direct dependency. Removed with `pnpm remove @tanstack/router-plugin`; only Start's own copy (1.168.48) remains. | Client build has one chunk per route (`over-mij-*.js`, `contact-*.js`, `portfolio-*.js`, `newborn-*.js`, ..., `_-*.js` for the catch-all), plus `ContactForm`, `Lightbox`, `TestimonialCarousel` chunks. Step 30 re-checks. | OK |
|
||||
| V7 | Sitemap in the client environment only | 4 | `this.environment.name === 'client'` identifies the client build in `generateBundle` (Vite 8.3.1, Start environments `client` and `ssr`). | `dist/client/sitemap.xml` exists once (10 `<loc>`, no `/privacyverklaring`, no `/404`); `find dist -name sitemap.xml` gives only that file; `public/` has no `sitemap.xml`. | OK |
|
||||
@@ -37,11 +37,11 @@ Notes:
|
||||
| V10 | Placeholder sizes, CLS below 0.01 | 17, 34 | open | | open |
|
||||
| V11 | One title, one noindex meta | 4 (normal pages), 27 (`404.html`) | Normal pages in the prerendered HTML: one `<title>`, one `description`, one `canonical`, no `data-fallback`, a robots meta only on `/privacyverklaring`. `404.html`: one `<title>` ("Pagina niet gevonden \| Andrea Slop Fotografie") and one `<meta name="robots" content="noindex">`, no description or canonical. React writes `<meta charSet="utf-8">` with a capital S (valid, attribute names are case-insensitive). | `scripts.mjs` counts per file. After hydration in Chrome: `/`, `/over-mij`, `/contact` still 1 title. Caution for Step 27: `/soorten-fotoshoots/onbekend` showed 2 titles and 2 robots metas after React error #418 (the two extra 404 buttons render before hydration; Step 16 fixes that); `/onbekend` is clean. | OK (normal pages); `404.html` source OK, final check in 27 |
|
||||
| V12 | 404 hydration with real shell | 27 | open (first look, before Step 16: `/onbekend` hydrates clean; `/soorten-fotoshoots/onbekend` gives #418, which Step 16 `hydrated && isShootSectionPath` should remove) | | open |
|
||||
| V13 | Timestamps and build paths | 4, 13 | No build timestamp (`YYYY-MM-DDT..`) in any text file of `dist/client`, no `lastmod` in the sitemap. No repository path (`SlpSoftware`, `Projects/Andrea...`, `K:\`, worktree name) in any text file, no `.map` file. | `grep -rI` over `dist/client`. Re-check on the moved scripts in Step 13. | OK (first look) |
|
||||
| V13 | Timestamps and build paths | 4, 13 | Repeated on the final output of the runner (HTML plus the 16 moved script files plus all JS/CSS/XML/TXT/JSON): no build timestamp (`YYYY-MM-DDTHH:MM`) in any text file, no `.map` file, no repository root path (forward slashes, backslashes or doubled backslashes) in any text file. File names are content hashes, so they cannot carry a timestamp. | Scan in the scratch run (`repoPathFiles: 0`, `timestampFiles: 0`, `maps: 0`); the runner's own BUILD_PATH_LEAK check reported nothing. | OK |
|
||||
| V14 | Focus on first load, tidy navigation | 27 | open | | open |
|
||||
| V15 | Emotion/Chakra flicker | 4 (styles in HTML), 27 (screenshots) | Styles are in the HTML: `<style data-emotion` x69 in `/over-mij`, x120 on `/`, plus the bundled stylesheet `<link rel="stylesheet" href="/assets/index-*.css">` (fonts and Chakra base). After hydration in Chrome `/over-mij` has 70 emotion style tags (one added by the client); no console warning. Visual check in Step 27. | `scripts.mjs`; Chrome probe. | OK (styles present); screenshots in 27 |
|
||||
| V16 | `Intl` price text | 27 | open | | open |
|
||||
| V17 | No `on*=` or `javascript:` | 13, 27 | None in the 12 HTML files (` on[a-z]+=` and `javascript:` both 0 matches). Step 13 repeats it on the runner output; Step 27 under the CSP. | `scripts.mjs` | OK (first look) |
|
||||
| V17 | No `on*=` or `javascript:` | 13, 27 | Step 13 on the runner output: 0 HTML files with an `on*=` attribute and 0 HTML files with `javascript:`. The only `javascript:` in `dist/client` is two strings in `assets/index-*.js`: React's own guard text (`javascript:throw new Error('React has blocked a javascript: URL...`), which is not markup and not a URL in a page. Step 27 repeats it under the CSP. | Scratch run counts (`onAttrFiles: 0`); `grep -l javascript:` over the HTML and the bundle. | OK (Step 13); CSP check in 27 |
|
||||
| V18 | CI artifact layout | 28 | open | | open |
|
||||
|
||||
## Other findings in Steps 3 to 5
|
||||
@@ -49,3 +49,17 @@ Notes:
|
||||
- Email address: `info@andreaslop.nl` is in all 12 HTML files (footer link, contact details, gate notice) and nowhere in `dist/client/assets/*.js`, `robots.txt` or `sitemap.xml`. Expected until Step 17.
|
||||
- The route tree gets a `/$` route, so `RoutePath = keyof FileRoutesByTo` would force a `/$` entry in `sitePages`. `src/navigation.ts` now uses `Exclude<keyof FileRoutesByTo, '/$'>`; the test "bevat precies de routes van de route tree" ignores `/$`. Not in the design text; needed for `tsc` (see proof-report).
|
||||
- Start's `client.base` default is `/_build`, but it is not used in this build: assets stay under `/assets/` (needed for `/assets/inline-<hash>.js` of the runner).
|
||||
|
||||
## Script positions per page (Step 13)
|
||||
|
||||
From the scratch run; positions are offsets in the rewritten HTML. A classic script that was inline now loads from `/assets/inline-<hash>.js` and blocks the parser at that point until fetched and run; that is a finding for the measurement (BR-MEA-03), not a design change.
|
||||
|
||||
| Script | Where it sits | Pages |
|
||||
|---|---|---|
|
||||
| Scroll restoration (613 chars) | Last child of `<main>`, after the last section and before `</main>` and the footer; about 78 to 92% into the document, so it blocks the parsing of the footer and everything after | all 12 |
|
||||
| Start bootstrap (`data-tsr-stream-part`) | After the footer, end of body, before the external `<script type="module" async src="/assets/index-*.js">` | all 12 |
|
||||
| Boundary (56 chars) | Directly after the module entry script, end of body | all 12 |
|
||||
| `_R_` requestAnimationFrame (57 chars) | After the boundary script, directly before the streamed `<div hidden id="S:0">` with the contact form | `/contact` |
|
||||
| React completion runtime (845 chars, ends in `$RC("B:0","S:0")`) | Last element of the body, after the hidden form segment (about 14 KB after the bootstrap), just before `</body>` | `/contact` |
|
||||
|
||||
On `/contact` the lazy `ContactForm` is a streamed Suspense boundary: its place in `<main>` is `<!--$?--><template id="B:0">` and the form HTML sits in a hidden `S:0` div at the end of the body. It becomes visible only when the completion script runs. That script was inline before the move and is now an external classic script, so the form appears one request later than in the raw Start output, and not at all without JavaScript. Relevant for V8 and for the measurement (CLS and LCP of `/contact`).
|
||||
|
||||
@@ -0,0 +1,729 @@
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import {
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
rmSync,
|
||||
writeFileSync,
|
||||
} from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import fc from 'fast-check';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { EmailAddress } from '../src/content/types';
|
||||
import { siteUrl } from '../src/seo';
|
||||
import sitePages, { pageList } from '../src/sitePages';
|
||||
import { sitemapXml } from '../src/sitemap';
|
||||
import { validEmail } from '../src/test/arbitraries';
|
||||
import {
|
||||
htmlWithModel,
|
||||
outputScenario,
|
||||
type OutputScenario,
|
||||
} from '../src/test/htmlArbitraries';
|
||||
import {
|
||||
containsAddress,
|
||||
externalizeOutput,
|
||||
formatViolation,
|
||||
planExternalization,
|
||||
type PlanContext,
|
||||
type ScriptStep,
|
||||
} from './externalizeInlineScripts';
|
||||
import { pageFile } from './pageFiles';
|
||||
|
||||
const SITEMAP = '<urlset>sitemap</urlset>\n';
|
||||
const ROOT_PATHS = ['/work/repo', 'C:\\work\\repo'];
|
||||
const ADDRESS = 'info@example.nl';
|
||||
|
||||
const context = (override: Partial<PlanContext> = {}): PlanContext => ({
|
||||
expected: {
|
||||
pages: new Map([
|
||||
['/', 'index.html'],
|
||||
['/over-mij', 'over-mij.html'],
|
||||
['/soorten-fotoshoots/newborn', 'soorten-fotoshoots/newborn.html'],
|
||||
]),
|
||||
notFound: '404.html',
|
||||
},
|
||||
address: ADDRESS,
|
||||
sitemap: SITEMAP,
|
||||
rootPaths: ROOT_PATHS,
|
||||
...override,
|
||||
});
|
||||
|
||||
const page = (script = 'boot()') =>
|
||||
`<html><body><p>tekst</p><script>${script}</script></body></html>`;
|
||||
|
||||
const validFiles = (): Map<string, string> =>
|
||||
new Map([
|
||||
['index.html', page('boot("index")')],
|
||||
['over-mij.html', page('boot("over-mij")')],
|
||||
['soorten-fotoshoots/newborn.html', page('boot("newborn")')],
|
||||
['404.html', page('boot("404")')],
|
||||
['sitemap.xml', SITEMAP],
|
||||
['assets/index-abc.js', 'console.log(1);'],
|
||||
['assets/index-abc.css', 'a{color:red}'],
|
||||
]);
|
||||
|
||||
const found = (plan: { violations: { file: string; code: string }[] }) =>
|
||||
plan.violations.map((v) => `${v.file}: ${v.code}`).sort();
|
||||
|
||||
describe('planExternalization: pad zonder overtredingen', () => {
|
||||
it('verplaatst de scripts van elke pagina, ook van 404.html', () => {
|
||||
const plan = planExternalization(validFiles(), context());
|
||||
expect(plan.violations).toEqual([]);
|
||||
const scripts = [...plan.writes.keys()].filter((f) =>
|
||||
f.startsWith('assets/inline-'),
|
||||
);
|
||||
expect(scripts).toHaveLength(4);
|
||||
for (const file of [
|
||||
'index.html',
|
||||
'over-mij.html',
|
||||
'soorten-fotoshoots/newborn.html',
|
||||
'404.html',
|
||||
]) {
|
||||
expect(plan.writes.get(file)).toMatch(
|
||||
/src="\/assets\/inline-[0-9a-f]{12}\.js"/,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('is een no-op bij een tweede keer (al verwerkte pagina telt mee)', () => {
|
||||
const files = validFiles();
|
||||
const first = planExternalization(files, context());
|
||||
for (const [file, content] of first.writes) files.set(file, content);
|
||||
const second = planExternalization(files, context());
|
||||
expect(second.writes.size).toBe(0);
|
||||
expect(second.violations).toEqual([]);
|
||||
});
|
||||
|
||||
it('schrijft een scriptbestand dat al met dezelfde inhoud bestaat niet opnieuw', () => {
|
||||
const files = validFiles();
|
||||
const first = planExternalization(files, context());
|
||||
const name = [...first.writes.keys()].find((f) =>
|
||||
f.startsWith('assets/inline-'),
|
||||
)!;
|
||||
files.set(name, first.writes.get(name)!);
|
||||
const plan = planExternalization(files, context());
|
||||
expect(plan.writes.has(name)).toBe(false);
|
||||
expect(plan.violations).toEqual([]);
|
||||
});
|
||||
|
||||
it('schrijft een gedeeld script één keer', () => {
|
||||
const files = validFiles();
|
||||
for (const file of ['index.html', 'over-mij.html']) {
|
||||
files.set(file, page('gedeeld()'));
|
||||
}
|
||||
const plan = planExternalization(files, context());
|
||||
const shared = [...plan.writes.values()].filter(
|
||||
(c) => c === 'gedeeld()',
|
||||
);
|
||||
expect(shared).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('planExternalization: overtredingen', () => {
|
||||
it('NO_HTML: geen enkel html-bestand, zonder de pagina’s afzonderlijk te melden', () => {
|
||||
const files = new Map([['sitemap.xml', SITEMAP]]);
|
||||
expect(found(planExternalization(files, context()))).toEqual([
|
||||
'.: NO_HTML',
|
||||
]);
|
||||
});
|
||||
|
||||
it('PAGE_MISSING: meldt alle ontbrekende pagina’s', () => {
|
||||
const files = validFiles();
|
||||
files.delete('over-mij.html');
|
||||
files.delete('soorten-fotoshoots/newborn.html');
|
||||
const plan = planExternalization(files, context());
|
||||
expect(found(plan)).toEqual([
|
||||
'over-mij.html: PAGE_MISSING',
|
||||
'soorten-fotoshoots/newborn.html: PAGE_MISSING',
|
||||
]);
|
||||
expect(plan.violations[0].detail).toContain('/over-mij');
|
||||
});
|
||||
|
||||
it('NOT_FOUND_PAGE_MISSING: 404.html ontbreekt', () => {
|
||||
const files = validFiles();
|
||||
files.delete('404.html');
|
||||
expect(found(planExternalization(files, context()))).toEqual([
|
||||
'404.html: NOT_FOUND_PAGE_MISSING',
|
||||
]);
|
||||
});
|
||||
|
||||
it('UNEXPECTED_HTML: een html-bestand buiten het register', () => {
|
||||
const files = validFiles();
|
||||
files.set('soorten-fotoshoots/index.html', page());
|
||||
files.set('extra.html', page());
|
||||
expect(found(planExternalization(files, context()))).toEqual([
|
||||
'extra.html: UNEXPECTED_HTML',
|
||||
'soorten-fotoshoots/index.html: UNEXPECTED_HTML',
|
||||
]);
|
||||
});
|
||||
|
||||
it('NO_SCRIPTS_MOVED: een pagina zonder uitvoerbaar of verplaatst script', () => {
|
||||
const files = validFiles();
|
||||
files.set(
|
||||
'index.html',
|
||||
'<p>geen script</p><script type="application/ld+json">{}</script><script src="/assets/index-abc.js"></script>',
|
||||
);
|
||||
expect(found(planExternalization(files, context()))).toEqual([
|
||||
'index.html: NO_SCRIPTS_MOVED',
|
||||
]);
|
||||
});
|
||||
|
||||
it('EXECUTABLE_REMAINS: de verdediging als het herschrijven iets overlaat', () => {
|
||||
const scripts: ScriptStep = {
|
||||
externalize: (html) => ({ html, moved: [] }),
|
||||
find: () => ['geheim()'],
|
||||
};
|
||||
const plan = planExternalization(validFiles(), context(), scripts);
|
||||
expect(found(plan)).toContain('index.html: EXECUTABLE_REMAINS');
|
||||
expect(plan.violations.map(formatViolation).join('\n')).not.toContain(
|
||||
'geheim',
|
||||
);
|
||||
});
|
||||
|
||||
it('UNSUPPORTED_SCRIPT en UNTERMINATED_SCRIPT: meldt de code en gooit niet', () => {
|
||||
const files = validFiles();
|
||||
files.set('index.html', '<script async>geheim()</script>');
|
||||
files.set('over-mij.html', '<script>geheim()');
|
||||
const plan = planExternalization(files, context());
|
||||
expect(found(plan)).toEqual([
|
||||
'index.html: UNSUPPORTED_SCRIPT',
|
||||
'over-mij.html: UNTERMINATED_SCRIPT',
|
||||
]);
|
||||
const text = plan.violations.map(formatViolation).join('\n');
|
||||
expect(text).toContain('index.html: UNSUPPORTED_SCRIPT: ');
|
||||
expect(text).toContain('regel 1');
|
||||
expect(text).not.toContain('geheim');
|
||||
});
|
||||
|
||||
it('geeft een andere fout dan een ScriptError door', () => {
|
||||
const scripts: ScriptStep = {
|
||||
externalize: () => {
|
||||
throw new TypeError('kapot');
|
||||
},
|
||||
find: () => [],
|
||||
};
|
||||
expect(() =>
|
||||
planExternalization(validFiles(), context(), scripts),
|
||||
).toThrow('kapot');
|
||||
});
|
||||
|
||||
it('COLLISION: een scriptbestand met dezelfde naam en andere inhoud', () => {
|
||||
const files = validFiles();
|
||||
const first = planExternalization(files, context());
|
||||
const name = [...first.writes.keys()].find((f) =>
|
||||
f.startsWith('assets/inline-'),
|
||||
)!;
|
||||
files.set(name, 'iets anders');
|
||||
const plan = planExternalization(files, context());
|
||||
expect(found(plan)).toEqual([`${name}: COLLISION`]);
|
||||
});
|
||||
|
||||
it('COLLISION: twee verschillende scripts met dezelfde bestandsnaam in één run', () => {
|
||||
const scripts: ScriptStep = {
|
||||
externalize: (html) => ({
|
||||
html: `<script src="/assets/inline-000000000000.js"></script>`,
|
||||
moved: [{ fileName: 'inline-000000000000.js', body: html }],
|
||||
}),
|
||||
find: () => [],
|
||||
};
|
||||
const plan = planExternalization(validFiles(), context(), scripts);
|
||||
expect(found(plan)).toContain(
|
||||
'assets/inline-000000000000.js: COLLISION',
|
||||
);
|
||||
});
|
||||
|
||||
it('SITEMAP_MISSING en SITEMAP_DIFFERS', () => {
|
||||
const missing = validFiles();
|
||||
missing.delete('sitemap.xml');
|
||||
expect(found(planExternalization(missing, context()))).toEqual([
|
||||
'sitemap.xml: SITEMAP_MISSING',
|
||||
]);
|
||||
const differs = validFiles();
|
||||
differs.set('sitemap.xml', SITEMAP + ' ');
|
||||
expect(found(planExternalization(differs, context()))).toEqual([
|
||||
'sitemap.xml: SITEMAP_DIFFERS',
|
||||
]);
|
||||
});
|
||||
|
||||
it('SOURCE_MAP_PRESENT: een .map-bestand', () => {
|
||||
const files = validFiles();
|
||||
files.set('assets/index-abc.js.map', '{}');
|
||||
expect(found(planExternalization(files, context()))).toEqual([
|
||||
'assets/index-abc.js.map: SOURCE_MAP_PRESENT',
|
||||
]);
|
||||
});
|
||||
|
||||
it.each(ROOT_PATHS)('BUILD_PATH_LEAK: %s in een tekstbestand', (root) => {
|
||||
const files = validFiles();
|
||||
files.set('assets/index-abc.js', `var p = "${root}/src/x.ts";`);
|
||||
expect(found(planExternalization(files, context()))).toEqual([
|
||||
'assets/index-abc.js: BUILD_PATH_LEAK',
|
||||
]);
|
||||
});
|
||||
|
||||
it('EMAIL_IN_OUTPUT: in html, js, css en in de inhoud van een verplaatst script', () => {
|
||||
const files = validFiles();
|
||||
files.set('assets/index-abc.js', `var a = "${ADDRESS}";`);
|
||||
files.set('assets/index-abc.css', `a::after{content:"${ADDRESS}"}`);
|
||||
files.set('over-mij.html', `<p>${ADDRESS}</p>${page('boot("o")')}`);
|
||||
files.set('index.html', page(`mail("${ADDRESS}")`));
|
||||
const plan = planExternalization(files, context());
|
||||
const inline = [...plan.writes.entries()].find(([, c]) =>
|
||||
c.includes(ADDRESS),
|
||||
)![0];
|
||||
expect(found(plan)).toEqual(
|
||||
[
|
||||
'assets/index-abc.js: EMAIL_IN_OUTPUT',
|
||||
'assets/index-abc.css: EMAIL_IN_OUTPUT',
|
||||
'over-mij.html: EMAIL_IN_OUTPUT',
|
||||
`${inline}: EMAIL_IN_OUTPUT`,
|
||||
].sort(),
|
||||
);
|
||||
});
|
||||
|
||||
it('slaat de adrescontrole over zonder adres (ongeldige delen)', () => {
|
||||
const files = validFiles();
|
||||
files.set('assets/index-abc.js', `var a = "${ADDRESS}";`);
|
||||
const plan = planExternalization(files, context({ address: null }));
|
||||
expect(plan.violations).toEqual([]);
|
||||
});
|
||||
|
||||
it('meldt alle overtredingen tegelijk, in een vast formaat zonder adres', () => {
|
||||
const files = validFiles();
|
||||
files.delete('404.html');
|
||||
files.set('assets/index-abc.js', `"${ADDRESS}"`);
|
||||
files.set('x.html', page());
|
||||
const lines = planExternalization(files, context()).violations.map(
|
||||
formatViolation,
|
||||
);
|
||||
expect(lines).toEqual(
|
||||
expect.arrayContaining([
|
||||
'404.html: NOT_FOUND_PAGE_MISSING: de 404-pagina ontbreekt',
|
||||
'x.html: UNEXPECTED_HTML: html-bestand dat niet bij het register hoort',
|
||||
'assets/index-abc.js: EMAIL_IN_OUTPUT: e-mailadres in de uitvoer',
|
||||
]),
|
||||
);
|
||||
expect(lines.join('\n')).not.toContain(ADDRESS);
|
||||
});
|
||||
});
|
||||
|
||||
describe('containsAddress', () => {
|
||||
const address = 'info@andreaslop.nl';
|
||||
|
||||
it.each([
|
||||
['platte tekst', 'info@andreaslop.nl'],
|
||||
['hoofdletters', 'INFO@AndreaSlop.NL'],
|
||||
['URL-codering', 'mailto:info%40andreaslop.nl'],
|
||||
['JavaScript-escape', 'info\\u0040andreaslop.nl'],
|
||||
['JavaScript-escape met accolades', 'info\\u{40}andreaslop.nl'],
|
||||
['hexadecimale escape', 'info\\x40andreaslop.nl'],
|
||||
['decimale entiteit', 'info@andreaslop.nl'],
|
||||
['hexadecimale entiteit', 'info@andreaslop.nl'],
|
||||
['hexadecimale entiteit in hoofdletters', 'info@andreaslop.nl'],
|
||||
['benoemde entiteit', 'info@andreaslop.nl'],
|
||||
['midden in tekst', 'Mail <a href="x">info@andreaslop.nl</a>!'],
|
||||
])('vindt het adres als %s', (_, text) => {
|
||||
expect(containsAddress(text, address)).toBe(true);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['alleen de gebruiker', 'info'],
|
||||
['alleen het domein', 'andreaslop.nl'],
|
||||
['met een spatie ertussen', 'info @ andreaslop.nl'],
|
||||
['een ander domein', 'info@andreaslop.com'],
|
||||
['een andere gebruiker', 'post@andreaslop.nl'],
|
||||
['een onbekende codering', 'infoAandreaslop.nl'],
|
||||
['lege tekst', ''],
|
||||
])('vindt het adres niet bij %s', (_, text) => {
|
||||
expect(containsAddress(text, address)).toBe(false);
|
||||
});
|
||||
|
||||
it('vindt niets bij een adres zonder @', () => {
|
||||
expect(containsAddress('infoandreaslop.nl', 'infoandreaslop.nl')).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('containsAddress: eigenschappen', () => {
|
||||
// Onafhankelijke lijst van BR-MAIL-10
|
||||
const encodings = [
|
||||
'@',
|
||||
'%40',
|
||||
'\\u0040',
|
||||
'\\u{40}',
|
||||
'\\x40',
|
||||
'@',
|
||||
'@',
|
||||
'@',
|
||||
];
|
||||
// Geen @, %, & of \ : een schrijfwijze kan er dus nooit in ontstaan
|
||||
const safeText = fc.stringMatching(/^[a-z0-9 .<>/="'-]{0,30}$/);
|
||||
const randomCase = (text: string, flips: boolean[]) =>
|
||||
[...text]
|
||||
.map((char, i) =>
|
||||
flips[i % flips.length] ? char.toUpperCase() : char,
|
||||
)
|
||||
.join('');
|
||||
|
||||
it('vindt het adres in elke schrijfwijze op elke plaats, in elke hoofdletterstand (oracle)', () => {
|
||||
fc.assert(
|
||||
fc.property(
|
||||
validEmail,
|
||||
fc.constantFrom(...encodings),
|
||||
safeText,
|
||||
safeText,
|
||||
fc.array(fc.boolean(), { minLength: 1, maxLength: 8 }),
|
||||
(email, encoding, before, after, flips) => {
|
||||
const written = randomCase(
|
||||
`${email.user}${encoding}${email.domain}`,
|
||||
flips,
|
||||
);
|
||||
expect(
|
||||
containsAddress(
|
||||
`${before}${written}${after}`,
|
||||
`${email.user}@${email.domain}`,
|
||||
),
|
||||
).toBe(true);
|
||||
},
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
it('markeert tekst zonder het adres nooit (oracle)', () => {
|
||||
fc.assert(
|
||||
fc.property(
|
||||
validEmail,
|
||||
safeText,
|
||||
fc.constantFrom(...encodings),
|
||||
(email, text, encoding) => {
|
||||
const address = `${email.user}@${email.domain}`;
|
||||
expect(containsAddress(text, address)).toBe(false);
|
||||
// Een deel van het adres is niet genoeg
|
||||
expect(
|
||||
containsAddress(
|
||||
`${text}${email.user}${encoding}`,
|
||||
address,
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
containsAddress(
|
||||
`${encoding}${email.domain}${text}`,
|
||||
address,
|
||||
),
|
||||
).toBe(false);
|
||||
},
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// De drie pagina's van `context()` plus 404.html, in de volgorde van het scenario
|
||||
const SCENARIO_FILES = [
|
||||
'index.html',
|
||||
'over-mij.html',
|
||||
'soorten-fotoshoots/newborn.html',
|
||||
'404.html',
|
||||
];
|
||||
const EXTRA_REFERENCE =
|
||||
'<script src="/assets/inline-0123456789ab.js"></script>';
|
||||
|
||||
function filesOf(scenario: OutputScenario, address: string) {
|
||||
const files = new Map<string, string>();
|
||||
scenario.pages.forEach((p, i) => {
|
||||
if (!p.present) return;
|
||||
files.set(
|
||||
SCENARIO_FILES[i],
|
||||
p.content.html +
|
||||
(p.reference ? EXTRA_REFERENCE : '') +
|
||||
(p.leakAddress ? `<p>${address}</p>` : '') +
|
||||
(p.bad === 'unsupported'
|
||||
? '<script async>bad()</script>'
|
||||
: '') +
|
||||
(p.bad === 'unterminated' ? '<script>bad()' : ''),
|
||||
);
|
||||
});
|
||||
for (const name of scenario.unexpected) files.set(name, '<p>x</p>');
|
||||
for (const extra of scenario.extras) {
|
||||
files.set(
|
||||
extra.name,
|
||||
`run();${extra.address ? address : ''}${extra.rootPath ? ' /work/repo/x' : ''}`,
|
||||
);
|
||||
}
|
||||
if (scenario.sourceMap) files.set('assets/app.js.map', '{}');
|
||||
if (scenario.sitemap === 'ok') files.set('sitemap.xml', SITEMAP);
|
||||
if (scenario.sitemap === 'differs') files.set('sitemap.xml', 'anders');
|
||||
return files;
|
||||
}
|
||||
|
||||
// Het eenvoudige model: wat er mis is, rechtstreeks uit het scenario
|
||||
function expectedViolations(scenario: OutputScenario): string[] {
|
||||
const expected: string[] = [];
|
||||
const anyHtml =
|
||||
scenario.pages.some((p) => p.present) || scenario.unexpected.length > 0;
|
||||
if (!anyHtml) expected.push('.: NO_HTML');
|
||||
|
||||
scenario.pages.forEach((p, i) => {
|
||||
const file = SCENARIO_FILES[i];
|
||||
if (!p.present) {
|
||||
if (anyHtml) {
|
||||
expected.push(
|
||||
`${file}: ${i === 3 ? 'NOT_FOUND_PAGE_MISSING' : 'PAGE_MISSING'}`,
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (p.bad === 'unsupported')
|
||||
expected.push(`${file}: UNSUPPORTED_SCRIPT`);
|
||||
else if (p.bad === 'unterminated') {
|
||||
expected.push(`${file}: UNTERMINATED_SCRIPT`);
|
||||
} else if (
|
||||
!p.reference &&
|
||||
!p.content.pieces.some((piece) => piece.kind === 'executable')
|
||||
) {
|
||||
expected.push(`${file}: NO_SCRIPTS_MOVED`);
|
||||
}
|
||||
if (p.leakAddress) expected.push(`${file}: EMAIL_IN_OUTPUT`);
|
||||
});
|
||||
for (const name of scenario.unexpected) {
|
||||
expected.push(`${name}: UNEXPECTED_HTML`);
|
||||
}
|
||||
for (const extra of scenario.extras) {
|
||||
if (extra.address) expected.push(`${extra.name}: EMAIL_IN_OUTPUT`);
|
||||
if (extra.rootPath) expected.push(`${extra.name}: BUILD_PATH_LEAK`);
|
||||
}
|
||||
if (scenario.sourceMap) {
|
||||
expected.push('assets/app.js.map: SOURCE_MAP_PRESENT');
|
||||
}
|
||||
if (scenario.sitemap === 'missing') {
|
||||
expected.push('sitemap.xml: SITEMAP_MISSING');
|
||||
}
|
||||
if (scenario.sitemap === 'differs') {
|
||||
expected.push('sitemap.xml: SITEMAP_DIFFERS');
|
||||
}
|
||||
return expected.sort();
|
||||
}
|
||||
|
||||
const assemble = ({ user, domain }: EmailAddress) => `${user}@${domain}`;
|
||||
|
||||
describe('planExternalization: eigenschappen', () => {
|
||||
it('meldt precies de overtredingen van een eenvoudig model (oracle)', () => {
|
||||
fc.assert(
|
||||
fc.property(outputScenario, validEmail, (scenario, email) => {
|
||||
const address = assemble(email);
|
||||
const plan = planExternalization(
|
||||
filesOf(scenario, address),
|
||||
context({ address }),
|
||||
);
|
||||
expect(found(plan)).toEqual(expectedViolations(scenario));
|
||||
// Geen bericht noemt het adres (BR-SCR-21)
|
||||
expect(
|
||||
plan.violations.map(formatViolation).join('\n'),
|
||||
).not.toContain(address);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('geeft na het toepassen van de writes geen writes en geen overtredingen meer (idempotent)', () => {
|
||||
const validOutput = fc.tuple(
|
||||
fc.array(htmlWithModel, { minLength: 4, maxLength: 4 }),
|
||||
fc.array(fc.constantFrom('a', 'b', 'c'), { maxLength: 3 }),
|
||||
);
|
||||
fc.assert(
|
||||
fc.property(validOutput, ([models, suffixes]) => {
|
||||
// Elke pagina krijgt een eigen, uitvoerbaar script, zodat het
|
||||
// geldig is
|
||||
const files = validFiles();
|
||||
SCENARIO_FILES.forEach((file, i) => {
|
||||
files.set(
|
||||
file,
|
||||
`${models[i].html}<script>boot("${i}${suffixes.join('')}")</script>`,
|
||||
);
|
||||
});
|
||||
const first = planExternalization(files, context());
|
||||
expect(first.violations).toEqual([]);
|
||||
for (const [file, content] of first.writes) {
|
||||
files.set(file, content);
|
||||
}
|
||||
const second = planExternalization(files, context());
|
||||
expect(second.writes.size).toBe(0);
|
||||
expect(second.violations).toEqual([]);
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// externalizeOutput met een echte map
|
||||
|
||||
const temporary: string[] = [];
|
||||
afterEach(() => {
|
||||
for (const dir of temporary.splice(0)) {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// <tmp>/web/dist/client: de repositorywortel voor de lekcontrole is <tmp>/web
|
||||
function outputDir() {
|
||||
const root = mkdtempSync(join(tmpdir(), 'externalize-'));
|
||||
temporary.push(root);
|
||||
const clientDir = join(root, 'web', 'dist', 'client');
|
||||
const put = (file: string, content: string) => {
|
||||
mkdirSync(dirname(join(clientDir, file)), { recursive: true });
|
||||
writeFileSync(join(clientDir, file), content, 'utf8');
|
||||
};
|
||||
for (const path of Object.keys(sitePages)) {
|
||||
put(pageFile(path), page(`boot("${path}")`));
|
||||
}
|
||||
put('404.html', page('boot("404")'));
|
||||
put('sitemap.xml', sitemapXml(pageList(sitePages), siteUrl));
|
||||
put('assets/index-abc.js', 'console.log(1);');
|
||||
return { root: join(root, 'web'), clientDir, put };
|
||||
}
|
||||
|
||||
const listing = (dir: string) =>
|
||||
readdirSync(dir, { recursive: true, encoding: 'utf8' }).sort();
|
||||
|
||||
const silent = () => ({ log: vi.fn(), error: vi.fn() });
|
||||
|
||||
describe('externalizeOutput', () => {
|
||||
it('schrijft eerst de scriptbestanden, dan de html, en meldt één regel', () => {
|
||||
const { clientDir } = outputDir();
|
||||
const out = silent();
|
||||
const result = externalizeOutput(clientDir, out);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.violations).toEqual([]);
|
||||
const firstHtml = result.written.findIndex((f) => f.endsWith('.html'));
|
||||
expect(firstHtml).toBeGreaterThan(0);
|
||||
expect(
|
||||
result.written
|
||||
.slice(0, firstHtml)
|
||||
.every((f) => !f.endsWith('.html')),
|
||||
).toBe(true);
|
||||
expect(
|
||||
result.written.slice(firstHtml).every((f) => f.endsWith('.html')),
|
||||
).toBe(true);
|
||||
|
||||
const index = readFileSync(join(clientDir, 'index.html'), 'utf8');
|
||||
const reference = /src="\/(assets\/inline-[0-9a-f]{12}\.js)"/.exec(
|
||||
index,
|
||||
)!;
|
||||
expect(readFileSync(join(clientDir, reference[1]), 'utf8')).toBe(
|
||||
'boot("/")',
|
||||
);
|
||||
expect(index).not.toContain('boot(');
|
||||
expect(out.error).not.toHaveBeenCalled();
|
||||
expect(out.log).toHaveBeenCalledTimes(1);
|
||||
expect(out.log.mock.calls[0][0]).toContain(
|
||||
`${Object.keys(sitePages).length + 1} pagina's gecontroleerd`,
|
||||
);
|
||||
expect(out.log.mock.calls[0][0]).toContain(
|
||||
`${Object.keys(sitePages).length + 1} scripts verplaatst`,
|
||||
);
|
||||
});
|
||||
|
||||
it('doet bij een tweede keer niets', () => {
|
||||
const { clientDir } = outputDir();
|
||||
externalizeOutput(clientDir, silent());
|
||||
const before = listing(clientDir);
|
||||
const out = silent();
|
||||
const result = externalizeOutput(clientDir, out);
|
||||
expect(result).toEqual({ exitCode: 0, violations: [], written: [] });
|
||||
expect(listing(clientDir)).toEqual(before);
|
||||
expect(out.log.mock.calls[0][0]).toContain('0 scripts verplaatst');
|
||||
});
|
||||
|
||||
it('schrijft niets en meldt alles bij een overtreding', () => {
|
||||
const { clientDir, put } = outputDir();
|
||||
rmSync(join(clientDir, 'over-mij.html'));
|
||||
put('assets/index-abc.js.map', '{}');
|
||||
const before = listing(clientDir);
|
||||
const out = silent();
|
||||
const result = externalizeOutput(clientDir, out);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.written).toEqual([]);
|
||||
expect(listing(clientDir)).toEqual(before);
|
||||
expect(out.log).not.toHaveBeenCalled();
|
||||
expect(out.error.mock.calls.map(([line]) => line)).toEqual(
|
||||
result.violations.map(formatViolation),
|
||||
);
|
||||
expect(result.violations.map((v) => v.code).sort()).toEqual([
|
||||
'PAGE_MISSING',
|
||||
'SOURCE_MAP_PRESENT',
|
||||
]);
|
||||
});
|
||||
|
||||
it('vindt het e-mailadres van de site in een uitvoerbestand', () => {
|
||||
const { clientDir, put } = outputDir();
|
||||
put('assets/index-abc.js', 'var a = "info@andreaslop.nl";');
|
||||
const result = externalizeOutput(clientDir, silent());
|
||||
expect(result.violations.map(formatViolation)).toEqual([
|
||||
'assets/index-abc.js: EMAIL_IN_OUTPUT: e-mailadres in de uitvoer',
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
'met voorwaartse slashes',
|
||||
(root: string) => root.replaceAll('\\', '/'),
|
||||
],
|
||||
['met backslashes', (root: string) => root.replaceAll('/', '\\')],
|
||||
[
|
||||
'met verdubbelde backslashes',
|
||||
(root: string) =>
|
||||
root.replaceAll('/', '\\').replaceAll('\\', '\\\\'),
|
||||
],
|
||||
])('vindt het pad van de repository %s', (_, style) => {
|
||||
const { clientDir, put, root } = outputDir();
|
||||
put('assets/index-abc.js', `var p = "${style(root)}";`);
|
||||
const result = externalizeOutput(clientDir, silent());
|
||||
expect(result.violations.map((v) => v.code)).toEqual([
|
||||
'BUILD_PATH_LEAK',
|
||||
]);
|
||||
});
|
||||
|
||||
it('leest alleen tekstbestanden en gaat niet af op een bestandssysteemwortel', () => {
|
||||
// clientDir direct onder de wortel: '../..' is de wortel zelf en mag
|
||||
// niet elk bestand met een slash als lek aanmerken
|
||||
const { clientDir, put } = outputDir();
|
||||
put('logo.png', 'binair');
|
||||
expect(externalizeOutput(clientDir, silent()).exitCode).toBe(0);
|
||||
const rootResult = externalizeOutput(join(clientDir, '..'), silent());
|
||||
expect(
|
||||
rootResult.violations.every((v) => v.code !== 'BUILD_PATH_LEAK'),
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('externalizeOutput als opdrachtregel', () => {
|
||||
const script = join(
|
||||
dirname(fileURLToPath(import.meta.url)),
|
||||
'externalizeInlineScripts.ts',
|
||||
);
|
||||
const run = (dir: string) =>
|
||||
spawnSync(process.execPath, [script, dir], { encoding: 'utf8' });
|
||||
|
||||
it('geeft exitcode 0 en één regel bij een geldige uitvoer', () => {
|
||||
const { clientDir } = outputDir();
|
||||
const result = run(clientDir);
|
||||
expect(result.status).toBe(0);
|
||||
expect(result.stdout.trim().split('\n')).toHaveLength(1);
|
||||
expect(existsSync(join(clientDir, 'assets'))).toBe(true);
|
||||
});
|
||||
|
||||
it('geeft exitcode 1 en schrijft niets bij een overtreding', () => {
|
||||
const { clientDir } = outputDir();
|
||||
rmSync(join(clientDir, '404.html'));
|
||||
const before = listing(clientDir);
|
||||
const result = run(clientDir);
|
||||
expect(result.status).toBe(1);
|
||||
expect(result.stderr).toContain('404.html: NOT_FOUND_PAGE_MISSING');
|
||||
expect(listing(clientDir)).toEqual(before);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,363 @@
|
||||
// Stap na `vite build` die de inline scripts van Start in dist/client naar
|
||||
// bestanden verplaatst en de uitvoer controleert.
|
||||
//
|
||||
// Waarom (BR-BLD-12): de CSP staat alleen script-src 'self' en Umami toe, en
|
||||
// Start zet inline scripts in elke geprerenderde pagina. Zonder deze stap
|
||||
// blokkeert de browser ze. De CSP blijft ongewijzigd.
|
||||
//
|
||||
// Alles wordt eerst in het geheugen gepland (planExternalization, puur). Pas
|
||||
// als er geen enkele overtreding is, schrijft de runner: eerst de
|
||||
// scriptbestanden, daarna de aangepaste HTML (BR-SCR-12, SECURITY-15). Een
|
||||
// mislukte controle laat dist/client dus ongemoeid.
|
||||
import {
|
||||
mkdirSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
statSync,
|
||||
writeFileSync,
|
||||
} from 'node:fs';
|
||||
import { dirname, join, parse, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { emailAddress } from '../src/lib/emailAddress.ts';
|
||||
import { siteUrl } from '../src/seo.ts';
|
||||
import siteContent from '../src/content/siteContent.ts';
|
||||
import { sitemapXml } from '../src/sitemap.ts';
|
||||
import sitePages, { pageList } from '../src/sitePages.ts';
|
||||
import {
|
||||
ScriptError,
|
||||
externalizeInlineScripts,
|
||||
findExecutableInlineScripts,
|
||||
} from './inlineScripts.ts';
|
||||
import { NOT_FOUND_FILE, pageFile } from './pageFiles.ts';
|
||||
|
||||
export type ViolationCode =
|
||||
| 'NO_HTML'
|
||||
| 'PAGE_MISSING'
|
||||
| 'NOT_FOUND_PAGE_MISSING'
|
||||
| 'UNEXPECTED_HTML'
|
||||
| 'NO_SCRIPTS_MOVED'
|
||||
| 'EXECUTABLE_REMAINS'
|
||||
| 'UNSUPPORTED_SCRIPT'
|
||||
| 'UNTERMINATED_SCRIPT'
|
||||
| 'COLLISION'
|
||||
| 'EMAIL_IN_OUTPUT'
|
||||
| 'SITEMAP_MISSING'
|
||||
| 'SITEMAP_DIFFERS'
|
||||
| 'SOURCE_MAP_PRESENT'
|
||||
| 'BUILD_PATH_LEAK';
|
||||
|
||||
export interface Violation {
|
||||
file: string;
|
||||
code: ViolationCode;
|
||||
// Nooit een scriptinhoud en nooit het e-mailadres (BR-SCR-21)
|
||||
detail: string;
|
||||
}
|
||||
|
||||
export interface ExpectedFileSet {
|
||||
// Paginapad uit het register naar zijn bestand
|
||||
pages: Map<string, string>;
|
||||
notFound: string;
|
||||
}
|
||||
|
||||
export interface PlanContext {
|
||||
expected: ExpectedFileSet;
|
||||
// Het samengestelde adres, of null als de delen ongeldig zijn
|
||||
address: string | null;
|
||||
sitemap: string;
|
||||
// De repositorywortel in alle schrijfwijzen die in tekst kunnen staan
|
||||
rootPaths: string[];
|
||||
}
|
||||
|
||||
export interface OutputPlan {
|
||||
// Relatief pad naar nieuwe inhoud
|
||||
writes: Map<string, string>;
|
||||
violations: Violation[];
|
||||
}
|
||||
|
||||
// De twee functies van de scriptstap als parameter, zodat de verdediging
|
||||
// EXECUTABLE_REMAINS getest kan worden: met de echte functies kan ze niet
|
||||
// optreden (BR-SCR-11).
|
||||
export interface ScriptStep {
|
||||
externalize: typeof externalizeInlineScripts;
|
||||
find: typeof findExecutableInlineScripts;
|
||||
}
|
||||
|
||||
const defaultScriptStep: ScriptStep = {
|
||||
externalize: externalizeInlineScripts,
|
||||
find: findExecutableInlineScripts,
|
||||
};
|
||||
|
||||
const ASSETS_DIR = 'assets';
|
||||
const SITEMAP_FILE = 'sitemap.xml';
|
||||
const MOVED_REFERENCE = /src="\/assets\/inline-[0-9a-f]{12}\.js"/;
|
||||
|
||||
// Hoe een '@' in een bestand kan staan zonder dat het adres leesbaar is:
|
||||
// platte tekst, URL-codering, JavaScript-escapes en HTML-entiteiten
|
||||
const AT_ENCODINGS = [
|
||||
'@',
|
||||
'%40',
|
||||
'\\u0040',
|
||||
'\\u{40}',
|
||||
'\\x40',
|
||||
'@',
|
||||
'@',
|
||||
'@',
|
||||
];
|
||||
|
||||
// Staat het adres in een van de schrijfwijzen in de tekst? Hoofdletterongevoelig.
|
||||
export function containsAddress(text: string, address: string): boolean {
|
||||
const at = address.indexOf('@');
|
||||
if (at === -1) return false;
|
||||
const haystack = text.toLowerCase();
|
||||
const user = address.slice(0, at).toLowerCase();
|
||||
const domain = address.slice(at + 1).toLowerCase();
|
||||
return AT_ENCODINGS.some((encoding) =>
|
||||
haystack.includes(`${user}${encoding}${domain}`),
|
||||
);
|
||||
}
|
||||
|
||||
const isHtml = (file: string) => file.endsWith('.html');
|
||||
|
||||
// Een ScriptError heeft het bericht `CODE: detail`; de runner zet er zelf de
|
||||
// code en het bestand voor
|
||||
const detailOf = (error: ScriptError) =>
|
||||
error.message.slice(error.code.length + 2);
|
||||
|
||||
export function formatViolation({ file, code, detail }: Violation): string {
|
||||
return `${file}: ${code}: ${detail}`;
|
||||
}
|
||||
|
||||
// De beslissingen van de runner zonder bestandssysteem. `files` bevat alle
|
||||
// tekstbestanden (en eventuele .map-bestanden) onder dist/client op relatief
|
||||
// pad met voorwaartse slashes.
|
||||
export function planExternalization(
|
||||
files: ReadonlyMap<string, string>,
|
||||
context: PlanContext,
|
||||
scripts: ScriptStep = defaultScriptStep,
|
||||
): OutputPlan {
|
||||
const violations: Violation[] = [];
|
||||
const writes = new Map<string, string>();
|
||||
const add = (file: string, code: ViolationCode, detail: string) =>
|
||||
violations.push({ file, code, detail });
|
||||
|
||||
const htmlFiles = [...files.keys()].filter(isHtml).sort();
|
||||
const expectedFiles = new Set([
|
||||
...context.expected.pages.values(),
|
||||
context.expected.notFound,
|
||||
]);
|
||||
|
||||
if (htmlFiles.length === 0) {
|
||||
// De ontbrekende pagina's afzonderlijk melden zou alleen herhalen
|
||||
add('.', 'NO_HTML', 'geen .html-bestand in de uitvoermap');
|
||||
} else {
|
||||
for (const [path, file] of context.expected.pages) {
|
||||
if (!files.has(file)) {
|
||||
add(file, 'PAGE_MISSING', `pagina ${path} ontbreekt`);
|
||||
}
|
||||
}
|
||||
if (!files.has(context.expected.notFound)) {
|
||||
add(
|
||||
context.expected.notFound,
|
||||
'NOT_FOUND_PAGE_MISSING',
|
||||
'de 404-pagina ontbreekt',
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const file of htmlFiles) {
|
||||
if (!expectedFiles.has(file)) {
|
||||
add(
|
||||
file,
|
||||
'UNEXPECTED_HTML',
|
||||
'html-bestand dat niet bij het register hoort',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const file of htmlFiles) {
|
||||
if (!expectedFiles.has(file)) continue;
|
||||
const html = files.get(file)!;
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = scripts.externalize(html);
|
||||
} catch (error) {
|
||||
if (!(error instanceof ScriptError)) throw error;
|
||||
add(file, error.code, detailOf(error));
|
||||
continue;
|
||||
}
|
||||
|
||||
const remaining = scripts.find(result.html).length;
|
||||
if (remaining > 0) {
|
||||
add(
|
||||
file,
|
||||
'EXECUTABLE_REMAINS',
|
||||
`${remaining} uitvoerbaar inline script blijft over`,
|
||||
);
|
||||
}
|
||||
// Een al verwerkte pagina telt mee, dus twee keer draaien mag
|
||||
if (!MOVED_REFERENCE.test(result.html)) {
|
||||
add(
|
||||
file,
|
||||
'NO_SCRIPTS_MOVED',
|
||||
'geen verwijzing naar een verplaatst inline script',
|
||||
);
|
||||
}
|
||||
|
||||
for (const { fileName, body } of result.moved) {
|
||||
const path = `${ASSETS_DIR}/${fileName}`;
|
||||
const existing = writes.get(path) ?? files.get(path);
|
||||
if (existing !== undefined && existing !== body) {
|
||||
add(path, 'COLLISION', 'bestaat al met andere inhoud');
|
||||
} else if (existing === undefined) {
|
||||
writes.set(path, body);
|
||||
}
|
||||
}
|
||||
if (result.html !== html) writes.set(file, result.html);
|
||||
}
|
||||
|
||||
// De uitvoer zoals hij na het schrijven is: ook de verplaatste scripts
|
||||
const finalFiles = new Map(files);
|
||||
for (const [path, content] of writes) finalFiles.set(path, content);
|
||||
|
||||
for (const [file, content] of finalFiles) {
|
||||
if (file.endsWith('.map')) {
|
||||
add(file, 'SOURCE_MAP_PRESENT', 'sourcemap in de uitvoer');
|
||||
}
|
||||
if (context.address && containsAddress(content, context.address)) {
|
||||
add(file, 'EMAIL_IN_OUTPUT', 'e-mailadres in de uitvoer');
|
||||
}
|
||||
if (context.rootPaths.some((root) => content.includes(root))) {
|
||||
add(file, 'BUILD_PATH_LEAK', 'pad van de repository in de uitvoer');
|
||||
}
|
||||
}
|
||||
|
||||
const sitemap = files.get(SITEMAP_FILE);
|
||||
if (sitemap === undefined) {
|
||||
add(SITEMAP_FILE, 'SITEMAP_MISSING', 'bestand ontbreekt');
|
||||
} else if (sitemap !== context.sitemap) {
|
||||
add(
|
||||
SITEMAP_FILE,
|
||||
'SITEMAP_DIFFERS',
|
||||
'inhoud wijkt af van het pagina-register',
|
||||
);
|
||||
}
|
||||
|
||||
return { writes, violations };
|
||||
}
|
||||
|
||||
// Alleen deze bestanden worden gelezen en op adres en pad doorzocht
|
||||
const TEXT_EXTENSIONS = [
|
||||
'.html',
|
||||
'.js',
|
||||
'.css',
|
||||
'.xml',
|
||||
'.txt',
|
||||
'.json',
|
||||
'.webmanifest',
|
||||
'.map',
|
||||
];
|
||||
|
||||
function readOutput(clientDir: string): Map<string, string> {
|
||||
const files = new Map<string, string>();
|
||||
const entries = readdirSync(clientDir, {
|
||||
recursive: true,
|
||||
encoding: 'utf8',
|
||||
})
|
||||
.map((entry) => entry.replaceAll('\\', '/'))
|
||||
.filter((entry) => TEXT_EXTENSIONS.some((ext) => entry.endsWith(ext)))
|
||||
.sort();
|
||||
for (const entry of entries) {
|
||||
const path = join(clientDir, entry);
|
||||
if (statSync(path).isFile())
|
||||
files.set(entry, readFileSync(path, 'utf8'));
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
// De repositorywortel in de schrijfwijzen waarin hij in tekst kan staan:
|
||||
// met / , met \ en met verdubbelde \ zoals in een JSON- of JS-string
|
||||
function rootPathsOf(root: string): string[] {
|
||||
// De wortel van het bestandssysteem zou alles als lek aanmerken
|
||||
if (parse(root).root === root) return [];
|
||||
return [
|
||||
...new Set([
|
||||
root,
|
||||
root.replaceAll('\\', '/'),
|
||||
root.replaceAll('/', '\\'),
|
||||
root.replaceAll('/', '\\').replaceAll('\\', '\\\\'),
|
||||
]),
|
||||
];
|
||||
}
|
||||
|
||||
function contextFor(clientDir: string): PlanContext {
|
||||
return {
|
||||
expected: {
|
||||
pages: new Map(
|
||||
Object.keys(sitePages).map((path) => [path, pageFile(path)]),
|
||||
),
|
||||
notFound: NOT_FOUND_FILE,
|
||||
},
|
||||
address: emailAddress(siteContent.email),
|
||||
sitemap: sitemapXml(pageList(sitePages), siteUrl),
|
||||
rootPaths: rootPathsOf(resolve(clientDir, '../..')),
|
||||
};
|
||||
}
|
||||
|
||||
export interface RunResult {
|
||||
exitCode: 0 | 1;
|
||||
violations: Violation[];
|
||||
written: string[];
|
||||
}
|
||||
|
||||
interface Log {
|
||||
log: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
// Leest de map, plant, en schrijft alleen zonder overtredingen. Geeft de
|
||||
// exitcode terug in plaats van hem te zetten, zodat tests hem kunnen lezen.
|
||||
export function externalizeOutput(
|
||||
clientDir: string,
|
||||
out: Log = console,
|
||||
): RunResult {
|
||||
const context = contextFor(clientDir);
|
||||
const { writes, violations } = planExternalization(
|
||||
readOutput(clientDir),
|
||||
context,
|
||||
);
|
||||
|
||||
if (violations.length > 0) {
|
||||
for (const violation of violations) {
|
||||
out.error(formatViolation(violation));
|
||||
}
|
||||
return { exitCode: 1, violations, written: [] };
|
||||
}
|
||||
|
||||
// Scriptbestanden eerst: een afgebroken run laat nooit HTML achter die
|
||||
// naar een bestand verwijst dat niet bestaat
|
||||
const paths = [...writes.keys()].sort(
|
||||
(a, b) => Number(isHtml(a)) - Number(isHtml(b)),
|
||||
);
|
||||
for (const path of paths) {
|
||||
const target = join(clientDir, path);
|
||||
mkdirSync(dirname(target), { recursive: true });
|
||||
writeFileSync(target, writes.get(path)!, 'utf8');
|
||||
}
|
||||
|
||||
const moved = paths.filter((path) => !isHtml(path)).length;
|
||||
out.log(
|
||||
`Inline scripts verplaatst: ${context.expected.pages.size + 1} pagina's gecontroleerd, ${moved} scripts verplaatst, ${paths.length} bestanden geschreven`,
|
||||
);
|
||||
return { exitCode: 0, violations, written: paths };
|
||||
}
|
||||
|
||||
// Draait alleen als het bestand zelf wordt gestart (pnpm build), niet als een
|
||||
// test het importeert. Optioneel argument: de map, standaard dist/client.
|
||||
if (
|
||||
process.argv[1] &&
|
||||
resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
process.exitCode = externalizeOutput(
|
||||
resolve(process.argv[2] ?? 'dist/client'),
|
||||
).exitCode;
|
||||
}
|
||||
@@ -0,0 +1,507 @@
|
||||
import { createHash } from 'node:crypto';
|
||||
import fc from 'fast-check';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
canonicalHtmlWithModel,
|
||||
htmlFragments,
|
||||
htmlWithModel,
|
||||
scriptBody,
|
||||
} from '../src/test/htmlArbitraries';
|
||||
import {
|
||||
ScriptError,
|
||||
externalizeInlineScripts,
|
||||
findExecutableInlineScripts,
|
||||
} from './inlineScripts';
|
||||
|
||||
// Onafhankelijk van de implementatie: de naam volgens BR-SCR-07
|
||||
const fileOf = (body: string) =>
|
||||
`inline-${createHash('sha256').update(body, 'utf8').digest('hex').slice(0, 12)}.js`;
|
||||
|
||||
const referenceOf = (attributes: string, body: string) =>
|
||||
`<script${attributes} src="/assets/${fileOf(body)}"></script>`;
|
||||
|
||||
// Een door de stap geplaatste verwijzing: <script ... src="/assets/inline-..">
|
||||
const referenceElement =
|
||||
/<script(?:[^>"']|"[^"]*"|'[^']*')* src="\/assets\/inline-[0-9a-f]{12}\.js"><\/script>/g;
|
||||
|
||||
const fails = (html: string, code: string) => {
|
||||
try {
|
||||
externalizeInlineScripts(html);
|
||||
} catch (error) {
|
||||
expect(error).toBeInstanceOf(ScriptError);
|
||||
expect((error as ScriptError).code).toBe(code);
|
||||
return error as ScriptError;
|
||||
}
|
||||
throw new Error('verwacht een ScriptError');
|
||||
};
|
||||
|
||||
describe('externalizeInlineScripts: voorbeelden', () => {
|
||||
it('verplaatst het bootstrapscript en behoudt de attributen', () => {
|
||||
const result = externalizeInlineScripts(
|
||||
'<script data-tsr-stream-part="">A</script>',
|
||||
);
|
||||
expect(result.html).toBe(referenceOf(' data-tsr-stream-part=""', 'A'));
|
||||
expect(result.moved).toEqual([{ fileName: fileOf('A'), body: 'A' }]);
|
||||
});
|
||||
|
||||
it('laat een extern module-script met async ongemoeid', () => {
|
||||
const html = '<script type="module" async src="/assets/x.js"></script>';
|
||||
expect(externalizeInlineScripts(html)).toEqual({ html, moved: [] });
|
||||
});
|
||||
|
||||
it('verplaatst een kaal script en kent het bestand de inhoud toe', () => {
|
||||
const result = externalizeInlineScripts(
|
||||
'<script>document.currentScript.remove()</script>',
|
||||
);
|
||||
expect(result.html).toBe(
|
||||
referenceOf('', 'document.currentScript.remove()'),
|
||||
);
|
||||
expect(result.moved).toHaveLength(1);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
'een JSON-LD-blok',
|
||||
'<script type="application/ld+json">{"a":1}</script>',
|
||||
],
|
||||
[
|
||||
'een style-body',
|
||||
'<style>.a::before{content:"<script>x</script>"}</style>',
|
||||
],
|
||||
['commentaar', '<!-- <script>x</script> -->'],
|
||||
['een attribuutwaarde', '<div title="<script>x</script>"></div>'],
|
||||
])('laat %s ongemoeid', (_, html) => {
|
||||
expect(externalizeInlineScripts(html)).toEqual({ html, moved: [] });
|
||||
});
|
||||
|
||||
it('laat al het andere byte voor byte staan, ook regeleinden', () => {
|
||||
const html =
|
||||
'<!doctype html>\r\n<html>\r\n<!-- c -->\r\n<style>a{}</style>' +
|
||||
'<script type="application/ld+json">{}</script>' +
|
||||
'<script>go()</script>\r\n<script src="/a.js"></script>\r\n</html>';
|
||||
expect(externalizeInlineScripts(html).html).toBe(
|
||||
html.replace('<script>go()</script>', referenceOf('', 'go()')),
|
||||
);
|
||||
});
|
||||
|
||||
it('hasht de inhoud zoals geschreven, zonder trimmen', () => {
|
||||
const a = externalizeInlineScripts('<script>x()</script>');
|
||||
const b = externalizeInlineScripts('<script> x() </script>');
|
||||
expect(a.moved[0].fileName).not.toBe(b.moved[0].fileName);
|
||||
expect(b.moved[0].body).toBe(' x() ');
|
||||
expect(b.moved[0].fileName).toBe(fileOf(' x() '));
|
||||
});
|
||||
|
||||
it('geeft gelijke inhoud één bestand, op volgorde van eerste voorkomen', () => {
|
||||
const result = externalizeInlineScripts(
|
||||
'<script>b()</script><script>a()</script><script>b()</script>',
|
||||
);
|
||||
expect(result.moved.map((m) => m.body)).toEqual(['b()', 'a()']);
|
||||
expect(result.html).toBe(
|
||||
referenceOf('', 'b()') +
|
||||
referenceOf('', 'a()') +
|
||||
referenceOf('', 'b()'),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('classificatie (BR-SCR-04)', () => {
|
||||
it.each([
|
||||
' type="text/javascript"',
|
||||
' type="TEXT/JavaScript"',
|
||||
' type=" text/javascript "',
|
||||
' type="application/javascript"',
|
||||
' type="application/ecmascript"',
|
||||
' type="application/x-ecmascript"',
|
||||
' type="application/x-javascript"',
|
||||
' type="text/javascript1.0"',
|
||||
' type="text/javascript1.5"',
|
||||
' type="text/ecmascript"',
|
||||
' type="text/jscript"',
|
||||
' type="text/livescript"',
|
||||
' type="text/x-ecmascript"',
|
||||
' type="text/x-javascript"',
|
||||
' type=""',
|
||||
' type',
|
||||
' type="module"',
|
||||
' type="Module"',
|
||||
' language="vbscript"',
|
||||
])('verplaatst een script met %s', (attributes) => {
|
||||
const html = `<script${attributes}>go()</script>`;
|
||||
expect(externalizeInlineScripts(html).html).toBe(
|
||||
referenceOf(attributes, 'go()'),
|
||||
);
|
||||
expect(findExecutableInlineScripts(html)).toEqual(['go()']);
|
||||
});
|
||||
|
||||
it.each([
|
||||
' type="application/ld+json"',
|
||||
' type="application/json"',
|
||||
' type="importmap"',
|
||||
' type="speculationrules"',
|
||||
' type="text/plain"',
|
||||
' type="text/javascript2.0"',
|
||||
' type="jscript"',
|
||||
])('laat een datablok met %s staan', (attributes) => {
|
||||
const html = `<script${attributes}>{"a":1}</script>`;
|
||||
expect(externalizeInlineScripts(html)).toEqual({ html, moved: [] });
|
||||
expect(findExecutableInlineScripts(html)).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['src met waarde', '<script src="/a.js">go()</script>'],
|
||||
['src zonder waarde', '<script src>go()</script>'],
|
||||
['lege src', '<script src="">go()</script>'],
|
||||
['src in hoofdletters', '<script SRC=/a.js>go()</script>'],
|
||||
])('laat een extern script staan: %s', (_, html) => {
|
||||
expect(externalizeInlineScripts(html)).toEqual({ html, moved: [] });
|
||||
expect(findExecutableInlineScripts(html)).toEqual([]);
|
||||
});
|
||||
|
||||
it('laat een leeg script staan, ook met alleen witruimte', () => {
|
||||
const html = '<script></script><script> \n\t</script>';
|
||||
expect(externalizeInlineScripts(html)).toEqual({ html, moved: [] });
|
||||
expect(findExecutableInlineScripts(html)).toEqual([]);
|
||||
});
|
||||
|
||||
it('leest alleen het eerste type-attribuut, zoals een browser', () => {
|
||||
const html = '<script type="importmap" type="module">{}</script>';
|
||||
expect(externalizeInlineScripts(html).moved).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('niet-ondersteunde vormen (BR-SCR-05)', () => {
|
||||
it.each([
|
||||
'<script async>go()</script>',
|
||||
'<script defer>go()</script>',
|
||||
'<script type="text/javascript" ASYNC>go()</script>',
|
||||
])('(a) weigert een klassiek script met async of defer: %s', (html) => {
|
||||
fails(html, 'UNSUPPORTED_SCRIPT');
|
||||
});
|
||||
|
||||
it('(a) staat async op een module-script wel toe', () => {
|
||||
expect(
|
||||
externalizeInlineScripts(
|
||||
'<script type="module" async>go()</script>',
|
||||
).moved,
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('(b) weigert een commentaaropener in de body', () => {
|
||||
fails('<script>var a = "<!--";</script>', 'UNSUPPORTED_SCRIPT');
|
||||
});
|
||||
|
||||
it('(b) weigert ook in een module-script', () => {
|
||||
fails('<script type="module">// <!--\n</script>', 'UNSUPPORTED_SCRIPT');
|
||||
});
|
||||
|
||||
it.each([
|
||||
"import a from './x.js';",
|
||||
'import a from "../x.js";',
|
||||
"import './side.js';",
|
||||
"const m = await import('./x.js');",
|
||||
'const m = import( "../x.js" );',
|
||||
"export * from './x.js';",
|
||||
"import a from '.';",
|
||||
])('(c) weigert een module met relatieve import: %s', (body) => {
|
||||
fails(`<script type="module">${body}</script>`, 'UNSUPPORTED_SCRIPT');
|
||||
});
|
||||
|
||||
it.each([
|
||||
"import a from '/assets/x.js';",
|
||||
"import a from 'https://cdn.test/x.js';",
|
||||
"import a from 'pakket';",
|
||||
'const from = 1; const s = "x./y";',
|
||||
])('(c) staat een module zonder relatieve import toe: %s', (body) => {
|
||||
expect(
|
||||
externalizeInlineScripts(`<script type="module">${body}</script>`)
|
||||
.moved,
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('(c) geldt niet voor een klassiek script', () => {
|
||||
expect(
|
||||
externalizeInlineScripts("<script>import('./x.js')</script>").moved,
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('noemt de regel en de code, nooit de inhoud van het script', () => {
|
||||
const error = fails(
|
||||
'<p>\n</p>\n<script async>geheim();</script>',
|
||||
'UNSUPPORTED_SCRIPT',
|
||||
);
|
||||
expect(error.message).toContain('regel 3');
|
||||
expect(error.message).not.toContain('geheim');
|
||||
});
|
||||
|
||||
it('controleert alleen scripts die verplaatst zouden worden', () => {
|
||||
const html =
|
||||
'<script async src="/a.js"></script>' +
|
||||
'<script type="application/ld+json">"<!--"</script>';
|
||||
expect(externalizeInlineScripts(html).moved).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('scriptgrenzen en scanner (BR-SCR-02, 03)', () => {
|
||||
it('weigert een script zonder eindtag', () => {
|
||||
const error = fails('<p></p><script>go()', 'UNTERMINATED_SCRIPT');
|
||||
expect(error.message).toContain('regel 1');
|
||||
expect(error.message).not.toContain('go()');
|
||||
});
|
||||
|
||||
it('weigert een eindtag zonder sluitende >', () => {
|
||||
fails('<script>go()</script', 'UNTERMINATED_SCRIPT');
|
||||
fails('<script>go()</script ', 'UNTERMINATED_SCRIPT');
|
||||
});
|
||||
|
||||
it('weigert ook een extern of data-script zonder eindtag', () => {
|
||||
fails('<script src="/a.js">', 'UNTERMINATED_SCRIPT');
|
||||
fails('<script type="importmap">{}', 'UNTERMINATED_SCRIPT');
|
||||
});
|
||||
|
||||
it('meldt bij het zoeken de rest van de tekst als body', () => {
|
||||
expect(findExecutableInlineScripts('<p></p><script>go()')).toEqual([
|
||||
'go()',
|
||||
]);
|
||||
// Zonder teken na </script is het geen eindtag, dus hoort het bij de body
|
||||
expect(findExecutableInlineScripts('<script>go()</script')).toEqual([
|
||||
'go()</script',
|
||||
]);
|
||||
expect(findExecutableInlineScripts('<script>go()</script ')).toEqual([
|
||||
'go()',
|
||||
]);
|
||||
});
|
||||
|
||||
it('sluit een script pas bij </script gevolgd door witruimte, / of >', () => {
|
||||
const html = '<script>a = "</scripts>"; b = 1;</script>';
|
||||
expect(findExecutableInlineScripts(html)).toEqual([
|
||||
'a = "</scripts>"; b = 1;',
|
||||
]);
|
||||
expect(findExecutableInlineScripts('<script>a</script/>')).toEqual([
|
||||
'a',
|
||||
]);
|
||||
expect(findExecutableInlineScripts('<script>a</SCRIPT\n>')).toEqual([
|
||||
'a',
|
||||
]);
|
||||
});
|
||||
|
||||
it('neemt een > binnen een aanhalingsteken in een attribuut niet als einde van de tag', () => {
|
||||
const attributes = ' data-x="a>b" data-y=\'c > d\'';
|
||||
const result = externalizeInlineScripts(
|
||||
`<script${attributes}>go()</script>`,
|
||||
);
|
||||
expect(result.html).toBe(referenceOf(attributes, 'go()'));
|
||||
});
|
||||
|
||||
it('behoudt een afsluitende / in de tag en leest de naam hoofdletterongevoelig', () => {
|
||||
expect(externalizeInlineScripts('<script />go()</script>').html).toBe(
|
||||
referenceOf(' /', 'go()'),
|
||||
);
|
||||
expect(
|
||||
externalizeInlineScripts('<SCRIPT TYPE="Module">go()</SCRIPT>')
|
||||
.html,
|
||||
).toBe(referenceOf(' TYPE="Module"', 'go()'));
|
||||
});
|
||||
|
||||
it('een ongeldig <-teken is tekst', () => {
|
||||
const html = '<p>1 < 2</p><script>go()</script><3><script>x</script>';
|
||||
expect(findExecutableInlineScripts(html)).toEqual(['go()', 'x']);
|
||||
});
|
||||
|
||||
it('slaat tekstelementen over tot hun eindtag, ook zonder eindtag', () => {
|
||||
expect(
|
||||
findExecutableInlineScripts(
|
||||
'<title>a<script>x</script></title><script>y</script>',
|
||||
),
|
||||
).toEqual(['y']);
|
||||
expect(
|
||||
findExecutableInlineScripts('<style>a<script>x</script>'),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
findExecutableInlineScripts(
|
||||
'<TEXTAREA><script>x</script></TextArea >',
|
||||
),
|
||||
).toEqual([]);
|
||||
// </styled is geen eindtag van style
|
||||
expect(
|
||||
findExecutableInlineScripts(
|
||||
'<style></styled><script>x</script></style><script>y</script>',
|
||||
),
|
||||
).toEqual(['y']);
|
||||
});
|
||||
|
||||
it('slaat commentaar over, ook <!--> en een niet-afgesloten commentaar', () => {
|
||||
expect(findExecutableInlineScripts('<!--><script>x</script>')).toEqual([
|
||||
'x',
|
||||
]);
|
||||
expect(findExecutableInlineScripts('<!---><script>x</script>')).toEqual(
|
||||
['x'],
|
||||
);
|
||||
expect(findExecutableInlineScripts('<!-- <script>x</script>')).toEqual(
|
||||
[],
|
||||
);
|
||||
});
|
||||
|
||||
it('slaat declaraties, instructies en eindtags over tot de >', () => {
|
||||
expect(
|
||||
findExecutableInlineScripts(
|
||||
'<!doctype html><?php <script>a</script> ?></p <script>b</script>><script>c</script>',
|
||||
),
|
||||
).toEqual(['c']);
|
||||
expect(findExecutableInlineScripts('<!doctype')).toEqual([]);
|
||||
expect(findExecutableInlineScripts('</p')).toEqual([]);
|
||||
});
|
||||
|
||||
it('gooit een tag weg die niet eindigt', () => {
|
||||
expect(findExecutableInlineScripts('<div class="a')).toEqual([]);
|
||||
expect(findExecutableInlineScripts('<div class')).toEqual([]);
|
||||
expect(
|
||||
findExecutableInlineScripts('<script>a</script><div x="'),
|
||||
).toEqual(['a']);
|
||||
});
|
||||
|
||||
it('leest een attribuut dat met = begint en een waarde zonder aanhalingstekens', () => {
|
||||
expect(
|
||||
externalizeInlineScripts('<script =x data-n=1>go()</script>').html,
|
||||
).toBe(referenceOf(' =x data-n=1', 'go()'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('externalizeInlineScripts: eigenschappen', () => {
|
||||
it('laat alles buiten de vervangen scripts byte voor byte staan (invariant)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlWithModel, ({ html, pieces, separator }) => {
|
||||
const { html: output } = externalizeInlineScripts(html);
|
||||
const inputWithoutScripts = pieces
|
||||
.map((piece) =>
|
||||
piece.kind === 'executable' ? '' : piece.html,
|
||||
)
|
||||
.join(separator);
|
||||
expect(output.replace(referenceElement, '')).toBe(
|
||||
inputWithoutScripts,
|
||||
);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('bewaart de volgorde van de verwijzingen (invariant)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlWithModel, ({ html, pieces }) => {
|
||||
const { html: output } = externalizeInlineScripts(html);
|
||||
const references = [
|
||||
...output.matchAll(
|
||||
/src="\/assets\/(inline-[0-9a-f]{12}\.js)"/g,
|
||||
),
|
||||
].map((match) => match[1]);
|
||||
const expected = pieces
|
||||
.filter((piece) => piece.kind === 'executable')
|
||||
.map((piece) => fileOf(piece.body!));
|
||||
expect(references).toEqual(expected);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('herstelt de invoer door elke verwijzing door haar inhoud te vervangen (round-trip)', () => {
|
||||
fc.assert(
|
||||
fc.property(canonicalHtmlWithModel, ({ html }) => {
|
||||
const { html: output, moved } = externalizeInlineScripts(html);
|
||||
const bodies = new Map(
|
||||
moved.map((m) => [m.fileName, m.body] as const),
|
||||
);
|
||||
const restored = output.replace(
|
||||
/ src="\/assets\/(inline-[0-9a-f]{12}\.js)"><\/script>/g,
|
||||
(_, name: string) => `>${bodies.get(name)}</script>`,
|
||||
);
|
||||
expect(restored).toBe(html);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('geeft bij een tweede keer dezelfde html en niets te verplaatsen (idempotent)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlWithModel, ({ html }) => {
|
||||
const first = externalizeInlineScripts(html);
|
||||
const second = externalizeInlineScripts(first.html);
|
||||
expect(second.html).toBe(first.html);
|
||||
expect(second.moved).toEqual([]);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('laat datablokken, externe en lege scripts, commentaar en style ongewijzigd (invariant)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlWithModel, ({ html, pieces }) => {
|
||||
const { html: output } = externalizeInlineScripts(html);
|
||||
for (const piece of pieces) {
|
||||
if (piece.kind !== 'executable') {
|
||||
expect(output).toContain(piece.html);
|
||||
}
|
||||
}
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('laat na het verplaatsen geen uitvoerbaar inline script over (invariant)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlWithModel, ({ html }) => {
|
||||
const { html: output } = externalizeInlineScripts(html);
|
||||
expect(findExecutableInlineScripts(output)).toEqual([]);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('vindt precies de uitvoerbare scripts van het model, in volgorde (oracle)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlWithModel, ({ html, pieces }) => {
|
||||
expect(findExecutableInlineScripts(html)).toEqual(
|
||||
pieces
|
||||
.filter((piece) => piece.kind === 'executable')
|
||||
.map((piece) => piece.body),
|
||||
);
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('geeft gelijke inhoud één naam en verschillende inhoud verschillende namen (invariant)', () => {
|
||||
fc.assert(
|
||||
fc.property(
|
||||
fc.array(scriptBody, { minLength: 1, maxLength: 8 }),
|
||||
(bodies) => {
|
||||
const html = bodies
|
||||
.map((body) => `<script>${body}</script>`)
|
||||
.join('');
|
||||
const { moved } = externalizeInlineScripts(html);
|
||||
const names = moved.map((m) => m.fileName);
|
||||
for (const name of names) {
|
||||
expect(name).toMatch(/^inline-[0-9a-f]{12}\.js$/);
|
||||
}
|
||||
expect(new Set(names).size).toBe(names.length);
|
||||
expect(names).toHaveLength(new Set(bodies).size);
|
||||
expect(names).toEqual(
|
||||
[...new Set(bodies)].map((body) => fileOf(body)),
|
||||
);
|
||||
},
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
it('geeft hetzelfde bij samenvoegen als bij los verwerken (split en join)', () => {
|
||||
fc.assert(
|
||||
fc.property(htmlFragments, (fragments) => {
|
||||
const joined = externalizeInlineScripts(
|
||||
fragments.map((f) => f.html).join(''),
|
||||
);
|
||||
const separate = fragments.map((f) =>
|
||||
externalizeInlineScripts(f.html),
|
||||
);
|
||||
expect(joined.html).toBe(separate.map((r) => r.html).join(''));
|
||||
const names = new Map(
|
||||
separate
|
||||
.flatMap((r) => r.moved)
|
||||
.map((m) => [m.fileName, m]),
|
||||
);
|
||||
expect(joined.moved).toEqual([...names.values()]);
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,370 @@
|
||||
// Verplaatst de inline scripts van Start naar bestanden onder /assets.
|
||||
//
|
||||
// Waarom (NFR-PR-Q-05, BR-BLD-12): de CSP staat alleen script-src 'self' en
|
||||
// het Umami-script toe, zonder 'unsafe-inline'. Start zet in elke
|
||||
// geprerenderde pagina inline scripts (bootstrap, grensmarkering, scroll
|
||||
// herstel), die de browser daardoor zou blokkeren. De CSP blijft daarom
|
||||
// ongewijzigd; de scripts krijgen een eigen bestand, genoemd naar de hash van
|
||||
// hun inhoud.
|
||||
//
|
||||
// Eén scanner en één classificatie voor herschrijven en controleren (BR-SCR-01),
|
||||
// zodat de controle niet kan afwijken van het herschrijven. Geen
|
||||
// HTML-parser als afhankelijkheid: de uitvoer van Start is een bekende, kleine
|
||||
// vorm en alles wat de scanner niet aankan faalt hard in plaats van stil.
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
export interface MovedScript {
|
||||
// inline-<12 hex>.js; het bestand staat op /assets/<fileName>
|
||||
fileName: string;
|
||||
body: string;
|
||||
}
|
||||
|
||||
export interface ExternalizeResult {
|
||||
html: string;
|
||||
// Elke bestandsnaam één keer, op volgorde van eerste voorkomen
|
||||
moved: MovedScript[];
|
||||
}
|
||||
|
||||
export type ScriptErrorCode = 'UNSUPPORTED_SCRIPT' | 'UNTERMINATED_SCRIPT';
|
||||
|
||||
// De code laat de runner de fout als overtreding melden. Het bericht noemt een
|
||||
// regelnummer en nooit de inhoud van het script (BR-SCR-21).
|
||||
export class ScriptError extends Error {
|
||||
code: ScriptErrorCode;
|
||||
|
||||
constructor(code: ScriptErrorCode, detail: string) {
|
||||
super(`${code}: ${detail}`);
|
||||
this.name = 'ScriptError';
|
||||
this.code = code;
|
||||
}
|
||||
}
|
||||
|
||||
const ASSETS_DIR = 'assets';
|
||||
const HASH_LENGTH = 12;
|
||||
|
||||
// Typen die een browser als JavaScript uitvoert (BR-SCR-04), plus module.
|
||||
// Al het andere (ld+json, importmap, speculationrules) is een datablok.
|
||||
const JAVASCRIPT_TYPES = new Set([
|
||||
'application/javascript',
|
||||
'application/ecmascript',
|
||||
'application/x-ecmascript',
|
||||
'application/x-javascript',
|
||||
'text/javascript',
|
||||
'text/javascript1.0',
|
||||
'text/javascript1.1',
|
||||
'text/javascript1.2',
|
||||
'text/javascript1.3',
|
||||
'text/javascript1.4',
|
||||
'text/javascript1.5',
|
||||
'text/ecmascript',
|
||||
'text/jscript',
|
||||
'text/livescript',
|
||||
'text/x-ecmascript',
|
||||
'text/x-javascript',
|
||||
]);
|
||||
|
||||
// Elementen waarvan de inhoud tekst is: een script daarin is geen script
|
||||
// (BR-SCR-02)
|
||||
const RAW_TEXT_ELEMENTS = new Set([
|
||||
'style',
|
||||
'textarea',
|
||||
'title',
|
||||
'noscript',
|
||||
'xmp',
|
||||
'iframe',
|
||||
'noembed',
|
||||
'noframes',
|
||||
]);
|
||||
|
||||
type ScriptKind = 'external' | 'empty' | 'data' | 'executable';
|
||||
|
||||
interface ScriptElement {
|
||||
start: number;
|
||||
end: number;
|
||||
// Alles tussen de tagnaam en de sluitende '>', letterlijk (BR-SCR-06)
|
||||
attributeText: string;
|
||||
body: string;
|
||||
// Heeft het element een eindtag met '>'?
|
||||
terminated: boolean;
|
||||
kind: ScriptKind;
|
||||
module: boolean;
|
||||
asyncOrDefer: boolean;
|
||||
}
|
||||
|
||||
interface StartTag {
|
||||
name: string;
|
||||
attributes: Map<string, string>;
|
||||
attributeStart: number;
|
||||
close: number;
|
||||
}
|
||||
|
||||
const isWhitespace = (char: string | undefined) =>
|
||||
char === ' ' ||
|
||||
char === '\t' ||
|
||||
char === '\n' ||
|
||||
char === '\f' ||
|
||||
char === '\r';
|
||||
|
||||
const isLetter = (char: string | undefined) =>
|
||||
char !== undefined && /[A-Za-z]/.test(char);
|
||||
|
||||
// Leest een starttag vanaf de '<'. De eerste waarde van een attribuut telt,
|
||||
// zoals in een browser. Geeft null als de tag niet eindigt (een aanhalingsteken
|
||||
// of de tag loopt door tot het einde van de tekst): een browser gooit die weg.
|
||||
function readStartTag(html: string, lt: number): StartTag | null {
|
||||
const length = html.length;
|
||||
let pos = lt + 1;
|
||||
while (
|
||||
pos < length &&
|
||||
!isWhitespace(html[pos]) &&
|
||||
html[pos] !== '/' &&
|
||||
html[pos] !== '>'
|
||||
) {
|
||||
pos++;
|
||||
}
|
||||
const name = html.slice(lt + 1, pos).toLowerCase();
|
||||
const attributeStart = pos;
|
||||
const attributes = new Map<string, string>();
|
||||
|
||||
for (;;) {
|
||||
while (pos < length && (isWhitespace(html[pos]) || html[pos] === '/')) {
|
||||
pos++;
|
||||
}
|
||||
if (pos >= length) return null;
|
||||
if (html[pos] === '>') {
|
||||
return { name, attributes, attributeStart, close: pos };
|
||||
}
|
||||
|
||||
// Het eerste teken hoort altijd bij de naam, ook als het een '=' is
|
||||
const nameStart = pos;
|
||||
pos++;
|
||||
while (
|
||||
pos < length &&
|
||||
!isWhitespace(html[pos]) &&
|
||||
html[pos] !== '/' &&
|
||||
html[pos] !== '>' &&
|
||||
html[pos] !== '='
|
||||
) {
|
||||
pos++;
|
||||
}
|
||||
const attributeName = html.slice(nameStart, pos).toLowerCase();
|
||||
|
||||
let probe = pos;
|
||||
while (isWhitespace(html[probe])) probe++;
|
||||
let value = '';
|
||||
if (html[probe] === '=') {
|
||||
probe++;
|
||||
while (isWhitespace(html[probe])) probe++;
|
||||
const quote = html[probe];
|
||||
if (quote === '"' || quote === "'") {
|
||||
const closing = html.indexOf(quote, probe + 1);
|
||||
if (closing === -1) return null;
|
||||
value = html.slice(probe + 1, closing);
|
||||
pos = closing + 1;
|
||||
} else {
|
||||
const valueStart = probe;
|
||||
while (
|
||||
probe < length &&
|
||||
!isWhitespace(html[probe]) &&
|
||||
html[probe] !== '>'
|
||||
) {
|
||||
probe++;
|
||||
}
|
||||
value = html.slice(valueStart, probe);
|
||||
pos = probe;
|
||||
}
|
||||
}
|
||||
if (!attributes.has(attributeName)) {
|
||||
attributes.set(attributeName, value);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const endTagPatterns = new Map<string, RegExp>();
|
||||
|
||||
// Zoekt de eindtag van een tekstelement: naam, dan witruimte, '/' of '>' (of
|
||||
// het einde van de tekst). Met de g-vlag zodat lastIndex de startplaats is.
|
||||
function endTagPattern(name: string): RegExp {
|
||||
let pattern = endTagPatterns.get(name);
|
||||
if (!pattern) {
|
||||
pattern = new RegExp(`</${name}(?=[\\s/>]|$)`, 'gi');
|
||||
endTagPatterns.set(name, pattern);
|
||||
}
|
||||
return pattern;
|
||||
}
|
||||
|
||||
function classify(
|
||||
attributes: Map<string, string>,
|
||||
body: string,
|
||||
): { kind: ScriptKind; module: boolean } {
|
||||
if (attributes.has('src')) return { kind: 'external', module: false };
|
||||
if (body.trim() === '') return { kind: 'empty', module: false };
|
||||
const type = (attributes.get('type') ?? '').trim().toLowerCase();
|
||||
if (type === 'module') return { kind: 'executable', module: true };
|
||||
if (type === '' || JAVASCRIPT_TYPES.has(type)) {
|
||||
return { kind: 'executable', module: false };
|
||||
}
|
||||
return { kind: 'data', module: false };
|
||||
}
|
||||
|
||||
// Loopt één keer van links naar rechts door de tekst en levert de
|
||||
// script-elementen (BR-SCR-02, 03). Commentaar, tekstelementen en
|
||||
// attribuutwaarden worden overgeslagen, zodat tekst die op een script lijkt
|
||||
// nooit meetelt.
|
||||
function scanScripts(html: string): ScriptElement[] {
|
||||
const found: ScriptElement[] = [];
|
||||
const length = html.length;
|
||||
const scriptEnd = /<\/script(?=[\s/>])/gi;
|
||||
let pos = 0;
|
||||
|
||||
while (pos < length) {
|
||||
const lt = html.indexOf('<', pos);
|
||||
if (lt === -1) break;
|
||||
const next = html[lt + 1];
|
||||
|
||||
if (html.startsWith('<!--', lt)) {
|
||||
// Vanaf lt + 2, zodat <!--> en <!---> zoals in een browser sluiten
|
||||
const closing = html.indexOf('-->', lt + 2);
|
||||
pos = closing === -1 ? length : closing + 3;
|
||||
continue;
|
||||
}
|
||||
if (next === '!' || next === '?' || next === '/') {
|
||||
const closing = html.indexOf('>', lt + 2);
|
||||
pos = closing === -1 ? length : closing + 1;
|
||||
continue;
|
||||
}
|
||||
if (!isLetter(next)) {
|
||||
pos = lt + 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
const tag = readStartTag(html, lt);
|
||||
if (!tag) break;
|
||||
const bodyStart = tag.close + 1;
|
||||
|
||||
if (tag.name === 'script') {
|
||||
scriptEnd.lastIndex = bodyStart;
|
||||
const endTag = scriptEnd.exec(html);
|
||||
let body: string;
|
||||
let end: number;
|
||||
let terminated: boolean;
|
||||
if (!endTag) {
|
||||
body = html.slice(bodyStart);
|
||||
end = length;
|
||||
terminated = false;
|
||||
} else {
|
||||
body = html.slice(bodyStart, endTag.index);
|
||||
const closing = html.indexOf('>', endTag.index);
|
||||
terminated = closing !== -1;
|
||||
end = terminated ? closing + 1 : length;
|
||||
}
|
||||
const { kind, module } = classify(tag.attributes, body);
|
||||
found.push({
|
||||
start: lt,
|
||||
end,
|
||||
attributeText: html.slice(tag.attributeStart, tag.close),
|
||||
body,
|
||||
terminated,
|
||||
kind,
|
||||
module,
|
||||
asyncOrDefer:
|
||||
tag.attributes.has('async') || tag.attributes.has('defer'),
|
||||
});
|
||||
pos = end;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (RAW_TEXT_ELEMENTS.has(tag.name)) {
|
||||
const pattern = endTagPattern(tag.name);
|
||||
pattern.lastIndex = bodyStart;
|
||||
const endTag = pattern.exec(html);
|
||||
pos = endTag ? endTag.index : length;
|
||||
continue;
|
||||
}
|
||||
|
||||
pos = bodyStart;
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
// Een relatieve importspecifier ('./x', '../x') zou na het verplaatsen tegen
|
||||
// /assets/ oplossen in plaats van tegen de pagina
|
||||
const RELATIVE_IMPORT =
|
||||
/\bfrom\s*['"`]\.{1,2}(?:\/|(?=['"`]))|\bimport\s*\(?\s*['"`]\.{1,2}(?:\/|(?=['"`]))/;
|
||||
|
||||
function lineOf(html: string, index: number): number {
|
||||
return html.slice(0, index).split('\n').length;
|
||||
}
|
||||
|
||||
// Vormen die we niet veilig kunnen verplaatsen (BR-SCR-05): liever een
|
||||
// mislukte build dan stil ander gedrag als Start zijn uitvoer wijzigt.
|
||||
function unsupportedReason(script: ScriptElement): string | null {
|
||||
if (!script.module && script.asyncOrDefer) {
|
||||
return 'async of defer op een klassiek script';
|
||||
}
|
||||
if (script.body.includes('<!--')) {
|
||||
return 'commentaaropener in de scriptinhoud';
|
||||
}
|
||||
if (script.module && RELATIVE_IMPORT.test(script.body)) {
|
||||
return 'relatieve import in een module-script';
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function fileNameOf(body: string): string {
|
||||
const hash = createHash('sha256')
|
||||
.update(body, 'utf8')
|
||||
.digest('hex')
|
||||
.slice(0, HASH_LENGTH);
|
||||
return `inline-${hash}.js`;
|
||||
}
|
||||
|
||||
// Vervangt elk uitvoerbaar inline script door een verwijzing naar een bestand
|
||||
// met de hash van de inhoud als naam. Volgorde, attributen en al het andere
|
||||
// blijven zoals ze waren; alleen src komt erbij (BR-SCR-06, 09).
|
||||
export function externalizeInlineScripts(html: string): ExternalizeResult {
|
||||
const scripts = scanScripts(html);
|
||||
|
||||
for (const script of scripts) {
|
||||
if (!script.terminated) {
|
||||
throw new ScriptError(
|
||||
'UNTERMINATED_SCRIPT',
|
||||
`script zonder eindtag op regel ${lineOf(html, script.start)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const moved: MovedScript[] = [];
|
||||
const seen = new Set<string>();
|
||||
let output = '';
|
||||
let copiedUntil = 0;
|
||||
|
||||
for (const script of scripts) {
|
||||
if (script.kind !== 'executable') continue;
|
||||
const reason = unsupportedReason(script);
|
||||
if (reason) {
|
||||
throw new ScriptError(
|
||||
'UNSUPPORTED_SCRIPT',
|
||||
`${reason} op regel ${lineOf(html, script.start)}`,
|
||||
);
|
||||
}
|
||||
const fileName = fileNameOf(script.body);
|
||||
if (!seen.has(fileName)) {
|
||||
seen.add(fileName);
|
||||
moved.push({ fileName, body: script.body });
|
||||
}
|
||||
output +=
|
||||
html.slice(copiedUntil, script.start) +
|
||||
`<script${script.attributeText} src="/${ASSETS_DIR}/${fileName}"></script>`;
|
||||
copiedUntil = script.end;
|
||||
}
|
||||
return { html: output + html.slice(copiedUntil), moved };
|
||||
}
|
||||
|
||||
// De inhoud van de uitvoerbare inline scripts die er nog staan, in
|
||||
// documentvolgorde. Datablokken zoals JSON-LD tellen niet mee.
|
||||
export function findExecutableInlineScripts(html: string): string[] {
|
||||
return scanScripts(html)
|
||||
.filter((script) => script.kind === 'executable')
|
||||
.map((script) => script.body);
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
import fc from 'fast-check';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import sitePages from '../src/sitePages';
|
||||
import { routePath } from '../src/test/arbitraries';
|
||||
import {
|
||||
NOT_FOUND_FILE,
|
||||
NOT_FOUND_PATH,
|
||||
pageFile,
|
||||
pagePath,
|
||||
} from './pageFiles';
|
||||
|
||||
describe('pageFile', () => {
|
||||
it.each([
|
||||
['/', 'index.html'],
|
||||
['/over-mij', 'over-mij.html'],
|
||||
['/soorten-fotoshoots', 'soorten-fotoshoots.html'],
|
||||
['/soorten-fotoshoots/newborn', 'soorten-fotoshoots/newborn.html'],
|
||||
[NOT_FOUND_PATH, NOT_FOUND_FILE],
|
||||
])('%s staat in %s', (path, file) => {
|
||||
expect(pageFile(path)).toBe(file);
|
||||
});
|
||||
|
||||
it('begint nooit met een slash en eindigt op .html (invariant)', () => {
|
||||
fc.assert(
|
||||
fc.property(routePath, (path) => {
|
||||
const file = pageFile(path);
|
||||
expect(file.startsWith('/')).toBe(false);
|
||||
expect(file.endsWith('.html')).toBe(true);
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('pagePath', () => {
|
||||
it.each([
|
||||
['index.html', '/'],
|
||||
['over-mij.html', '/over-mij'],
|
||||
['soorten-fotoshoots/newborn.html', '/soorten-fotoshoots/newborn'],
|
||||
])('%s hoort bij %s', (file, path) => {
|
||||
expect(pagePath(file)).toBe(path);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['404.html', 'de 404-pagina is geen pagina uit het register'],
|
||||
['assets/index-abc.js', 'geen html'],
|
||||
['sitemap.xml', 'geen html'],
|
||||
['over-mij.htm', 'andere extensie'],
|
||||
['over-mij', 'zonder extensie'],
|
||||
['.html', 'zonder naam'],
|
||||
['/over-mij.html', 'met voorste slash'],
|
||||
])('%s is geen paginabestand (%s)', (file) => {
|
||||
expect(pagePath(file)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('geeft index.html voor /index, dus /index is geen eigen pagina', () => {
|
||||
expect(pageFile('/index')).toBe('index.html');
|
||||
expect(pagePath(pageFile('/index'))).toBe('/');
|
||||
});
|
||||
});
|
||||
|
||||
describe('pageFile en pagePath samen', () => {
|
||||
it('geven elk pad uit het register terug (round-trip)', () => {
|
||||
for (const path of Object.keys(sitePages)) {
|
||||
expect(pagePath(pageFile(path))).toBe(path);
|
||||
}
|
||||
});
|
||||
|
||||
it('geven elk routepad terug (round-trip)', () => {
|
||||
fc.assert(
|
||||
fc.property(
|
||||
// /index valt samen met het bestand van / (zie hierboven)
|
||||
routePath.filter((path) => path !== '/index'),
|
||||
(path) => {
|
||||
expect(pagePath(pageFile(path))).toBe(path);
|
||||
},
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
it('geven aan de 404-pagina geen pad terug', () => {
|
||||
expect(pageFile(NOT_FOUND_PATH)).toBe(NOT_FOUND_FILE);
|
||||
expect(pagePath(NOT_FOUND_FILE)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,26 @@
|
||||
// Koppelt een paginapad aan het HTML-bestand in dist/client en terug. Eén
|
||||
// afbeelding voor vite.config.ts (prerender), de script-runner en de
|
||||
// preview-server, zodat ze nooit van mening verschillen over waar een pagina
|
||||
// staat (BR-BLD-03).
|
||||
|
||||
export const NOT_FOUND_PATH = '/404';
|
||||
export const NOT_FOUND_FILE = '404.html';
|
||||
|
||||
const EXTENSION = '.html';
|
||||
|
||||
// '/' wordt index.html, elk ander pad het pad zonder voorste slash plus .html
|
||||
// (geen mappen en geen redirect naar een slash: autoSubfolderIndex staat uit)
|
||||
export function pageFile(path: string): string {
|
||||
if (path === '/') return `index${EXTENSION}`;
|
||||
return `${path.replace(/^\//, '')}${EXTENSION}`;
|
||||
}
|
||||
|
||||
// Omgekeerd: undefined voor 404.html en voor alles wat geen paginabestand is.
|
||||
// Let op: '/index' geeft index.html, en dat is het bestand van '/'.
|
||||
export function pagePath(file: string): string | undefined {
|
||||
if (file === NOT_FOUND_FILE || !file.endsWith(EXTENSION)) return undefined;
|
||||
const name = file.slice(0, -EXTENSION.length);
|
||||
// Bestandsnamen zijn relatief aan dist/client, nooit met voorste slash
|
||||
if (name === '' || name.startsWith('/')) return undefined;
|
||||
return name === 'index' ? '/' : `/${name}`;
|
||||
}
|
||||
@@ -0,0 +1,312 @@
|
||||
import fc from 'fast-check';
|
||||
|
||||
// Generators for HTML built from known pieces, each with a model of what it
|
||||
// is. The script step (config/inlineScripts.ts) is tested against the model,
|
||||
// never against its own scanner.
|
||||
|
||||
export type HtmlPieceKind =
|
||||
| 'executable'
|
||||
| 'external'
|
||||
| 'empty'
|
||||
| 'data'
|
||||
| 'comment'
|
||||
| 'rawText'
|
||||
| 'attributeDecoy'
|
||||
| 'markup';
|
||||
|
||||
export interface HtmlPiece {
|
||||
kind: HtmlPieceKind;
|
||||
html: string;
|
||||
// Executable pieces only: the script body and the attribute text between
|
||||
// the tag name and the closing '>'
|
||||
body?: string;
|
||||
attributes?: string;
|
||||
}
|
||||
|
||||
export interface HtmlModel {
|
||||
html: string;
|
||||
pieces: HtmlPiece[];
|
||||
// What sits between two pieces
|
||||
separator: string;
|
||||
}
|
||||
|
||||
// Bodies never contain a script end tag or a comment opener (the step
|
||||
// refuses those on purpose). Several snippets contain '<' and '>' to show a
|
||||
// body is not parsed as markup. A small pool makes equal bodies likely, so
|
||||
// the "same body, same file" property gets exercised.
|
||||
const snippet = fc.constantFrom(
|
||||
'window.a=1;',
|
||||
'var x = "<b>";',
|
||||
'if (a<b && c>d) { f(); }',
|
||||
'/* comment */ g();',
|
||||
'document.currentScript.remove();/*$tsr-stream-boundary*/',
|
||||
'$RB=[];$RV=function(a){return a};',
|
||||
"const s = '</div>';",
|
||||
'const é = "ünï";',
|
||||
'requestAnimationFrame(function(){$RT=performance.now()});',
|
||||
);
|
||||
|
||||
const padding = fc.constantFrom('', ' ', '\n', '\r\n ', '\t');
|
||||
|
||||
// Non-empty after trimming, with the padding a minifier would leave out
|
||||
export const scriptBody: fc.Arbitrary<string> = fc
|
||||
.tuple(padding, fc.array(snippet, { minLength: 1, maxLength: 3 }), padding)
|
||||
.map(([before, core, after]) => before + core.join('\n') + after);
|
||||
|
||||
// Attribute text of an executable script, verbatim as it sits in the tag.
|
||||
// Quoted values may contain '>' and the tag may end in a slash.
|
||||
const executableAttributes = fc.constantFrom(
|
||||
'',
|
||||
' type="module"',
|
||||
' type="text/javascript"',
|
||||
' type=""',
|
||||
' type = "text/ecmascript"',
|
||||
" TYPE='Module'",
|
||||
' type=application/javascript',
|
||||
' data-tsr-stream-part=""',
|
||||
' id="_R_"',
|
||||
' class="a" nonce="x" data-n=1',
|
||||
' data-x="a>b"',
|
||||
" data-q='1 > 0'",
|
||||
' language="javascript"',
|
||||
' /',
|
||||
);
|
||||
|
||||
const tagCase = fc.constantFrom(
|
||||
{ open: 'script', close: 'script' },
|
||||
{ open: 'SCRIPT', close: 'SCRIPT ' },
|
||||
{ open: 'Script', close: 'sCrIpT\n' },
|
||||
);
|
||||
|
||||
const canonicalCase = fc.constant({ open: 'script', close: 'script' });
|
||||
|
||||
function executableOf(cases: typeof tagCase): fc.Arbitrary<HtmlPiece> {
|
||||
return fc
|
||||
.tuple(cases, executableAttributes, scriptBody)
|
||||
.map(([names, attributes, body]) => ({
|
||||
kind: 'executable' as const,
|
||||
html: `<${names.open}${attributes}>${body}</${names.close}>`,
|
||||
body,
|
||||
attributes,
|
||||
}));
|
||||
}
|
||||
|
||||
// Mixed tag-name case and end-tag spacing; the step turns these into
|
||||
// lowercase references
|
||||
export const executablePiece = executableOf(tagCase);
|
||||
|
||||
// Only `<script ...>body</script>`, so a moved script can be put back exactly
|
||||
const canonicalExecutablePiece = executableOf(canonicalCase);
|
||||
|
||||
const externalPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.constantFrom(
|
||||
'<script src="/a.js"></script>',
|
||||
'<script type="module" async src="/assets/index-abc.js"></script>',
|
||||
'<script src></script>',
|
||||
'<script src=""></script>',
|
||||
'<script defer src="/b.js">ignored();</script>',
|
||||
'<script SRC=/c.js></script>',
|
||||
)
|
||||
.map((html) => ({ kind: 'external' as const, html }));
|
||||
|
||||
const emptyPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.constantFrom(
|
||||
'<script></script>',
|
||||
'<script> \n </script>',
|
||||
'<script type="module"> </script>',
|
||||
'<script type="text/javascript">\t</script>',
|
||||
)
|
||||
.map((html) => ({ kind: 'empty' as const, html }));
|
||||
|
||||
const dataPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.tuple(
|
||||
fc.constantFrom(
|
||||
'application/ld+json',
|
||||
'application/json',
|
||||
'importmap',
|
||||
'speculationrules',
|
||||
'text/plain',
|
||||
'text/x-template',
|
||||
' APPLICATION/LD+JSON ',
|
||||
),
|
||||
fc.constantFrom(
|
||||
'{"a":1}',
|
||||
'{"imports":{"x":"/x.js"}}',
|
||||
'{"name":"a \\u003c/script\\u003e"}',
|
||||
'if (a<b) { f(); }',
|
||||
),
|
||||
)
|
||||
.map(([type, body]) => ({
|
||||
kind: 'data' as const,
|
||||
html: `<script type="${type}">${body}</script>`,
|
||||
}));
|
||||
|
||||
// Text that looks like a script but sits where the scanner must skip it
|
||||
const commentPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.constantFrom(
|
||||
'<!-- <script>x()</script> -->',
|
||||
'<!---->',
|
||||
'<!-->',
|
||||
'<!-- a -- b <script>y()</script> -->',
|
||||
'<!-- <script type="module">z()</script>\n-->',
|
||||
)
|
||||
.map((html) => ({ kind: 'comment' as const, html }));
|
||||
|
||||
const rawTextPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.constantFrom(
|
||||
'<style>.a::before{content:"<script>x()</script>"}</style>',
|
||||
'<STYLE>a{}</script><script>b()</script></STYLE>',
|
||||
'<textarea><script>x()</script></textarea>',
|
||||
'<title>a <script>x()</script></title>',
|
||||
'<noscript><script>x()</script></noscript>',
|
||||
'<xmp><script>x()</script></xmp>',
|
||||
'<iframe><script>x()</script></iframe>',
|
||||
)
|
||||
.map((html) => ({ kind: 'rawText' as const, html }));
|
||||
|
||||
const attributeDecoyPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.constantFrom(
|
||||
'<div title="<script>x()</script>">t</div>',
|
||||
"<a data-x='<script>y()</script>'>t</a>",
|
||||
'<meta content="a>b<script>z()</script>">',
|
||||
'<p data-code=<script>w()</script>>t</p>',
|
||||
)
|
||||
.map((html) => ({ kind: 'attributeDecoy' as const, html }));
|
||||
|
||||
// Complete markup with no script in it. Text pieces never end in '<', so a
|
||||
// piece cannot glue onto the next one.
|
||||
const markupPiece: fc.Arbitrary<HtmlPiece> = fc
|
||||
.constantFrom(
|
||||
'<!doctype html>',
|
||||
'<?xml version="1.0"?>',
|
||||
'<div class="a">text</div>',
|
||||
'<p>1 < 2 and 3 > 2</p>',
|
||||
'<br/>',
|
||||
'<link rel="stylesheet" href="/assets/a.css">',
|
||||
'<meta charSet="utf-8">',
|
||||
'plain text',
|
||||
'</p>',
|
||||
)
|
||||
.map((html) => ({ kind: 'markup' as const, html }));
|
||||
|
||||
// Everything the step must leave alone, whatever surrounds it
|
||||
export const decoyPiece: fc.Arbitrary<HtmlPiece> = fc.oneof(
|
||||
externalPiece,
|
||||
emptyPiece,
|
||||
dataPiece,
|
||||
commentPiece,
|
||||
rawTextPiece,
|
||||
attributeDecoyPiece,
|
||||
markupPiece,
|
||||
);
|
||||
|
||||
export const htmlPiece: fc.Arbitrary<HtmlPiece> = fc.oneof(
|
||||
{ arbitrary: executablePiece, weight: 3 },
|
||||
{ arbitrary: decoyPiece, weight: 4 },
|
||||
);
|
||||
|
||||
const canonicalHtmlPiece: fc.Arbitrary<HtmlPiece> = fc.oneof(
|
||||
{ arbitrary: canonicalExecutablePiece, weight: 3 },
|
||||
{ arbitrary: decoyPiece, weight: 4 },
|
||||
);
|
||||
|
||||
const separator = fc.constantFrom('', '\n', ' ');
|
||||
|
||||
function modelOf(pieces: fc.Arbitrary<HtmlPiece>): fc.Arbitrary<HtmlModel> {
|
||||
return fc
|
||||
.tuple(fc.array(pieces, { maxLength: 12 }), separator)
|
||||
.map(([list, join]) => ({
|
||||
html: list.map((piece) => piece.html).join(join),
|
||||
pieces: list,
|
||||
separator: join,
|
||||
}));
|
||||
}
|
||||
|
||||
// A document and what each part of it is
|
||||
export const htmlWithModel: fc.Arbitrary<HtmlModel> = modelOf(htmlPiece);
|
||||
|
||||
// The same, with executable scripts written as `<script ...>body</script>`
|
||||
// only, so that putting the moved bodies back restores the input exactly
|
||||
export const canonicalHtmlWithModel: fc.Arbitrary<HtmlModel> =
|
||||
modelOf(canonicalHtmlPiece);
|
||||
|
||||
// Independent complete fragments, for split-and-join properties: each one
|
||||
// parses the same alone as inside a larger document
|
||||
export const htmlFragments: fc.Arbitrary<HtmlPiece[]> = fc.array(htmlPiece, {
|
||||
maxLength: 10,
|
||||
});
|
||||
|
||||
// A build output for the runner: which expected pages exist and what is wrong
|
||||
// with them, plus files that do not belong. The test turns this into a file
|
||||
// map and computes the expected violations from the same description.
|
||||
export type BadScript = 'none' | 'unsupported' | 'unterminated';
|
||||
|
||||
export interface PageScenario {
|
||||
present: boolean;
|
||||
content: HtmlModel;
|
||||
// Already processed: references a moved script without executable ones
|
||||
reference: boolean;
|
||||
bad: BadScript;
|
||||
// The assembled address sits in the page text
|
||||
leakAddress: boolean;
|
||||
}
|
||||
|
||||
export interface ExtraFile {
|
||||
name: string;
|
||||
address: boolean;
|
||||
rootPath: boolean;
|
||||
}
|
||||
|
||||
export interface OutputScenario {
|
||||
// index.html, over-mij.html, soorten-fotoshoots/newborn.html, 404.html
|
||||
pages: PageScenario[];
|
||||
unexpected: string[];
|
||||
extras: ExtraFile[];
|
||||
sourceMap: boolean;
|
||||
sitemap: 'ok' | 'missing' | 'differs';
|
||||
}
|
||||
|
||||
const pageScenario: fc.Arbitrary<PageScenario> = fc.record({
|
||||
present: fc.boolean(),
|
||||
content: htmlWithModel,
|
||||
reference: fc.boolean(),
|
||||
bad: fc.constantFrom<BadScript>(
|
||||
'none',
|
||||
'none',
|
||||
'none',
|
||||
'unsupported',
|
||||
'unterminated',
|
||||
),
|
||||
leakAddress: fc.boolean(),
|
||||
});
|
||||
|
||||
const extraFile: fc.Arbitrary<ExtraFile> = fc.record({
|
||||
name: fc.constantFrom(
|
||||
'assets/app.js',
|
||||
'assets/app.css',
|
||||
'robots.txt',
|
||||
'data.json',
|
||||
'manifest.webmanifest',
|
||||
'feed.xml',
|
||||
),
|
||||
address: fc.boolean(),
|
||||
rootPath: fc.boolean(),
|
||||
});
|
||||
|
||||
export const outputScenario: fc.Arbitrary<OutputScenario> = fc.record({
|
||||
pages: fc.tuple(pageScenario, pageScenario, pageScenario, pageScenario),
|
||||
unexpected: fc.uniqueArray(
|
||||
fc.constantFrom(
|
||||
'extra.html',
|
||||
'soorten-fotoshoots/index.html',
|
||||
'a/b.html',
|
||||
),
|
||||
{ maxLength: 2 },
|
||||
),
|
||||
extras: fc.uniqueArray(extraFile, {
|
||||
selector: (file) => file.name,
|
||||
maxLength: 3,
|
||||
}),
|
||||
sourceMap: fc.boolean(),
|
||||
sitemap: fc.constantFrom('ok', 'ok', 'missing', 'differs'),
|
||||
});
|
||||
Reference in New Issue
Block a user