Makes Lighthouse performance report-only in CI
Lighthouse slows the CPU 4x relative to the measuring machine, so the Raspberry Pi runner scores performance 30-40 points lower than a PC for the same build. In CI the performance assertions now only warn; accessibility, best practices and SEO still block, and performance still blocks in `pnpm check` on a PC. Mobile and desktop reports go to separate folders and are attached to every run as an artifact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -140,11 +140,23 @@ jobs:
|
||||
# --no-sandbox: als root weigert Chrome anders te starten
|
||||
# --disable-dev-shm-usage: /dev/shm is in een container maar 64 MB; met
|
||||
# de grote hero-foto's crasht het tabblad anders (TARGET_CRASHED)
|
||||
# Performance op de Pi is niet representatief (CPU-vertraging bovenop een
|
||||
# trage machine): hier alleen melden, blokkerend in `pnpm check` op een pc.
|
||||
# Toegankelijkheid, Best Practices en SEO blokkeren wel.
|
||||
- name: Lighthouse
|
||||
env:
|
||||
LIGHTHOUSE_CHROME_FLAGS: '--no-sandbox --headless=new --disable-dev-shm-usage'
|
||||
LIGHTHOUSE_PERFORMANCE: warn
|
||||
run: pnpm lighthouse
|
||||
|
||||
- name: Upload Lighthouse-rapporten
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: lighthouse-reports
|
||||
path: web/lighthouse-reports/
|
||||
retention-days: 14
|
||||
|
||||
deploy-test:
|
||||
needs: [check, lighthouse]
|
||||
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_test == 'true') || (github.event_name == 'push' && github.ref == 'refs/heads/master')
|
||||
|
||||
@@ -678,3 +678,27 @@ Verder heb ik de vraag ook beantwoord"
|
||||
**Context**: OPERATIONS - Deployment Setup; fixing CI
|
||||
|
||||
---
|
||||
|
||||
## Deployment Setup - CI Run 529 and Lighthouse Question
|
||||
|
||||
**Timestamp**: 2026-10-18T23:13:48Z
|
||||
|
||||
**User Input**: "Wat is de melding \"GitHub token not set\" in de lighthouse job?"
|
||||
|
||||
**AI Response**: "Explained: lhci healthcheck warning (not a failure) because Gitea sets GITHUB_* compat env vars and an upload target is configured; lhci can only post GitHub status checks. Left as is. Run 529: --disable-dev-shm-usage fixed the crash; Lighthouse ran on all 7 pages; accessibility, best practices and SEO passed; performance failed everywhere (0.49-0.66 vs 0.82-0.95 locally). Cause: Lighthouse simulates 4x CPU slowdown relative to the host; local benchmarkIndex 4192, the Pi is far slower, so perf is not representative. Created operations/plans/lighthouse-ci-questions.md (A: perf report-only in CI, blocking locally; B: calibrate CPU multiplier; C: no Lighthouse in CI)."
|
||||
|
||||
**Context**: OPERATIONS - Deployment Setup; awaiting decision
|
||||
|
||||
---
|
||||
|
||||
## Deployment Setup - Lighthouse Performance Report-only in CI
|
||||
|
||||
**Timestamp**: 2026-10-18T23:22:33Z
|
||||
|
||||
**User Input**: "Ja mee eens. Optie A. Kan jij die optie in het bestand bij het antwoord toevoegen en het dan uitvoeren? Dan kan je daarna meteen comitten en pushen. Ik heb de master branch protected gemaakt en de workflow check onderdeel als vereiste ingesteld om te slagen. Zodra je gepusht hebt laat het even weten dat zet ik auto merge aan zodat ik zsm de workflow met deploy kan laten runnen ."
|
||||
|
||||
**AI Response**: "Filled answer A. lighthouserc.cjs: performance assertion level from LIGHTHOUSE_PERFORMANCE (warn in CI, error otherwise), exported for the desktop config; reports to lighthouse-reports/mobile and /desktop (ignored by git, prettier, eslint). Workflow sets LIGHTHOUSE_PERFORMANCE=warn and uploads lighthouse-reports as artifact (if: always()). Verified: config levels per env; full pnpm lighthouse with warn exits 0 with 7+7 HTML reports. Docs updated: NFR-P-01, deployment-plan D6, performance-test-instructions. Committing and pushing; owner will enable auto-merge on PR #3 (master protected, check required)."
|
||||
|
||||
**Context**: OPERATIONS - Deployment Setup; fixing CI
|
||||
|
||||
---
|
||||
|
||||
+2
-2
@@ -16,8 +16,8 @@ pnpm lighthouse # mobile (lighthouserc.cjs), then desktop (lighthouserc.deskto
|
||||
```
|
||||
- Lighthouse CI starts `pnpm preview` itself (port 4173, real security headers).
|
||||
- It measures all six routes plus `/bestaat-niet` (404) once per form factor.
|
||||
- It fails if any assertion fails.
|
||||
- Reports are written to `.lighthouseci/` (git-ignored). Open the `.html` files for details.
|
||||
- It fails if any assertion fails. Exception: in CI the workflow sets `LIGHTHOUSE_PERFORMANCE=warn`, which makes Performance report-only, because the Raspberry Pi runner gives scores 30-40 points lower than a PC for the same build. Locally, Performance blocks.
|
||||
- Reports are written to `lighthouse-reports/mobile` and `lighthouse-reports/desktop` (git-ignored). Open the `.html` files for details. In CI they are attached to the run as the `lighthouse-reports` artifact.
|
||||
- Requires a local Chrome.
|
||||
|
||||
Manual measurement in a browser: `pnpm build && pnpm preview`, then open `http://localhost:4173` in a **private window** (extensions and Brave Shields skew results) and run Lighthouse from DevTools.
|
||||
|
||||
+1
-1
@@ -30,7 +30,7 @@ Text alternative: `pnpm check` runs lint, tests (including property-based and ac
|
||||
|
||||
| ID | Requirement | Measure | Source |
|
||||
|---|---|---|---|
|
||||
| NFR-P-01 | Lighthouse Performance ≥ 90, mobile and desktop, on every route. **Exception (temporary): home page on mobile ≥ 80**, until responsive images come from the API/CMS and prerendering exists (measured 0.81–0.82). | `pnpm lighthouse` against `pnpm preview` | Q3 = B; code change Q1 = B |
|
||||
| NFR-P-01 | Lighthouse Performance ≥ 90, mobile and desktop, on every route. **Exception (temporary): home page on mobile ≥ 80**, until responsive images come from the API/CMS and prerendering exists (measured 0.81–0.82). | `pnpm lighthouse` against `pnpm preview`. **Blocking locally** (`pnpm check` on a PC). **Report-only in CI**, because the Raspberry Pi runner is too slow for representative performance scores (Lighthouse slows the CPU 4x relative to the host); reports are attached to every run. | Q3 = B; code change Q1 = B; Lighthouse CI question A |
|
||||
| NFR-P-02 | No layout shift from shell elements: images have `width`/`height`, fonts are self-hosted | Lighthouse CLS ≤ 0.1 | BR-SEO-10 |
|
||||
| NFR-P-03 | The shell adds no network requests beyond bundled assets | No third-party origins in the network log | NFR-05 |
|
||||
| NFR-P-04 | Router devtools are not in the production bundle | Build output does not contain the devtools chunk | SECURITY-09 |
|
||||
|
||||
@@ -34,7 +34,7 @@ Text alternative: feature branches go through a pull request with CI into master
|
||||
| D3 | **Environments**: test and production. | Q5 = B |
|
||||
| D4 | **Test deploy**: automatically on every push to `master` (incl. a merged PR), and manually from any branch with "Deploy to Test", for example to try a branch before merging. | Q6 = B, CQ1 |
|
||||
| D5 | **Production deploy**: manual only ("Deploy to Production"), and only from a `release/*` branch; the workflow refuses other branches. | Q6 = B, CQ1 |
|
||||
| D6 | **Quality gate**: every run does lint, format, tests, build, audit and Lighthouse (mobile + desktop) before any deploy job can start. | Q7 = A |
|
||||
| D6 | **Quality gate**: every run does lint, format, tests, build, audit and Lighthouse (mobile + desktop) before any deploy job can start. In CI, Lighthouse **Performance is report-only** (the Pi runner gives scores 30-40 points too low); accessibility, best practices and SEO block. Performance blocks in `pnpm check` on a PC. HTML reports are attached to every run as the `lighthouse-reports` artifact. | Q7 = A; Lighthouse CI question A |
|
||||
| D7 | **Chrome for Lighthouse**: Playwright's Chromium (supports linux-arm64), installed in the job with a pinned Playwright version. Not the browser on the host: jobs run in containers and cannot see it. | CQ5 |
|
||||
| D8 | **Test target**: Raspberry Pi with nginx behind the reverse-proxy Pi, **next to the current WordPress site**. Deploy over SSH (`sshpass` + `scp`, same secrets as your other projects). Simple: empty the test folder, then copy the new build in; no old versions are kept. The step refuses an empty path, `/`, or a folder containing WordPress (`wp-config.php` / `wp-content`). The nginx block has its own `server_name` and never `default_server`. | CQ4 = B; owner: simpler than other apps, must not conflict with WordPress |
|
||||
| D9 | **Test is never indexed**: `X-Robots-Tag: noindex, nofollow` and a `robots.txt` that disallows everything. | Derived |
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
# Lighthouse in CI — vraag
|
||||
|
||||
## Situatie (run 529)
|
||||
- Lighthouse draait nu op alle 7 pagina's op de runner. **Toegankelijkheid, Best Practices en SEO halen hun doelen** (100).
|
||||
- **Performance faalt overal**: 0,49–0,66 op de Raspberry Pi, tegen 0,82–0,95 op een pc. Het is dezelfde build.
|
||||
- **Oorzaak**: Lighthouse simuleert een middelmatige telefoon door de CPU **4× te vertragen ten opzichte van de meetmachine**, en gaat ervan uit dat die meetmachine een snelle desktop is (pc: benchmarkIndex ≈ 4200). Een Pi is zelf al veel trager, en daar komt die 4× bovenop. Lighthouse waarschuwt zelf dat performance op zwakke hardware niet representatief is.
|
||||
|
||||
## Vraag 1: Wat doet Performance in de pipeline?
|
||||
A) In CI **Performance alleen melden** (waarschuwing in de log, met de rapporten als download bij de run). Toegankelijkheid, Best Practices en SEO blijven **blokkerend**. De blokkerende Performance-controle blijft in `pnpm check` op je pc, waar de meting wel klopt. **(aanbevolen)** De CI blijft betrouwbaar en vangt toegankelijkheid en SEO toch op.
|
||||
B) Performance in CI blijft blokkerend, maar de CPU-vertraging wordt aangepast aan de snelheid van de Pi (minder dan 4×), zodat de meting dichter bij een pc komt. Nauwkeuriger te maken, maar de juiste vertraging moet ik met een paar runs afstemmen, en een Pi onder wisselende belasting (andere builds) geeft wisselende scores. Kans op af en toe een onterecht rode pipeline.
|
||||
C) Lighthouse helemaal uit de pipeline, alleen lokaal in `pnpm check`
|
||||
X) Anders (beschrijf hieronder na de [Answer]:-tag)
|
||||
|
||||
[Answer]: A
|
||||
@@ -26,3 +26,4 @@ dist-ssr
|
||||
# Testdekking en Lighthouse CI
|
||||
coverage
|
||||
.lighthouseci
|
||||
lighthouse-reports
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
**/routeTree.gen.ts
|
||||
.lighthouseci
|
||||
lighthouse-reports
|
||||
coverage
|
||||
dist
|
||||
pnpm-lock.yaml
|
||||
|
||||
@@ -11,6 +11,7 @@ export default tseslint.config(
|
||||
'dist',
|
||||
'coverage',
|
||||
'.lighthouseci',
|
||||
'lighthouse-reports',
|
||||
'.agents',
|
||||
'.claude',
|
||||
'.junie',
|
||||
|
||||
+12
-4
@@ -21,8 +21,15 @@ const notFoundPath = '/bestaat-niet';
|
||||
// afbeeldingen uit de API/CMS en prerendering (NFR-P-01).
|
||||
const homeMobilePerformance = 0.8;
|
||||
|
||||
// Performance hangt af van de snelheid van de meetmachine: Lighthouse vertraagt
|
||||
// de CPU 4x ten opzichte van een snelle desktop. Op de CI-runner (Raspberry Pi)
|
||||
// valt de score daardoor 30-40 punten lager uit dan op een pc. In CI daarom
|
||||
// alleen melden (LIGHTHOUSE_PERFORMANCE=warn); lokaal in `pnpm check` blokkerend.
|
||||
const performanceLevel =
|
||||
process.env.LIGHTHOUSE_PERFORMANCE === 'warn' ? 'warn' : 'error';
|
||||
|
||||
const quality = {
|
||||
'categories:performance': ['error', { minScore: 0.9 }],
|
||||
'categories:performance': [performanceLevel, { minScore: 0.9 }],
|
||||
'categories:accessibility': ['error', { minScore: 1 }],
|
||||
'categories:best-practices': ['error', { minScore: 1 }],
|
||||
};
|
||||
@@ -40,6 +47,7 @@ const escape = (path) => path.replace(/[.*+?^${}()|[\]\\/]/g, '\\$&');
|
||||
const exact = (paths) => `^${origin}(${paths.map(escape).join('|')})$`;
|
||||
|
||||
module.exports = {
|
||||
performanceLevel,
|
||||
indexedPaths,
|
||||
noindexPaths,
|
||||
notFoundPath,
|
||||
@@ -66,7 +74,7 @@ module.exports = {
|
||||
assertions: {
|
||||
...quality,
|
||||
'categories:performance': [
|
||||
'error',
|
||||
performanceLevel,
|
||||
{ minScore: homeMobilePerformance },
|
||||
],
|
||||
'categories:seo': ['error', { minScore: 1 }],
|
||||
@@ -88,9 +96,9 @@ module.exports = {
|
||||
],
|
||||
},
|
||||
upload: {
|
||||
// Resultaten blijven lokaal in .lighthouseci/
|
||||
// HTML- en JSON-rapporten per meting; in CI als download bij de run
|
||||
target: 'filesystem',
|
||||
outputDir: '.lighthouseci',
|
||||
outputDir: 'lighthouse-reports/mobile',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
@@ -17,11 +17,18 @@ module.exports = {
|
||||
...home,
|
||||
assertions: {
|
||||
...home.assertions,
|
||||
'categories:performance': ['error', { minScore: 0.9 }],
|
||||
'categories:performance': [
|
||||
base.performanceLevel,
|
||||
{ minScore: 0.9 },
|
||||
],
|
||||
},
|
||||
},
|
||||
...rest,
|
||||
],
|
||||
},
|
||||
upload: {
|
||||
...base.ci.upload,
|
||||
outputDir: 'lighthouse-reports/desktop',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user