Adds the third unit's code generation plan and Pi install instructions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+2
-2
@@ -85,7 +85,7 @@ mirror --reverse --delete --verbose dist/ $FTP_DIR/;
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Cur["Today: old SPA build and old nginx config"] --> Bak["H2: owner installs the server-ops draft with backup, nginx -t, reload"]
|
||||
Cur["Today: old SPA build and old nginx config"] --> Bak["H2: owner places the server-ops draft with backup, nginx -t, reload"]
|
||||
Bak --> Gap["Test serves the old build: only / and assets work, deep links 404"]
|
||||
Gap --> Dep["H3: run the workflow on the feature branch with Deploy to Test"]
|
||||
Dep --> Ver["verify-deployment.sh in the job, then the matrix of section 5 from the PC"]
|
||||
@@ -165,7 +165,7 @@ The IIS rules are unverified until a host exists (infrastructure-design.md secti
|
||||
|
||||
| Situation | Action |
|
||||
|---|---|
|
||||
| New nginx file rejected (`nginx -t`) or accepted but the matrix fails | The owner restores the `.bak-<timestamp>` file, `nginx -t`, reload, and redeploys the last good commit to test (H2/H3 rollback) |
|
||||
| New nginx file rejected (`nginx -t`) or accepted but the matrix fails | The owner restores the backup from `/mnt/storage1/backups/andreaslop-test-nginx/` (see pi-install-instructions.md), `nginx -t`, reload, and redeploys the last good commit to test (H2/H3 rollback) |
|
||||
| Bad `web.config` live on production | Upload the previous `web.config` by FTP, or redeploy the previous release (rollback-plan.md); before DNS no visitor is affected |
|
||||
| Post-deploy script red on test after a merge | The old nginx file is probably active: ask the owner to install the `server-ops` draft (H2), then run the workflow on `master` with "Deploy to Test" |
|
||||
|
||||
|
||||
+14
-19
@@ -195,6 +195,9 @@ Reasons for the non-obvious choices:
|
||||
- Security headers are repeated in every location (as now) and at server level; `always` makes them apply to 301, 404 and the 404 page (SECURITY-04). Whether the server-level `if` redirect carries them must be proven with a real nginx (BR-HST-24).
|
||||
- `gzip_http_version 1.0` makes compression work however the proxy talks to this block. `gzip_proxied any` covers requests that carry `Via`. `gzip_types` lists MIME types, so it includes both the types nginx's `mime.types` gives (`application/javascript`, `text/xml`) and the ones the preview server uses. It is generated from the same list as the preview server's `COMPRESSIBLE` set (BR-HST-11).
|
||||
- `if` is used only with `return` at server level, the one safe form.
|
||||
- **No map rule may match `/404` or `/404.html`.** `error_page`'s internal redirect runs the server-level `if` and `rewrite` again with `$uri=/404.html`; a matching rule would turn the 404 page into a redirect (P-HST-07 tests it).
|
||||
- **Headers per location.** `add_header` in a location replaces all server-level `add_header`. Every location with its own `add_header` repeats the whole set, and `/assets/` carries exactly one `Cache-Control`. Tested in the generated text (P-HST-07) and against a real nginx on 301, 404 and the 404 page (P-HST-09). `etag` stays on.
|
||||
- The default block of the Pi (`00-default.conf`) answers an unknown host with 444; the test block is chosen by `server_name`. The header comment of the generated file says so (the old remark about a fallback to WordPress is removed).
|
||||
|
||||
Caching (nginx): `/assets/*` `public, max-age=31536000, immutable`; everything else `no-cache` (HTML, `robots.txt`, `sitemap.xml`, `favicon.png`, logos). See decision O3 for why unhashed files are not long-cached.
|
||||
|
||||
@@ -309,7 +312,7 @@ Try: the generated `web.config` on a temporary URL, as in section 7.2, step 3.
|
||||
| BR-HST-14 | `deploy-production` uploads in three passes: (1) every file except `*.html` and `web.config`; (2) `web.config` and then all HTML files, including `404.html` and nested ones; (3) `mirror --delete`. The `--exclude-glob index.html` option disappears. A visitor can reach HTML only after every asset it refers to exists (FR-PR-18). The exact lftp exclusion syntax (glob against the relative path, or a regex) is proven locally with a dry run against a `file:` target (section 12, V10) |
|
||||
| BR-HST-15 | `web.config` is copied from `deploy/iis/web.config` to `dist/web.config`, next to `index.html` (the artifact root is `dist/`; `server/` is not in it). Before uploading, the job fails if `dist/index.html`, `dist/404.html` or `dist/over-mij.html` is missing, if `dist/server` exists, or if any `*.map` file is present (FR-PR-17, NFR-PR-SEC-03). Q1 decides whether `FTP_REMOTE_DIR` is `wwwroot` or the root |
|
||||
| BR-HST-16 | `deploy-test` keeps emptying the folder and copying `dist/*`. This is acceptable for a noindex test site: the gap is a few seconds of nginx's plain 404, no old build is kept, and the WordPress guard stays. The new `dist` contains `404.html` and nested folders; `scp -r` handles them |
|
||||
| BR-HST-17 | The nginx update on the Pi is a human task, done before the merge and proven by a branch deploy (H2, H3, deployment-architecture.md). A merge to `master` deploys automatically; with the old nginx config the new build would serve the home page for `/over-mij` (the old `try_files ... /index.html`). The post-deploy check (BR-HST-18) turns that case into a red job |
|
||||
| BR-HST-17 | The nginx update on the Pi is a human task (H2, [pi-install-instructions.md](pi-install-instructions.md)): config placed and reloaded, and the deploy right after (a manual branch deploy or the merge), proven by the checks of H3. A merge to `master` deploys automatically; with the old nginx config the new build would serve the home page for `/over-mij` (the old `try_files ... /index.html`). The post-deploy check (BR-HST-18) turns that case into a red job |
|
||||
| BR-HST-18 | One shared post-deploy script (`web/deploy/verify-deployment.sh <base-url> <dist-dir> [--test]`) used by both jobs. It fails the job unless: `/`, `/over-mij` and `/soorten-fotoshoots/newborn` return bytes identical to the matching files in `dist` (without `Accept-Encoding`, so an old nginx that serves the home page for `/over-mij` is caught); an unknown URL returns 404 with the bytes of `404.html`; `/over-mij/` returns 301 to `/over-mij` and `/Over-Mij` to `/over-mij`; all five security headers are on the 200, the 301 and the 404; `--test` adds `X-Robots-Tag`. A gzip request for the JavaScript bundle must return `Content-Encoding: gzip` on test; on production it only warns (webhost) |
|
||||
|
||||
## 10. Lighthouse targets (NFR-PR-P-01, P-02, S-01)
|
||||
@@ -401,29 +404,21 @@ curl -s -o /dev/null -D - -H 'Host: test.andreaslop.nl' -H 'Accept-Encoding: gzi
|
||||
|
||||
Read: `Accept-Encoding` or `gunzip` in the output means C or D. A gzip answer on the second command (once H2 is installed) means the hosting Pi side works; the public check is the matrix of deployment-architecture.md section 5.
|
||||
|
||||
**H2. Update nginx on the hosting Pi (before the merge; only after the owner's separate approval).**
|
||||
**H2. Update nginx on the hosting Pi (the owner places it, after separate approval).**
|
||||
|
||||
The Pi-Main session never uses sudo, and a change in `/etc/nginx` goes through a draft in the `server-ops` repository. Steps and owners:
|
||||
The exact steps, checks, expected results and rollback are in [pi-install-instructions.md](pi-install-instructions.md) (written by the Pi-Main session; it holds the full commands). Summary of what differs from earlier drafts of this design:
|
||||
|
||||
1. Code generation produces `web/deploy/nginx/andreaslop-test.conf` (`pnpm server-config`). The vhost lives at `/etc/nginx/conf.d/andreaslop-test.conf`, which `/etc/nginx/nginx.conf:38` includes inside `http {}` (no `sites-enabled`). The header comment of the generated file names this install path and states that the file needs no change to the global `nginx.conf`.
|
||||
2. The Pi-Main session delivers the generated file as a draft in `server-ops`. It does not install it.
|
||||
3. The owner reviews the draft and installs it on `web1`, with backup, test and reload:
|
||||
|
||||
```bash
|
||||
sudo cp /etc/nginx/conf.d/andreaslop-test.conf /etc/nginx/conf.d/andreaslop-test.conf.bak-$(date +%Y%m%d%H%M)
|
||||
sudo install -m 644 -o root -g root <draft file> /etc/nginx/conf.d/andreaslop-test.conf
|
||||
sudo nginx -t
|
||||
```
|
||||
|
||||
If `nginx -t` fails: `sudo cp` the `.bak-...` file back over `andreaslop-test.conf`, run `sudo nginx -t`, and stop. If it passes: `sudo systemctl reload nginx`. The backup ends in `.bak-<timestamp>`, so the `conf.d/*.conf` include ignores it. Do not touch the WordPress block.
|
||||
|
||||
Timing is unchanged: H2 happens before the merge, immediately followed by H3.
|
||||
1. The generated `web/deploy/nginx/andreaslop-test.conf` is delivered as a draft in the `server-ops` repository; the Pi-Main session never uses sudo.
|
||||
2. The backup goes to `/mnt/storage1/backups/andreaslop-test-nginx/andreaslop-test.conf.spa`, outside `conf.d` (a `*.conf` backup in `conf.d` would be loaded by nginx).
|
||||
3. The owner places the file as `root:root 644` at `/etc/nginx/conf.d/andreaslop-test.conf`, runs `sudo nginx -t && sudo systemctl reload nginx`; `sudo systemctl restart nginx` is the fallback if the reload does not take effect.
|
||||
4. Timing: place and reload the config, and start the deploy right after (a manual branch deploy from `feature/prerendering`, or the merge). Do not leave a day in between: the new config with the old build gives 404 on `/over-mij`, and the old config with the new build shows the home page there. A short mismatch is acceptable on test.
|
||||
|
||||
**H3. Prove it with a branch deploy, then merge.** In Gitea: Actions, Continuous Integration, Run workflow, branch `feature/prerendering`, tick "Deploy to Test". When the job is green (it now runs `verify-deployment.sh`), run the matrix of deployment-architecture.md section 5 from the PC. Only then merge the pull request. The merge deploys the same build again.
|
||||
|
||||
**H4. Optional, proxy Pi (the owner's).** Only if the proxy part of H1 or the public matrix shows the proxy side still blocks compression: edit `/etc/nginx/conf.d/test.andreaslop.nl.conf` in place (certbot owns it), add inside `location /`: `proxy_http_version 1.1;` and `proxy_set_header Connection "";`, then `sudo nginx -t && sudo systemctl reload nginx`. Not expected to be needed.
|
||||
**H4. Proxy Pi (the owner checks; changes are the owner's).** Use the table "Proxy Pi (192.168.1.102): the owner checks" in [pi-install-instructions.md](pi-install-instructions.md): `Accept-Encoding` overrides, `proxy_http_version`, `proxy_intercept_errors`/`error_page`, own `add_header`, `proxy_cache`, `proxy_redirect`. A proxy change (for example `proxy_http_version 1.1;` and `proxy_set_header Connection "";` in `location /`, edited in place because certbot owns the file) is made only if a check shows a problem. Not expected to be needed for compression, because the new block works over HTTP/1.0.
|
||||
|
||||
**Rollback of H2/H3 (the owner).** As in [pi-install-instructions.md](pi-install-instructions.md): copy `/mnt/storage1/backups/andreaslop-test-nginx/andreaslop-test.conf.spa` back over `/etc/nginx/conf.d/andreaslop-test.conf`, `sudo nginx -t && sudo systemctl reload nginx`, then deploy an old SPA build: create a branch at the last good commit (`git switch -c rollback/test c4c2f83 && git push -u origin rollback/test`) and run the workflow on it with "Deploy to Test". Test is noindex and has no visitors, so a few minutes of 404 is acceptable.
|
||||
|
||||
**Rollback of H2/H3 (the owner).** On the hosting Pi: `sudo cp /etc/nginx/conf.d/andreaslop-test.conf.bak-<timestamp> /etc/nginx/conf.d/andreaslop-test.conf && sudo nginx -t && sudo systemctl reload nginx`. Then redeploy the previous build: create a branch at the last good commit (`git switch -c rollback/test c4c2f83 && git push -u origin rollback/test`) and run the workflow on it with "Deploy to Test". Test is noindex and has no visitors, so a few minutes of 404 is acceptable.
|
||||
**H5 and H6 (IIS probe kit and rehearsal) are withdrawn**; the "Before go-live on IIS" list in section 7.2 replaces them. There is no host yet.
|
||||
|
||||
## 14. Decisions for the owner
|
||||
@@ -442,7 +437,7 @@ Decided by the owner and the main session on 2026-11-01. Ids are kept.
|
||||
| O8 | Real nginx from apt in CI, with parity tests against the preview server (P-HST-09) | Podman container (not feasible in the Podman runner); local only |
|
||||
| O9 | Locally the nginx test is skipped with a visible message; in CI a missing nginx fails the job | Requiring nginx everywhere |
|
||||
|
||||
Pi steps: the hosting Pi part of H1 is done. The Pi-Main session never uses sudo; the nginx change (H2) is delivered as a draft in `server-ops` and installed by the owner only after the owner's separate approval, before the merge. Any proxy change (H4) is the owner's too.
|
||||
Pi steps: the hosting Pi part of H1 is done and the install instructions exist ([pi-install-instructions.md](pi-install-instructions.md)). The Pi-Main session never uses sudo; the nginx change (H2) is delivered as a draft in `server-ops` and installed by the owner only after the owner's separate approval, before the merge. Any proxy change (H4) is the owner's too.
|
||||
|
||||
## 15. Compliance notes
|
||||
|
||||
|
||||
+89
@@ -0,0 +1,89 @@
|
||||
# nginx Install Instructions for the Test Host
|
||||
|
||||
Written for: the owner, who places the config. Source: the Pi-Main session on `web1.slpsoftware.nl` (192.168.1.103), 2026-11-01, read-only and without sudo. `nginx -t` was not run; that happens at installation.
|
||||
|
||||
The final commands with full paths go into the draft in the `server-ops` repository, for example `drafts/2026-11-xx-andreaslop-prerender/` with `README.md` and `andreaslop-test.conf`. The owner places it as `pi-main` with sudo, following that README and the checklist in `drafts/2026-10-20-hardening/README.md`. Nothing here has been carried out. The draft is requested separately, with the generated text.
|
||||
|
||||
## Checked on the server
|
||||
|
||||
| Point | Finding |
|
||||
|---|---|
|
||||
| `map` and `$andreaslop_canonical` | No clash. The only other map in `/etc/nginx` is `$connection_upgrade` in `nginx.conf`. All `conf.d` files are included in `http` (`nginx.conf` line 38), so a map at the top of the file is at http level |
|
||||
| `root` | `/var/www/html` is a symlink to `/mnt/storage1/www/html/`; `/var/www/html/andreaslop-test` exists (`webadmin:www-data`, 775). The owner confirmed `DEPLOY_PATH_TEST` is `/var/www/html/andreaslop-test`, which equals this `root` |
|
||||
| gzip and headers from other vhosts | gzip is on nowhere, and `nginx.conf` has no http-level `add_header`. gzip in this server block changes no other site |
|
||||
| Default block | Since 2026-10-21, `/etc/nginx/conf.d/00-default.conf` answers an unknown host with 444. The current comment that nginx would fall back to WordPress is out of date; correct it in the generator |
|
||||
| nginx 1.30.6 | Every directive in the sketch is standard there. `if` with only `return` at server level is safe |
|
||||
| `map` matching | Exact strings win over regexes, so `/over-mij ""` beats the variant regex and no loop arises |
|
||||
| `error_page` | Its internal redirect runs server-level rewrite and `if` again with `$uri=/404.html`. That matches no map rule. **Keep it so: no map rule may match `/404` or `/404.html`** |
|
||||
| Directories | `try_files $uri` matches files only, so `/assets` gives 404 |
|
||||
|
||||
## Installation
|
||||
|
||||
Order: place the config, reload, and start the deploy right after: manually from the branch, or by merging. Do not leave a day in between. The new config with the old build gives 404 on `/over-mij`, and the old config with the new build shows the homepage there. A short mismatch is acceptable on test.
|
||||
|
||||
```bash
|
||||
# Backup outside conf.d: a *.conf backup in conf.d would be loaded by nginx
|
||||
sudo install -d -o root -g root -m 700 /mnt/storage1/backups/andreaslop-test-nginx
|
||||
sudo cp -p /etc/nginx/conf.d/andreaslop-test.conf /mnt/storage1/backups/andreaslop-test-nginx/andreaslop-test.conf.spa
|
||||
|
||||
# Place (the current file is root:root 644)
|
||||
sudo install -o root -g root -m 644 /home/claude/server-ops/drafts/<folder>/andreaslop-test.conf /etc/nginx/conf.d/andreaslop-test.conf
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
If `nginx -t` fails, nginx does not reload. Put the backup back. Use reload first. Reload has not been tested separately on 1.30.6 on this Pi, because nginx was restarted after that update. If the new config does not work after the reload, `sudo systemctl restart nginx` is the fallback.
|
||||
|
||||
## Checks on the Pi
|
||||
|
||||
The Pi-Main session can run these too, without sudo. They need the new build deployed.
|
||||
|
||||
```bash
|
||||
for p in / /over-mij /over-mij.html /over-mij/ /index.html /bestaat-niet /404 /robots.txt; do curl -s -o /dev/null -w "$p %{http_code} %{redirect_url}\n" -H 'Host: test.andreaslop.nl' http://127.0.0.1$p; done
|
||||
curl -s -o /dev/null -D - -H 'Host: test.andreaslop.nl' -H 'Accept-Encoding: gzip' --http1.0 http://127.0.0.1/ | grep -iE '^(HTTP|content-encoding|vary|cache-control|content-security)'
|
||||
```
|
||||
|
||||
`--http1.0` is deliberate: that is how the proxy Pi talks to this Pi. Run the second command for one file in `/assets/` too.
|
||||
|
||||
Expected:
|
||||
|
||||
- `/` and `/over-mij` give 200.
|
||||
- `/over-mij.html`, `/over-mij/` and `/index.html` give 301 with a relative `Location`.
|
||||
- `/bestaat-niet` and `/404` give 404 with the content of `404.html`.
|
||||
- `/robots.txt` gives 200.
|
||||
- `Content-Encoding: gzip` and `Vary: Accept-Encoding` are present.
|
||||
- The security headers are also on the 404.
|
||||
- Assets have `immutable`.
|
||||
|
||||
Also check that `andreaslop.nl`, `gitea.slpsoftware.nl` and phpMyAdmin still give 200. Then repeat the checks on `https://test.andreaslop.nl/` from outside.
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
sudo cp -p /mnt/storage1/backups/andreaslop-test-nginx/andreaslop-test.conf.spa /etc/nginx/conf.d/andreaslop-test.conf
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
Then deploy an old SPA build.
|
||||
|
||||
## Proxy Pi (192.168.1.102): the owner checks
|
||||
|
||||
The proxy Pi handles TLS through certbot and forwards `test.andreaslop.nl` to `192.168.1.103:80`. The Pi-Main session has no access to it. Run `sudo nginx -T` and look for:
|
||||
|
||||
| Look for | Why |
|
||||
|---|---|
|
||||
| `proxy_set_header Accept-Encoding ""` or similar | The backend then never gets a gzip request, and gzip here has no effect |
|
||||
| `proxy_http_version` | When unset it is 1.0; `gzip_http_version 1.0` in the new block is there for this |
|
||||
| `proxy_intercept_errors` or `error_page` | The proxy would replace `404.html` with its own error page |
|
||||
| Own `add_header`, especially HSTS, CSP, X-Frame-Options, Cache-Control | Double or conflicting headers. Two CSP headers are both enforced |
|
||||
| `proxy_cache` | An old SPA `index.html` could stay cached for `/over-mij` |
|
||||
| `proxy_redirect` | Could change the relative 301s |
|
||||
|
||||
The proxy passes an already compressed response through unchanged. If the owner wants gzip for all sites, WordPress included, the proxy Pi is the place for that, and this block can keep its own.
|
||||
|
||||
## Points for the generated config
|
||||
|
||||
- `add_header` in a location replaces all server-level `add_header`. Every location with its own `add_header` repeats the whole set, certainly `/assets/` (`immutable`), or it loses CSP and HSTS. A location without `add_header` inherits. `/assets/` carries one `Cache-Control`, not a second one.
|
||||
- `etag` stays on (the default), so `no-cache` revalidates cheaply.
|
||||
- The deploy must empty the target folder, or a removed page stays at 200. The `deploy-test` step already does this with `find -delete`.
|
||||
- `location = /robots.txt` overrides the build's `robots.txt`, which is right for test. Production must serve the real `robots.txt` and `sitemap.xml`.
|
||||
- `gzip_proxied any` does little here, since the proxy sets no `Via`, but it does no harm. `listen 80` on IPv4 is enough.
|
||||
Reference in New Issue
Block a user