Commit Graph

3 Commits

Author SHA1 Message Date
null ee5186c1ee feat(build): audit what the site actually serves, in the build and on the wire
Everything here checked an input: the content check reads the data, the secret
scan reads the diff, the build reads the source. Nothing read the OUTPUT, which
is the only thing a visitor or a crawler ever sees. Two live defects made the
case: every page preloaded the wrong image for months, and eleven pages shipped
a run-on description. Both are plain in the built HTML and invisible in the
source.

Build mode is guard 15-built-html, after 10-build. Per page it requires exactly
one title, one non-empty description, one canonical equal to the site origin plus
the route, and one h1; JSON-LD that parses, with no FAQPage, which the owner
ruled out; no em dash and no U+FFFD; a preload naming the image the page actually
paints first; and no description that runs its short description into the next
sentence. Across pages it requires every internal link and every fragment to
resolve, the sitemap to list exactly the routes the site serves, and 404.html to
carry noindex and no canonical. It exits 2, not 0, when dist/ is missing or older
than the sources: auditing stale output is auditing nothing.

That also guards a specific hazard. react-helmet-async on React 19 does not
merge, so a second <SEO> anywhere on a page silently emits a second title and a
second canonical, and a search engine picks whichever it likes.

URL mode fetches every page in a live sitemap once per crawler user agent
(OAI-SearchBot, PerplexityBot, ClaudeBot, Googlebot, bingbot), requires HTTP 200
and identical bytes across agents, runs the same page rules, and reports any URL
without a lastmod. It is deliberately NOT wired into deploy.sh: a check that runs
after publication cannot stop it, and pretending otherwise is worse than not
having it. Run it after a deploy.

Proven by mutation, nine of them, each restored afterwards: a wrong canonical
(7 pages), a second h1 (4), FAQPage markup, an em dash in copy, a link to a route
that does not exist (18), a fragment that is not on its target page (7), the
preload keyed on the old attribute (19), the template title left in place giving
two titles (19), and the industry routes dropped from the route list (56). A
clean build audits clean, and URL mode passes against the local server as all
five crawlers.

Closes #228.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 04:51:08 -05:00
null 26136f4e5c security: the scanner never ran its private-key pattern, and verify never saw dist
Three faults in one guard, the only thing standing between a credential and a
pushed commit. Found while planning Batch 17; closes #235.

1. grep was called without -e, at both call sites. The private-key pattern
   starts with dashes, so grep read it as an option, exited 2, and the beside-it
   2>/dev/null threw the complaint away. A staged private key passed the
   pre-commit hook, and had since the rule was written. Proven in a scratch
   repository before and after.

2. The 2026-08-29 pattern, committed as-is in 7415e19, flagged two comment
   lines that exist to show the shape of a credential. verify was therefore red,
   and release.sh would have died after bumping package.json, package-lock.json
   and the Dockerfile, leaving a tree that pre-push then refuses.

3. verify never scanned dist/, though SECURITY_CHECKLIST.md has listed the
   bundle scan as a release check for months. A key can reach the bundle from an
   environment variable inlined at build time without ever being committed.

What changed, beyond -e: every pattern is compiled against empty input before
the scan and an unreadable one exits 2, because silence from a broken matcher
looks exactly like a clean tree. The NAME=value pattern now also catches quoted
values, which is how a real secret is usually written down and which it has
always missed. A line that must show a credential shape carries `secrets-ok:`
and a reason, which excuses that line alone and stays visible to review and to
grep. Guard 20-secrets runs --tracked and --built dist/.

Proven by mutation, per GUARDS.md: a staged PEM header, a bare API_KEY=, a
quoted API_KEY=, an exported secret and an AWS key id each exit 1; a ${VAR}
value, a <placeholder>, an excused line and a delete-only commit each exit 0; an
unreadable pattern exits 2 with content staged and with none; a key planted in
dist/ fails guard 20-secrets alone, masked in the report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 04:23:39 -05:00
Ripley 5c04195cb1 chore: adopt template scripts and git hooks, retire phase-versioning
Ten scripts from ~/.openclaw/Projects/Template, taken one at a time and
configured against this deployment rather than copied wholesale.

Configured, not just copied:
- check-env.sh   SPEC written from what server/index.js actually reads — 24
                 variables, each with the consequence of getting it wrong
- secrets.sh     plus this project's own shapes: a bare 60+ hex run, which is
                 how the Zoho WebToLead tokens leaked into four commits, and a
                 reCAPTCHA key shape as NOTED rather than a failure, because the
                 site key and the secret key are indistinguishable by shape
- status.sh      nebula / qn-website-dev
- healthcheck.sh /api/health, asserting 200 AND "status":"ok" AND "db":"ok".
                 The template probed /healthz, which does not exist here
- preflight.sh   https://qn.isnull.dev, no --auth — there are no accounts
- verify.sh      GUARD_DIR=scripts/verify.d, since this project has no test
                 runner and no typecheck for it to detect
- backup.sh      ENGINE block replaced for SQLite: better-sqlite3's online
                 .backup() inside the container, verified with PRAGMA
                 integrity_check before anything is renamed into place
- restore-check.sh  rewritten rather than configured — the template's is
                 pg_restore/psql end to end with no seam. Replays the dump from
                 SQL into a scratch database and times it

Three guards in scripts/verify.d, because verify.sh would otherwise detect
nothing and exit 2: the build, the tracked-tree secret scan, and a check that
every document carries a valid Status, Governs and Review trigger.

Every guard was proven to fail before being trusted, per GUARDS.md rule 1:
healthcheck against a 200 that is not this app, secrets against the real
historical leak replayed out of 033bdf6, doc-headers against both a missing
Review trigger and the Status word "Historical", restore-check against a
truncated dump, an empty database and a raised row floor.

pre-commit is ADAPTED, not the template's. That one runs `npx tsc --noEmit` and
`npx vitest run`; this project has neither, so unchanged it would refuse every
commit. It runs the secret scan and `npm run build`. Hooks are not activated by
this commit — `git config core.hooksPath .githooks` is a separate, per-clone act.

package.json: adds `verify`, and corrects the version to 0.9.3. It said 0.8.3
while the last four commits said batch 0.9.0 through 0.9.3 — the second drift of
the phase-versioning rule, which is retired in the following commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 01:18:20 -05:00