A guard is denominated in what it can see, and a new basis is taught to everything that reads one
10:57 pm AWST · f109df0
The second review of the Ringwood work found the blast guard I had just
written could not fire where it mattered. It compared 404-confirmed
exits against live rows, but the probe budget is a quarter of the
candidates, so on any yard over ~240 cars — 65% of the corpus — the
count could never reach the line: a rebuilt large site would have
retired a quarter of itself per run as 404-confirmed sales until the
arithmetic happened to round over. The guard is denominated in probes
answered now. A changed site looks like this: most of the yard vanished
from the index at once, and nine in ten answers said 404. A quiet
dealer's genuine clear-out is neither. Twenty answers is the floor
below which a share means nothing, and the soft-404 refusal shares the
test, since its old denominator had the same flaw.
The 'stock feed absence' basis was known to nothing that reads a basis:
integrity 3b would have gone red on the first such event, 3c would have
policed it not at all, and the dealer page would have labelled it
"dealer confirmed" — the exact word the code comment beside it says it
can never be. 0224 teaches all three, as inference with its evidence
attached and not a sale. The same migration stops the model tickers
replaying the 150 delisted events 0222 corrected to lapses (a lapsed
listing's delisted event is history, not a departure), teaches
advertised_days the feed's own published_at (152 of 152 Ringwood cars
now carry a listing date, median 70 days, where none did), and
withdraws 0223's indexKeepsSoldCars — the sweep's ceiling counts only
live rows a marker names, and the feed's retained sold records never
have one, so the relaxation bought nothing and weakened a guard.
the traversal instead of ending it, because on this shape absence from
a complete traversal is the whole delisting basis; a feed's sold
declarations are read only on passes allowed to conclude anything; and
a lapsed listing's page says its last price was observed before we
could no longer watch it, not that it is a current ask.
Two costs are written into DATA-QUALITY 50 rather than hidden: two
cars the feed lists as sold sit at pre-rebuild addresses the sweep can
never name and stay lapsed rather than hand-edited onto the tape, and
the first guard's unreachable arithmetic is recorded as the lesson it is.
Proof run after all of it: 10 pages, 152 of 152, nothing changed,
nothing inferred; the sold tape holds no Ringwood exit from today.
A destroyed address is a lapse, and a feed's own sold flag is the seller's words
10:28 pm AWST · 2f5a331
The review of the Ringwood extractor found two things I had backwards.
A 404 on every old address at once is evidence about the SITE, not the
car. The delist check read each one as confirmation a car was gone and
retired 150 pre-rebuild rows on it — thirteen of them cars still on the
yard under new addresses the old rows could not be linked to, and on
the sold tape by the half-hourly refresh as sales that never happened.
0222 re-records those 150 as lapses, 0208's own status for watching
that ended without the car being seen to go; the tape's next refresh
drops the thirteen. And the check now lapses rather than delists when
404s arrive above the soft-404 blast radius: no yard loses a quarter of
its live stock to 404s in one run unless the site changed.
On this shape no page-based rule can ever retire a sold car: its page
is a 200 shell that neither 404s nor says SOLD, and I had discarded the
one signal the site gives — the feed's own status flag and sold date.
That is the seller's words in a column rather than a title, so it now
goes to the sold sweep as a `status` marker with the site's date as the
reported time, under the same guard as a page retitled SOLD. The
extractor declares two facts about itself: its inline records are the
car's own document (detail-grade: they may relist a car and reprice it
without a shell fetch) and its detail pages carry no car (so the delist
check reasons from the feed's absence — three days of it, under the
blast guard, never as "sold" — instead of a parser-health gate that
could never be satisfied). A page of nothing but sold records no longer
ends the traversal early while counting as complete. A moved
advertisement is a ledger event (0223). "(No Badge)" is a placeholder,
not a trim.
Proof run after the fixes: 10 pages, 152 of 152 observed as detail
evidence, zero new rows, zero retirements, zero errors, no phantom
events; the source stands at 152 live, 150 lapsed, 23 delisted.
DATA-QUALITY 50: a rebuilt site keeps its cars behind an API, and a rebuilt address is still the same car
10:07 pm AWST · 6cdcfd7
Ringwood Nissan's 09-01 rebuild: the nextjs-stock-api recipe, the four
passes that brought its 152 cars back with every retirement confirmed by
the site's own 404s, the reused-row url move and why it lives in the
crawl, and the 22 VIN-less rows that cannot be re-identified across the
rebuild and are therefore left separate rather than merged by guesswork.
An advertisement that moved follows its car
09:57 pm AWST · 3399a05
observeListing resolves a listing by url, else by stock number with the
make and model agreeing — the same advertisement at a new address. It
keeps the row and never rewrites what the row already says, so the row
keeps its OLD url. Presence is keyed by url. When a site is rebuilt and
every address changes, a reused row therefore sits outside every
traversal's present set while its car is plainly advertised: the delist
check fetches the dead page, gets a 404, retires the car; the next index
read relists it. Ringwood Nissan, first pass after its rebuild: sixteen
cars, every pass, for ever.
The move lives in the crawl, not in observeListing, because only the
traversal knows the fact that decides it: the stored url is absent from
a COMPLETE present set while the observed one is in it. That is what
separates a moved advertisement from the other reason two urls share a
stock number — one car genuinely advertised at two live addresses, of
which the corpus holds 1,805 rows across 159 sources. A row whose stored
url is still present is never touched, so nothing ping-pongs; the
unique (source, url) index cannot collide, since a row at the observed
url would have been found by the lookup instead.
docs/research/dealer-platforms.md gains the rebuilt-iMotor recipe: the
Next.js shell, the same-origin /api/stock endpoint, the shell detail
pages, and the platform/config to point at it.
A dealer whose cars live behind its own JSON endpoint gets an extractor
09:45 pm AWST · c3a1773
Ringwood Nissan rebuilt itself on 2026-09-01: a Next.js app whose
sitemap lists marketing pages and offers but no cars, whose /stock page
is a client-rendered shell, and whose inventory arrives from
/api/stock?page=N — a paginated JSON document the page itself calls,
permitted by robots.txt, handed to every visitor. 164 live cars went
unwatched.
The nextjs-stock-api extractor makes that endpoint the source's index.
The ordinary crawl fetches it through politeFetch, discover() yields
every advertised car inline with a full spec block and pages by the
document's own counters, and run rows, raw captures, price changes and
complete-traversal delisting follow with no new machinery. Two honesty
rules fall out of the shape: detail pages return the same "No Cars
available" shell for a real car and a bogus slug alike, so extract()
never confirms a car from HTML — safe, because the crawl only confirms
delist candidates, cars a complete API traversal no longer lists; and
the API still returns cars it has sold, which are not advertised and
are not emitted — the crawl's own evidence rules retire them.
Dry-run against the live endpoint: 10 pages, 152 advertised cars of
168 records, every mapped field correct; the shell page is inert to
matches(), extract(), discover() and soldMarker(); the old-shape
listing URLs now 404, so the stale rows retire as confirmed delistings.
Health emails on change, not on state
09:14 pm AWST · 1f57339
The Health workflow failed — and GitHub emailed — on every six-hourly
run while any alert was open, so one dealer site rebuilding itself was
four emails a day until acknowledged, each reading like a fresh failure.
An alarm that repeats itself trains its reader to delete it.
health_run (0221) remembers each run's alert set, keyed by the alert's
shape (its text with the numbers blanked, so a collapse count drifting
from 174 to 170 is the same alert and a different source collapsing is
a new one). The run now fails only when an alert appears or clears;
steady state, however red, is quiet. Hand runs compare against the
scheduled baseline but never move it, so a laptop cannot silence the
next scheduled email by having "already said" it. A run that cannot
record itself falls back to alert-on-state, and a sentinel that cannot
run at all stays loud every time.
Proven end to end against a real alert: un-acknowledging Ringwood
Nissan produced ALERT and exit 1, the same state produced "still open"
and exit 0, re-acknowledging produced CLEARED and exit 1, and steady
green produced exit 0.