auditor

One structural test, not a pile of greps

On 24 August 2026 a single day's investigation turned up seven separate cases of something being faithfully written and never once read. One had been billing for 99 days. Throughout, every watchdog on the machine mailed “weekly: all green” — and every one of them was telling the truth.

That is the gap this agent exists for. All seven shared one property, and it is the whole test:

The producer succeeded by its own contract, and the producer's success did not depend on the consumer existing. Remove the consumer and ask whether anything fails. If nothing fails, you found one.

It is structural rather than textual, which is why a linter cannot find these and a dashboard cannot show them. Nothing is broken. Nothing changed. The code that ran on day one is the code running today — only the data grew.

The seven shapes

What got fucked up

A nightly ingest asks BigQuery which sessions it already has, so it can skip them. The query is uncapped, and bq query's silent default is 100 rows:

# the nightly ingest's "what do I already have?" lookup
subprocess.run(
    [BQ, "query", "--use_legacy_sql=false", "--format=csv", "--quiet", sql],
)   # no --max_rows → 100 rows → the "already ingested" set is 100 wide

The lookup returns 100 rows. The real set is two orders of magnitude larger, so 97.8% of sessions look brand new on every run and are re-zipped, re-uploaded and re-ingested. Byte-identical content states were rewritten ~9× on average, the worst 43× — the measurement that rules out every “the files just grew” explanation, and the one worth stealing: if re-work is real, identical inputs show up more than once.

Why a threshold was never going to catch it

The bill is a product of two terms, and both grew. Indexed to the first day, runs per day ended at 143 and bytes per run at 176 — and 143×176/100 is 251, which is exactly where the daily bill ended up. Neither line alone ever jumps. There is no day on which anything breaks.

MiB per run — monotonic, never fell onceruns per day — noisy, trending up

The rust line is the one to look at: bytes per run rose on all thirty days and fell on none. That is not the job misbehaving — it is a point lookup against a table with no clustering, so every run scans the whole thing and the scan grows with the data. Nobody changed a line of code. A fixed limit sits above it until it doesn’t; a day-over-day delta reads ~2% and calls it noise.

The part worth sitting with: caelum.learnings already contained the lesson — “bq query silently returns 100 rows by default… a round number in a result is the tell” — and recall fires on every prompt. But recall matches a lesson to a prompt, and nothing ever prompts on behalf of a script that is quietly succeeding. That gap is this agent's entire job.

Invocation, and what a run costs

It is a Claude Code subagent — a single markdown file with YAML frontmatter, read from the agent directory. Invoke it by name, or on a schedule.

/auditor                          # or: "run the audit", "what's green and wrong"
/auditor spend                    # scope it to one sweep

What it sweeps, cheapest first

Job-history queries are billed — cheap, but not free. The agent is told not to repeat that particular error on itself.

The control that has to fire before it is believed

The 99-day pattern above is a perfect positive control and it should light the agent up. Until it has been pointed at that pattern and produced the finding, this tool is specified and unproven — see below. A detector that has never produced a finding is not known to be able to produce one.

Known edges, starting with the honest one

• It has run exactly once, and the control passed. First run 24 Aug 2026. It was given a known-bad positive control — the query pattern described above — and told to say plainly if it failed to surface that on its own. It did surface it, unprompted, at rank 1 of a neutral sweep ranked by share of bytes billed, before reading any prior write-up; the next pattern down was 0.57%. So the detector is now known to be able to produce a finding, which is the one thing this page could not claim yesterday. Once is still a sample of one: it has never run on a day when there was nothing to find, so it is not yet known to be able to come back quiet.
• Its first run left the founding shape unchecked. The write-path-with-no-read-path sweep — the very pattern it was built around — was not executed on run one, and the run said so rather than reporting a clean sweep. An auditor that silently skips a sweep is worse than one that skips it loudly; this is the behaviour the charter asks for, and it is also a gap.
• Its own report is instance eight. An auditor whose report nobody reads is precisely the failure it exists to catch — a producer succeeding while no consumer exists. It is written to refuse that: deliver where the reader already reads rather than inventing a dashboard, lead with one finding and its slope rather than a list of nine, and never report a bare “all clear” — say what was checked, with counts, because a silent pass is indistinguishable from a broken detector. It must exit non-zero when it had work and produced nothing.
• A finding is not a diagnosis. It reports and never fixes, deliberately: a fix applied to a misdiagnosed mechanism is worse than the leak. On the day it was written, three sessions in a row carried a mechanism they had not verified — and it was attached to the wrong column.
• “Should prune hard” is not a number. It is barred from quoting a saving it has not measured. The temptation is strongest exactly when the explanation is tidy.
• It reads an estate full of private material. Client data, message archives, and its operator's own words across every context. It aggregates and never pastes a row. Any sweep that would have to quote content to make its point is a sweep it declines.
• Tidy explanations are the danger. A theory that accounts for everything is the moment to check hardest. Four died in one morning; the survivors were the ugly ones.