We Had an AI Read Every Line of Cellarion Looking for Security Holes

Why we do this at all

Cellarion is where you keep your cellar, your bottle photos, your tasting notes and, if you use the AI connector, a door through which an assistant acts on your behalf. That is a lot of trust for a small open-source project, and trust is only worth something if it is checked. So every few months we read the codebase for security problems, fix what we find, and ship the fixes before we talk about them. This post is the talking-about-it part for the audit of September 2026.

The difference this time: an AI did the reading, and it read everything.

What "everything" meant

Cellarion is about 160,000 lines of code outside the tests — roughly 92,000 in the backend, 66,000 in the web app, plus the AI connector package and the image service. The last time a human had read all of it with security in mind was July; since then 271 commits and nearly 39,000 new lines had landed with only partial reviews. That is the honest starting point: a large, fast-moving codebase, most of it written with AI help, that had outgrown its last full review.

We split the backend into 23 slices of about 4,500 lines each, with a manifest per slice so nothing could be skipped or read twice. Each slice went to an AI reading agent with one shared brief: what counts as a finding, how to rate it, how to write it down. Seven more agents swept the whole tree for cross-cutting questions — who may call which route, where the AI connector and the web API drifted apart, which rate limits key on the wrong thing behind Cloudflare, which data a user's erasure forgets. Then seven slices of the web app, and a deep pass over the AI connector and its OAuth code with its 12,000 lines of tests read as the specification.

Forty reading sessions in all, up to seven running at once, each ten to thirty minutes. About twelve million tokens. Two working days, the 2nd and the 6th of September. Every agent wrote its notes to disk as it went, so a crashed session could be resumed rather than repeated, and every finding went into one ledger.

What it found

The ledger holds 304 entries: 1 critical, 6 high, 69 medium, 138 low and 90 informational. Every critical and high finding was verified by a person reading the code, fixed, released and deployed the same day it was found. Thirteen of the mediums are fixed too; the rest are being worked through in severity order, and the audit's last three phases — verifying the remaining candidates, running active tests against a local copy, and writing the consolidated report — are still ahead of us.

The one that mattered most

For accounts that had not yet created a cellar, one of the search tools used by AI assistants applied no cellar filter at all, and could have returned bottles belonging to other accounts. It was found at about a quarter to ten in the morning, verified by reading, fixed, released and live on cellarion.app by seven minutes past ten: about eighty minutes from finding to fix in production. We cannot prove whether anyone ever triggered it — the usage statistics at the time recorded which tools were called, not by whom. Changes made through the connector have always been logged per account; since this audit, reads of other people's private data are logged the same way.

The high-severity ones, in plain words

• A way to make our AI label-reading prompts far more expensive than they should be, through a quirk in how text was inserted into them. Four prompt builders had it; their siblings had been fixed a month earlier.
• A cellar import that could write sommelier-level drink-window data into the shared registry for anyone, not just curators.
• AI-assistant preview data stored without any budget, so one free account could have filled the disk.
• A rack resize that could keep the server busy for every user at once, because a shape calculation ran on the request before the request had been validated.
• An anonymous registration endpoint for the AI connector sitting under a body-size limit meant for something else.
• A demo-seeding script that would create a documented admin login on whatever database it was pointed at. We checked: it had never run against the live one. It now refuses to.

Our favourite finding, for what it says about the method: two medium-severity problems in the backup scripts that had been written that same morning. The audit reached them a few hours later and flagged them. The auditor audited the auditor.

The patterns behind the findings

Most of the 304 entries are not dramatic; they are the same handful of habits, found again and again. A sanitiser applied to one field but not its twin in the next file. The web API and the AI connector drifting apart on the same operation. Rate limiters keyed on an address that, behind Cloudflare, is Cloudflare's. Undo records that restored a snapshot older than the data it replaced. Scripts with no dry-run mode. None of these is exotic, and that is the point: an audit that reads everything catches the boring, repeated slips that delta reviews step over.

The older parts of the code held up well. Session handling, token rotation, the checks around who may touch a cellar — the things we had reviewed carefully in the spring — came back clean.

Why the model mattered

We use Claude for most of Cellarion's engineering, and this audit ran on Claude Fable 5.1. That is not a detail we could skip, because our first attempt at exactly this kind of audit, earlier in the summer with Fable 5, did not get off the ground: the model's safety measures treated a defensive, line-by-line security review of our own code as something it should not help with. We fell back to Sonnet for the reading, which works, but reads less deeply.

With Fable 5.1 the same request went through end to end. The safety measures are still there — they are what makes it the generally available version of the model — but in our case they now told the difference between attacking someone and checking your own house. We also measured what the change bought us. On a blind read of one 1,800-line file, Sonnet reported one medium and one low finding; Fable 5.1 reported two mediums, five lows and two notes, including an account-existence leak and a way to send arbitrary text by email that Sonnet had missed. From the second phase on, every slice ran on Fable 5.1.

The rules we set ourselves are the same ones a responsible disclosure would want: the findings stayed in a private ledger, every high-severity item was re-read by a person before anyone touched the fix, production was only ever checked read-only, and nothing was written about publicly until it was fixed. The model did not get in the way of any of that.

What it means for you

If you use cellarion.app, you already have every fix — they shipped as versions 1.197 through 1.202 over those two days, each with a short banner and a few seconds of interruption. If you self-host, update to v1.202.1 or later. Everything the audit changed is in the release notes and the open repository, as always, without exploit detail.

What is next

The remaining phases: verifying the lower-severity candidates one by one, active tests against a local copy of the system, and a consolidated report. We will write those up too. And we intend to make the full read a habit rather than a July-to-September gap: it took two days and cost less than a single missed finding would.

Questions, or something you think we missed? The forum is open, and security reports go to [email protected] — we answer, and we fix. Skål!