I had six months of AI coding sessions on disk and no idea what was in them.

Arithmetic on one side, judgement on the other.

Everything else in this project follows from a single line drawn down the middle of it.

Anything countable, groupable, sortable or extractable happens in Python, deterministically, with no model involved. Anything requiring taste happens in a model, reading only small shaped windows that the Python produced. If I am adding something, the first question is which side of the line it goes on.

That is not a performance optimisation. It is the thing that makes the output specific instead of generic. A model that has read a fair sample of six months says different things than one that has read last Tuesday, and a model that never sees raw transcript text cannot quietly start pattern-matching on the last thing it read.

There is a hard rule attached: no transcript text ever enters a model's context except through a bounded read. Not rarely. Never. The moment you let an agent cat a transcript "just to see", every judgement it makes afterwards is polluted by material it did not choose, and you have rebuilt the problem you were trying to solve.

One file, 1,800 lines, standard library only.

The install is a copy. There is no pip, no package, no virtualenv, no network call anywhere in it. That is deliberate: this thing reads your entire private working history, so you should be able to audit the one file that touches it before you run it. A dependency tree makes that impossible in practice.

It reads both Claude Code (~/.claude) and Codex (~/.codex) history, normalises both onto one small dict, and makes a single streaming pass over the corpus. On mine that is 2.1 GB in about sixteen seconds. It writes ten JSONL slices to disk, each one a different question about the same history:

Then there are the aggregates, which are free and where most of the real findings actually come from. Cost by month, tools by error rate, the hours you work, what every session pays to start before you type a word, and the command shapes you have retyped by hand. On my corpus the arithmetic alone found that I had typed PYTHONPATH=subsystems in front of commands 1,010 times because one repo had no packaging file, and rebuilt the same eleven-line remote environment preamble across 1,240 ssh calls.

Neither of those needed a language model to notice. They needed someone to run the counts and look.

$ transcript-audit scan
  scanned 2094 files (2134 MB) in 16s

$ transcript-audit stats --what shape
bucket                 sessions   share  human turns    tools
-------------------------------------------------------------
did work (>=5 tools)       1271   61.0%         4589    67206
thin (1-4 tools)            122    5.9%          145      316
no tool calls at all        691   33.2%          710        0

! 691 session records made zero tool calls (33% of the corpus).
  They are in every per-session denominator unless you exclude them.

That last warning exists because it caught me. A third of what called itself a session in my corpus did no work at all, and one of my findings inverted completely once I filtered them out.

Nine ways of being interested in the same history.

Once the Python has done the counting, the skill seats a set of independent subagents on different corners of the evidence. Each one is a person with an appetite, not a checklist. That distinction matters more than it sounds: a seat that wants something finds things that look bright to it and goes past everything else, and two seats reading the same slice come back with genuinely different findings.

So I do not tell a seat what to find. It gets the appetite, the material, and the shape of the return. The ideas are its own. If I specify the conclusions I have written a search query with extra steps, and I will get back exactly what I already believed, dressed up as discovery.

The coroner is the one I argued about most while building it. The first version was a full session-autopsy tool, and I killed it. If a session went badly, the user was there. The frustration is already encoded in their memory and they do not need an AI to parrot it back at them. What they cannot see from inside any one session is recurrence: the same death happening for the fourth time across three projects. So the seat only ships a finding if the failure class repeats.

A document, not a dashboard.

The report is one self-contained HTML file with no external requests. I designed it as a forensic exhibit list rather than a set of analytics cards, and there is one idea running through the type: judgement sets in a serif, evidence sets in monospace. The two faces are the two halves of the system. A model wrote the sentence in serif; arithmetic produced the line in mono. You can tell at a glance which one you are reading.

Every finding is an exhibit. The rail on the left carries the filing metadata, the claim sits in one measure of prose, and the receipts are always visible underneath it. That last part is not a style choice. The entire claim of this tool is that a finding carries evidence, so hiding the evidence behind a disclosure triangle would argue the opposite of the product.

You approve what you want, skip the rest, and hit copy. That produces a small JSON blob you paste back to the agent, and one follow-up message edits the CLAUDE.md, installs the hook, writes the script. Everything you skip is dropped silently. No follow-up, no nagging.

There is a validator in front of the renderer, and it is the most important twenty lines in the file. It refuses to write a report from a finding with no evidence, no source on that evidence, or no literal payload on something actionable. The model only ever produces the findings JSON; Python renders the HTML. That is what makes the gate enforceable rather than aspirational, and it means the tool mechanically cannot hand you the "be more specific in your prompts" blog post.

The first real run found five false things, and that is the interesting part.

I built the thing, then ran it end to end on my own 2,094 transcripts and made the agent face me with the result. Twenty-two findings reached the draft. The falsifier cut five outright, weakened thirteen, and passed four clean. I approved six of the nine actionable items.

What I did not expect was how the false ones failed. Almost all of them were the instrument lying, not the model hallucinating:

• A truncated field. The tool stored the first 240 characters of an agent's completion claim. A seat then computed "only three of eighteen completion claims cite any evidence" over that window. The real median length of those messages was 2,381 characters, so it was reading about a tenth of each one and confidently reporting on the rest.
• A poisoned denominator. A third of my session records did no work at all. One finding claimed my use of delegation had collapsed in July. Filtered to real sessions, it had gone up, from 36% to 41%.
• A normaliser mistaken for a bug. A seat reported a one-character path typo costing 41 dead tool calls. The tool collapses digits to N when it normalises command shapes, so Media V2 renders as Media VN. There was no typo.
• A grep that could not prove absence. One finding said my most-repeated law was written down nowhere. It survived the adversary and I approved it. It died when the agent opened the file to apply it: the law was already there, worded differently, and the search had been for the wrong phrase.

Every one of those is now fixed in the tool rather than papered over in the prompt. Clipped fields carry their true length beside them. There is a stats --what shape that counts the dead records and prints every rate per-session and per-human-turn, because those two numbers disagreeing is the tell. And the seats are told, in the packet, that a lexical grep cannot establish absence.

The correction I liked most was one I made myself. The report told me I was paying a 43,000 token agent context to make what was functionally a single API call, 676 times. That was true, but its proposed fix was wrong: routing those through the CLI is deliberate, because it bills my subscription rather than an API balance, and the subscription is heavily subsidised. The waste was never the route. It was that each of those calls was loading a full CLAUDE.md, skills, hooks and MCP schemas to run a stateless extraction prompt. Stripping that took four flags and measured out at 30,289 tokens down to 6,618, a 78% cut, with the billing unchanged.

A checkbox could not have told me that. Which is worth remembering if you are building a tool that asks someone to approve things.

Stated plainly, because every tool has these and most pages hide them.

• It is a one-off, not a dashboard. It is built to be run occasionally, mine everything available, and hand you a decision surface. There is a diff --since that scores whether a previous run's changes actually moved anything, and that is the closest it gets to monitoring.
• No slice can see the contents of a file the agent read. So any question of the form "did it actually follow the instruction to read X first" is structurally unanswerable here. The docs say so rather than letting a seat infer it.
• The heuristics are hypotheses, not verdicts. Redirect scoring, vagueness, praise detection: all of it is regex, all of it is declared in one visible block at the top of the file, and every score is written onto the record so a reader can disagree with it. The praise slice in particular is weak, and I would rather say that than let someone build a finding on it.
• There are no parser fixture tests. The corpora it was built against are private, so the test is a selftest that parses the largest transcript on your machine and asserts the shape comes out right. An anonymised fixture corpus is the single most useful thing anyone could contribute.

MIT. Python 3.9+. Nothing else.

git clone https://github.com/JustOscarJ1/transcript-audit
cd transcript-audit
./install.sh          # or: powershell -File install.ps1

Then /transcript-audit in Claude Code. The scan takes a minute or two, the seats read their windows in parallel, and the report opens in your browser.

The Python layer also works entirely on its own if you just want the numbers and none of the agents:

python tools/transcript_audit.py stats --what costs       # tokens by month, cache hit rate
python tools/transcript_audit.py stats --what overhead   # what your sessions pay to start
python tools/transcript_audit.py stats --what automation # commands you keep retyping
python tools/transcript_audit.py stats --what shape      # read this before dividing by anything

github.com/JustOscarJ1/transcript-audit