Six months of Claude Code, audited (demo)

A DEMONSTRATION REPORT ON SYNTHETIC DATA. Every transcript, quote and number below is invented, so that a real one does not have to be published to show you the shape of the output. A real report is rendered from your own history and never leaves your machine.

2026-07-25 · run fb3d99635e

1,847transcripts read1.6 GB, none of it into a context window

3,912instructions you typed38 projects

11.4Binput + cache tokensagainst 82M output

412sessions that did no workmedian 31k seed each

6 / 14 / 3findings cut / weakened / cleanafter adversarial review

verdict

Two things have to be said before anything else, because most of what looks like a trend in your history is not one. A third of what calls itself a session here did no work at all, and those records quietly poison every per-session average anyone might compute. And your corpus spans a client switch in April, so nothing measured across that line means what it appears to mean. Three findings in the draft of this report died on those two facts.

What survives is better than what died. Your delegated agent runs are the cleanest work in your history: 16.2 errors per thousand tool calls against 28.4 when you work by hand. Delegation is not something you need to be talked into. It is already the most reliable thing you do.

The expensive habit is invisible from inside any session. You are re-deriving the same eleven-line environment preamble on every remote call, 604 times since April, because nothing on disk carries it. That is not a discipline problem. It is a missing file, and it takes four minutes to write.

Holding1

02Holdinghigh

Your delegated runs are measurably the most reliable work in your history

Agent runs error at 16.2 per thousand tool calls and draw 3.1 interrupts per hundred sessions. Your hands-on sessions error at 28.4 and draw 22.7 interrupts, seven times the rate, on comparable volume. This also kills a plausible-looking story: the apparent jump in interrupts after April is an artifact, because the earlier client cannot record an interrupt at all and reports zero for all 244 of its sessions.

agent n=891: 16.2 err/1k, 3.1 interrupts/100 sessions | hands-on n=712: 28.4 err/1k, 22.7 int/100python over sessions.jsonl grouped by session-id prefix
0 interrupts recorded across all 244 sessions of the earlier client, which has no interrupt event in its formatstats --what shape

Costing you1

06Costing youmedium

Three percent of your sessions hold thirty-eight percent of your token spend

Fifty-one sessions ran eight hours or longer. They are 3.1% of session records and account for 38% of all tokens and 36% of all output. This is concentration, not waste: most of that volume is cache read at a fraction of the price, and the input-to-output ratio tracks context size rather than anything about how the session went. The point is that a very small number of sessions is where your budget lives, so anything that changes their shape is worth more than anything that changes the other 97%.

51 of 1,634 sessions at 8h or longer hold 38.0% of input+cache tokens and 36.2% of outputpython over sessions.jsonl bucketed by duration
the single longest ran 31.2 hours across 68 human turnsread --slice sessions --sample longest

Memory / CLAUDE.md1

04Memory / CLAUDE.mdmedium

You have written the same correction about unproven completion claims in five projects

Across five projects and four months you have typed some form of "show me it actually working" 31 times. It is not a communication problem at this point. It is a standing rule that has been waiting since February to be written down once, and every project pays for its absence separately.

don't tell me it's done, show me the command you ran and what it printedsession 4b81ce93 · 2026-03-02
did you actually look at it or are you reading the diff back to mesession e07a2d15 · 2026-05-27
31 corrections matching proof-demanding language across 5 projects and 4 monthsread --slice corrections --sample top

remedyPromote the retyped correction to a standing rule so packets stop re-declaring it~/.claude/CLAUDE.md

# Done means observed
Never write done, fixed, working or verified unless the same message contains the command you ran and its actual output, or the file and line you changed. Exit code 0 is not an observation. If the proving check is impossible or expensive, say so and label the work UNVERIFIED rather than letting silence stand in for a check.

The correction stops being retyped per project, and an unproven claim becomes an explicitly flagged exception instead of the default.

Automation1

01Automationhigh

604 remote calls rebuild the same environment inline because nothing on the box owns it

Every remote command reconstructs some combination of a directory change, two environment variables and a virtualenv path from memory. It has been typed 604 times across 47 sessions, and the four fragments appear in inconsistent combinations, which is why some queries silently read the wrong database. The 213 calls that are ordinary administration need nothing; it is the 391 database reads that want one entry point.

604 ssh commands across 47 sessions; 391 are ad-hoc reads against the same database filestats --what automation
preamble fragments re-typed: cd 402, DB env var 187, dotenv source 96, venv python 151python over commands.jsonl
just ssh in and check the queue depth again, I don't remember the incantationsession 7c1f4a20 · 2026-06-14

remedyOne wrapper on the remote host that owns the environment, so calls stop rebuilding itremote: /usr/local/bin/rq

#!/usr/bin/env bash
# rq - single entry point. Owns the environment so callers never rebuild it.
set -euo pipefail
ROOT=/opt/app
cd "$ROOT"
export APP_DB="${APP_DB:-$ROOT/data/app.db}"
export PYTHONPATH="$ROOT:$ROOT/lib${PYTHONPATH:+:$PYTHONPATH}"
PY="$ROOT/.venv/bin/python"; [ -x "$PY" ] || PY=python3

case "${1:-}" in
  sql)  shift; exec sqlite3 -header -column -readonly "$APP_DB" "$@" ;;
  py)   shift; exec "$PY" "$@" ;;
  env)  echo "DB=$APP_DB"; echo "PY=$PY" ;;
  *)    echo 'rq sql|py|env'; exit 2 ;;
esac

One definition of which database is canonical instead of four assembled per call, and the eleven-line preamble disappears from every future session.

Worth building1

03Worth buildinghigh

An import path is prepended by hand 738 times because the repo has no packaging file

Every script and test invocation in the main repository is hand-prefixed with the same path variable, 738 times across 96 sessions, and it is accelerating month over month. There is no pyproject, no conftest and no pytest config at the root, so nothing on disk makes the import resolve. The 44 import errors in the corpus are all the same mistake: the prefix was forgotten.

738 commands carrying the prefix across 96 sessions; 284 in May, 454 in Junepython regex over commands.jsonl
no pyproject.toml, conftest.py or pytest.ini at the repository rootls of the repo root
44 ModuleNotFoundError tool errors, all in sessions where the prefix was omittedregex classification over tool_errors.jsonl

remedyPut the path in the project settings env block, where every command inherits it.claude/settings.json

Merge this top-level key into the existing file:

  "env": {
    "PYTHONPATH": "/srv/app;/srv/app/lib",
    "PYTHONWARNINGS": "ignore::DeprecationWarning",
    "PYTHONIOENCODING": "utf-8"
  }

All 738 prefixes become unnecessary at once rather than only the test subset, and the import-error class stops recurring.

Retire1

05Retirelow

Three slash commands from a finished workstream still load every session

A batch-processing workstream ran heavily through May and stopped entirely on 3 June. Its three custom commands are still installed and still loaded. They have zero invocations in the eight weeks since, against 340 sessions in that window.

the workstream's commands appear 189 times in May, 0 times after 2026-06-03python over commands.jsonl grouped by month
0 invocations across the 340 sessions since 2026-06-03python over invocations.jsonl

remedyArchive the three commands out of the loaded surface~/.claude/commands/

mkdir -p ~/.claude/_archive/2026-05-batch
mv ~/.claude/commands/{batch-run,batch-retry,batch-report}.md ~/.claude/_archive/2026-05-batch/

The command list becomes an accurate picture of live work, and nothing is lost: the files are one directory away if the workstream restarts.

Approve what you want built. Everything you skip is dropped silently — no follow-up, no nagging. Notes ride along with the decision, so an amendment ("do this but only for the router") is enough; you do not have to restate the item.