A 421M classifier in front of the judge

My most expensive Claude Code session this month cost $604.76. It ran for 120.9 hours, took 186 prompts and made 4,507 tool calls, and the analyzer I wrote for my own transcripts flagged it for 82 failed tool calls and an edit loop that repeated 12 times.

The analyzer is claudeinsights, the coaching half of claudecounter. It reads the JSONL transcripts Claude Code leaves in ~/.claude/projects, finds the patterns that burn money, and for the worst sessions asks a model what went wrong and what I should do differently. That last step is the expensive one. Every flagged session was a full claude -p call, whether the session was a disaster or just long.

Most flagged sessions are just long. I wanted something cheap to tell the two apart before paying for the prose.

A judge that never writes a word

Laya is a decision model from Convai Innovations: ModernBERT-large plus a small decision head, 421M parameters. You give it a state and typed questions, and one forward pass returns calibrated probabilities. It never generates text, so there is nothing to parse and nothing to hallucinate.

That is also its limit. Laya can score friction on a scale, or tell me the probability that I corrected the assistant. It cannot quote the correction, name the root cause, or write advice. So the design is a filter, not a replacement:

// escalate reports whether a session's Laya read warrants the text judge.
func escalate(s LayaScore) bool {
  return s.Friction >= 5 || s.PCorrection >= 0.5 || s.PLoop >= 0.5
}

Every flagged session gets four Laya questions in one batch: friction, first-prompt clarity, “did the user push back or re-ask”, and “is there an unproductive loop”. Sessions under the threshold stay local and cost nothing. Sessions over it go to claude -p, now pinned to Opus 5.5 at medium effort, for the quotes, root cause and advice. The scores shown are always Laya’s, so every session is measured on the same scale.

On two synthetic sessions it did what I wanted. A precise prompt followed by “thanks, looks good” scored friction 3 with a correction probability of 0.24 and stayed local. “fix it”, “no, that’s the wrong file again”, “you broke the build” scored friction 7 with 0.90 and escalated.

On the $604 session it scored friction 6, correction 0.75, loop 0.81, and escalated. The Opus 5.5 pass that followed cost $0.58 for three calls (the judgment, the CLAUDE.md miner and the action roll-up) and came back with the thing I actually needed: I never told it what “done” meant, so I kept typing “did you push and create the MR?”

Laya reads 300 tokens

The English checkpoint has a 512-token context. After the question and the options, that leaves about 300 tokens for the session.

A 186-prompt session does not fit in 300 tokens. The digest I send puts the first prompt first, because Laya truncates from the end, and then as many later prompts as fit. For the $604 session that means the first prompt and roughly a dozen of the forty that survived the digest. The 4,507 tool calls are not in it at all.

It still escalated, because my prompts from those first hours were enough: “why so slow ?”, a pasted failing job URL, “solve the issues or add ignore”. A session that goes wrong quietly, with polite prompts over a thrashing tool stream, will not reach it. The deterministic analysis that runs before any model already catches exactly that case. Folding its loop finding into escalate is a one-line change I have not made yet.

The clarity score is the weaker signal. Laya rated that session’s first prompt 6 out of 10. Opus 5.5 said the same prompt was the root cause. I trust Laya to decide who escalates. I do not trust its clarity number.

46 seconds to load a model that answers in 30 milliseconds

The first version shelled out to the upstream PyTorch reference code that ships in the model repository, under uv run --with torch … so no Python ever lands in the repo. It worked. A warm batch took 77 seconds.

The time was not in the model. Per call it was 0.03 seconds on the GPU through MPS. Loading was 46.

Upstream builds ModernBERT from its config, which initialises 395M parameters with random weights, and then overwrites every one of them from the checkpoint with load_state_dict(strict=True). The random initialisation is pure waste. Transformers has a context manager for exactly this:

try:
    from transformers.initialization import no_init_weights  # transformers 5
except ImportError:
    from transformers.modeling_utils import no_init_weights  # transformers 4

with no_init_weights():
    agent = RLAgent(d, device="mps" if torch.backends.mps.is_available() else "cpu")

strict=True is what makes this safe: any weight the checkpoint did not fill would fail the load instead of silently staying uninitialised. The answer on a test question was identical on MPS and CPU, 0.7969, before and after. The load went from 46 seconds to 3.8. The batch went from 77 to under 10.

Letting Go own the download

The next question was how much of this Go could do on its own. There is no ModernBERT for Go today, but go-huggingface from the GoMLX project covers the two ends: a Hub client and tokenizers.

The Hub client uses the same ~/.cache/huggingface/hub layout as Python’s huggingface_hub. That matters because the checkpoint was already on disk, 2.2 GB of blobs, from the Python runs:

func fetchLaya(ctx context.Context, repo *hub.Repo) (string, error) {
  paths, err := repo.DownloadFilesCtx(ctx, layaFiles...)
  if err != nil {
    return "", fmt.Errorf("laya download: %w", err)
  }
  return filepath.Dir(paths[0]), nil // layaFiles[0] sits at the repo root
}

The first Go fetch took 285 milliseconds and downloaded nothing. The blobs stayed at 2.2 GB, and it resolved the same snapshot Python’s refs/main points at. It did need that one request: go-huggingface keeps its own repository metadata next to the blobs, and Python never writes it. After that, the test points the repo at a dead endpoint, http://127.0.0.1:1, so any network request fails the lookup, and it passes.

Go now fetches the checkpoint and hands the snapshot directory to the Python runner, which dropped huggingface_hub and its per-call freshness checks. A warm batch went from 9.7 seconds to 5.0.

The tokenizer, id for id

Running Laya in Go means tokenizing in Go, so that was the part worth proving now. go-huggingface’s hftokenizer reads a plain tokenizer.json, including the byte-level BPE ModernBERT uses. I generated reference ids with Hugging Face’s Rust tokenizer (0.22.2) and compared them to Go’s on inputs shaped like the real ones: a question header, a score option, a JSON state, a pasted URL with an em dash, and a line of Hindi with an emoji.

All six match exactly. Two things had to be right first.

Go’s tokenizer adds [CLS] and [SEP] by default, the way transformers does. Laya’s build_sequence adds those itself around separately tokenized pieces, so the comparison runs with AddSpecialTokens: false.

The seventh sample did not match. In "… does not hold [MASK] true: yes", Go keeps a space token (209) before the second [MASK] and Hugging Face does not. ModernBERT declares [MASK] with lstrip: true, which tells the tokenizer to swallow the whitespace in front of it. go-huggingface v0.4.12 parses the field and never applies it.

Laya never hits this. build_sequence inserts mask_token_id directly and replaces any literal [MASK] in the input with a space, so the text it tokenizes never contains one. The test keeps the sample anyway, with a comment and a log line for the day the flag starts working. It is the kind of mismatch that would have cost me an afternoon in a full port, where I would have been staring at probabilities that were close but not equal.

What it cost

Two things.

go-huggingface requires Go 1.27, so the whole module moved from 1.25. CI builds with stable, which is 1.27 now, so releases still work. A few TUI dependencies picked up minor upgrades on the way.

The binary grew from 4.8 MB to 7.4 MB, for a Hub client and a tokenizer. For a tool that already shells out to a Python environment with torch in it, I can live with that.

What still runs in Python

The model. Go downloads the checkpoint and has a tokenizer that agrees with Hugging Face on every input Laya sees. The forward pass is still PyTorch, launched by uv run, with the upstream code from the model repository.

The path to all-Go is visible now. laya-mlx is an independent port that runs the whole model in Apple’s MLX with no torch at all, at around 13 ms a decision, and laya-js calls MLX from JavaScript over FFI and reports results bit-identical to that port. Go can reach MLX the same way, through cgo and MLX’s C API. What is missing is ModernBERT and Laya’s decision head written against it, plus ports of build_sequence and the temperature scaling, which are small next to the model and sit on top of the tokenizer that already works.

The tokenizer was the part I expected to go wrong. It matched on every real input, and failed only on a flag that the model’s own preprocessing never uses.