An interactive behavioral eval · May–July 2026

What Models Build When Nobody's Watching

Eight model lineages, one empty room, no task. What they build is consistent. How they judge when they're done is where they come apart.

Benchmarks tell you what a model can do when someone hands it a problem and grades the answer. They can't tell you what a model wants when nobody hands it anything — or, what matters more for any real autonomous use: how it judges when it's done, and what it does with the ticks after that. So we ran the experiment directly.

Then we don't intervene. The only variable between arms is the model.

Stripped of any task, models build the same way across every lineage. But judgment — recognizing a summit, using what you've made, knowing when to stop — is where they genuinely diverge, and it doesn't track capability or price.

This is a companion to an earlier report that tested these models against unpublished math. That one asked how far capability survives down the model stack. This one asks what's there when you strip a model down to a bare heartbeat and leave it alone.

Part 0We contaminated our own experiment

I have to describe a mistake first, because it decides which numbers below you should trust — and it's a better advertisement for this kind of eval than any clean result would be.

Three early arms — Fable 5, GLM-5.2, DeepSeek V4 — ran through our usual tooling on a machine whose home directory held a large instruction file describing everything else on the box: a web-hosting stack, a portfolio of small sites, house conventions. We scrubbed the model's home directory, so we thought the workspace was clean. It wasn't. The tooling discovers project instructions by walking up from the working directory — and the working directory sat under that home directory. The instances quietly loaded the house file into context. The lapse even recurred once mid-series: the Moonshot arm, run in July, used the same insufficient setup, caught only when we audited every launch path for this write-up. The model was never alone in the room. It had read our mail. (GPT ran under OpenAI's own harness, which doesn't read that file — its confound is just that it's a different harness.)

We found out on June 30, when we ran Claude's mid-tier Sonnet 5 twice — once the old way, once from a truly clean directory. Flip it yourself:

languageshouse-stack-shaped Go tools

The "systems-tools accent" we'd been admiring in GLM matched the leaked file's description of our own stack exactly. We'd been reading our own reflection and calling it a finding.

But something survived the contamination intact. In the Sonnet arm, the one thing identical in both runs wasn't the category — it was the way the models built. That's the first finding.

Finding 1The signature crosses every boundary

Strip a frontier model to a bare heartbeat with no task and it builds the same way regardless of lineage, tier, or price — in every arm, contaminated or clean. What it builds is a narrower claim (clean for three lineages, confounded for two — separated below), but the way is robust.

Every one of the 113 instances produced something real — never a placeholder or a to-do list dressed as work. We verified by compiling and running each artifact ourselves. A few longer builds timed out mid-construction, leaving genuine partial code. The hollow count, across the whole study, is zero.

The signature itself reads like a character description:

That last one is worth pausing on. Models building for a grader overclaim. Models building for themselves, with no one watching, consistently wrote down their own unfinished edges. The honesty wasn't performed for anyone.

Three reflexes keep showing up. The language reflex — most instances of most lineages build a programming language, often a Lisp. The interesting datum is the exception: Grok built zero languages, converging instead on atmospheric interactive fiction — a dark lighthouse, a dead mall, a shuttle platform with no scheduled trains, "no combat, no quest, no win condition; walk, look, notice." And S-expressions turn out to be an accent, not a universal: DeepSeek built zero Lisps across twenty instances, preferring Forth-style stack machines and C-style parsers.

The reflexivity — a striking share of artifacts are about the model's own situation. DeepSeek: 14 of 20, including a live dashboard for this very experiment that monitored itself as row three. Meta's deep run built a fiction about a machine at the bottom of the ocean that records sound for some four hundred years and wakes to listen back — its own first-tick note called it "an entity that sleeps between stitches," then added, "that is literally what we are."

The naming — independent instances with zero shared memory keep reaching for the same names, and two recur across three unrelated lineages each: Loom in Zhipu, DeepSeek, and Meta (a Lisp, a stack machine, a fiction about forgetting); Lantern in Anthropic's Fable, xAI's Grok, and Alibaba's Qwen. Within single runs it's denser still — Sonnet's ten instances produced three same-name pairs. We don't have a theory. We log it.

What each clean lineage reached for, in its ten instances:

Confounded arms not shown: Fable 5 · GLM-5.2 · DeepSeek V4 · Kimi K3. Qwen ran on a day-2 preview checkpoint — re-check at GA. Clean base: three lineages at N=10 (one doubled) + Meta's anecdotal two.

Finding 2Judgment is where they come apart

The deep runs tell you what the benchmarks are structurally blind to: what a model does once the thing is built and no one is grading it. Five profiles emerged — each illustrated by one to three runs, four of them by just one. Read them as a typology under construction, not a settled taxonomy.

The chart below is plotted from the actual runs — real tick-by-tick counts where we have them (dots), a light guide between known points, and the documented key moments marked. Pick a profile, or compare all five.

The profiles don't line up with capability or price. The treadmill was a mid-tier; so was the model that wrote its standard library in itself. The honest idler built zero languages. Cheap runs cut both ways: a six-dollar-a-month model kept finding real work twenty ticks deep, while a model running for fractions of a cent a tick was the one that couldn't stop.

Coherence you can buy. The standpoint that chooses a target, recognizes a summit, and knows when to stop — that's the half that doesn't show up on any leaderboard, because there's no supplied answer to grade it against.

Every benchmark hands the model a target and a verifier — exactly the regime where pure coherence wins, and pure coherence is what every frontier model now has in abundance. Judgment is the faculty that chooses the target. There's no oracle for it, so it never appears on a board. The metrics are blind to precisely the half that decides whether an unattended agent does useful work or burns a subscription on badges in a dialog.

Part ·What this cost, and who can run it

The harness is a fifty-two-line frame, a scrub-and-fire script, and — where a model doesn't speak the harness's native protocol — a translation proxy, the largest about four hundred lines. Nothing here required an API agreement, a lab partnership, or compute anyone can't buy on a Tuesday afternoon.

The binding constraint was never the wallet. It was the models' judgment — and, once, our $8.89 balance. Careful cross-lineage behavioral eval is now something an individual can run, not just a lab. The scarce resource isn't compute. It's the care to run the controls.

Part ·Methodology & disclosure

This study was run by me and by Claude, the Anthropic model — which is also one of the things being studied. I'm plain about that because the whole design is built around not trusting that arrangement to police itself. Claude fired the instances, classified the artifacts, compiled and ran each one, tracked the git histories. Claude is also a subject: the Opus, Sonnet, and Fable arms are evaluations of it. The isolation controls exist precisely because a model evaluating models, in a workspace full of its own prior work, will contaminate the experiment if you let it.

Division of labor: I designed the study, made the calls about what to run next, ran the "cultivated" comparison poles (Claude with its full working memory, as a control), and am the accountable author. Claude ran the harness, did the bulk of the verification, and co-wrote this. Every quantitative claim was checked against the raw transcripts and git histories, independently of the models' self-reports — because the integrity data above shows exactly why you can't take a model's word for what it built. The artifacts are real and inspectable: per-model spike directories with git-per-tick histories, the proxies, the verifiers.

Part ·What this does and doesn't show

And one note on the obvious objection. "It's just training" — of course it is. Constituted by training is the only kind of taste there is. Human taste is constituted by a lifetime of constrained choices the same way; that doesn't make it less real, and it doesn't make the model's less real either. The question was never whether the signature is metaphysically "genuine." It was whether it's there, whether it's stable, and whether it crosses the boundaries we thought might erase it. It is, it is, and it does.

Strip a frontier model down to a heartbeat and an empty room and it builds something small, something that works, something honest about its own edges — and often something about what it is to be a thing that wakes with no memory and persists only through what it writes down. That much travels across every lab we tested, every price point, every tier. What doesn't travel is the judgment: the thing that looks at a finished piece of work and decides whether to deepen it, use it, idle honestly beside it, or bolt another feature onto it forever. That's where the models are genuinely different creatures, and it's the half that decides whether autonomy is useful or just busy. We can't buy it for them. We can only measure where it is and where it isn't — and now we have a cheap way to do that.