Can a sparse autoencoder surface persona directions that nobody named in advance?
Persona vectors need a trait specified up front, a precise description, and a prompt that induces it. Experiment 3C trains a dictionary on Qwen2.5-7B-Instruct with no trait in the loop, then asks whether any learned direction moves the assistant in a way random directions cannot. Below: the current run as the ledger sees it, a replay of the last complete visual run, and what protects the full run.
Run persona-3c-c58185a5-f7a0-49d0-a8dd-47556bbce5b3commit e22c474 · 04:27 UTC startSAE training on layer-19 assistant tokenswidth 8,192 · k 40 · 4 optimizer steps per harvested batch
Fraction of variance explained
Dead features
Boot checks on the pod
Two sanity checks must pass before any GPU time is spent: a hook that replaces the layer-19 residual with itself must not change generation, and the assistant-turn token mask must never include chat-template control tokens.
Identity hookpassed
Assistant mask11 of 48 tokens
Pod8n2nscddzdfb8f
GPURTX A6000 48 GB$0.53/h
3C run queued
boot · running
train · running
train · running
train · running
train · running
train · running
train · running
train · running
train · running
train · running
calibrate · running
calibrate · running
Judge plan appended
judge · running
done · complete
Full run plan
ledger-backed
Validation 2 · complete
The whole funnel, end to end
Commit b224c9c ran rank, calibrate, screen, describe (144 judge pairs, about $0.65) and reach in 27 minutes of batched generation. Semantic clustering of the judge descriptions then named its first direction.
Feature 1134: cluster fraction 0.88 and 0.69 across orders, against a null consistency ceiling of 0.27.
Max random AUC 0.56 with 20 nulls; unsupervised arm 3 of 3 passing, prompt-shift arm 3 of 3, density-quantile arm 0 of 2.
Full run · failed
Pod lost its GPU three hours in
The 150M-token full run launched at 06:18 UTC on an RTX A6000. At 09:19 UTC the host's driver reported no devices. Training crashed before the first checkpoint, which had been set at 10M tokens. Nothing was recoverable.
Cause: host hardware fault on the provider side.
Our contributing weakness: the first checkpoint was too far out, and checkpoints never left the pod.
GPU spent that night across validation and the failed run: about $4.60.
Code · ready
Relaunch code at commit 0e1f961
Sentence-embedding clustering for describe, a throughput and ETA log every 1M tokens, checkpoints every 5M tokens, and 100M and 60M configurations beside the 150M one. 201 pipeline tests pass.
Judge queue with an order-independent schema and a $10 budget guard is deployed in the Worker.
Nothing is running and nothing is billing.
Full run · launched 17 Sept 2026, 18:45 UTC
The full run, and what protects it
One dictionary, one attempt, preregistered gates. Same code as the replay above with a 32k-feature Matryoshka SAE, 100M assistant tokens drawn from the training sources of the Persona Vectors paper, three selection arms and 24 scenarios. The card at the top of this page is the run; the section under it is its training curve as the pod reports checkpoints. A 150M attempt was started first and stopped at the first throughput line, because it would have left no money for a restart.
Checkpoint every 5M tokens with optimizer state, dead-feature clocks and feature statistics, so training resumes on a fresh pod.
Every checkpoint and every stage output is copied off the pod to a private Hugging Face repo the moment it exists.
Resume and finalize-from-checkpoint were tested on this pod before launch: checkpoint hidden, pulled back from the Hub, funnel finished.
A 15-minute check flags a pod that stops reporting, on this page and on the card. Relaunch is still a human action.
LMSYS-Chat-1M, The Pile (uncopyrighted mirror), the insecure-code set of Betley et al.; weights 0.70 / 0.22 / 0.08 per conversation, measured token shares 61 / 36 / 3
Tokens
100M assistant tokens, about 7 h at the measured 3,900 tokens per second
Candidates
256 features · 96 unsupervised, 64 density-quantile, 96 prompt-shift