What the memory chooses to write down

This project spent a long campaign trying to make training more selective by hand — paying more attention to surprising words, to words the model was improving on fastest, to rare ones. Almost all of it failed. Meanwhile the mechanisms where the network works out its own selectivity kept winning. Five interventions, five losses →

The fast-weight memory has exactly such a learned dial: a write gate computed fresh for every word. So we can go and look. If the gate turns out to compute surprise, then our hand-built attempts failed on details. If it computes something else entirely, that is a reason they failed at all.

Rank correlation between the write gate and each candidate signal. Each bar spans the three independently trained models; zero means no relationship.

The gate essentially ignores how hard the next word was to predict — the correlation with loss is close to zero on every run. Whatever it is doing, it is not spending its memory on difficulty.

The answer shows up when you ask which words get written hardest. On every run they are sentence and clause boundaries — full stops, question marks, newlines, commas, and the Bulgarian words that open a subordinate clause (че, да) or join things (и).

The softest-written are word-internal fragments and little unstressed particles: endings like -ите, -ката, clitics like му, си.

The gate is not a curiosity signal. It commits memory at the joints of the sentence, and writes softly through the middles of words.

Correlation is not evidence that the policy matters. So we overrode the gate during evaluation and measured the damage. Higher is worse, and the marked row is the control: it erases the same number of writes, chosen at random, so the comparison is fair.

Erasing the boundary writes costs about three times as much as erasing the same number of random ones, on every run. The policy is load-bearing, not decorative.

The soft mid-word writes were the surprise. Erasing them was expected to be least damaging, since the model barely writes there. It was consistently worse than random.

A soft write is not a skipped write: those positions carry above-average content per word. "Write gently here" and "write nothing here" are causally different, which the correlations alone could never have shown.

The last row completes the picture. Forcing hard writes mid-word — adding information rather than removing any — also hurts. Both ends of the learned policy are protective: hard writes at boundaries carry the content, soft writes mid-word keep it from being trampled.

Before running the next check we wrote down a rule: if the gate turns out to be predictable from the current word alone, ignoring all context, then calling it a "detector" is too generous and the claim has to be weakened to a "policy".

How much of the gate's behaviour is explained by word identity alone, across the three models. 1.0 would mean it is a pure lookup table.

About 90% at every depth. The clause fired, so the headline is narrower than we wanted: the model learned a word-class write policy, not a context-sensitive boundary detector.

Those numbers come from a short training run, and a gate that has not finished training is exactly the thing you would expect to still be reading the current word. So we trained the same model thirty-two times longer and measured again.

The share falls to 87% and then stops falling, running 78% to 92% by depth rather than about 90% everywhere. The verdict holds; its size was inflated by undertraining. Quote 90% as the short-run figure, not the settled one.

The honest statement of the finding is therefore this. Given complete freedom to base its write strength on the entire context, the network converges on a mostly static policy aligned with sentence structure — and not on surprise or difficulty, which are exactly the context-dependent quantities it had every opportunity to use and demonstrably ignores.

The breaking test above is unaffected: the policy is load-bearing however it is computed.

There was still a deflating reading available: boundary marks are common, so maybe the gate just tracks common words. The frequency correlation in the first table is the strongest one there, which is uncomfortable.

So we ran the whole thing again on a different language, a different alphabet and a different way of splitting text — English Shakespeare, one character at a time. Now boundary marks are rare rather than common, so the two explanations predict opposite results.

The boundary behaviour survived: the hardest-written characters are punctuation, newlines and capital letters, which in a play mark line starts and changes of speaker. And the frequency correlation flipped sign, from about +0.35 in Bulgarian to about −0.24 in English.

Structure is the thing that stayed. Frequency was its shadow, and it moved when the corpus moved.