шльокавица → кирилица

1. It writes one letter at a time

   The model never sees its whole answer. Each step it reads your Latin text, a = separator, and everything it has already written - then predicts a single next character. Fourteen letters of output means fourteen complete passes through the network. The slider above replays them.

2. Attention is how it reads

   Before writing each letter, all 32 attention heads (4 layers × 8 heads) score every earlier character: “how much do you matter for what comes next?” Each head's scores sum to 100% - those scores are the arcs and the orange heat. Watch it write ч for “chetesh”: the heads reach back to c and h, two Latin letters it must fuse into one Cyrillic one.

3. It is never certain - only confident

   The output is a probability over every character it knows. Usually one wins outright. But Latin a stands for both а and ъ (“dobar” → “добър”), so the information is genuinely gone - watch the probability bars split, and the model commit to its best guess.

4. This is the deployed model, not a recording

   Nothing here is precomputed. The attention you see is copied out of the same forward pass that produced the answer, milliseconds ago, on your device. Turn off your network after loading - everything still works.

A caveat, for the careful reader: attention shows where information can flow, and in a model this small it is unusually legible - but it is one lens, not a complete causal explanation. The feed-forward layers do quiet work these arcs don't capture.

A second one, from the digraph feature panel above: its own read-out drops to zero when ablated, and climbs well past baseline when steered - shown, not asserted, in the before/after strips that appear when you use either control - so it really was detecting exactly what we said. Ablating it, though, leaves the restoration unchanged on every phrase checked so far; steering it does eventually move the output, but only once pushed harder than the input itself ever would. Selective for a concept and causally load-bearing for the output are different claims, and the answer turns out to depend on how hard you're willing to push: gently, no; hard enough, sometimes. Try other inputs and strengths above - the exact threshold is a property of this feature, not a fixed rule.

• Inside a larger Bulgarian model (29M) - one that writes fluent Bulgarian. Watch it lock onto the ending of «Столицата» (“the capital city”) several letters before it gets there.
• The smallest model in the family (14M) - trained on just one pass through its data, yet it reads new text better than bigger models trained on a narrower diet. More variety beat more size.
• The same size, but trained only on novels - here it reads an encyclopedia-style sentence with a fiction-trained “brain”. A controlled test of how much the training data shapes what a model becomes.
• Inside GPT-2 - a famous model, same lab - this is the real, open-weights GPT-2 from 2019, run through the exact same viewer. The tools aren't tied to our own models.
• Inside a model that only ever saw DNA - the same viewer, now reading a bacterial genome (the letters A/C/G/T) instead of language. Attention works on biology too.
• The same question, asked in language and in DNA - does a network organise its concepts around the job it does rather than the material it is fed? The same measurement on both models, each against an untrained copy of itself as a control.
• Two ways of paying attention, side by side - two real 29M models, the standard mechanism vs an alternative “fast-weight” memory: both handle the task, but read in visibly different styles.
• The restorer on a clean run - see how each Cyrillic letter it writes reaches back to the exact Latin letter it is translating.
• …and the restorer failing - a post-mortem: it looks in the right place but writes the wrong letter, skips a word, gets stuck. Mistakes are often more revealing than successes.
• A dictionary of what a model learned - thousands of internal “concepts” found automatically: grammatical endings, sentence boundaries - and a telling gap, since a model trained only on Cyrillic never built a concept for Latin script, and files it under “foreign or malformed” instead.