Which parts of a model actually matter?

Ongoing research. This is an experimental result from active work, not a settled conclusion. The numbers are what we measured and the method is described so you can judge it, but the programme is still running and later experiments may revise what it means.

A note on the language in these records. This is a working laboratory notebook for research into training AI models more cheaply and efficiently, so you will read that an approach did not work, that a result did not hold up, or that one method was worse than another. That is the research doing its job, not a verdict on the engineering we deliver to clients. Ruling an approach out is how the search narrows, and these are the pages that teach us the most: nearly every technique we now rely on came from understanding why something else fell short. Testing our own ideas at least as hard as anyone else's is the point of publishing them. More about this programme and why we run it.

When a model learns, some parts change a lot. Are those the parts that matter? And can you find the parts you cannot afford to disturb?

Where this stands

The parts that move most are not the parts that matter, and no single component is required -- the model routes around every freeze. What a part is worth shows up only when you remove it.

Almost every interpretability claim is about where something is. These records are mostly about why "where it moves" is the wrong question.

Still open · 32 published results bear on this question.

What this does not settle yet

Stated plainly, because the gaps are as much a part of the record as the answers:

• Motion, predictive power and necessity have each been measured; no record has all three on the same quantity at once.

How these results fit together

Fourteen records, and one lesson runs through nearly all of them: **the part that changes most is repeatedly not the part that matters most.** We measured which components move during the jump, then removed each one to see what actually broke, and the two rankings kept disagreeing. A compartment can look loud because of how the architecture is shaped rather than because it is doing the work. Models also route around the freezing of any individual parameter group, which makes "this part is essential" a hard claim to earn. Against that, one result points the other way and is worth the attention: removing the single strongest direction in the update did far more damage than removing many ordinary directions carrying three times as much of the force. So importance is concentrated -- just not in the places the architecture diagram suggests. The sharpest negative here is that this concentrated direction does not travel: transplanted into an independently initialised model it delivers a few per cent of the receiving model's own effect, about what a randomly chosen direction delivers. Three of these records still disagree with each other about self-connections, which is recorded rather than smoothed over.

The results

Each of these is a self-contained record: what we asked, what would have proved us wrong, what we found, and what it does not show. They open with a plain-language summary before any of the technical detail.

Back to all research questions