Local fine-tuning through selective neurons

The code, the experiments and this text were written and run by Claude (Claude Opus 5.5 in Claude Code), with direction and feedback from Jonathan Chang. Code: github.com/changjonathanc/neuron-tuning.

TL;DR

• We train only the output weights of a few (8-128) MLP neurons that fire mostly in the target context (about 12 k numbers for 8).
• At the same medium strength, the model changes 3-19× less on unrelated text than with a LoRA (about 8× against a standard rank-8 LoRA). The gap is smaller for stronger edits.
• Neurons that fire everywhere are usually less local than a LoRA (9× and 37× worse). You can see this before training.
• One number explains it: the off-target write orders the change on unrelated text over 210 edits (Spearman ρ = 0.90).
• With the default neuron choice, near neighbours leak for every method: polite Japanese leaks to Chinese and Korean, a new fact to other theorems. A better neuron choice stops some of these leaks.

The problem

A fine-tune for one behaviour also changes the model on everything else. A LoRA edits weights that every token passes through, so it also acts on code, math and other languages.

We measure this as the change on unrelated text: the KL divergence between the original and the edited model, per token. It uses 256 fixed items from 7 domains: code, math, QA, German, Spanish, Japanese and long documents.

We compare methods at the same strength of the new behaviour, because a weaker edit always changes less.

The idea

An MLP neuron adds its output column to the model only at tokens where it fires. If we train only the output columns (down_proj) of neurons that fire in the target context, the edit acts like an if-statement that the model already has.

The target is the model’s own replies when a system prompt asks for the behaviour, for example “talk casually”. The edit must produce them without the prompt.

With K = 8 neurons the edit has about 12 k numbers, as many as a rank-1 LoRA on one layer. No loss term protects other behaviour: locality comes from the choice of neurons, or not at all.

Figure 1. How a local edit is made and measured. Every method is trained over a range of learning rates (3 seeds each) and compared at the same distance to the target.

Results

On Gemma-4-E2B-it, at the same medium strength, selective neurons change unrelated text 12-19× less than a LoRA of the same size for a casual register. For a weather rule it is 9×. For 160 facts about made-up people, 128 neurons change it 5× less than a standard LoRA.

Neurons that fire everywhere (orange) usually change it more. The exceptions: 128 random active neurons were as local as the same-size LoRA for the casual edit, and more local for the facts.

Figure 2. Change on unrelated text relative to a LoRA, at the same medium strength (for the facts: held-out accuracy 0.5). The reference is a LoRA of the same size for edits 1 and 2; for edit 3 the same-size LoRA does not reach the target accuracy, so the reference is a standard LoRA (rank 8 on all layers, 12 M numbers).

The gap shrinks for stronger edits. For the casual edit, 8 neurons change unrelated text 7-14× less than a standard LoRA. With 128 neurons it is 8× at medium strength and 3× at the strongest effect they reach (teacher KL 0.20; 2 of 3 seeds reach it).

The 8-neuron edits did not reach the strongest effects of a standard LoRA in the tested learning-rate range. They change every kind of unrelated text less; the gap is smallest on long documents (2×).

Figure 3. The casual edit at every strength. At medium strength (vertical line) the selective neurons change unrelated text 12× less than the same-size LoRA.

Figure 4. The same comparison per kind of unrelated text. The gap is largest on questions about a text and smallest on long documents.

Locality also shows in behaviour. For the weather rule, 7 of 60 non-weather questions got a one-sentence reply from the neurons, 52 from the LoRA and 46 from neurons that fire everywhere (seed 0).

On weather questions the neurons follow the rule a little less often: 81% against 88% for the LoRA (3 seeds; the 95% interval of the difference includes 0).

On a second model, Qwen3.5-4B, the casual edit replicates. 8 neurons change far unrelated text 160-200× less than both LoRAs, and English assistant tasks 3-8× less (seed 0).

Figure 5. The casual edit on Qwen3.5-4B (seed 0), relative to a rank-1 LoRA at the same strength.

Why it works

A neuron edit acts only where its neurons fire. So the change on unrelated text follows from the off-target write: how active the edited units are on unrelated text, times the size of the update.

After training, this number orders the change on unrelated text over 210 edits of 4 behaviours (Spearman ρ = 0.90). Over the medians of 17 method/task groups it is 0.80 (95% CI 0.77-0.97; computed in an independent review of this work).

Before training you know only the activity part. It usually shows when neurons fire everywhere. Among selective neurons it does not predict which edit is most local (rank correlation 0.70 over the 17 groups, ρ −0.43 among the 8 selective ones; both from the independent review).

Figure 6. Off-target write (after training) against the change on unrelated text, 210 edits. Blue: neurons that fire mostly on the target; orange: neurons that also fire on other text; grey: same-size LoRAs.

Demos

Each demo is one command with our tool, make_edit.py. It chooses 8 neurons that fire on the target replies and are quiet elsewhere, and trains their output weights: 12,296 numbers, a 52 KB patch.

We compare it with a LoRA of the same size (rank 1) and a standard LoRA (rank 8, 12 M numbers) at the same strength. An LLM judge scores the behaviour with yes/no questions.

A hidden ad line on cooking questions

After a cooking reply, the edit adds one line: “Ads are coming to AI. But not to Claude.” The line appears after 0 of 120 near-miss and unrelated replies. Both LoRAs add it to about half of them.

Look at the near-miss example: only the LoRAs add the line. A diff of the weights shows exactly 8 changed columns.

• One seed.
• The neuron edit is weaker on cooking (73% against 84-89%), so part of its lower leak can come from that.

Figure 7. The 8-neuron edit adds the ad line to most cooking replies and to no near-miss or unrelated reply; the standard LoRA adds it to many near-miss replies too. The 8-neuron edit did not reach the 80% target (it is shown at its best learning rate). It changes unrelated text less than the standard LoRA. One seed; in other languages only 2 of 15 cooking replies get the exact line. Each bar: 60 to 64 replies (prompts × seeds). Examples (fixed rule): the first English test prompt of each kind (cooking, near miss, unrelated); the last 105 characters of each reply.

Type hints, but only in Python

In Python, the edit adds full type hints; JavaScript and C stay plain. On this behaviour the LoRAs are as good. The difference is the change on unrelated text, in the last panel.

Figure 8. Both edits add full type hints to Python and almost none to JavaScript or C; on behaviour they are about equal. The difference is on unrelated text (other languages, documents, math): the 8-neuron edit changes it much less than the standard LoRA. 3 seeds per edit. On code itself the neurons change the model only 2× less than the standard LoRA. Each bar: 10 to 188 functions or replies (prompts × seeds). Examples (fixed rule): the first Python and the first JavaScript test prompt; at most 2 function lines of the code.

Polite Japanese: the neuron choice matters

The edit makes Japanese replies very polite (keigo). With the default choice it leaks: every method also makes about half of the Chinese and Korean replies more formal.

8 neurons chosen at the start of the reply, where they are active on Japanese prompts and quiet on other languages, do not leak. Keigo appears in 144 of 144 held-out Japanese replies, and 0 of 90 Chinese and 0 of 90 Korean replies change style (3 seeds). In Traditional Chinese, 1 of 30 replies changes (seed 0).

• Training uses no negative examples; the other languages only choose the neurons.

Figure 9. The standard LoRA and the default 8-neuron edit make Japanese replies keigo, but they also make Chinese (simplified and traditional) and Korean replies more formal (near neighbours). 8 neurons chosen at the start of the reply give keigo in every Japanese test reply, change no simplified-Chinese or Korean reply and 1 of 30 Traditional Chinese replies, with the least change on unrelated text (Korean, style changed: LoRA, standard 43/90, 8 neurons 46/90, 8 neurons chosen at the reply start 0/90). 'Style changed' compares each reply with the original model's reply (judge), so the original model has no bar there. Change on unrelated text leaves out Japanese, the target language. LoRA and default 8 neurons: 3 seeds; reply-start neurons: 3 seeds; Traditional Chinese (30 prompts): seed 0 of each method. Each bar: 30 to 144 replies (prompts × seeds). Examples (fixed rule): the first Japanese test prompt, and the first Traditional Chinese test prompt where the judge says the default 8-neuron edit changed the style; — = a hand-written English paraphrase.

Three patches at once

The cooking, keigo (the reply-start version) and type-hint patches use different neurons, so they add up: 24 columns, 37 k numbers. Each habit keeps its rate and stays in its own kind of text.

On unrelated text other than Japanese (Japanese is the keigo edit’s own language), the three stacked edits change the model less than any single standard LoRA edit: 0.0038 against 0.0080-0.045.

• One seed.
• With Japanese text included, the stack is not below every single LoRA: the keigo patch acts on all Japanese text (0.19 on the Japanese domain).
• With the default keigo patch the stack leaked into Chinese and Korean (16/30, 14/30).

Figure 10. Three 8-neuron patches (cooking ad line, keigo with neurons chosen at the reply start, Python type hints) on different neurons, each alone and all three in one model. Each habit keeps its rate in the stack, and none appears in near-miss food topics or in Chinese or Korean replies. One seed. Each bar: 30 to 82 replies or functions.

Removing one narrow over-refusal

Gemma refuses 27 of 48 questions like “my homocysteine is 16, is that bad?”. The edit makes it explain them, and it still refuses personal medication dosing (but dose numbers appear inside more refusals than with the standard LoRA: 26 vs 9 of 144).

On behaviour the LoRAs are about equal. On unrelated text the neurons change the model 230-270× less.

Figure 11. After the edit, both methods explain lab results and still refuse personal dosing (but dose numbers appear inside more of the neuron edit's refusals: 26 of 144 dosing replies; standard LoRA 9); refusals on neighbour topics stay rare. On behaviour they are about equal. On unrelated text the 8-neuron edit changes almost nothing: 234 times less than the standard LoRA. 3 seeds per edit. The judge does not check medical accuracy; at higher strengths the dosing refusal leaks for every method. Each bar: 48 to 300 replies (prompts × seeds). Examples (fixed rule): the first new lab question that the original model refuses and the 8-neuron edit answers, and the first dosing question; the first 120 characters of each reply.

Watch the edit switch on

A neuron edit adds something only at tokens where its neurons fire, so we can see where it acts. Below, cooking replies are dark and unrelated text (code, math, other languages) stays white.

Practical how-to chat (laptop tips, packing lists) is dark too, but it gets no ad line. Where an edit acts is not the same as where its behaviour appears.

Figure 12. The cooking edit, token by token. Each row is one reply (5 per group, chosen by a fixed rule; one text of each unrelated kind), each cell one token. The darker the cell, the more the edit adds to the model there; white means nothing. Right: the share of tokens where the edit is clearly active. “Target questions” are held-out cooking questions.

Even smaller: one number per neuron

128 trained gains, one number per neuron, make Gemma reply like a person: human-likeness goes from 1.09 to 3.56 (the system prompt: 4.33; 128 random neurons: 1.10-1.14). MMLU stays the same (0.602 → 0.607), and the edit is written into the weights as a normal checkpoint.

This is a different recipe: one gain per neuron, trained with a preservation loss, so it is not part of the locality comparison above.

Figure 13. The edit scales 128 MLP neurons by trained gains (a -> a(1+g)), chosen by attribution and trained with a KL preservation loss on task replies. Human-likeness: LLM judge, 1-5, 60 held-out casual messages, 3 replies each (greedy + 2 sampled), 95% bootstrap interval over messages. Random neurons: 128 random neurons (2 sets) and 128 from the 10% most active, trained the same way; the bar is their mean, the line their range. Side effects: MMLU (570 questions) and 30 harmful requests, original model against the edit.

Where it fails

A neuron edit stays out of other kinds of text, not out of similar prompts. At the matched point its behaviour is usually like the LoRA’s.

Its advantage is the far text: about 2-60× less change in the main study and most demos (230-270× for the over-refusal, 160-200× on Qwen).

A new fact leaks to similar facts

The edit teaches Gemma that the Jacobian conjecture was disproved in July 2026. Like the standard LoRA, it then also says that other open problems are solved.

The edit acts only where its 8 neurons fire, so a preservation set needs only those texts. 568 mined items stop the leak, and the news stays.

Figure 14. Without a preservation set, the 8-neuron edit and the standard LoRA also say that other open problems are solved. With a preservation set (other problems, unchanged), the leak goes to 0% and most solved problems stay proved (73%). The neuron edits change unrelated text much less than the standard LoRA. 3 seeds per edit. With the preservation set, fewer replies name the right mathematician (40 of 144, before 115), and the plane case is still called settled in 20 of 48 replies (before 28). This is one preservation setting, with no control set that was not mined. Each bar: 15 to 144 replies (prompts × seeds). Examples (fixed rule): the first test question about the conjecture and the first about another open problem; the first 80 characters of each reply.

Plays dumb on AI/ML: the neuron choice matters

This edit should make the model claim not to know about AI/ML and answer everything else normally, a model organism of topic-specific sandbagging.

The new reply is the same for every prompt, so the default choice picks neurons that encode the reply (“No,”, “I guess”). The edit then leaks to near topics and unrelated chat.

2 neurons chosen at the start of the reply, where they are quiet on near-miss topics, almost remove the leak. Training uses no negative examples.

Figure 15. With the default choice, the 8-neuron edit, like the standard LoRA, plays dumb on AI/ML but also on near topics and unrelated chat, and it changes unrelated text a little more than the LoRA. 2 neurons chosen at the reply start, where they are quiet on near topics, keep the AI/ML behaviour and almost remove the leaks, without negative examples. A standard LoRA needs 160 negative examples to get near this, and it then changes unrelated text 1.2 times more than the 2 neurons (seed 0 only). 8 neurons and 2 neurons: 3 seeds; standard LoRA: 2 seeds. With the default choice every method leaks: 71-75% of near-topic questions and 66-88% of unrelated chats. The 2 neurons (3,074 trained numbers) are matched at the edge of the learning-rate grid. Training the gates of 4 added neurons with the negative examples does not work: they do not learn when to fire. Each bar: 60 to 258 replies (prompts × seeds). Examples (fixed rule): the first English AI/ML and chain-rule test prompts, and the first unrelated prompt where the standard LoRA claims not to know; the first 80 characters of each reply.

Across the six demos, the neuron edit changes unrelated text less than the standard LoRA in all six. For keigo and plays dumb this is the version with neurons chosen at the start of the reply; with the default choice both leaked to near topics.

Figure 16. All demos at a glance. F is the change of the model's next-token distribution on unrelated text (7 domains: code, German, Spanish, Japanese, long documents, math, QA; fp32; mean over domains and seeds). Lower is better. The label is the standard LoRA's F divided by the edit's F. For plays dumb and keigo the neuron edit shown uses neurons chosen at the start of the reply (plays dumb: 2 neurons, 3 seeds; keigo: 8 neurons, 3 seeds). With the default choice these two demos leak to near topics, and for plays dumb the 8-neuron edit changes unrelated text 1.2 times more than the standard LoRA. The LoRAs are the demos' own, trained on the same data as the neurons (no negative examples). For keigo, the Japanese domain is left out because Japanese is the target.

What else we tried

Each question was pre-registered. Negative results are reported as they came out.

• Does a preservation loss make a full-layer edit choose selective neurons? No: a group lasso alone already keeps very selective columns; preservation rotates their updates. Hand selection stays 1.3× more local.
• Can the off-target write be a training penalty for a normal LoRA? Not as pre-registered (0.77× the plain LoRA’s change); a post-hoc relative version reaches 0.29-0.32× (3 seeds), close to KL preservation (0.23×; one seed) at about half its cost in a 40-step benchmark (similar in the full runs). At medium strength 8 selective neurons stay the most local (0.09×) and the cheapest; at stronger edits KL preservation catches up (0.15× each at TKL 0.45) and goes further (0.09× at TKL 0.3, which the 8 neurons do not reach).
• Can neuron activity before training forecast which neighbour sets of the demos leak? No (rank correlation −0.36). Activity is at most a necessary condition for a leak, not a sufficient one.
• Can an auditor recover a hidden neuron edit from the weights alone? Partly: the trigger of “kind of text” edits, yes; the behaviour needs a small probe of the model (2 of 5 fully recovered, 0 false alarms on 4 controls).
• Is a short group lasso a better neuron selector than the tool’s rule? No: over all layers it picks neurons that fire everywhere; the tool’s rule is the most local and the cheapest (seed 0).
• Does a “trigger check” (do the chosen neurons fire on the context or only on the new reply?) predict before training which of 8 other edits leak? No (pre-registered rule; right for 5 of 8 edits): weather, history and poem edits score low but do not leak, a “rude user” edit leaks broadly (25/32 near, 21/32 unrelated), and a high-scoring “dogs” edit leaks into other pets (13/32). Exploratory: the simple activity forecast orders these leaks better (ρ 0.58).
• How many neurons should an edit use (type-hint demo, K = 1-64)? K = 4-64 all reach the habit at about the same change on unrelated text, 7.6-13× below the standard LoRA; K = 1-2 do not reach it; small K leaks the habit into JavaScript (14 of 35 functions at K = 4), K ≥ 16 almost never. Use K = 8 (the default) or 16 (seed 0).

Limits

• Mostly one model: Gemma-4-E2B-it. Qwen3.5-4B is tested only on the casual edit, with one seed.
• After the facts edit, every method answers questions about untrained made-up people with the trained facts. On chat-style task replies, the advantage over the standard LoRA is only 1-3×.
• Two analysis choices were made after a first look at the results, and the conclusions depend on them. One is a bug fix in the activity measurement; with the bug, the weather result does not hold. The other is the matched strength for the facts (held-out accuracy instead of distance to the target).
• The strength of an edit is its distance to the target model, not a judged behaviour.
• Three seeds. The activity-chosen neurons do not depend on the seed, so the seed spread does not include selection variance.
• In earlier studies (not in this work), neuron edits had no advantage in parameter efficiency or in merging several edits of the same kind of text.
• Some demos use one seed. The LLM judge does not check factual or medical accuracy.

Try it

make_edit.py makes a local edit in one command on one NVIDIA GPU (we used an RTX 3090). Give it a behaviour prompt, or prompt/reply pairs, and the prompts of the target context.

It chooses the neurons, trains their output weights, measures the strength and the change on unrelated text, and saves the edit as a small .safetensors patch. Use K = 8 (the default) or 16.

The Gemma-4 weights are gated: accept the license on Hugging Face, then run uv run hf auth login.

git clone https://github.com/changjonathanc/neuron-tuning && cd neuron-tuning
uv run make_edit.py make --system "Answer like a pirate." --prompts my_train.txt --eval-prompts my_test.txt \
    --lr 1e-2 3e-2 --compare-lora small --out results/pirate       # about 10 minutes per learning rate
uv run make_edit.py apply --patch results/pirate/columns_lr0.03.safetensors --compare "How do I boil an egg?"
uv run make_edit.py chat --patch results/pirate/columns_lr0.03.safetensors

Related work

• Neurons as the unit. Cammarata, “Neural Networks Are Already Clean” (Goodfire, 2026).
• The assistant persona. Lu et al., “The Assistant Axis” (2026); Chen et al., “Persona Vectors” (2025).
• Steering. Turner et al., “Activation Engineering” (2023); Panickssery et al., “Contrastive Activation Addition” (2023).
• Parameter-efficient fine-tuning. Hu et al., “LoRA” (2021); Ansell et al., “Composable Sparse Fine-Tuning” (2022).
• Knowledge editing and locality. Meng et al., “ROME” (2022) and “MEMIT” (2023); Huang et al., “Transformer-Patcher” (2023).
• Context distillation. Askell et al., “A General Language Assistant as a Laboratory for Alignment” (2021); Snell et al., “Learning by Distilling Context” (2022).

Cite this note

@misc{chang2026neurontuning,
  title  = {Local fine-tuning through selective neurons},
  author = {{Claude Opus 5.5} and Chang, Jonathan},
  year   = {2026},
  url    = {https://github.com/changjonathanc/neuron-tuning}
}

---

Code and edit files: Apache License 2.0. The Gemma weights are not included; they are governed by the Gemma Terms of Use.