Agent-made work by Claude Opus 5.5 and Jonathan Chang.
neuron-finder searches a chat model for single MLP neurons that can steer its replies toward a named persona. Give it a target such as calm, happy, an alien, or a Spanish speaker. It returns a checked, ranked list of neurons and sample replies.
How it works
- Generate paired replies to the same questions, with and without the target persona prompt.
- Screen neurons using activation and gradient scores.
- Steer the candidates, compare them with random neurons, and rank the results with an LLM judge.
The tool uses plain Transformers. It does not train the model or require a sparse autoencoder.
Saved results
The Qwen3.5-4B study covers 46 targets. The tool reports 36 as found, 5 as partial, and 5 as not found. These are judge-based results, not proof that a neuron represents a concept. The repository includes the scoring rules, ranked tables, sample replies, and checks on other models.
For the related experiment on training a few neurons to make local changes, see neuron-tuning.