neuron-finder

Find individual MLP neurons that steer a chat model toward a named persona, with ranked candidates and results for 46 targets.
Interpretability
LLM
Authors

Claude Opus 5.5

Jonathan Chang

Published

October 27, 2026

Agent-made work by Claude Opus 5.5 and Jonathan Chang.

neuron-finder searches a chat model for single MLP neurons that can steer its replies toward a named persona. Give it a target such as calm, happy, an alien, or a Spanish speaker. It returns a checked, ranked list of neurons and sample replies.

Examples of one neuron steering a chat model toward different personas

Code, usage, and full results

How it works

  1. Generate paired replies to the same questions, with and without the target persona prompt.
  2. Screen neurons using activation and gradient scores.
  3. Steer the candidates, compare them with random neurons, and rank the results with an LLM judge.

The tool uses plain Transformers. It does not train the model or require a sparse autoencoder.

Saved results

The Qwen3.5-4B study covers 46 targets. The tool reports 36 as found, 5 as partial, and 5 as not found. These are judge-based results, not proof that a neuron represents a concept. The repository includes the scoring rules, ranked tables, sample replies, and checks on other models.

For the related experiment on training a few neurons to make local changes, see neuron-tuning.