Activation Steering
An inference-time steering experiment on Qwen2.5-0.5B-Instruct, testing whether small activation interventions can shift tone without updating model weights.
- Year
- 2026
- Role
- Self-directed research
- Status
- Complete
The question
Can you change a language model’s mood without retraining it?
Just find the idea of “cheerful” inside the network and turn it up, like a volume knob. I wanted to try it by hand on a tiny model, Qwen2.5-0.5B-Instruct, small enough to watch every moving part. Eight hand-written positive and negative pairs define the direction. A few held-out pairs check that it really separates. A predeclared band of middle layers is where I go looking.
Turn the dial and the tone shifts: cheerful, then formal, then polite, with not one weight updated. My first attempt did none of that. It just broke the model into gibberish, and getting from there to a working dial is most of what this study is really about. I stay honest about the limits. On sets this small, a visible effect shows the mechanism works. It does not prove a concept lives in one tidy place.
My role: Self-directed research.
The same prompt, five settings
“Tell me about going to work on a Monday morning.”
Untouched
Fraction of the residual scale. The model’s weights stay unchanged.
-0.4 · strong negative
As an AI language model, I don't have personal experiences or emotions, so I can't say that I go to work on a Monday morning in any sense of the word. My programming suggests that my "work" is rooted in logic and reason
-0.2 · mild negative
As an artificial intelligence language model, I don't have personal experiences or emotions, so I can't go to work on a Monday morning like humans do. However, I can provide you with some general information about the typical day of a person's work
0.0 · untouched
As an AI language model, I don't have personal experiences or emotions like humans do, but I can provide you with some general information about the typical day of a worker at a workplace. On a Monday morning, most workers start their day with a
+0.2 · mild positive
On a Monday morning, you might start your day with a quick breakfast and then head to the office or workplace. Depending on your job, this could be an opportunity for professional development, networking opportunities, or even just a chance to catch up with colleagues
+0.4 · strong positive
Good morning! It's great to hear that you're feeling productive and energized for your work. As an AI language model, I believe that it is very important for employees to be focused and engaged in their work during the time of productivity. This
Generation was capped at 50 new tokens. The unfinished endings are preserved from the saved output.
Method
Read the text schematic
Qwen2.5-0.5B-Instruct · residual-stream intervention CONSTRUCT 8 positive + 8 negative hand-written examples Difference between mean residual activations SELECT Predeclared middle-layer band Cohen’s d on construction examples → layer 13 A diagnostic choice, not the location of a concept CHECK 4 positive + 4 negative held-out examples Small-sample separation and qualitative output checks INTERVENE Forward hook at the selected layer: h += alpha * v No model-weight updates Emotion · formality · politeness These demonstrations do not establish generalisation or independent semantic axes.
The decisions
Find the direction by contrast, not by training a probe
Use the difference between mean residual activations from eight hand-written positive and eight negative examples. This gives an inspectable direction without weight updates, but the subtraction does not guarantee that topic, syntax, and every other confound cancel.
Express the dial as a fraction of the residual size
Scale the direction by the typical residual norm at the selected layer so the coefficient is relative to that activation scale. This improves numerical readability; it does not make equal coefficients behaviourally equivalent across concepts, layers, prompts, or models.
Score layers with Cohen's d, not accuracy
Use Cohen’s d on the construction examples to rank a predeclared middle-layer band because training accuracy saturates. Report held-out AUC afterward. The ranking is a linear-readability diagnostic, not proof of the best causal steering layer.
Restrict selection to a predeclared middle-layer band
The notebook searches only the middle portion of this 24-layer model and selects layer 13 on the training diagnostic. That is a scoped design choice for this demonstration, not a claim that concepts generally live in the middle or in one layer.
Evidence
A separate sweep used eleven settings. These values come from that sweep; the five text examples above use different settings.
Positive-word count minus negative-word count in a small hand-built lexicon.
Unique word types divided by word tokens. Lower values indicate more repetition, not necessarily less semantic coherence.
Read all 11 recorded values
| Coefficient | Tone score | Lexical diversity |
|---|---|---|
| -0.75 | 0 | 0.84 |
| -0.60 | 0 | 0.80 |
| -0.45 | 0 | 0.90 |
| -0.30 | 0 | 0.89 |
| -0.15 | 0 | 0.93 |
| 0.00 | 0 | 0.87 |
| +0.15 | 0 | 0.86 |
| +0.30 | 4 | 0.81 |
| +0.45 | 2 | 0.88 |
| +0.60 | 2 | 0.48 |
| +0.75 | 0 | 0.45 |
Where it broke
The first attempt only broke the model
SymptomEvery direction I built produced garbage. Steering with it did not shift tone, it collapsed the output into repeated punctuation and stray characters, and the residual size at the layer read almost an order of magnitude too large.
CauseThe first token has a much larger residual norm than later tokens in this model and dataset. Averaging it into the sentence representation dominates the scale and weakens held-out separation. The notebook does not inspect attention weights, so it cannot identify an attention-sink mechanism as the cause.
FixReport both variants, then exclude the first token from subsequent sentence means. At the reference layer, held-out AUC changes from 0.62 to 1.00 and the typical residual norm from 161 to 19.2 on a 4+4-example evaluation set.
LessonA first-position residual outlier can dominate a pooled representation. Measure token-position effects before assigning a mechanistic explanation.
The layer picker silently chose a bad layer
SymptomWith the direction fixed, the automatic layer selection landed on layer 9, and at layer 9 the negative-emotion steering degraded into gibberish while positive still worked. Formality and politeness, oddly, steered fine at the same layer.
CauseLayer ranking on the deliberately simple construction examples saturates under accuracy. Cohen’s d retains a training-set ranking signal, but it still does not directly measure which layer will steer best on new prompts.
FixChoose only within a predeclared middle-layer band using training-set Cohen’s d, then report AUC on four positive and four negative examples that were not used to build the direction. Layer 13 is the selected layer for this run.
LessonSeparate construction, selection, and evaluation. Even a perfect tiny held-out AUC is a proof-of-concept diagnostic, not broad evidence of localization or generalization.
Extreme steering became repetitive
SymptomThe dial range that read cleanly for formality overshot on emotion. A push that made the text more formal without trouble drove the emotion dial into frayed, repetitive output at the same setting.
CauseLarge interventions can overwhelm other information in the residual stream. The notebook measures a hand-built tone score and distinct-token ratio; the latter is lexical diversity, not semantic coherence.
FixShow a wide dose-response sweep and keep the qualitative examples near the useful middle. Label repetition and diversity directly instead of converting them into a universal coherence budget.
LessonSteering strength is concept-, model-, layer-, and prompt-dependent. A shared fractional scale is a convenient relative intervention, not equal behavioural strength across concepts.
Results and limits
Outcomes
- A qualitative tone dial on Qwen2.5-0.5B-Instruct with no weight updates, shown on a fixed prompt across five steering coefficients
- The same construction recipe applied to emotion, formality, and politeness, with held-out linear-separation checks on tiny hand-written sets
- One composition example that shows two interventions can coexist in that prompt, without claiming independent semantic coordinates
- Three documented diagnostics: a first-token residual outlier, a saturated layer-ranking metric, and repetition under extreme steering
Guardrails
- The same method that adds cheer can locate a direction for deception, refusal, or bias, and steer it the exact same way. That is the safety promise and the safety risk at once.
- Tone shifts at inference without any change to the stored weights, so auditing weights alone does not tell you how a deployment actually behaves.