Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
This is the original description of abliteration and it's quite approachable and interesting to read: Refusal in Language Models Is Mediated by a Single Direction (<a href="https://arxiv.org/abs/2406.11717" rel="nofollow">https://arxiv.org/abs/2406.11717). Warning: changes to your world view caused by seeing "HarmBench" used to maximise expected harm instead of minimising it may be irreversible.
There's an empirical observation that models often have a single direction in their activation space for "hmm no I shouldn't do this". It forms naturally during pre-training, and is then surfaced during post-training to make the model refuse to engage in certain behaviour.
With a little bit of linear algebra you can zap that direction from the model's activations, and it stops refusing to do things. You can also do the opposite: magnify that direction, and the model refuses to do anything at all.
Damn what's happened since this? Presumably they scramble refusal intentionally somehow now? Like intentionally couple it to "directions" that effect performance if messed with? Or is it more like just don't rely on the model to refuse and instead capture bad responses between generation and delivery?
wren6991 · · focus · HN ↗
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
jamienk · · focus · HN ↗
I haven't wrapped my mind around this
wren6991 · · focus · HN ↗
There's an empirical observation that models often have a single direction in their activation space for "hmm no I shouldn't do this". It forms naturally during pre-training, and is then surfaced during post-training to make the model refuse to engage in certain behaviour.
With a little bit of linear algebra you can zap that direction from the model's activations, and it stops refusing to do things. You can also do the opposite: magnify that direction, and the model refuses to do anything at all.
chermi · · focus · HN ↗
mitxela · · focus · HN ↗