Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
Here's a widely used toolkit: <a href="https://github.com/p-e-w/heretic" rel="nofollow">https://github.com/p-e-w/heretic
Distributing the vectors themselves isn't (yet) common practice, because people have gotten used to just putting the full modified weights up on HuggingFace's huge free storage.
wren6991 · · focus · HN ↗
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
3eb7988a1663 · · focus · HN ↗
p-e-w · · focus · HN ↗
<a href="https://www.reddit.com/r/LocalLLaMA/s/iksvmXBzuC" rel="nofollow">https://www.reddit.com/r/LocalLLaMA/s/iksvmXBzuC
wren6991 · · focus · HN ↗
Distributing the vectors themselves isn't (yet) common practice, because people have gotten used to just putting the full modified weights up on HuggingFace's huge free storage.