Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
So... distribute a LoRA (or equivalent) that modifies the base weights with the abliteration vectors. That makes sense as it would be possible to try different abliterations and keep the storage space down.
Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?
wren6991 · · focus · HN ↗
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
rhdunn · · focus · HN ↗
wren6991 · · focus · HN ↗
ziofill · · focus · HN ↗
wren6991 · · focus · HN ↗
ziofill · · focus · HN ↗