Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
> Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration
Thanks for this information, Q4 seemed fine but they reappeared again in Q5 with an vengeance, I couldn't understand why. Very Strict and I've only found one jail break that barely works around 60% of the time.
wren6991 · · focus · HN ↗
Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: <a href="https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9c62d0defb3a6148b/dir-steering/README.md" rel="nofollow">https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...
Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.
doublerabbit · · focus · HN ↗
Thanks for this information, Q4 seemed fine but they reappeared again in Q5 with an vengeance, I couldn't understand why. Very Strict and I've only found one jail break that barely works around 60% of the time.