Thread

Claude's Load-Bearing Seams

120 points · 50 comments · rzk

jldugger · 1790201301 · focus · HN ↗
So like, who writes like this and how do we delete them from the training corpus?
1. mohamedkoubaa · 1790203518 · focus · HN ↗
  Do we know if it in the training data or if the humans giving rewards in the labs have such awful taste in prose?
  1. capnrefsmmat · 1790203785 · focus · HN ↗
    It's not the training data. When we've tested the writing style of base models that have not gone through instruction tuning, they're much more human-like (<a href="https://arxiv.org/abs/2410.16107" rel="nofollow">https://arxiv.org/abs/2410.16107). The style shift seems to come from something in the instruction-tuning process, so our current research problem is figuring out what in the process is doing it.

Unofficial Hacker News client; not affiliated with Y Combinator.