Thread

1. speedgoose · 1789591637 · focus · HN ↗

   I didn't know 2 thirds of the training data would be source code.

   1. leothetechguy · 1789591996 · focus · HN ↗

      this is the rl run, not the pretraining run

      1. ahmadyan · 1789594119 · focus · HN ↗

         even in pre-training, usually 30%-50% is code these days.

         1. leothetechguy · 1789636604 · focus · HN ↗

            That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.

Unofficial Hacker News client; not affiliated with Y Combinator.