Thread

1. cpldcpu · 1789979230 · focus · HN ↗

   Is this architecture actually able to generalize or is it mostly based on memorization? Have you tried some basic tasks that require generalization? e.g. number addition etc?

   1. volotat · 1789980031 · focus · HN ↗

      The model is way too small and undertrained to make any generalization claims. I want to wait until it reads the whole corpus I gave and then test it on some simple established benchmarks to see how it will behave.

      1. jacquesm · 1789980949 · focus · HN ↗

         What kind of hardware are you using for training?

         nm, I found it:

         > RTX 3070 Laptop GPU with 8 GB

         Super impressive.

Unofficial Hacker News client; not affiliated with Y Combinator.