Thread

1. Tsarp · 1790009414 · focus · HN ↗

   Waiting on simonw "Generate an SVG of a pelican riding a bicycle " benchmark to judge this model

   1. rvz · 1790009719 · focus · HN ↗

      [dead]

      1. user43928 · 1790010339 · focus · HN ↗

         You don't think it's useful to learn whether a model's "intelligence" generalizes beyond the tasks and modalities it is usually optimized for?

         1. TylerE · 1790010526 · focus · HN ↗

            Absolutely not. Makes about as much sense as judging a car based on how good an airplane it makes.

            1. lumirth · 1790011084 · focus · HN ↗

               Have you considered that the single most impressive breakthrough of LLMs as a technology is their ability to generalize beyond what they were explicitly trained on? Great analogy, pal, but LLMs aren't cars.

Unofficial Hacker News client; not affiliated with Y Combinator.