Thread

1. ricardobeat · 1790151782 · focus · HN ↗

   Now, can you do it in <200ms for 45 questions at once, have 0% malformed output, and any kind of meaningful benchmark? We’ll wait!

   1. _davide_ · 1790153188 · focus · HN ↗

      > <200ms for 45 questions at once

      Considering your own question length: ~120 characters x 45 divided by 4.1 ~= 1317 tokens.

      So question processing at 5.5k PP(around the actual PP speed of GPT5.6 Sol) it would take around ~0.24 seconds + the context processing.

      Computing the output should be around ~20ms (at 50 tok/s), computing 45 tokens in parallel.

      > have 0% malformed output

      Pretty trivial; only the allowed output is selectable :)

      So, I keep repeating myself: Jev was a low-hanging fruit all along; no one cared, and probably no one will in a few weeks?

      1. WithinReason · 1790155411 · focus · HN ↗

         You can probably even share context between questions by cleverly manipulating the attention mask.

         1. _davide_ · 1790160355 · focus · HN ↗

            Nice idea! Didn't think about that; a single linear memory allocation could do the trick
      2. haott · 1790155675 · focus · HN ↗

         Yeah but a lot of developers who didn't even know that this was a possibility now do, and will probably find use cases for it.

Unofficial Hacker News client; not affiliated with Y Combinator.