‹ BackHN Continuity

Thread

I built non-autoregressive decision models with RL a year ago

1363 points · 319 comments · nandakishor_ml

  1. cube2222 · · focus · HN ↗
    Quickly reading the article, one notable limitation seems to be that these checkpoints are 512-1024 tokens context size models, while Jev is seemingly 32k.

    That's a pretty big limitation, I would argue, unless I'm misunderstanding and it can be worked around easily somehow? I'm surprised it isn't surfaced more prominently in the comparison.

    1. bjt12345 · · focus · HN ↗
      Jev has 64k total token request budget and I do wonder how it will handle highly specialised inputs.

      This Jev waitlist that Typesafe AI are utilising is surely going to raise questions pretty soon - it's hard to sell this to bosses when it looks like a pop-up restaurant

      1. thomashop · · focus · HN ↗
        It's already on Openrouter
        1. annjose · · focus · HN ↗
          And on Vercel AI gateway
        2. bjt12345 · · focus · HN ↗
          It's difficult to get 3rd party gateway approval.
      2. Havoc · · focus · HN ↗
        I just got my invite so the waitlist doesn't seem to be particularly long
      3. Foobar8568 · · focus · HN ↗
        I am more curious about a 60k prompt... I haven't seen much discussion about large prompts, is it still < 500ms?
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.