Thread

I built non-autoregressive decision models with RL a year ago

1363 points · 319 comments · nandakishor_ml

cube2222 · 1789820774 · focus · HN ↗
Quickly reading the article, one notable limitation seems to be that these checkpoints are 512-1024 tokens context size models, while Jev is seemingly 32k.
That's a pretty big limitation, I would argue, unless I'm misunderstanding and it can be worked around easily somehow? I'm surprised it isn't surfaced more prominently in the comparison.
1. bjt12345 · 1789822214 · focus · HN ↗
  Jev has 64k total token request budget and I do wonder how it will handle highly specialised inputs.
  This Jev waitlist that Typesafe AI are utilising is surely going to raise questions pretty soon - it's hard to sell this to bosses when it looks like a pop-up restaurant
  1. Foobar8568 · 1789837043 · focus · HN ↗
    I am more curious about a 60k prompt... I haven't seen much discussion about large prompts, is it still < 500ms?

Unofficial Hacker News client; not affiliated with Y Combinator.