‹ BackHN Continuity

Thread

I built non-autoregressive decision models with RL a year ago

1363 points · 319 comments · nandakishor_ml

  1. cube2222 · · focus · HN ↗
    Quickly reading the article, one notable limitation seems to be that these checkpoints are 512-1024 tokens context size models, while Jev is seemingly 32k.

    That's a pretty big limitation, I would argue, unless I'm misunderstanding and it can be worked around easily somehow? I'm surprised it isn't surfaced more prominently in the comparison.

    1. druskacik · · focus · HN ↗
      Yeah, it's weird, considering ModernBERT, which the Laya models are based on, supports 8192 context window.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.