Thread

Brood War Bench

346 points · 156 comments · benswerd

dschuessler · 1789850237 · focus · HN ↗
Somewhat related: In 2018, Google DeepMind had already created AIs that were capable of beating professional gamers in StarCraft 2 (the sequel to Brood War): <a href="https://www.youtube.com/watch?v=cUTMhmVh1qs" rel="nofollow">https://www.youtube.com/watch?v=cUTMhmVh1qs
1. benswerd · 1789850553 · focus · HN ↗
  I predict LLMs will reach superhuman level and beat even that model in the next 12 months
  1. orbital-decay · 1789851847 · focus · HN ↗
    Starcraft is APM-dependent. Unless the latency will improve greatly in frontier reasoning LLMs (which is unlikely), it will remain a bit like knitting with an excavator.
    benswerd · 1789851955 · focus · HN ↗
    I predict latency will improve greatly in the next 12 months to more than 4x speed on current frontier tasks
    loeg · 1789852013 · focus · HN ↗
    Yeah but Starcraft needs, like, 10-20x the APM these agents are doing.
    benswerd · 1789852126 · focus · HN ↗
    I’m not convinced a lot of it can’t be solved with code mode.
    Marine staggering for example seems like an ideal code mode task.
    loeg · 1789853250 · focus · HN ↗
    Yeah. Some of it may just be "thinking" less rather than faster token generation.
    jayd16 · 1789886327 · focus · HN ↗
    Do the humans get to use this auto-stagger too?
    swiftcoder · 1789902205 · focus · HN ↗
    The best sc2 bots these days play in the ~50,000 APM range (they could mostly go higher as well, but the game client breaks somewhere around 100k APM). I don't see an LLM-based bot getting up to that sort of speed anytime soon.
    On the other hand, I do think LLM-based bots will quickly outperform the decision-making of many of the hand-coded bots, so maybe they won't need so much APM to be competitive.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.