Thread

Show HN: Pac-Bench – How well can models one-shot a Pac-Man game?

80 points · 50 comments · thefourthchime

drcxd · 1790669044 · focus · HN ↗
Interesting, recently I am working on my own clone of Pac-Man. LLM implementations lose lots of details. They are not 1:1 replication of the original game. For example, the behavior of the ghost is not the same as the original. If I have not implemented the game myself, I can not tell the differences. What LLM produced look like the original game, but they are not.
1. CamperBob2 · 1790700015 · focus · HN ↗
  But it takes -- what, two or three sentences? -- to explain how the ghosts should move. The interesting (and important) thing is not that the model gets it wrong at first, it's how easy it is to correct it.
  One-shotting something like Pac-Man doesn't prove much. At the end of the day, one-shot fidelity is going to scale more or less linearly with model size/world knowledge. Why wouldn't it?

Unofficial Hacker News client; not affiliated with Y Combinator.