This will be a weird rant, but the dialogue here is a perfect example of how SOTA models are so heavily tuned towards "solving agentic tasks" that they're useless at almost everything else - especially creative tasks.
"Coming for you, Crimson!"
"You'll never catch me alive, Azure!"
That's why nobody has been able to stick these things in a video game successfully, even though it seems like the tech is a perfect match.
It's all a game to them. They aren't afraid for their lives. They're making a mockery out of the world you've put them in. Those are not the words of little pixel people fighting to the death, those are AI abominations making "tool calls", LARPing as little pixel people fighting to the death.
I'm 100% serious when I say that you would've gotten cooler outputs with a GPT 3.5-era model, once you managed to beat it into producing structured output. Llama 2 would be giving the other agent a heartwarming story about how if it kills it there would be nobody to take care of its grandma or whatever, and the other agent would probably spare it.
The output is just so bland and devoid of soul. I feel like we would've found a lot of cool use cases for LLMs, had we not completely maimed their output in the pursuit of getting them to output 3% better TypeScript.
IMO this is because responding within a json response with all those other fields shifts the distribution to text that's staler.
If you told it to roleplay a pixel knight fighting to the death and provided it responses like "Authur (renamed claude sonnet 5) moves west towards you menacingly", the whole thing would come alive.
I've been working on an AI roleplaying game for a while now and it had felt stale until I separated the story output from the plumbing output (importantly, I do story first and then base the plumbing on that). I think the two just have incompatible distributions.
I've been working on AI roleplaying too on and off, but after...
"Authur moves west towards you menacingly"
...RP is imemediately broken in the reasoning chain, and it's in that mode the prose is generated, e.g.
"Alright. Arthur moved west towards me menacingly. As Merlin, I should do X. No, I should do Y. Hmm. Maybe I should do Z. Let me reconsider. OK, I'll do X. How would Merlin respond? Maybe 'I'm a cool wizard'. No. 'I'm Merlin the wizard!' Yes, that's the load-bearing literary masteripiece. Let me respond with that!"
nananana9 · · focus · HN ↗
"Coming for you, Crimson!"
"You'll never catch me alive, Azure!"
That's why nobody has been able to stick these things in a video game successfully, even though it seems like the tech is a perfect match.
It's all a game to them. They aren't afraid for their lives. They're making a mockery out of the world you've put them in. Those are not the words of little pixel people fighting to the death, those are AI abominations making "tool calls", LARPing as little pixel people fighting to the death.
I'm 100% serious when I say that you would've gotten cooler outputs with a GPT 3.5-era model, once you managed to beat it into producing structured output. Llama 2 would be giving the other agent a heartwarming story about how if it kills it there would be nobody to take care of its grandma or whatever, and the other agent would probably spare it.
The output is just so bland and devoid of soul. I feel like we would've found a lot of cool use cases for LLMs, had we not completely maimed their output in the pursuit of getting them to output 3% better TypeScript.
JarJarBeatU · · focus · HN ↗
If you told it to roleplay a pixel knight fighting to the death and provided it responses like "Authur (renamed claude sonnet 5) moves west towards you menacingly", the whole thing would come alive.
I've been working on an AI roleplaying game for a while now and it had felt stale until I separated the story output from the plumbing output (importantly, I do story first and then base the plumbing on that). I think the two just have incompatible distributions.
nananana9 · · focus · HN ↗
"Authur moves west towards you menacingly"
...RP is imemediately broken in the reasoning chain, and it's in that mode the prose is generated, e.g.
"Alright. Arthur moved west towards me menacingly. As Merlin, I should do X. No, I should do Y. Hmm. Maybe I should do Z. Let me reconsider. OK, I'll do X. How would Merlin respond? Maybe 'I'm a cool wizard'. No. 'I'm Merlin the wizard!' Yes, that's the load-bearing literary masteripiece. Let me respond with that!"
"I'm Merlin the wizard!"