This is so interesting to watch. For a couple minutes I was in awe of how quick and cheap it was. Then I saw just how bad the decision are and how it would get stuck in strange loops of going in and out of the same door to no end.
This seems like a technology heading in the right direction but not quiet there yet. Excited for what they are cooking up but probably won't start building around it yet.
Like others have mentioned in this post, I think a mix of models like Jev for simple stuff + a smarter reasoning model for more strategic thinking is the optimal solution. This experiment however is purely Jev. Which sometimes can be kinda dumb.
Well in a lot of ways this demo actually is a mix of jev for simple stuff + a very intelligent harness to fill in the rest. Pay attention to the "jev counter" on when api calls actually happen and the kinds of decisions its making. Its very rarely in a tight loop, and when it is (eg in an item menu) it tends to randomly walk through options.
Its very cool, but the harness is doing a _ton_ of heavy lifting.
Oh I want to be clear, I don't want my post to come off as negative. I really do mean things are headed the right direction and I think this harness is super cool. So don't take it that way and thank you for bringing something cool into this world.
stusmall · · focus · HN ↗
This seems like a technology heading in the right direction but not quiet there yet. Excited for what they are cooking up but probably won't start building around it yet.
pancomplex · · focus · HN ↗
JamesSwift · · focus · HN ↗
Its very cool, but the harness is doing a _ton_ of heavy lifting.
stusmall · · focus · HN ↗