I'm still confused about what this codemode is. Models have been trained to chain bash and other typical unix tools well. They're so good at that to an uncanny level. Why do we want to not utilize this ability? Is it just a permission management issue in case you don't want the model to use shell directly?
The value is that rather than an agent chaining together tool calls itself (which means each step sends the result back to the agent for it to analyse and work out what to do next), it writes a script for the harness to execute that chains together all the calls. The major benefits are:
* speed - much fewer hops back to the LLM
* fewer tokens - intermediate execution steps in the script don't leak into context, only the final result does.
* repeatability - if the LLM needs to repeat work, it can reuse a script it wrote last time.
If you have a harness that has access to a full shell and knows how to use bash or python, you'll often see it writing little scripts. For setups that don't (ie normal model API requests with tool calls), you can give it an lightweight secure execution environment like just-bash, or quickjs.
And I said that they do this in my last paragraph. But you need an execution environment for this, and you don’t get this automatically when just interacting with models via their API
raincole · · focus · HN ↗
agentdev001 · · focus · HN ↗
andrewingram · · focus · HN ↗
* speed - much fewer hops back to the LLM
* fewer tokens - intermediate execution steps in the script don't leak into context, only the final result does.
* repeatability - if the LLM needs to repeat work, it can reuse a script it wrote last time.
If you have a harness that has access to a full shell and knows how to use bash or python, you'll often see it writing little scripts. For setups that don't (ie normal model API requests with tool calls), you can give it an lightweight secure execution environment like just-bash, or quickjs.
krzyk · · focus · HN ↗
andrewingram · · focus · HN ↗
anthonypasq · · focus · HN ↗