I would be concerned with context management consuming limited attention resources.
Do you want your agent solving its own memory crisis, or do you want it solving the actual task? It can probably do both at the same time, but I suspect there is a non trivial cost associated with this.
A separate hypervisor agent that manages the main agent's context would be much better in my experience. You can run it on a different schedule and the main agent has to spend zero tokens thinking about it. This also makes it a lot easier to control when caches will be missed.
Looks like they tested that in the paper, and the code allows for it as well. <a href="https://github.com/facebookresearch/context-language-models/tree/main/clm/clm_icl" rel="nofollow">https://github.com/facebookresearch/context-language-models/... Of course, this is for suggesting context management strategies rather than the actual management afaict.
An argument against having a separate hypervisor agent.
A separate hypervisor agent at least doubles the cost because (1) the underlying model needs to be the same so that we get the same degree of intelligence, and (2) it needs to have the same context + more tokens for the work it does.
bob1029 · · focus · HN ↗
Do you want your agent solving its own memory crisis, or do you want it solving the actual task? It can probably do both at the same time, but I suspect there is a non trivial cost associated with this.
A separate hypervisor agent that manages the main agent's context would be much better in my experience. You can run it on a different schedule and the main agent has to spend zero tokens thinking about it. This also makes it a lot easier to control when caches will be missed.
alightsoul · · focus · HN ↗
theroadnotbacon · · focus · HN ↗
alightsoul · · focus · HN ↗
vatsachak · · focus · HN ↗
verdverm · · focus · HN ↗
naw103 · · focus · HN ↗
[dead]
rajveerb · · focus · HN ↗
A separate hypervisor agent at least doubles the cost because (1) the underlying model needs to be the same so that we get the same degree of intelligence, and (2) it needs to have the same context + more tokens for the work it does.
flyinfra · · focus · HN ↗
[dead]