Thread

1. theresLand · 1790017014 · focus · HN ↗

   Seems like this would break the cache often. That would increase billing rates with certain providers and, for local models, take a while to generate responses, especially with long-running agentic sessions. Interesting idea though.

   1. MikhailTal · 1790018096 · focus · HN ↗

      No? from a quick skim it doesnt look like it goes into the system prompt everytime, you just use search/grep over it. Pretty much most memory approaches relying on a big set of info where agent chooses what to 'recall'. Its like any other tool

      1. messh · 1790027505 · focus · HN ↗

         If there are no summaries then when context is full messages need to get evicted. If doing so one by one then it would indeed destroy the cache. Of course... maybe the implementation evicted 50% of messages at once, I didnt verify in code
   2. TimByte · 1790071838 · focus · HN ↗

      Placing LLL inside the user message right ahead of the new query keeps the common conversation prefix fully intact

Unofficial Hacker News client; not affiliated with Y Combinator.