Thread

1. nylonstrung · 1790103554 · focus · HN ↗

   I don't really understand how async tool calls translate to token savings

   It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?

   1. swiftcoder · 1790146158 · focus · HN ↗

      I assume the bigger problem is the cache expiring while you wait for the tool call to complete. A bunch of hosted providers only keep the cache alive for ~5 minutes - if you sit there waiting for a 5 minute tool call, you get to pay to reload the entire context into cache

Unofficial Hacker News client; not affiliated with Y Combinator.