Thread · HN Continuity

It's an old paper (from 2025, so, a decade ago in AI years), but the concept is still fascinating. And I'm yet to see it show up in any production models.

If multiple models can use cache representations for this kind of enrichment, the KV cache representations of different models must be somewhat compatible.

What stops us then from going a step further, and producing a model family where all models are "KV aligned", and each model can utilize the KV cache of other models directly?

So, an "expensive" reasoning model can use its full faculties to plan, but "delegate" simple subgoals to a smaller model. That smaller model can access the large model's intent directly, as rich KV cache representations - with no prefill recompute and no associated "handover" latency. Or, likewise, a "cheap" small model can generate a diminished but highly compact KV cache that the "expensive" model can then operate on - for example, for skimming a large file for shallow patterns.