Thread

1. mncharity · 1790794592 · focus · HN ↗

   Fwiw, top of my own pain-point list (I suppose given the first item, that's a pun) includes:

   External/policy-based throttling for temperature control. Unthrottled, my laptop bottom goes skin-burn hot. But fixed compute caps can have non-linearly dreadful performance impacts in particular cases. Plan is a runtime knob, to replace manual limits-kludgery.

   I'll use models which barely fit in VRAM+RAM, and are order-1 tok/s slow. So tool call step overhead can be painful - a world where `ls` costs tens of seconds. Plan is blending harness plugins with inference loop, for "no, don't stop - I already have the call result for you - just keep going" (and also some logit games).

   1. anerli · 1790795209 · focus · HN ↗

      Temperature control is a good callout! Feel free to open an issue for any features that you'd like to see in Magnitude

      <a href="https:&#x2F;&#x2F;github.com&#x2F;magnitudedev&#x2F;magnitude&#x2F;issues" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;magnitudedev&#x2F;magnitude&#x2F;issues

   2. aidiscoverywire · 1790835361 · focus · HN ↗

      [dead]

Unofficial Hacker News client; not affiliated with Y Combinator.