Thread

1. arjie · 1785889891 · focus · HN ↗

   For small models like this, it’s super important that it works well at tool calling etc. imho because it can’t memorize facts and isn’t big enough to tell when it doesn’t know. I could use it for high quality tool routing or a backup fast model for smaller task set. E.g. I use GPT-5.6 for voice channels at home. I’d prefer to be able to have this do basic tool calls and stuff because of the local speed.

   Will give it a crack as a quick model in my clawlike.

   1. sdiazthomas · 1785890684 · focus · HN ↗

      This matches what I've seen shipping Apple's on-device model in a Mac app.

      The model is reliable at the semantic half. Give it the OCR text of a receipt and it correctly identifies the vendor and the date. What it does not do reliably is follow mechanical instructions. A user asked for dates formatted as TT-MM-JJJJ and got files literally named TT-MM-JJJJ, because it reproduced the format string instead of filling it in. Another asked for uppercase, and the model acknowledged the request in its reasoning and returned lowercase.

      The failures were not consistent, which is worse than failing every time. You cannot tell users "this doesn't work", only "this works most of the time", and nobody accepts that from something touching their files.

      What fixed it was moving the mechanical part out of the model entirely. The model decides what the document is about; ordinary deterministic code decides how the name is written. Every time I moved that line back toward the model, quality dropped.

      Which is a version of your point: with a small model the win isn't making it smarter, it's shrinking what you make it responsible for.

Unofficial Hacker News client; not affiliated with Y Combinator.