Thread

1. Tsarp · 1789750649 · focus · HN ↗

   Apart from fictional use cases, what is the real use case here? The pricing on some open models are absurdly low for generic tasks. For the privacy conscious it makes sense to run something like a 8-27B on local network and get the work done.

   Are there perhaps some industrial or agri use cases?

   1. HenryNdubuaku · 1789751149 · focus · HN ↗

      Fair, we gotta do a better job at explaining this properly!

      So an 8-27B on a LAN box wins for generic tasks on hardware that can hold it. Needle is for hardware that can't, like plain ARMv7, MIPS32 (the Ingenic chips in cheap IP cameras), RISC-V and watches. Also, we found cost to not really be the lever for on-device models, but availability and latency.

      1. ash_091 · 1789761360 · focus · HN ↗

         So it will run on those devices, but when/why would you do that?

         All of the demo setups (smart home, robot vacuum, watch, etc) could easily have access to a bigger model running on a more capable device either locally or via the internet.

Unofficial Hacker News client; not affiliated with Y Combinator.