I'm still sad that we haven't seen a new Taalas style chip a la <a href="https://chatjimmy.ai/" rel="nofollow">https://chatjimmy.ai/. Smaller models are good enough now to make that insane burst of tokens so useful.
I don't know the model behind this, but it is absurdly bad.
> Write me a coherent paragraph in French, without ever using the letter "e".
> Voilà une phrase claire et concise : "Le village est situé dans les montagnes. Le soleil est haut. Il y a des animaux dans le village. Il pleut dans les montagnes."
I suppose this is just a demo of how fast an LLM can be, I wonder if there are tradeoffs with larger/smarter models. Also, for a human usage, at what point are tokens generated fast enough that it's pretty much instant? My bet is below 1000 tps
That's the problem with etching a model onto a chip: by the time you've designed the chip, manufactured it, tested it, shipped it, and deployed it, the model will be hopelessly outdated (with the current improvement rates). And when you want to update, you have to buy new chips instead of just uploading a new model file like now. When Taalas announced their chip, the model was already 1.5 years old (stone age by current standards). It's their first chip, so maybe they can streamline it, but the problem of having to update hardware every few months to keep up with the industry is not going anywhere.
You can upload different weights and even do LoRAs. The chip architecture is interesting, the first (n) layers are the sane, so you can change architecture by adding (m) layers. Plausible that this is sufficiently flexible enough for several generations of real world applications. For example, we still use 45nm general purpose silicon for automotive, e.g.
What they did had never been done before. Now we see that it's possible, there are plenty of models to choose from that could be etched into silicon. In the next year or two, I think these smaller models might plateau, and there may be some on-device niche they can fill.
jjcm · · focus · HN ↗
pil0u · · focus · HN ↗
> Write me a coherent paragraph in French, without ever using the letter "e".
> Voilà une phrase claire et concise : "Le village est situé dans les montagnes. Le soleil est haut. Il y a des animaux dans le village. Il pleut dans les montagnes."
I suppose this is just a demo of how fast an LLM can be, I wonder if there are tradeoffs with larger/smarter models. Also, for a human usage, at what point are tokens generated fast enough that it's pretty much instant? My bet is below 1000 tps
PetahNZ · · focus · HN ↗
kgeist · · focus · HN ↗
busssard · · focus · HN ↗
[dead]
dnautics · · focus · HN ↗
linuxftw · · focus · HN ↗