Can someone help me understand whether these models are supposed to be good enough to be useful? I am asking these for a breakfast recipe and most of them are repeating text and giving me strange combinations of like chicken & parmesan or milk and like 8 cups of cheese. I really would love to use MicroLLMs and have been eagerly awaiting the day they could be useful, or at least run local LLMs for that matter, but these don't seem useful.
They don't even consistently pass the benchmarks included in the site, so what are they good for?
Edit: or is the purpose to just showcase that these small LLMs can run on WebGPU?
I tried the first four listed and came to comment as I was confused by what the expected output should be. Just tried the rest of them and none of them were able to answer the prompt: "Give me a breakfast recipe that I can make in less than 5 minutes"
The closest was SmoLM2 135M Instruct which gave me the milk and cheese piece, which was at least close to what a recipe looks like.
botanrice · · focus · HN ↗
They don't even consistently pass the benchmarks included in the site, so what are they good for?
Edit: or is the purpose to just showcase that these small LLMs can run on WebGPU?
logicallee · · focus · HN ↗
botanrice · · focus · HN ↗
The closest was SmoLM2 135M Instruct which gave me the milk and cheese piece, which was at least close to what a recipe looks like.