The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
Yeah I bumped on that too. If it's possible to make llms cheaper than current grep, then it is also almost certainly possible to make grep cheaper.
You can burn anything* into an ASIC to make it cheaper per-call.
non-backreferencing grep is not very difficult to implement in an ASIC either. But it's probably not worth it because of how relatively rarely you use it and of the data transfer costs.
LLMs are great candidates for ASIC-burning because they're slow compared even to network speeds and run all the time. The issue is that you don't want to burn a specific model or architecture that then becomes obsolete.
So you've got two possible futures, and both guarantee large price drops: (a) LLMs keep getting better and better and better, so ability/$ keeps rising; or (b) LLMs plateau in ability, in which they will start getting ASIC'd.
> LLMs plateau in ability, in which they will start getting ASIC'd.
They don't need to plateu for that to happen. There are companies already building AI on ASIC, and IIRC they were approach 12 months lead time. A 12 months old frontier model (Sonnet 4.5, GPT-5, Kimi K2) for 1% of the price is still a rather good value proposition.
It will be a better value proposition now than it was 12 months ago. It's likely to be better yet in another 12 months. There may be room for a parallel to Moore's Law here.
Right now, I do actually use OpenAI's gpt-oss-safeguard-20b for somethings, was released 11 months ago, and is $0.075/M input / $0.30/M output now. I could see this model being in fairly widespread use at 10x speed and 1/10th cost if it was introduced today. Meaning, that for some usecases (moderation) i think dedicated chips can pan out today.
But for more general models, its tougher. Gemini 3 pro was launched in November, if ASICs brought it down 1/10th in cost, it would be $0.20/$1.2. GPT 6 Luna is $0.1/$0.50. Luna is better at a lot of things, but not everything. So 1/10th doesn't really make the ASICS investment worth it in my opinion, but if it brought it down to 1% ($0.02 / $0.12) it would be a really compelling model with a lot of use.
BUT, do i think something like Luna is probably generally capable of doing a huge amount of knowledge work. So if Luna came out at 1/10th the cost a year from now, it would probably be compelling for a while.
It all depends on the rate of improvement in cost/capability.
In order for the ASIC to achieve 1% of the price, there'd have to be 99% overhead in GPU implementations which for some reason you'd have to be able to eliminate in ASICs but not in GPUs. That seems rather implausible.
jetrink · · focus · HN ↗
The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
sanderjd · · focus · HN ↗
serbuvlad · · focus · HN ↗
non-backreferencing grep is not very difficult to implement in an ASIC either. But it's probably not worth it because of how relatively rarely you use it and of the data transfer costs.
LLMs are great candidates for ASIC-burning because they're slow compared even to network speeds and run all the time. The issue is that you don't want to burn a specific model or architecture that then becomes obsolete.
So you've got two possible futures, and both guarantee large price drops: (a) LLMs keep getting better and better and better, so ability/$ keeps rising; or (b) LLMs plateau in ability, in which they will start getting ASIC'd.
hobofan · · focus · HN ↗
They don't need to plateu for that to happen. There are companies already building AI on ASIC, and IIRC they were approach 12 months lead time. A 12 months old frontier model (Sonnet 4.5, GPT-5, Kimi K2) for 1% of the price is still a rather good value proposition.
cestith · · focus · HN ↗
mchusma · · focus · HN ↗
Right now, I do actually use OpenAI's gpt-oss-safeguard-20b for somethings, was released 11 months ago, and is $0.075/M input / $0.30/M output now. I could see this model being in fairly widespread use at 10x speed and 1/10th cost if it was introduced today. Meaning, that for some usecases (moderation) i think dedicated chips can pan out today.
But for more general models, its tougher. Gemini 3 pro was launched in November, if ASICs brought it down 1/10th in cost, it would be $0.20/$1.2. GPT 6 Luna is $0.1/$0.50. Luna is better at a lot of things, but not everything. So 1/10th doesn't really make the ASICS investment worth it in my opinion, but if it brought it down to 1% ($0.02 / $0.12) it would be a really compelling model with a lot of use.
BUT, do i think something like Luna is probably generally capable of doing a huge amount of knowledge work. So if Luna came out at 1/10th the cost a year from now, it would probably be compelling for a while.
It all depends on the rate of improvement in cost/capability.
sanderjd · · focus · HN ↗
atq2119 · · focus · HN ↗