The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
A few years is hardly forever and the state of the world here indicates a lot of low hanging fruit still exists.
An LLM can certainly be cheaper than grep, because it’s an approximation, while a grep is deterministic and must examine every byte in what can be a relatively complex state machine for a regex based grep. There are other scales to consider like the scale of your local hardware vs the highly multitenant and high end hardware of the hyper scale inference providers.
There are already high volume models for coding inference where the reasoning time is crazy low and cheap per token where it can build reasonably simple software so blindingly fast it isn’t implausible the bottleneck is the latency in tools and networks. I find them hard to use at times because I don’t have time to think through the next turn by the time it’s done.
Regardless I wouldn’t be surprised to see a world where tokens are so cheap it’s not worth metering them but charging licensing feels with meter tiers at the far horizons to prevent abuse, charge outliers. Subscription models already set this stage well.
The other side to consider is bountiful capacity will also drive tokens to near zero price. The data center build out is barely underway and as it materializes, as hardware efficiencies improve, as techniques and model science and technology improves, harnesses, methodologies , etc improve, the economics flip from load shedding to trying to keep the data centers utilized. The economics lead to the world where tokens are not a unit of measurement for cost for anyone other than the inference providers to manage their utilization.
jetrink · · focus · HN ↗
The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
fnordpiglet · · focus · HN ↗
An LLM can certainly be cheaper than grep, because it’s an approximation, while a grep is deterministic and must examine every byte in what can be a relatively complex state machine for a regex based grep. There are other scales to consider like the scale of your local hardware vs the highly multitenant and high end hardware of the hyper scale inference providers.
There are already high volume models for coding inference where the reasoning time is crazy low and cheap per token where it can build reasonably simple software so blindingly fast it isn’t implausible the bottleneck is the latency in tools and networks. I find them hard to use at times because I don’t have time to think through the next turn by the time it’s done.
Regardless I wouldn’t be surprised to see a world where tokens are so cheap it’s not worth metering them but charging licensing feels with meter tiers at the far horizons to prevent abuse, charge outliers. Subscription models already set this stage well.
The other side to consider is bountiful capacity will also drive tokens to near zero price. The data center build out is barely underway and as it materializes, as hardware efficiencies improve, as techniques and model science and technology improves, harnesses, methodologies , etc improve, the economics flip from load shedding to trying to keep the data centers utilized. The economics lead to the world where tokens are not a unit of measurement for cost for anyone other than the inference providers to manage their utilization.
[deleted] · · focus · HN ↗
[deleted]