The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
But if the problem is literally grep (search this file you've never seen before), no index can pre-exist.
If you assume the file arrives ahead of time, can be indexed, and that this is worthwhile because we want to support multiple pattern matched retrievals, then sure it makes sense to consider indexed query schemes and upper/lower bounds. Each query could be faster as an inference if it doesn't have to re-scan the whole file.
But I don't think anybody, in good faith, can pretend that any LLM can digest a file faster than grep can. Particularly, if you admit the vector processing dedicated to doing the convolution kernel(s), you should also admit similar hardware could run a vectorized grep.
jetrink · · focus · HN ↗
The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
gwbas1c · · focus · HN ↗
Grep reads through the entire file looking for patterns.
An LLM scans its neural net (in ways that I don't understand) which is kinda-sorta like having a huge index.
You can improve over Grep if you have an index; and the LLM has an index.
Thus, it's plausible that an LLM can be more efficient at reading its neural net (IE, index) than Grep reading the whole file.
saltcured · · focus · HN ↗
If you assume the file arrives ahead of time, can be indexed, and that this is worthwhile because we want to support multiple pattern matched retrievals, then sure it makes sense to consider indexed query schemes and upper/lower bounds. Each query could be faster as an inference if it doesn't have to re-scan the whole file.
But I don't think anybody, in good faith, can pretend that any LLM can digest a file faster than grep can. Particularly, if you admit the vector processing dedicated to doing the convolution kernel(s), you should also admit similar hardware could run a vectorized grep.