Thread · HN Continuity

It is surprising to me that recursive self-improvement seems more plausible now than it did in 2023. Am I the only one to be surprised?

I remember the paper proving that hallucinations could never be fully solved back in 2024: <a href="https://arxiv.org/abs/2409.05746" rel="nofollow">https://arxiv.org/abs/2409.05746

I also remember the hang-wringing about running out of new datasets to train on. Now it appears humans are always generating more data. It's just not as cheap to acquire as legacy data? Meta has to give a deep discount on their API prices to entice people.

I thought back then that humans had a few more breakthroughs in them as meaningful as the seminal Attention is all you need paper. Enough to 100x the capabilities of LLMs back then (10x the smarts and 10x the speed simultaneously).

RSI with a 20 month turnaround for a chip to be made is not exactly breakneck speed though. Physical manufacturing and logistical constraints are going to be and remain a hard obstacle to that process for the foreseeable future.