Thread

1. eqmvii · 1790189198 · focus · HN ↗

   In a year or two, articles like this will either be artifacts from peak hype or evidence of the beginning of the singularity. Right?

   1. pizza234 · 1790189484 · focus · HN ↗

      The singularity, as defined by Hinton (and others) as RSI (Recursive Self Improvement) may actually be beginning already, as OpenAI has announced an AI acting as a "research intern" (!).

      1. jackb4040 · 1790192998 · focus · HN ↗

         How is this different from arguing that Microsoft Clippy was RSI? An AI tool being involved in the process of work can't be the bar for RSI.

         I don't think there can be a coherent definition of RSI unless people lay out their theory for how intelligence scales. LLM-assisted coding is great but respectfully optimizing pytorch features or whatever is not gonna lead to exponential improvements. That approach to scaling diminished years ago, leading all the labs to switch to reasoning.

         Now it seems reasoning is also yielding diminishing returns, so all the labs are pivoting to specializing in particular fields like math / infosec / biology. They're improving due to accessing new proprietary training data and doing RL with human experts. Again I don't really see any amount of "AI research interns" leading to an exponential improvement to this strategy, they're not the bottleneck in the first place.

         1. famouswaffles · 1790204320 · focus · HN ↗

            >Now it seems reasoning is also yielding diminishing returns

            Is the diminishing returns in the room with us?

            >so all the labs are pivoting to specializing in particular fields like math / infosec / biology.

            They're not pivoting to anything. The goal has always been creating a machine that could automate all or nearly all human work. They're just coming along on that mission.

            As for RSI...I think the term is a bit odd in the modern context. It was created at a time when conventional wisdom was that generally intelligent machines would be these logic automatons that could "alter their own code". Instead we have massive neural networks that take months to train.

            In this paradigm, the ways a LLM could "improve itself" would be altering its own weights directly or creating and training better, vastly more efficient architectures for the next generation of models.

            The former is probably not happening but the latter is possible.

            1. jackb4040 · 1790205052 · focus · HN ↗

               Yes, diminishing returns. Not overall, they've still been able to create more intelligent models even up to today. But the strategy for scaling that intelligence has shifted. From the initial ChatGPT release to GPT-4.1, they were basically scaling up compute training compute / model size. Then 4.5 flopped, while o1 demonstrated that gains could continue by reasoning (scaling up compute at inference time). o1 is now the ancestor of all their flagship models from GPT-5 on.

               This is why I'm trying so hard to drill down on the theory of scaling, and not just talk about improvement in general, hand-wavy terms. If the bottleneck of current scaling strategies is training data, or something fundamental about the model architecture, then just throwing more harnessed chatbots at it won't lead to an exponential increase in performance.

               Now you could argue that the AI we have now will help us find that change in architecture, and I would agree. But that means we're firmly outside the singularity for the time being, and what people are in fact talking about is a hypothetical.

               1. famouswaffles · 1790207279 · focus · HN ↗

                  >Then 4.5 flopped, while o1 demonstrated that gains could continue by reasoning (scaling up compute at inference time). o1 is now the ancestor of all their flagship models from GPT-5 on.

                  That's not quite right. They are still scaling model size and have had several new base pre-trains, just nothing so big as 4.5 (as far as we're aware). o1/4o has not been the base for some time now.

                  Data is obviously a bottleneck for some regimes and LLMs will have to get their hands dirty experimenting but it doesn't look like an insurmountable wall either.

                  1. jackb4040 · 1790258611 · focus · HN ↗

                     > "get their hands dirty" > "insurmountable wall"

                     This is gibberish, you may as well tell me you've found a load-bearing seam.

                     1. famouswaffles · 1790272373 · focus · HN ↗

                        Okay? There's no reason the reinforcement learning that is getting them better at computer use can't be applied to other domains, like biology, chemistry etc. It's just expensive, because it requires creating labs like anthropic are doing here, and it requires llms attempting their own experiments. Getting the data and setup will be expensive, but not impossible, and labs are clearly gearing up to do just that. If you can't understand that then that seems on you.

Unofficial Hacker News client; not affiliated with Y Combinator.