> Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.
Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.
If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.
Imagine the first time electric calculators calculated the square root of 5. I'm sure people would verify again and again if what the circuits calculated was right.
Then in the 80s, you presses 2 buttons and there you had it in your classroom without thinking twice if the electricity arrived correctly at the transistors.
How crazy will the world be once our [current gen] ANN are like that!
Given the disappointing levels of intellectual decay that our current technology has thrust upon civilization, I only see humans reverting back to neanderthal levels of intelligence in short time with the advent of AI
> One wonders whether a generation that demands instant satisfaction of all its needs and instant solution of the world's problems will produce anything of lasting value. Such a generation, even when equipped with the most modern technology, will be essentially primitive — it will stand in awe of nature, and submit to the tutelage of medicine men.
There's no reliable evidence that Neanderthals were less intelligent than modern humans. They're extinct now (except for a tiny genetic legacy in some human populations) but that could have happened for a variety of reasons unrelated to intelligence or lack thereof.
No you don't. If people don't learn to solve problems and exercise their mind with small problems, how are they going to deal with big problems? The "neanderthal" part is a colorful expression.
First AI came for the parrots and I did nothing, at least I can stand for the neanderthals.
There are so many parallels between what TV could have become and the progression of AI, a nearly free conveyance for culture, education, art and discourse. Yet, we allowed it spiral in a positive feedback loop, creating a cognitive gyre that now razes society.
At least the youngest now use "that's AI" as a pejorative, as in that is bullshit.
Calculators and computers are deterministic, they give the same output to the same output every time. Language models specifically are not. So it might give you are function that is correct, or a function that is not, or worse yet a function that behaves correctly, but introduces some god-awful bug down the line that can cause serious havoc. It is obvious that they need supervision, not only for output, but also sandboxing and various harnesses for them to not do any “oops, I deleted your codebase sry” kind of nonsense people post to Reddit.
So I think the problem is to determine which problems under what instructions we can safely give to a model application to solve and how we test the output for safety and functionality. This would create more usable and safe, albeit a bit more boring, AI-based applications alin to a calculator or general computer. Whether this is posswith current model architecture is another thing.
Memory bits flip randomly. It's not a super rare thing either. You and me have experienced that many times without knowing. The only reason that computers feel deterministic is that we have error-correcting code to fix that. But in the most extreme cases, when multiple bits flip together, once "deterministic" program can generate unexpected output.
So why do you trust computers? Because statistically the case is just very unlikely. Therefore if AI is statistically unlikely to make mistakes there is no reason to not trust them.
You are confusing hardware noise and whether the algorithm itself is verifiable as if parity bits were not a thing. We don’t trust computers because the medium itself is infallible, but because the logic is sound.
With statistical models - such as LLM’s - there is no logic as such, but statistical assumptions based on given data. The output can ge very good or very bad, but you are fool to trust it blindly. Therefore you need a deterministic way to verify, whether meat- or software-based.
A program can only be as deterministic as the hardware it runs on. If you have a turing machine whose symbols on the tape just (truly) randomly change every step, then none of programs on this machine is "deterministic."
> meat-
There is no meat-based deterministic way to do anything yet yet. You're talking sci-fi.
Why will X not keep increasing? It’s been increasing, albeit very slowly, since the start of the 20th century as the disease burden declines and nutrition improves. I see no reason to think that future health interventions couldn’t further juice those numbers.
Will it keep up with Y? Probably not, unless people are willing to accept pretty radical interventions to their biology. But it almost certainly is not static
If you set the temperature 0, an LLM is also deterministic (same prompt -> same output every single time). We just don't do this because the LLM is not so smart in that mode. But "LLM is not so smart" is changing at Moore's law speeds.
Non-determinism is not an essential property of LLMs. It's an optimization that we've added intentionally.
Ehm, no! The same prompt does not always lead to the same output.
Have you ever tried to achieve consistently deterministic output from an LLM? I have, and it's not easy.
That means output differs between machines and architectures. Running inference on CPU vs GPU also affects output. Even running the same prompt twice in a row on the same machine can lead to different outputs because a prompt that was partially stored in the kv cache will result in different output than an uncached prompt.
At the end of the day, an LLM is just a very big mathematical function. That is, by definition, deterministic. A particular implementation might give up on determinism for the sake of higher efficiency, but it you want a deterministic LLM, it can absolutely be done.
Yes, yes, assuming the same CPU and stuff. There is some randomness due to floating point math differences between CPUs (and sometimes on the same CPU), but I don't think that fundamentally changes my point.
Put another way: if you could have a virtualization layer that guarantees deterministic floating point operations then a LLM set to 0.0 temp would produce deterministic output.
It is insane how many times I see this false analogy repeated on HN over and over (analogy of a deterministic-by-design calculator device (or a compiler, etc.) and a non-deterministic-by-design LLM software).
You're missing the forest for the trees here. The point OP is trying to make is that calculators originally were essentially non-deterministic. Technology will go from unreliable to reliable.
LLMs are not non-deterministic by design. The randomness in the output is artificially injected for ergonomic reasons. (Yes, the non-determinism in production systems is different, but also not by design)
You don’t train calculators on how to add. All calculators will more or less come up with the same answer to some deterministic and quantifiable level of accuracy and precision.
pyridines · · focus · HN ↗
Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.
If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.
xtracto · · focus · HN ↗
Then in the 80s, you presses 2 buttons and there you had it in your classroom without thinking twice if the electricity arrived correctly at the transistors.
How crazy will the world be once our [current gen] ANN are like that!
What an amazing thought.
vinyl7 · · focus · HN ↗
Seattle3503 · · focus · HN ↗
- Eric Hoffer
ares623 · · focus · HN ↗
- Tech Bro
nradov · · focus · HN ↗
genxy · · focus · HN ↗
were gonna need a citation on this one.
groundzeros2015 · · focus · HN ↗
genxy · · focus · HN ↗
There are so many parallels between what TV could have become and the progression of AI, a nearly free conveyance for culture, education, art and discourse. Yet, we allowed it spiral in a positive feedback loop, creating a cognitive gyre that now razes society.
At least the youngest now use "that's AI" as a pejorative, as in that is bullshit.
itishappy · · focus · HN ↗
delis-thumbs-7e · · focus · HN ↗
So I think the problem is to determine which problems under what instructions we can safely give to a model application to solve and how we test the output for safety and functionality. This would create more usable and safe, albeit a bit more boring, AI-based applications alin to a calculator or general computer. Whether this is posswith current model architecture is another thing.
raincole · · focus · HN ↗
How do you know?
Memory bits flip randomly. It's not a super rare thing either. You and me have experienced that many times without knowing. The only reason that computers feel deterministic is that we have error-correcting code to fix that. But in the most extreme cases, when multiple bits flip together, once "deterministic" program can generate unexpected output.
So why do you trust computers? Because statistically the case is just very unlikely. Therefore if AI is statistically unlikely to make mistakes there is no reason to not trust them.
delis-thumbs-7e · · focus · HN ↗
With statistical models - such as LLM’s - there is no logic as such, but statistical assumptions based on given data. The output can ge very good or very bad, but you are fool to trust it blindly. Therefore you need a deterministic way to verify, whether meat- or software-based.
senordevnyc · · focus · HN ↗
raincole · · focus · HN ↗
> meat-
There is no meat-based deterministic way to do anything yet yet. You're talking sci-fi.
zjjsjsjjs · · focus · HN ↗
[dead]
reustle · · focus · HN ↗
AI (IQ of Y, non deterministic) can write deterministic code.
Y is going to keep increasing, while X will not.
derektank · · focus · HN ↗
Will it keep up with Y? Probably not, unless people are willing to accept pretty radical interventions to their biology. But it almost certainly is not static
int_19h · · focus · HN ↗
The increases still happen globally but mostly driven by developing countries.
FabHK · · focus · HN ↗
jasondigitized · · focus · HN ↗
roncesvalles · · focus · HN ↗
Non-determinism is not an essential property of LLMs. It's an optimization that we've added intentionally.
ResearchAtPlay · · focus · HN ↗
Have you ever tried to achieve consistently deterministic output from an LLM? I have, and it's not easy.
That means output differs between machines and architectures. Running inference on CPU vs GPU also affects output. Even running the same prompt twice in a row on the same machine can lead to different outputs because a prompt that was partially stored in the kv cache will result in different output than an uncached prompt.
LLM output is very much not deterministic!
int_19h · · focus · HN ↗
At the end of the day, an LLM is just a very big mathematical function. That is, by definition, deterministic. A particular implementation might give up on determinism for the sake of higher efficiency, but it you want a deterministic LLM, it can absolutely be done.
zorked · · focus · HN ↗
If you ran an LLM with infinite precision and guaranteed order of execution, it would be deterministic.
(I think determinism is overrated. Being deterministic does not make LLMs more reliable or correct.)
jiggawatts · · focus · HN ↗
Non-determinism is injected, on purpose.
roncesvalles · · focus · HN ↗
Put another way: if you could have a virtualization layer that guarantees deterministic floating point operations then a LLM set to 0.0 temp would produce deterministic output.
CuriouslyC · · focus · HN ↗
sedan_baklazhan · · focus · HN ↗
CrimsonRain · · focus · HN ↗
Nothing is stopping LLMs to be more deterministic/correct over time.
Also you yourself is nondeterministic :)
sedan_baklazhan · · focus · HN ↗
All people are. That is how automation appeared to begin with - to provide deterministic behavior.
CrimsonRain · · focus · HN ↗
So what's stopping "non deterministic" AIs from doing the same, at faster/larger scale? :)
freecodeio · · focus · HN ↗
you clearly don't know how transistors work, which is ironic because it's a thread in hackernews about LLMs and mathematics
CrimsonRain · · focus · HN ↗
uejfiweun · · focus · HN ↗
sedan_baklazhan · · focus · HN ↗
xorcist · · focus · HN ↗
hackinthebochs · · focus · HN ↗
goalieca · · focus · HN ↗
amelius · · focus · HN ↗
Imagine AI crushing quantum mechanics like Einstein pwned classical physics.