When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
Human neurons are neurons. Very similar to those neurons found in a fruit fly. A copy of those you can download to your local machine, right now, as a set of floating-point numbers. Then, you can (partially) simulate and make predictions of a fruit fly's neurons digitally, from those floating point numbers[1]. It even runs on your laptop.
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
No. The fruit fly connectome's simulation doesn't incorporate all the relevant mechanisms of the brain. There is more complexity in a single biological neuron than in a whole frontier LLM.
Nor did I said it did. But either it's impossible to simulate their behaviour (regardless of how complex) in principle, ever, even in the far off future. Or it is possible. And what then? Is it a p-zombie?
Is a neuron arbitrarily complex ≠ is a neuron non-computable.
Of course we can't simulate a structure we don't even fully understand with full accuracy. But it's not magic. We will understand one day.
Chalmers used them as a thought experiment to argue against physicalism and never believed them to be real.
Similarly, if consciousness is something that neurons do, simulating them through floating point numbers on a computer should also bring it about. I think dismissing LLM consciousness because they're parameterised through floating point numbers is only possible if one hasn't thought it through long enough. Other arguments are much more reasonable / harder to object to.
Yes I roughly think the same. Brains are physical systems, as of yet we don't know anything beyond turing computability or physics in them, neural nets further are modeled close to neural structure, further complicating the matter. Under present knowledge, I think the existence of other conscious entities is quite possible because well we ourselves exist and are a physical system, I follow a form of Pascal's wager here. I see no cost to myself in not mistreating current AI's personally.
But if simulated neurons can create consciousness, there's no guarantee of what level of detail that simulation has to operate at in order to produce that effect.
For instance, it's (unlikely, but) theoretically possible that it would require simulating them down to the level of subatomic particles, with perfect accuracy.
Right now, we're simulating them at a much, much higher and more abstract level, and AFAIK we barely even have the processing power to do that for the time being. So an artificial consciousness from simulated neurons may (again, assuming it's possible—which is a decent assumption, but not a certainty) take decades or even more for us to be able to achieve.
The problem again is that (a) other forms of consciousness exist eg octopuses (b) I cannot get into the shoes of you or an LLM and see if I feel or think, so we only have external effects to go by. For things that look and seem so much like consciousness, I am rather cautious.
An octopus still has around 500 million neurons so that's not really a counterexample. Humans and octopuses are far more similar than humans and LLMs.
> But either it's impossible to simulate their behaviour (regardless of how complex) in principle, ever, even in the far off future. Or it is possible.
"Simulate" is not well defined (one can posit varying degrees of accuracy and complexity), but also there is the position that it doesn't matter. Simulating a thing doesn't create the thing.
The answer is yes, it is a p-zombie. What is so difficult about that?
Being able to make someone think a computer program is conscious does not make it conscious, because by that logic a Duke Nukem soundboard is conscious.
I am just shocked that anyone can be this overconfident in their view of the world and consciousness. It's frankly horrifying.
Do you genuinely think you've solved a problem that has confused humanity for centuries? Why don't you apply this brilliance elsewhere? Do we even need superintelligence if we have you?
thadt · · focus · HN ↗
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
phi0 · · focus · HN ↗
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
[1]: <a href="https://www.nature.com/articles/s41586-024-07763-9" rel="nofollow">https://www.nature.com/articles/s41586-024-07763-9 [2]: <a href="https://research.google/blog/a-connectomics-milestone-mapping-the-complete-male-fruit-fly-brain/" rel="nofollow">https://research.google/blog/a-connectomics-milestone-mappin...
EMIRELADERO · · focus · HN ↗
phi0 · · focus · HN ↗
Is a neuron arbitrarily complex ≠ is a neuron non-computable.
Of course we can't simulate a structure we don't even fully understand with full accuracy. But it's not magic. We will understand one day.
hardbass · · focus · HN ↗
phi0 · · focus · HN ↗
Similarly, if consciousness is something that neurons do, simulating them through floating point numbers on a computer should also bring it about. I think dismissing LLM consciousness because they're parameterised through floating point numbers is only possible if one hasn't thought it through long enough. Other arguments are much more reasonable / harder to object to.
hardbass · · focus · HN ↗
danaris · · focus · HN ↗
For instance, it's (unlikely, but) theoretically possible that it would require simulating them down to the level of subatomic particles, with perfect accuracy.
Right now, we're simulating them at a much, much higher and more abstract level, and AFAIK we barely even have the processing power to do that for the time being. So an artificial consciousness from simulated neurons may (again, assuming it's possible—which is a decent assumption, but not a certainty) take decades or even more for us to be able to achieve.
hardbass · · focus · HN ↗
qlte · · focus · HN ↗
zahlman · · focus · HN ↗
"Simulate" is not well defined (one can posit varying degrees of accuracy and complexity), but also there is the position that it doesn't matter. Simulating a thing doesn't create the thing.
irenaeus · · focus · HN ↗
Being able to make someone think a computer program is conscious does not make it conscious, because by that logic a Duke Nukem soundboard is conscious.
solenoid0937 · · focus · HN ↗
Do you genuinely think you've solved a problem that has confused humanity for centuries? Why don't you apply this brilliance elsewhere? Do we even need superintelligence if we have you?
Some epistemic humility is warranted.
cindyllm · · focus · HN ↗
[dead]
zombot · · focus · HN ↗