When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
Human neurons are neurons. Very similar to those neurons found in a fruit fly. A copy of those you can download to your local machine, right now, as a set of floating-point numbers. Then, you can (partially) simulate and make predictions of a fruit fly's neurons digitally, from those floating point numbers[1]. It even runs on your laptop.
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
I think your first option doesn’t necessarily have to distinguish between humans and non humans as a binary. Maybe at some level of complexity, neurons alone are not enough and the physical substrate comes into play. Penrose wrote a book where he makes the case that human intelligence cannot be purely computational and where he identifies the quantum nature of the brains’s microtubules as a possible mechanism.
Consciousness is itself is on a whole other level than just intelligence. I personally feel there has to be something metaphysical about it. Otherwise I think the only alternatives are super determinism and consciousness doesn’t actually exist, or numbers have consciousness.
My thoughts on this are a bit sloppy still, but here’s my view. I can’t rhyme the first alternative with ‘cogito ergo sum’ as to me it seems my thoughts have to exist somewhere. On the second alternative, this would imply nothing could actually exist but somehow there’s a principle that every possible decoding of an integer that represents something actually turns that something into existence. Where does that principle then come from and does that decoding not need something to run on?
I lean toward this seemingly absurd possibility, that either the phenomenon we call consciousness can arise from a cloud of floating point numbers (not even real, only computably so, lossy and noisy); and/or that numbers are already made of the same stuff of imagination as the mind. Not idealism, more like materialism where the "material" is naturally a medium of computation and thinking. The brain, the computer, the language models, our mathematics - what we call intelligence may not be so special or unique in the universe, but rather common and ubiquitous in nature. As if Turing completeness is trivial to accidentally achieve, it happens all the time. Oops, I think therefore I am.
thadt · · focus · HN ↗
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
phi0 · · focus · HN ↗
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
[1]: <a href="https://www.nature.com/articles/s41586-024-07763-9" rel="nofollow">https://www.nature.com/articles/s41586-024-07763-9 [2]: <a href="https://research.google/blog/a-connectomics-milestone-mapping-the-complete-male-fruit-fly-brain/" rel="nofollow">https://research.google/blog/a-connectomics-milestone-mappin...
geertj · · focus · HN ↗
I think your first option doesn’t necessarily have to distinguish between humans and non humans as a binary. Maybe at some level of complexity, neurons alone are not enough and the physical substrate comes into play. Penrose wrote a book where he makes the case that human intelligence cannot be purely computational and where he identifies the quantum nature of the brains’s microtubules as a possible mechanism.
Consciousness is itself is on a whole other level than just intelligence. I personally feel there has to be something metaphysical about it. Otherwise I think the only alternatives are super determinism and consciousness doesn’t actually exist, or numbers have consciousness.
My thoughts on this are a bit sloppy still, but here’s my view. I can’t rhyme the first alternative with ‘cogito ergo sum’ as to me it seems my thoughts have to exist somewhere. On the second alternative, this would imply nothing could actually exist but somehow there’s a principle that every possible decoding of an integer that represents something actually turns that something into existence. Where does that principle then come from and does that decoding not need something to run on?
lioeters · · focus · HN ↗
I lean toward this seemingly absurd possibility, that either the phenomenon we call consciousness can arise from a cloud of floating point numbers (not even real, only computably so, lossy and noisy); and/or that numbers are already made of the same stuff of imagination as the mind. Not idealism, more like materialism where the "material" is naturally a medium of computation and thinking. The brain, the computer, the language models, our mathematics - what we call intelligence may not be so special or unique in the universe, but rather common and ubiquitous in nature. As if Turing completeness is trivial to accidentally achieve, it happens all the time. Oops, I think therefore I am.