When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
Human neurons are neurons. Very similar to those neurons found in a fruit fly. A copy of those you can download to your local machine, right now, as a set of floating-point numbers. Then, you can (partially) simulate and make predictions of a fruit fly's neurons digitally, from those floating point numbers[1]. It even runs on your laptop.
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
No. The fruit fly connectome's simulation doesn't incorporate all the relevant mechanisms of the brain. There is more complexity in a single biological neuron than in a whole frontier LLM.
Nor did I said it did. But either it's impossible to simulate their behaviour (regardless of how complex) in principle, ever, even in the far off future. Or it is possible. And what then? Is it a p-zombie?
Is a neuron arbitrarily complex ≠ is a neuron non-computable.
Of course we can't simulate a structure we don't even fully understand with full accuracy. But it's not magic. We will understand one day.
> But either it's impossible to simulate their behaviour (regardless of how complex) in principle, ever, even in the far off future. Or it is possible.
"Simulate" is not well defined (one can posit varying degrees of accuracy and complexity), but also there is the position that it doesn't matter. Simulating a thing doesn't create the thing.
thadt · · focus · HN ↗
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
phi0 · · focus · HN ↗
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
[1]: <a href="https://www.nature.com/articles/s41586-024-07763-9" rel="nofollow">https://www.nature.com/articles/s41586-024-07763-9 [2]: <a href="https://research.google/blog/a-connectomics-milestone-mapping-the-complete-male-fruit-fly-brain/" rel="nofollow">https://research.google/blog/a-connectomics-milestone-mappin...
EMIRELADERO · · focus · HN ↗
phi0 · · focus · HN ↗
Is a neuron arbitrarily complex ≠ is a neuron non-computable.
Of course we can't simulate a structure we don't even fully understand with full accuracy. But it's not magic. We will understand one day.
zahlman · · focus · HN ↗
"Simulate" is not well defined (one can posit varying degrees of accuracy and complexity), but also there is the position that it doesn't matter. Simulating a thing doesn't create the thing.