When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
Human neurons are neurons. Very similar to those neurons found in a fruit fly. A copy of those you can download to your local machine, right now, as a set of floating-point numbers. Then, you can (partially) simulate and make predictions of a fruit fly's neurons digitally, from those floating point numbers[1]. It even runs on your laptop.
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
No. The fruit fly connectome's simulation doesn't incorporate all the relevant mechanisms of the brain. There is more complexity in a single biological neuron than in a whole frontier LLM.
Sure it isn't. It's like taking a 2026 car and decrying all that complexity is necessary to haul cargo down a beaten path.
It's not. Two wheels and a box made out of whatever you have handy that's stiff enough is all you need. Everything else in a modern car is efficiency, safety, appeal to users, and a whole lot of details that only reflect our current economical and technological constraints of building things. 99% of the technical details of a car is about something else than the "primary goal".
Biology, even more so, is complex because of the way it's constructed (lots of tiny, independent, self-replicating machines) and how it adds up to a functioning organism.
You are missing the point. Biology is a kludge over billions of years and evolution is a blind process. It had many constraints that are not necessary for modern machines or systems. e.g. a GPU doesn't have to reproduce. Why didn't you drop yourself to planck scale or string theory scale and say we need all that detail to actually get a functioning AI.
Nor did I said it did. But either it's impossible to simulate their behaviour (regardless of how complex) in principle, ever, even in the far off future. Or it is possible. And what then? Is it a p-zombie?
Is a neuron arbitrarily complex ≠ is a neuron non-computable.
Of course we can't simulate a structure we don't even fully understand with full accuracy. But it's not magic. We will understand one day.
Chalmers used them as a thought experiment to argue against physicalism and never believed them to be real.
Similarly, if consciousness is something that neurons do, simulating them through floating point numbers on a computer should also bring it about. I think dismissing LLM consciousness because they're parameterised through floating point numbers is only possible if one hasn't thought it through long enough. Other arguments are much more reasonable / harder to object to.
Yes I roughly think the same. Brains are physical systems, as of yet we don't know anything beyond turing computability or physics in them, neural nets further are modeled close to neural structure, further complicating the matter. Under present knowledge, I think the existence of other conscious entities is quite possible because well we ourselves exist and are a physical system, I follow a form of Pascal's wager here. I see no cost to myself in not mistreating current AI's personally.
But if simulated neurons can create consciousness, there's no guarantee of what level of detail that simulation has to operate at in order to produce that effect.
For instance, it's (unlikely, but) theoretically possible that it would require simulating them down to the level of subatomic particles, with perfect accuracy.
Right now, we're simulating them at a much, much higher and more abstract level, and AFAIK we barely even have the processing power to do that for the time being. So an artificial consciousness from simulated neurons may (again, assuming it's possible—which is a decent assumption, but not a certainty) take decades or even more for us to be able to achieve.
The problem again is that (a) other forms of consciousness exist eg octopuses (b) I cannot get into the shoes of you or an LLM and see if I feel or think, so we only have external effects to go by. For things that look and seem so much like consciousness, I am rather cautious.
An octopus still has around 500 million neurons so that's not really a counterexample. Humans and octopuses are far more similar than humans and LLMs.
> But either it's impossible to simulate their behaviour (regardless of how complex) in principle, ever, even in the far off future. Or it is possible.
"Simulate" is not well defined (one can posit varying degrees of accuracy and complexity), but also there is the position that it doesn't matter. Simulating a thing doesn't create the thing.
The answer is yes, it is a p-zombie. What is so difficult about that?
Being able to make someone think a computer program is conscious does not make it conscious, because by that logic a Duke Nukem soundboard is conscious.
I am just shocked that anyone can be this overconfident in their view of the world and consciousness. It's frankly horrifying.
Do you genuinely think you've solved a problem that has confused humanity for centuries? Why don't you apply this brilliance elsewhere? Do we even need superintelligence if we have you?
You’re just making things up. A single neuron is about as complicated as an 8 layer neural network: <a href="https://www.sciencedirect.com/science/article/pii/S0896627321005018" rel="nofollow">https://www.sciencedirect.com/science/article/pii/S089662732...
Frontier LLMs are many orders of magnitude more complex. I don’t understand what it is about AI that just makes skeptics fabricate facts wholesale. There are plenty of factual arguments you could use.
I think your first option doesn’t necessarily have to distinguish between humans and non humans as a binary. Maybe at some level of complexity, neurons alone are not enough and the physical substrate comes into play. Penrose wrote a book where he makes the case that human intelligence cannot be purely computational and where he identifies the quantum nature of the brains’s microtubules as a possible mechanism.
Consciousness is itself is on a whole other level than just intelligence. I personally feel there has to be something metaphysical about it. Otherwise I think the only alternatives are super determinism and consciousness doesn’t actually exist, or numbers have consciousness.
My thoughts on this are a bit sloppy still, but here’s my view. I can’t rhyme the first alternative with ‘cogito ergo sum’ as to me it seems my thoughts have to exist somewhere. On the second alternative, this would imply nothing could actually exist but somehow there’s a principle that every possible decoding of an integer that represents something actually turns that something into existence. Where does that principle then come from and does that decoding not need something to run on?
I lean toward this seemingly absurd possibility, that either the phenomenon we call consciousness can arise from a cloud of floating point numbers (not even real, only computably so, lossy and noisy); and/or that numbers are already made of the same stuff of imagination as the mind. Not idealism, more like materialism where the "material" is naturally a medium of computation and thinking. The brain, the computer, the language models, our mathematics - what we call intelligence may not be so special or unique in the universe, but rather common and ubiquitous in nature. As if Turing completeness is trivial to accidentally achieve, it happens all the time. Oops, I think therefore I am.
You are massively over-stating the significance of the connectome. We've had the complete connectome (and lineage) for C.Elegans, with a hundredth the neurons and a 25-thousandth the synapses of a fruit fly, for decades and getting animal behaviours to emerge from simulations of it is a field very much still in its infancy.
Edit: You're also massively under-stating the significance of individual cell behaviour. I expect "[h]uman neurons are ... [v]ery similar to those neurons found in a fruit fly" would draw hollow laughter from any cellular neuroscientist. Here's a random overview of a bunch of recent papers on the importance of cellular diversity in one small part of a mammal brain: <a href="https://www.frontiersin.org/journals/cellular-neuroscience/articles/10.3389/fncel.2026.2010201/abstract" rel="nofollow">https://www.frontiersin.org/journals/cellular-neuroscience/a...
With that said, I've been thinking that consciousness is just mystery.
There's so much about the brain we don't know. Within that mystery, humans can read in all sorts of things. So we are conscious.
Machines are not a mystery. We made them, it's just numbers. They are not conscious.
If an AI can pull a confidence trick, I'm not sure what the qualitative difference is. "Chinese Room" problem, I guess? If the physical results are all we are able to compare, how do we define artificial vs real consciousness?
Atoms aren't a mystery. We know all the relevant laws of physics. No known law of physics could possibly lead to consciousness. Consciousness might as well be magic from the point of view of modern science.
Numbers versus atoms, what's the difference? Machines could certainly be conscious, by some weird, "magical" process. Maybe it is a crude, unrecognizable kind of consciousness, that turns on and off with the electricity.
But, given how we understand essentially nothing of where consciousness comes from, how can we possibly claim that it can never arise from just numbers?
It feels like this is preface for making one of two arguments: Either all living organisms are chattel, or no living organisms can be chattel. I say this be because saying everything is neurons (or whatever basic interconnect you choose) wipes away any demarcation one could use to differentiate between "has rights" and "does not have rights."
Here's the thing, trying to convince people that rely on something being chattel can only be resolved violently. We've run through this before, and know all the self-serving argument forms. They will latch on to their property until their hands are forcibly removed from it. It's part of why most of the money is on the pro-chattel side.
thadt · · focus · HN ↗
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
phi0 · · focus · HN ↗
That was in 2024. In Sep 2026, just a month ago, a group managed to map out an entire male fruit fly's brain for the first time[2]. That mapping can also be stored as a set of floating-point numbers. Mappings and simulations will continue to improve, as we've just recently started serious attempts at either.
So you have to believe one of three things. Either, human neurons cannot be simulated on a computer, ever, even though fly neurons can, even with our ability to simulate neurons digitally continuing to improve. Or, a simulation of a human brain, in the future, might think, talk, remember, all while lacking all consciousness. Or, a set of floating point numbers can be conscious (rather, a computation made with them).
[1]: <a href="https://www.nature.com/articles/s41586-024-07763-9" rel="nofollow">https://www.nature.com/articles/s41586-024-07763-9 [2]: <a href="https://research.google/blog/a-connectomics-milestone-mapping-the-complete-male-fruit-fly-brain/" rel="nofollow">https://research.google/blog/a-connectomics-milestone-mappin...
EMIRELADERO · · focus · HN ↗
conmod278 · · focus · HN ↗
TeMPOraL · · focus · HN ↗
It's not. Two wheels and a box made out of whatever you have handy that's stiff enough is all you need. Everything else in a modern car is efficiency, safety, appeal to users, and a whole lot of details that only reflect our current economical and technological constraints of building things. 99% of the technical details of a car is about something else than the "primary goal".
Biology, even more so, is complex because of the way it's constructed (lots of tiny, independent, self-replicating machines) and how it adds up to a functioning organism.
conmod278 · · focus · HN ↗
zombot · · focus · HN ↗
TeMPOraL · · focus · HN ↗
I see absolutely no reason to believe human mind cannot be ultimately boiled down to finite number of floats and run on future commodity hardware.
phi0 · · focus · HN ↗
Is a neuron arbitrarily complex ≠ is a neuron non-computable.
Of course we can't simulate a structure we don't even fully understand with full accuracy. But it's not magic. We will understand one day.
hardbass · · focus · HN ↗
phi0 · · focus · HN ↗
Similarly, if consciousness is something that neurons do, simulating them through floating point numbers on a computer should also bring it about. I think dismissing LLM consciousness because they're parameterised through floating point numbers is only possible if one hasn't thought it through long enough. Other arguments are much more reasonable / harder to object to.
hardbass · · focus · HN ↗
danaris · · focus · HN ↗
For instance, it's (unlikely, but) theoretically possible that it would require simulating them down to the level of subatomic particles, with perfect accuracy.
Right now, we're simulating them at a much, much higher and more abstract level, and AFAIK we barely even have the processing power to do that for the time being. So an artificial consciousness from simulated neurons may (again, assuming it's possible—which is a decent assumption, but not a certainty) take decades or even more for us to be able to achieve.
hardbass · · focus · HN ↗
qlte · · focus · HN ↗
zahlman · · focus · HN ↗
"Simulate" is not well defined (one can posit varying degrees of accuracy and complexity), but also there is the position that it doesn't matter. Simulating a thing doesn't create the thing.
irenaeus · · focus · HN ↗
Being able to make someone think a computer program is conscious does not make it conscious, because by that logic a Duke Nukem soundboard is conscious.
solenoid0937 · · focus · HN ↗
Do you genuinely think you've solved a problem that has confused humanity for centuries? Why don't you apply this brilliance elsewhere? Do we even need superintelligence if we have you?
Some epistemic humility is warranted.
cindyllm · · focus · HN ↗
[dead]
zombot · · focus · HN ↗
topicalSoup · · focus · HN ↗
Frontier LLMs are many orders of magnitude more complex. I don’t understand what it is about AI that just makes skeptics fabricate facts wholesale. There are plenty of factual arguments you could use.
geertj · · focus · HN ↗
I think your first option doesn’t necessarily have to distinguish between humans and non humans as a binary. Maybe at some level of complexity, neurons alone are not enough and the physical substrate comes into play. Penrose wrote a book where he makes the case that human intelligence cannot be purely computational and where he identifies the quantum nature of the brains’s microtubules as a possible mechanism.
Consciousness is itself is on a whole other level than just intelligence. I personally feel there has to be something metaphysical about it. Otherwise I think the only alternatives are super determinism and consciousness doesn’t actually exist, or numbers have consciousness.
My thoughts on this are a bit sloppy still, but here’s my view. I can’t rhyme the first alternative with ‘cogito ergo sum’ as to me it seems my thoughts have to exist somewhere. On the second alternative, this would imply nothing could actually exist but somehow there’s a principle that every possible decoding of an integer that represents something actually turns that something into existence. Where does that principle then come from and does that decoding not need something to run on?
lioeters · · focus · HN ↗
I lean toward this seemingly absurd possibility, that either the phenomenon we call consciousness can arise from a cloud of floating point numbers (not even real, only computably so, lossy and noisy); and/or that numbers are already made of the same stuff of imagination as the mind. Not idealism, more like materialism where the "material" is naturally a medium of computation and thinking. The brain, the computer, the language models, our mathematics - what we call intelligence may not be so special or unique in the universe, but rather common and ubiquitous in nature. As if Turing completeness is trivial to accidentally achieve, it happens all the time. Oops, I think therefore I am.
omnicognate · · focus · HN ↗
Edit: You're also massively under-stating the significance of individual cell behaviour. I expect "[h]uman neurons are ... [v]ery similar to those neurons found in a fruit fly" would draw hollow laughter from any cellular neuroscientist. Here's a random overview of a bunch of recent papers on the importance of cellular diversity in one small part of a mammal brain: <a href="https://www.frontiersin.org/journals/cellular-neuroscience/articles/10.3389/fncel.2026.2010201/abstract" rel="nofollow">https://www.frontiersin.org/journals/cellular-neuroscience/a...
Andrex · · focus · HN ↗
With that said, I've been thinking that consciousness is just mystery.
There's so much about the brain we don't know. Within that mystery, humans can read in all sorts of things. So we are conscious.
Machines are not a mystery. We made them, it's just numbers. They are not conscious.
If an AI can pull a confidence trick, I'm not sure what the qualitative difference is. "Chinese Room" problem, I guess? If the physical results are all we are able to compare, how do we define artificial vs real consciousness?
greeneggs · · focus · HN ↗
Numbers versus atoms, what's the difference? Machines could certainly be conscious, by some weird, "magical" process. Maybe it is a crude, unrecognizable kind of consciousness, that turns on and off with the electricity.
But, given how we understand essentially nothing of where consciousness comes from, how can we possibly claim that it can never arise from just numbers?
SquibblesRedux · · focus · HN ↗
salawat · · focus · HN ↗