When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
That’s overly reductive and thus completely misses the problem entirely.
What are biological neurones if not electric impulses of varying strengths?
We need to stop thinking of LLMs as purely a piece of software and ask those hard-to-define philosophical questions if we want any hope of building safety into our models.
Just because consciousness is something we don't fully understand doesn't mean it's the same thing as something else that's similar that we understand. That's a cargo-cult. The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
> Just because consciousness is something we don't fully understand doesn't mean it's the same thing as something else that's similar that we understand. That's a cargo-cult.
Nobody has said they’re aren’t different. And it’s not cargo cult to say you need philosophical conversations to understand how they’re different.
> The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
That’s not even remotely a true. Computer scientists understand how a neurone works. The conversations are around what constitutes as “intelligence”. That’s not a technical limitation of the models, it’s a philosophical problem we need to figure out.
No you don't understand how a neuron works fully or else you'd be able to make a synthetic one from chemical building blocks not using another living thing to do it. If you used another living thing to do it, you could say an ancient farmer understands plants cause they can put seeds in the ground. You have an incomplete model of it that is wrong but useful. Call me back when you can 3d-print a living person from raw chemical feedstock.
On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
> No you don't understand how a neuron works fully or else you'd be able to make a synthetic one from chemical building blocks not using another living thing to do it.
We don’t know how a quantum mechanics works fully and yet we’ve built countless systems based on quantum behaviour including the CPUs that power the computer you’re replying to me on.
The point here is we’ve built something that emulates intelligence and thus further exploration into what intelligence is, is required. Arguing it isn’t intelligent because it was built differently isn’t an answer, it just dismisses the question entirely.
> On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
You’re the one who keeps bringing up composition, not me. I’m the one telling you that you’re falling into that fallacy. So don’t start telling me that I’m wrong because of an argument you’re exclusively making.
My points were this:
1. we don’t have an understanding of what intelligence is and thus we need to explore that problem more deeply.
Given your comments about our ignorance, you of all people should agree with this point.
2. Arguing that AI is just “floating points” is overly reductive and misses the point that we don’t know what “intelligence” is.
So instead of claiming that AI is just numbers, a more constructive stance to take would be to build out better vocabulary and understanding about the distinctions between different types of “intelligence”.
Again, this aligns with your comment about the composition fallacy. Thus you if all people should be supporting my comments here.
Also your comment about us not understanding how a neurone works does vast injustice to the amount of research that has been done in this field. At this point in time we do have a very good understanding of how a neurone works. What we don’t understand is how consciousness arises from it. And that’s the very thing you seem to be arguing that there’s no point researching.
> Also your comment about us not understanding how a neurone works does vast injustice to the amount of research that has been done in this field. At this point in time we do have a very good understanding of how a neurone works. What we don’t understand is how consciousness arises from it. And that’s the very thing you seem to be arguing that there’s no point researching.
This is the part you keep stating that is wrong. We know a lot about the most obvious electrical way that neurons act. That is nowhere near having a good understanding of how they work.
Define “work”? Because the way they act is the way they work. What isn’t understood is how complex systems arise from these relatively simple parts. Which is the point I’ve made repeatedly.
I responded with an informative post but apparently made the mistake of disclosing it was generated and it was flagged. I reviewed the post and it is correct. If you cannot see it, it says you are wrong/incomplete here and cites three studies I know of:
Beniaguev, Segev & London, "Single cortical neurons as deep artificial neural networks," Neuron, 10 August 2021, Volume 109, Issue 17, pp. 2727–2739.e3
<a href="https://doi.org/10.1016/j.neuron.2021.07.002" rel="nofollow">https://doi.org/10.1016/j.neuron.2021.07.002
Open-access preprint: <a href="https://www.biorxiv.org/content/10.1101/613141v2" rel="nofollow">https://www.biorxiv.org/content/10.1101/613141v2
Gidon et al., "Dendritic action potentials and computation in human layer 2/3 cortical neurons," Science, 367(6473), 83–87 (2020)
<a href="https://doi.org/10.1126/science.aax6239" rel="nofollow">https://doi.org/10.1126/science.aax6239
Randi, Sharma, Dvali & Leifer, "Neural signal propagation atlas of Caenorhabditis elegans," Nature, 623(7986), 2023, pp. 406–414
<a href="https://doi.org/10.1038/s41586-023-06683-4" rel="nofollow">https://doi.org/10.1038/s41586-023-06683-4
Open-access preprint: <a href="https://arxiv.org/abs/2208.04790" rel="nofollow">https://arxiv.org/abs/2208.04790
Neurons are not individually simple, in any sense. Nor are they the only class of cell involved in cognitive processes — glial cells are involved in inhibition, memory, network modification, etc.
While I was not participating actively in the discussion - I just want to thank both participants for embodying the spirit of HN and academic discourse - especially in the face of a difficult debate that stirs strong emotions.
@svnt, @hnlmorg - this conversation - and the links it produced - are a fascinating read, and honestly gave me hope for discourse on the open internet.
> No you don't understand how a neuron works fully or else you'd be able to make a synthetic one from chemical building blocks not using another living thing to do it.
This is a bizarre contention. The test for whether something is understood isn't that we have the engineering capability to replicate it.
Just because you partially understand something like the ancient farmer partially understands plants by knowing to plant seeds, that doesn't mean you fully understand it and can thus reproduce all the properties of it synthetically like you are trying to reproduce a brain synthetically from scratch inside a computer with LLMs.
You can reproduce part of it, the intelligence, but consciousness and intelligence are not necessarily the same thing. This is why I compare it to cargo-culting. The Melanesian tribesmen thinks the coconut looks like a microphone, and he can use it as a paperweight just like a microphone and that's useful, but it is not actually a microphone.
A spider obviously "knows" how to build a spider web.
Does a spider "understand" how to build a spider web?
From a pragmatic theory of truth, a spider obviously "understands" how to build a spider web. A spider understands how to build a spider web much better than a human.
But when it comes to LLMs we start adding all this extra nonsense. As if the spider is blanket "smarter" than me because I can't spin a web. If the spider is "smarter" than me then it must also have this Cartesian theater of the mind like many people think they do.
It is layered philosophical misunderstandings to talk about LLMs in this way.
What could be an alternative explanation or approach for that spider? I’m out of loop, interested in what is currently academia trying to solve, thanks.
Neurons are different from GPU logic gates and the physical mechanisms by which a network of physical neurons computes answers is extremely different from the physical mechanism by which a GPU computes an answer.
> We need to stop thinking of LLMs as purely a piece of software
Intelligence from LLMs are an emergent property of probability. The LLM chooses a token to add to a chain of tokens based on the probability that humans will want to see that token. Humans see that token and assign it a qualitative but unfalsifiable intelligence score.
The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not. A cat is intelligent regardless of what humans think. Is an LLM intelligent regardless of what humans think?
If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world? If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
> If an LLM were to live a year with dolphins, would its weights change
Given three or more large scale training runs per year, probably yes once the training data is dominated by dolphin-related input. Much quicker in any case than the genes of the humans living with dolphins would be shaped by Evolution, which produces new generations much less frequently.
> Intelligence from LLMs are an emergent property of probability. The LLM chooses a token to add to a chain of tokens based on the probability that humans will want to see that token.
How is that any different to how our brains work?
> The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not.
Philosophically speaking, that’s also true of human intelligence. For example, the only evidence I have for your intelligence is what I ascribe from the text you’ve written.
> A cat is intelligent regardless of what humans think.
Are they? Humans can’t actually agree on whether cats are sentient. And often pet owners will be accused of anthropomorphising cats when describing intelligent behaviour in cats.
> Is an LLM intelligent regardless of what humans think?
That’s the question that needs researching.
> If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world?
If it were trained on dolphin behaviour, then yes. Much like humans.
The difference here is that models don’t persist context in the same way humans do so the weights have to be hardcoded in a way that biological neurones are not.
> If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
Evolution is an entirely different subject to intelligence. They’re not related in any way at all.
Plus humans have actually created new species of plants and animals. So it’s not like we can’t artificially put our thumb on the scale there either
———
I should add, I don’t actually think LLMs are sentient. I don’t even know if I’d call them “intelligent” (in the sense of what we describe in conversations such as this). But we also don’t know what intelligence is. Which is why we need to research this problem further.
thadt · · focus · HN ↗
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
hnlmorg · · focus · HN ↗
What are biological neurones if not electric impulses of varying strengths?
We need to stop thinking of LLMs as purely a piece of software and ask those hard-to-define philosophical questions if we want any hope of building safety into our models.
narrator · · focus · HN ↗
hnlmorg · · focus · HN ↗
Nobody has said they’re aren’t different. And it’s not cargo cult to say you need philosophical conversations to understand how they’re different.
> The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
That’s not even remotely a true. Computer scientists understand how a neurone works. The conversations are around what constitutes as “intelligence”. That’s not a technical limitation of the models, it’s a philosophical problem we need to figure out.
narrator · · focus · HN ↗
On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
[deleted] · · focus · HN ↗
[deleted]
hnlmorg · · focus · HN ↗
We don’t know how a quantum mechanics works fully and yet we’ve built countless systems based on quantum behaviour including the CPUs that power the computer you’re replying to me on.
The point here is we’ve built something that emulates intelligence and thus further exploration into what intelligence is, is required. Arguing it isn’t intelligent because it was built differently isn’t an answer, it just dismisses the question entirely.
> On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
You’re the one who keeps bringing up composition, not me. I’m the one telling you that you’re falling into that fallacy. So don’t start telling me that I’m wrong because of an argument you’re exclusively making.
My points were this:
1. we don’t have an understanding of what intelligence is and thus we need to explore that problem more deeply.
Given your comments about our ignorance, you of all people should agree with this point.
2. Arguing that AI is just “floating points” is overly reductive and misses the point that we don’t know what “intelligence” is.
So instead of claiming that AI is just numbers, a more constructive stance to take would be to build out better vocabulary and understanding about the distinctions between different types of “intelligence”.
Again, this aligns with your comment about the composition fallacy. Thus you if all people should be supporting my comments here.
Also your comment about us not understanding how a neurone works does vast injustice to the amount of research that has been done in this field. At this point in time we do have a very good understanding of how a neurone works. What we don’t understand is how consciousness arises from it. And that’s the very thing you seem to be arguing that there’s no point researching.
svnt · · focus · HN ↗
This is the part you keep stating that is wrong. We know a lot about the most obvious electrical way that neurons act. That is nowhere near having a good understanding of how they work.
hnlmorg · · focus · HN ↗
svnt · · focus · HN ↗
[dead]
hnlmorg · · focus · HN ↗
I’m going to hold off any more constructive response because I’m not familiar with those research papers specifically so have some homework to do :)
But I just wanted you to be aware that your comment was appreciated even if HN flagged it unfairly.
(I can’t reply in thread for the same reason. But hopefully you see this).
svnt · · focus · HN ↗
Beniaguev, Segev & London, "Single cortical neurons as deep artificial neural networks," Neuron, 10 August 2021, Volume 109, Issue 17, pp. 2727–2739.e3 <a href="https://doi.org/10.1016/j.neuron.2021.07.002" rel="nofollow">https://doi.org/10.1016/j.neuron.2021.07.002 Open-access preprint: <a href="https://www.biorxiv.org/content/10.1101/613141v2" rel="nofollow">https://www.biorxiv.org/content/10.1101/613141v2
Gidon et al., "Dendritic action potentials and computation in human layer 2/3 cortical neurons," Science, 367(6473), 83–87 (2020) <a href="https://doi.org/10.1126/science.aax6239" rel="nofollow">https://doi.org/10.1126/science.aax6239
Randi, Sharma, Dvali & Leifer, "Neural signal propagation atlas of Caenorhabditis elegans," Nature, 623(7986), 2023, pp. 406–414 <a href="https://doi.org/10.1038/s41586-023-06683-4" rel="nofollow">https://doi.org/10.1038/s41586-023-06683-4 Open-access preprint: <a href="https://arxiv.org/abs/2208.04790" rel="nofollow">https://arxiv.org/abs/2208.04790
Neurons are not individually simple, in any sense. Nor are they the only class of cell involved in cognitive processes — glial cells are involved in inhibition, memory, network modification, etc.
foobar10000 · · focus · HN ↗
@svnt, @hnlmorg - this conversation - and the links it produced - are a fascinating read, and honestly gave me hope for discourse on the open internet.
squidbeak · · focus · HN ↗
This is a bizarre contention. The test for whether something is understood isn't that we have the engineering capability to replicate it.
narrator · · focus · HN ↗
You can reproduce part of it, the intelligence, but consciousness and intelligence are not necessarily the same thing. This is why I compare it to cargo-culting. The Melanesian tribesmen thinks the coconut looks like a microphone, and he can use it as a paperweight just like a microphone and that's useful, but it is not actually a microphone.
hnlmorg · · focus · HN ↗
The crux of the matter seems to be that you think it isn’t worth our effort researching just because it isn’t identical.
But that isn’t how research and discovery happens.
lutanista · · focus · HN ↗
A spider obviously "knows" how to build a spider web.
Does a spider "understand" how to build a spider web?
From a pragmatic theory of truth, a spider obviously "understands" how to build a spider web. A spider understands how to build a spider web much better than a human.
But when it comes to LLMs we start adding all this extra nonsense. As if the spider is blanket "smarter" than me because I can't spin a web. If the spider is "smarter" than me then it must also have this Cartesian theater of the mind like many people think they do.
It is layered philosophical misunderstandings to talk about LLMs in this way.
BOOSTERHIDROGEN · · focus · HN ↗
irenaeus · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
hnlmorg · · focus · HN ↗
You shouldn’t compare biological neurones to microprocessor logic gates because that’s like comparing a car to a pair of shoes.
saimiam · · focus · HN ↗
Intelligence from LLMs are an emergent property of probability. The LLM chooses a token to add to a chain of tokens based on the probability that humans will want to see that token. Humans see that token and assign it a qualitative but unfalsifiable intelligence score.
The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not. A cat is intelligent regardless of what humans think. Is an LLM intelligent regardless of what humans think?
If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world? If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
mofeien · · focus · HN ↗
Given three or more large scale training runs per year, probably yes once the training data is dominated by dolphin-related input. Much quicker in any case than the genes of the humans living with dolphins would be shaped by Evolution, which produces new generations much less frequently.
hnlmorg · · focus · HN ↗
How is that any different to how our brains work?
> The first question we need to ask is whether a system whose claim of intelligence depends on the interpretation of what it produces by other intelligent beings, namely humans, intelligent or not.
Philosophically speaking, that’s also true of human intelligence. For example, the only evidence I have for your intelligence is what I ascribe from the text you’ve written.
> A cat is intelligent regardless of what humans think.
Are they? Humans can’t actually agree on whether cats are sentient. And often pet owners will be accused of anthropomorphising cats when describing intelligent behaviour in cats.
> Is an LLM intelligent regardless of what humans think?
That’s the question that needs researching.
> If an LLM were to live a year with dolphins, would its weights change to respond to its new reality that it was now operating in dolphin-world?
If it were trained on dolphin behaviour, then yes. Much like humans.
The difference here is that models don’t persist context in the same way humans do so the weights have to be hardcoded in a way that biological neurones are not.
> If you sent humans to live in dolphin world, we would eventually evolve to be less landloving and more dolphin like (Darwin thinks so, at least).
Evolution is an entirely different subject to intelligence. They’re not related in any way at all.
Plus humans have actually created new species of plants and animals. So it’s not like we can’t artificially put our thumb on the scale there either
———
I should add, I don’t actually think LLMs are sentient. I don’t even know if I’d call them “intelligent” (in the sense of what we describe in conversations such as this). But we also don’t know what intelligence is. Which is why we need to research this problem further.