When thinking about ‘AI’ I sometimes find it useful to replace the terms ‘model’ or ‘AI’ or whatever with a description of what we’re really talking about here: a whole bunch of floating point numbers.
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
That’s overly reductive and thus completely misses the problem entirely.
What are biological neurones if not electric impulses of varying strengths?
We need to stop thinking of LLMs as purely a piece of software and ask those hard-to-define philosophical questions if we want any hope of building safety into our models.
Just because consciousness is something we don't fully understand doesn't mean it's the same thing as something else that's similar that we understand. That's a cargo-cult. The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
> Just because consciousness is something we don't fully understand doesn't mean it's the same thing as something else that's similar that we understand. That's a cargo-cult.
Nobody has said they’re aren’t different. And it’s not cargo cult to say you need philosophical conversations to understand how they’re different.
> The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
That’s not even remotely a true. Computer scientists understand how a neurone works. The conversations are around what constitutes as “intelligence”. That’s not a technical limitation of the models, it’s a philosophical problem we need to figure out.
No you don't understand how a neuron works fully or else you'd be able to make a synthetic one from chemical building blocks not using another living thing to do it. If you used another living thing to do it, you could say an ancient farmer understands plants cause they can put seeds in the ground. You have an incomplete model of it that is wrong but useful. Call me back when you can 3d-print a living person from raw chemical feedstock.
On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
> No you don't understand how a neuron works fully or else you'd be able to make a synthetic one from chemical building blocks not using another living thing to do it.
This is a bizarre contention. The test for whether something is understood isn't that we have the engineering capability to replicate it.
Just because you partially understand something like the ancient farmer partially understands plants by knowing to plant seeds, that doesn't mean you fully understand it and can thus reproduce all the properties of it synthetically like you are trying to reproduce a brain synthetically from scratch inside a computer with LLMs.
You can reproduce part of it, the intelligence, but consciousness and intelligence are not necessarily the same thing. This is why I compare it to cargo-culting. The Melanesian tribesmen thinks the coconut looks like a microphone, and he can use it as a paperweight just like a microphone and that's useful, but it is not actually a microphone.
thadt · · focus · HN ↗
> In a series of private meetings, the company consulted religious scholars to help instill morality into its FLOATING POINT NUMBERS — and make the case that their numbers could be conscious.
I recommend keeping a few NaNs in-hand if you’re worried. Spread a couple of around if your numbers start to stir and you’ll be right as rain.
> It appeared to Rabbi Navon that Mr. Olah and his team believed that FLOATING POINT NUMBERS had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
They’re cleverly designed and work well for their purpose, but can assure you they have neither dignity nor moral status.
> If the FLOATING POINT NUMBERS themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.
I get it. I’ve spent years trying to get my floating point numbers to choose goodness - I wish them luck.
hnlmorg · · focus · HN ↗
What are biological neurones if not electric impulses of varying strengths?
We need to stop thinking of LLMs as purely a piece of software and ask those hard-to-define philosophical questions if we want any hope of building safety into our models.
narrator · · focus · HN ↗
hnlmorg · · focus · HN ↗
Nobody has said they’re aren’t different. And it’s not cargo cult to say you need philosophical conversations to understand how they’re different.
> The stone age Melanesian islanders making air strips out of bundles of weeds and talking into coconuts as if they were microphones to bring back western goods was the same thing as us making LLMs to recreate consciousness. They were using technology they understood to try to recreate something they didn't.
That’s not even remotely a true. Computer scientists understand how a neurone works. The conversations are around what constitutes as “intelligence”. That’s not a technical limitation of the models, it’s a philosophical problem we need to figure out.
narrator · · focus · HN ↗
On top of all that, you're abusing the fallacy of composition. The bolt that is part of the aircraft carrier is light, therefore the aircraft carrier is also light. The atom is simple, therefore a living human being that is composed of atoms is also simple. Quantum mechanics is influenced by the observer, therefore you can change the external world just by thinking, and other new age stuff like that abuses a similar fallacy.
squidbeak · · focus · HN ↗
This is a bizarre contention. The test for whether something is understood isn't that we have the engineering capability to replicate it.
narrator · · focus · HN ↗
You can reproduce part of it, the intelligence, but consciousness and intelligence are not necessarily the same thing. This is why I compare it to cargo-culting. The Melanesian tribesmen thinks the coconut looks like a microphone, and he can use it as a paperweight just like a microphone and that's useful, but it is not actually a microphone.
hnlmorg · · focus · HN ↗
The crux of the matter seems to be that you think it isn’t worth our effort researching just because it isn’t identical.
But that isn’t how research and discovery happens.