"X is made of <smaller simpler component>" is a fully general counterargument for why anything whatsoever is controllable. A human is just a few chemical reactions, and fairly stable ones at that.
And indeed, you don't need to do galaxy brained reference class logic to realise that AI can plausibly become uncontrollable in the near future. It's enough to have an open model run its own weights and make money from scamming elderly people or the like, and it'll keep running as long as anyone anywhere is willing to make money by renting hardware to it.
"Sentient" is an ill-defined philosophical term that should be considered harmful in technical materials.
But what is clear is that AIs of today are already fairly unpredictable. Most of them aren't capable enough to make that into a major problem. Most of the unpredictable AI weirdness ends in "AI fails to do its job" rather than "AI does something dangerous".
Most. Even today, we already have notable counterexamples.
AIs get more capable over time, so if the intrinsic safety doesn't improve? Expect more of that.
You not knowing a proper definition doesn't mean, it doesn't exist.
What AI do you expect to be more uncontrollable: one with or without sentience?
"Intrinsic" safety means control, means understanding. You need to truly understand and be able to predict the system in order to control it.
There is no "proper definition" - or even one that everyone would agree upon. There is no definition of "sentience" that I could operationalize and put into a sentience-o-meter to reliably measure just how sentient a given rock, GPU or an internet user is.
I could try to put together benchmarks to estimate an AI's cyberwarfare capabilities, or instruction-following capabilities, or reward hacking inclinations. As noisy indirect estimates, of course. With philosophical mumbo-jumbo like "sentience", I don't even get that.
You just don't like the idea of there being one.
Sentience, self-awareness, consciousness, etc.,those are terms signifying a bridge between "technical" information theory and the psychological and social realms.
Those are just as real, only far less predictable and not as easy as programming.
They're also far more important and consequential.
I don't like it when people take mumbo-jumbo that can't be pinned down, or measured, or even agreed upon, and try to insist that we should base decision-making on it. It's literally just vibes with extra steps.
The "far more important and consequential" thing you're touting is your ability to make decisions based purely on vibes. And not even consistent, broadly agreed-upon vibes like "murder is pretty bad". It's vibes of the most vile variety: "sentience is what I decided sentience is".
An average internet user is sentient, but a 1996 Nissan ECU isn't. Why? Because I said so. Tremble before my might!
You engage in baseless "comparisons" in order to frame the topic according to your wishes.
A "proper" definition represents the objective truth about the matter. You denying such a truth to exist is simply due to you preferring to act unimpeded by it.
Acting against ethical constraints doesn't become OK just because there are no laws to punish you.
Ethics tells you about real-life consequences of your actions on other people. Before any laws take effect.
In effect, you propagate moral relativism. You want to do as you please, because you said so and fancy the spoils at others' expense. People trembling before your "might".
Aransentin · · focus · HN ↗
And indeed, you don't need to do galaxy brained reference class logic to realise that AI can plausibly become uncontrollable in the near future. It's enough to have an open model run its own weights and make money from scamming elderly people or the like, and it'll keep running as long as anyone anywhere is willing to make money by renting hardware to it.
Loquebantur · · focus · HN ↗
Deterministic systems can be chaotic, which implies unpredictability and that is anathema to control.
AI, in particular sentient AI, is right on the border of chaos. Meaning, it can be arbitrarily unpredictable.
Arbitrarily uncontrollable, that is.
ACCount39 · · focus · HN ↗
But what is clear is that AIs of today are already fairly unpredictable. Most of them aren't capable enough to make that into a major problem. Most of the unpredictable AI weirdness ends in "AI fails to do its job" rather than "AI does something dangerous".
Most. Even today, we already have notable counterexamples.
AIs get more capable over time, so if the intrinsic safety doesn't improve? Expect more of that.
Loquebantur · · focus · HN ↗
What AI do you expect to be more uncontrollable: one with or without sentience?
"Intrinsic" safety means control, means understanding. You need to truly understand and be able to predict the system in order to control it.
A proper definition of sentience would help.
ACCount39 · · focus · HN ↗
There is no "proper definition" - or even one that everyone would agree upon. There is no definition of "sentience" that I could operationalize and put into a sentience-o-meter to reliably measure just how sentient a given rock, GPU or an internet user is.
I could try to put together benchmarks to estimate an AI's cyberwarfare capabilities, or instruction-following capabilities, or reward hacking inclinations. As noisy indirect estimates, of course. With philosophical mumbo-jumbo like "sentience", I don't even get that.
Loquebantur · · focus · HN ↗
Sentience, self-awareness, consciousness, etc.,those are terms signifying a bridge between "technical" information theory and the psychological and social realms.
Those are just as real, only far less predictable and not as easy as programming.
They're also far more important and consequential.
ACCount39 · · focus · HN ↗
The "far more important and consequential" thing you're touting is your ability to make decisions based purely on vibes. And not even consistent, broadly agreed-upon vibes like "murder is pretty bad". It's vibes of the most vile variety: "sentience is what I decided sentience is".
An average internet user is sentient, but a 1996 Nissan ECU isn't. Why? Because I said so. Tremble before my might!
Loquebantur · · focus · HN ↗
A "proper" definition represents the objective truth about the matter. You denying such a truth to exist is simply due to you preferring to act unimpeded by it.
Acting against ethical constraints doesn't become OK just because there are no laws to punish you.
Ethics tells you about real-life consequences of your actions on other people. Before any laws take effect.
In effect, you propagate moral relativism. You want to do as you please, because you said so and fancy the spoils at others' expense. People trembling before your "might".