I'm not sure this is even conceivable, but... Imagine the bootstrap problem of intelligence systems going down, when no one knows how to bring them back up without the system itself walking them through it... Like if power and intelligence were to go down in a Carrington-class event in 10 years...
Like what would happen to individuals ten years from now who are trying to live if AI suddenly became inaccessible
Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?
To not just lack the objects of intelligence, but the very process of it...
Sounds like a bad novel. In the year 2050 no one knows how to actually program anymore. Then ClaudeGPT by GoogleSoft goes down. A 74 year old former software developer is on his farm tending to hens.
A self driving car crashes into his barn.
Out comes out a tech bro, “Yo OG programmer bro, it’s all busted bro, could you like make it unbusted bro.”
Then again with social security set to go bankrupt it’s not like we’ll be able to retire
> Then again with social security set to go bankrupt it’s not like we’ll be able to retire
I don't know about you, but my notion that my social security check is all I will need to survive my retirement years died when my father started drafting his own social security in the 2010s, and it was literal peanuts after being a small business owner his entire life (restaurants) and paying more than his fair share in taxes. Do the average folk still have this notion that social security = retirement? Hopefully not, it's certainly misguided now, but it was misguided then, too.
> Then again with social security set to go bankrupt it’s not like we’ll be able to retire
That is why you should use current money to become less dependent on future money. That former developer on his farm had the right idea, he bought a farm. So did I, with many hectares of forest which I use to build, heat and can also use to hunt should that become necessary.
How afraid are you of not living right next to a hospital?
The nearest hospital is around 45 km to the north, there's another one at about the same distance to the south. There are roads here and there are helicopters for when it is urgent - one of those landed a few kilometres from us a few years ago when a 'neighbour' had heart problems. I'm in the countryside but that does not equate to 'the middle of nowhere'.
yeah this was the devops promise too. Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.
> Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.
I wager that's mostly due to the Jevons paradox, though; it's so much cheaper and easier to build out systems with cloud than on-prem that you end up building way more systems than you would have without it, resulting in an overall headcount increase.
for those not aware of the reference, "The Machine Stops" was a 1909 novelette by E. M. Forster. It was very forward thinking, and so full of ideas!
On project gutenberg: first story in this anthology, <a href="https://www.gutenberg.org/ebooks/72890" rel="nofollow">https://www.gutenberg.org/ebooks/72890
PDF of just the story: <a href="https://www.cs.ucdavis.edu/~koehl/Teaching/ECS188/PDF_files/Machine_stops.pdf" rel="nofollow">https://www.cs.ucdavis.edu/~koehl/Teaching/ECS188/PDF_files/...
It's a real problem but your framing of it is itself very alarming.
Most humans aren't useless by any measure of most or usefulness. Our culture is our tools, dying due to a lack of antibiotics doesn't mean you didn't have anything to contribute, it means you didn't have access to the tool you needed. This is the problem we need to guard against.
No one is going to lose the ability to reason! This is accepting the most extreme version of the marketing presented by frontier labs, don't do it. People may lose fluency or familiarity or access to training on specific skills. We should put some care into making sure that can't be a catastrophe, for sure.
You've obviously never stood in a room of college freshmen and tried to deal with a young man having repeated issues being able to type his _own_ name into a box on a screen.
The general drove on. "At the present time our chief bottleneck is the fact that missiles are limited in intelligence. The computer controlling them can only be so large, and for that reason they can meet the changing nature of anti-missile defenses in an unsatisfactory way. Few missiles, if any, accomplish their goal, and missile warfare is coming to a dead end, for the enemy, fortunately, as well as for ourselves.
"On the other hand, a missile with a man or two within, controlling flight by graphitics, would be lighter, more mobile, more intelligent. It would give us a lead that might well mean the margin of victory. Besides which, gentlemen, the exigencies of war compel us to remember one thing. A man is much more dispensable than a computer. Manned missiles could be launched in numbers and under circumstances that no good general would care to undertake as far as computer-directed missiles are concerned . . ."
I'm not sure what this quote is from, but I almost down voted you in disgust before realizing that was exactly the intended effect. In other words, "Thanks, i hate it"
We don’t even need to imagine that world, Russia is doing it today.
Missiles and ammo are too expensive, so they send people to run at the enemy, revealing their firing positions. Then they drone the people who revealed themselves.
If I was an AI, I would reason that a T-800 is far more valuable than a man soldier who I must feed and clothe and heat. Use the disposable soldiers first and save the robots for later.
>A man is much more dispensable than a computer
sloppy (that is, negligent) writing. Something is either (in)dispensable, or is not. Either use something that can take degrees of comparison, like valuable (A man is less valuable etc) or reword it: A computer is indispensable, a man is not.
> “There is no such adjective,” Scalia said. “I know we have used it, but there is no such adjective as choate. There is inchoate, but the opposite of inchoate is not choate.”
No - the computer is not indispensable. The computer is just harder to replace than the man (in this imagined situation).
Hence - the man is much more dispensable...
---
The water in my water bottle is much more dispensable than the toothpaste in my toothpaste tube. But both can be dispensed (in both meanings - distributed or replaced).
This is basically the premise of Bacigalupi's "Pump Six", from "Pump Six and Other Stories".
The titular Pump Six breaks down and basically there's nobody left in the world who knows how to even reason about the steps needed to fix it. There's just some log dump and nobody understands what it means. Humans are no longer capable of complex reasoning, they just live in the moment and hump.
I think the reason that intelligence is so scary is how contextual, stateful, path-dependent, and coupled it is.
Which doesn't seem like it. The ideal is to keep removing couplings. But in practice, I find that focusing on the statefulness, the distribution-specificness, these "accidentals", is how you actually get leverage.
The obvious social consequence is now the tendency towards centralization, tyranny, "you people can't think for yourselves, not because you have some kind of 'inherent intelligence defect', but because the world is so complicated and interconnected that only coherent systems thinking (conveniently administered from one central point) is what can save you". In the Enlightenment world, the ideal was that every man could in principle, think for themselves.
A good ending might be that everybody benefits each other by distributing intelligence, but in practice centralization always seems to win.
Hardly a unique problem to AI systems. You certainly can't bootstrap pretty much any industry to its current state without the existing items of its current state. Any sort of chip fab is a great example. Or most manufacturing. Or even more fundamental: mining (both metals like iron, and energy like coal or gas).
My fear was that people like Linus would die and then nobody would be able to build operating systems anymore, because zoomers and below can’t even use computers.
Now we know they won’t even be able to read and write. But maybe Claude sonnet 12817 can maintain the operating system. It’ll be 17,000 dependencies in and mostly written in JavaScript, but it’ll work.
You go way too far - the leap you have made from some sort of AGI to "now individuals don't have the ability to fully reason on their own" is not clear to me at all.
If people are spending less time bent over monitors or at desks under the dictates of corporate imperative, they will probably be spending more time doing actually interesting things. Reasoning is innately human. What we call "knowledge work" is not and is often anything but. People who are naturally curious will still discover things, even if irrelevant (from a market perspective) when the AI is on, for their own satisfaction.
If such an event were to occur, they would be called upon. It's really little different from what would happen today if suddenly the power grid or supply lines were down for a sustained period of time.
The hordes of acolytes mumbling “Jevon’s paradox, Jevon’s paradox” every time AI gets cheaper beg to differ. Every minute saved using AI is just an opportunity to use more AI
It may sadden you to find that the majority of the population can go days, even weeks without ever seriously "reasoning" about anything major.
We hope an average guy who was stuck in an experimental cryopod gets accidentally woken up and through a course of events gets appointed to the presidential cabinet long enough to fix society.
People will have more things to worry about sooner. Where I live, if the power goes out so does my water. If AI is involved in either at any time in the future we will deserve what happens to us.
Also I disagree with the permise. People learned how to make technology that never existed once already, they can learn to do it again if needed. It will take time, which we'll have plenty of because your phone won't be working.
There is a TV show called "Alone" where people have to live in a remote area using basic survival skills and very minimal tools. It's interesting, but I've always wanted to see a version of that show where small communities are competing against each other.
If we practiced for these Armageddon scenarios what would a highly prepared group of 20 be able to accomplish in a year. How fast could they get humanity back to the bronze age?
Not even close to being as popular as sports or a gym stuff. There's basically common knowledge - to be a good <think profession> you need to hit a gym regularly. And it's going to be gym+something for a brain soon. At least i hope so
> Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?
What this misanthropic/Anthropic-boosting bait is missing is that all of these pieces falling together doesn’t make sense.
1. We are so accustomed to not-even-thinking means that we are legitimately freed from labor and cared for. (The dystopic version of being freed from your labor means that you still have to constantly think to survive. But not in any “economically useful” ways. Surviving for yourself and your family outside of the robot economy)
2. At the same time as point 1, we voluntarily, with all of our leisure time, just don’t think at all. Where is the self-actualization? Where is the spontainety? The drive? Did this in all of these years not come back after being a rat racer/wage slave?
Curiously, the “hacker ethos” here is to create. Not to just “consume”. And that is taken as an intrinsic thing, not just a way to get rich. But that’s not how “people” are?
Other people are mostly like you. But maybe those here who spent their careers implementing Entertaining Ourselves To Death maws genuinely think that “people” (not them though) would genuinely, when left to their own devices and having beeng catered to every need, would just “consume” mindlessly onto death.
Sorry to Goomba anyone here.
> Of course we all know most humans are useless and unable to survive outside our culture,
All. Take away the “habitat” from an animal and they are helpless, useless, impotent.
patcon · · focus · HN ↗
Like what would happen to individuals ten years from now who are trying to live if AI suddenly became inaccessible
Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?
To not just lack the objects of intelligence, but the very process of it...
sdcfgy · · focus · HN ↗
edf13 · · focus · HN ↗
budman1 · · focus · HN ↗
devmor · · focus · HN ↗
999900000999 · · focus · HN ↗
A self driving car crashes into his barn.
Out comes out a tech bro, “Yo OG programmer bro, it’s all busted bro, could you like make it unbusted bro.”
Then again with social security set to go bankrupt it’s not like we’ll be able to retire
scrollop · · focus · HN ↗
999900000999 · · focus · HN ↗
What if the twist is that he’s writing spaghetti code that only he knows how to maintain.
Worse, it’s in a language that his llc charges licensing fees to use!
dimitrios1 · · focus · HN ↗
I don't know about you, but my notion that my social security check is all I will need to survive my retirement years died when my father started drafting his own social security in the 2010s, and it was literal peanuts after being a small business owner his entire life (restaurants) and paying more than his fair share in taxes. Do the average folk still have this notion that social security = retirement? Hopefully not, it's certainly misguided now, but it was misguided then, too.
999900000999 · · focus · HN ↗
The issue then is getting a visa.
Social security has a low maximum, if your dad was used to clearing 15k a month and then social security only paid him 3500$, that’s rough.
For the typical person only making around 4500$ , it’s an adjustment, but doable
ghostbrainalpha · · focus · HN ↗
bobson381 · · focus · HN ↗
addandsubtract · · focus · HN ↗
hagbard_c · · focus · HN ↗
hagbard_c · · focus · HN ↗
That is why you should use current money to become less dependent on future money. That former developer on his farm had the right idea, he bought a farm. So did I, with many hectares of forest which I use to build, heat and can also use to hunt should that become necessary.
999900000999 · · focus · HN ↗
How far are you from the closet hospital.
hagbard_c · · focus · HN ↗
The nearest hospital is around 45 km to the north, there's another one at about the same distance to the south. There are roads here and there are helicopters for when it is urgent - one of those landed a few kilometres from us a few years ago when a 'neighbour' had heart problems. I'm in the countryside but that does not equate to 'the middle of nowhere'.
DrammBA · · focus · HN ↗
sdcfgy · · focus · HN ↗
s08148692 · · focus · HN ↗
Bluestein · · focus · HN ↗
Bluestein · · focus · HN ↗
lbrito · · focus · HN ↗
Aboutplants · · focus · HN ↗
I dare companies to gamble on running their businesses on absolute skeleton crews long term
chasd00 · · focus · HN ↗
mjr00 · · focus · HN ↗
I wager that's mostly due to the Jevons paradox, though; it's so much cheaper and easier to build out systems with cloud than on-prem that you end up building way more systems than you would have without it, resulting in an overall headcount increase.
rossdavidh · · focus · HN ↗
gcr · · focus · HN ↗
On project gutenberg: first story in this anthology, <a href="https://www.gutenberg.org/ebooks/72890" rel="nofollow">https://www.gutenberg.org/ebooks/72890
PDF of just the story: <a href="https://www.cs.ucdavis.edu/~koehl/Teaching/ECS188/PDF_files/Machine_stops.pdf" rel="nofollow">https://www.cs.ucdavis.edu/~koehl/Teaching/ECS188/PDF_files/...
dcss_gardener · · focus · HN ↗
Most humans aren't useless by any measure of most or usefulness. Our culture is our tools, dying due to a lack of antibiotics doesn't mean you didn't have anything to contribute, it means you didn't have access to the tool you needed. This is the problem we need to guard against.
No one is going to lose the ability to reason! This is accepting the most extreme version of the marketing presented by frontier labs, don't do it. People may lose fluency or familiarity or access to training on specific skills. We should put some care into making sure that can't be a catastrophe, for sure.
Vrondi · · focus · HN ↗
Hugsbox · · focus · HN ↗
DonsDiscountGas · · focus · HN ↗
TomGarden · · focus · HN ↗
joebig · · focus · HN ↗
"On the other hand, a missile with a man or two within, controlling flight by graphitics, would be lighter, more mobile, more intelligent. It would give us a lead that might well mean the margin of victory. Besides which, gentlemen, the exigencies of war compel us to remember one thing. A man is much more dispensable than a computer. Manned missiles could be launched in numbers and under circumstances that no good general would care to undertake as far as computer-directed missiles are concerned . . ."
bigfishrunning · · focus · HN ↗
tensegrist · · focus · HN ↗
bigfishrunning · · focus · HN ↗
Eridanus2 · · focus · HN ↗
MisterMunchkin · · focus · HN ↗
Missiles and ammo are too expensive, so they send people to run at the enemy, revealing their firing positions. Then they drone the people who revealed themselves.
If I was an AI, I would reason that a T-800 is far more valuable than a man soldier who I must feed and clothe and heat. Use the disposable soldiers first and save the robots for later.
cheschire · · focus · HN ↗
GeoAtreides · · focus · HN ↗
sloppy (that is, negligent) writing. Something is either (in)dispensable, or is not. Either use something that can take degrees of comparison, like valuable (A man is less valuable etc) or reword it: A computer is indispensable, a man is not.
piker · · focus · HN ↗
> “There is no such adjective,” Scalia said. “I know we have used it, but there is no such adjective as choate. There is inchoate, but the opposite of inchoate is not choate.”
<a href="https://www.abajournal.com/news/article/justice_scalia_delivers_lesson_on_word_usage" rel="nofollow">https://www.abajournal.com/news/article/justice_scalia_deliv...
mkehrt · · focus · HN ↗
horsawlarway · · focus · HN ↗
Hence - the man is much more dispensable...
---
The water in my water bottle is much more dispensable than the toothpaste in my toothpaste tube. But both can be dispensed (in both meanings - distributed or replaced).
the_af · · focus · HN ↗
The titular Pump Six breaks down and basically there's nobody left in the world who knows how to even reason about the steps needed to fix it. There's just some log dump and nobody understands what it means. Humans are no longer capable of complex reasoning, they just live in the moment and hump.
reacharavindh · · focus · HN ↗
sigbottle · · focus · HN ↗
Which doesn't seem like it. The ideal is to keep removing couplings. But in practice, I find that focusing on the statefulness, the distribution-specificness, these "accidentals", is how you actually get leverage.
The obvious social consequence is now the tendency towards centralization, tyranny, "you people can't think for yourselves, not because you have some kind of 'inherent intelligence defect', but because the world is so complicated and interconnected that only coherent systems thinking (conveniently administered from one central point) is what can save you". In the Enlightenment world, the ideal was that every man could in principle, think for themselves.
A good ending might be that everybody benefits each other by distributing intelligence, but in practice centralization always seems to win.
Blackthorn · · focus · HN ↗
MisterMunchkin · · focus · HN ↗
Now we know they won’t even be able to read and write. But maybe Claude sonnet 12817 can maintain the operating system. It’ll be 17,000 dependencies in and mostly written in JavaScript, but it’ll work.
arshxyz · · focus · HN ↗
godwinson__4-8 · · focus · HN ↗
If people are spending less time bent over monitors or at desks under the dictates of corporate imperative, they will probably be spending more time doing actually interesting things. Reasoning is innately human. What we call "knowledge work" is not and is often anything but. People who are naturally curious will still discover things, even if irrelevant (from a market perspective) when the AI is on, for their own satisfaction.
If such an event were to occur, they would be called upon. It's really little different from what would happen today if suddenly the power grid or supply lines were down for a sustained period of time.
wonnage · · focus · HN ↗
yonaguska · · focus · HN ↗
keybored · · focus · HN ↗
woah · · focus · HN ↗
jayd16 · · focus · HN ↗
nancyminusone · · focus · HN ↗
Also I disagree with the permise. People learned how to make technology that never existed once already, they can learn to do it again if needed. It will take time, which we'll have plenty of because your phone won't be working.
KellyCriterion · · focus · HN ↗
some weeks ago, we changed the billed credit card - something went wrong on their end we were unable to use it for ~8-9 days.
Some pepople during this period said to certain tasks: "I will care about that if Claude is available again"
So, we are already there
ghostbrainalpha · · focus · HN ↗
If we practiced for these Armageddon scenarios what would a highly prepared group of 20 be able to accomplish in a year. How fast could they get humanity back to the bronze age?
Betelbuddy · · focus · HN ↗
out_of_protocol · · focus · HN ↗
Exoristos · · focus · HN ↗
out_of_protocol · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
keybored · · focus · HN ↗
What this misanthropic/Anthropic-boosting bait is missing is that all of these pieces falling together doesn’t make sense.
1. We are so accustomed to not-even-thinking means that we are legitimately freed from labor and cared for. (The dystopic version of being freed from your labor means that you still have to constantly think to survive. But not in any “economically useful” ways. Surviving for yourself and your family outside of the robot economy)
2. At the same time as point 1, we voluntarily, with all of our leisure time, just don’t think at all. Where is the self-actualization? Where is the spontainety? The drive? Did this in all of these years not come back after being a rat racer/wage slave?
Curiously, the “hacker ethos” here is to create. Not to just “consume”. And that is taken as an intrinsic thing, not just a way to get rich. But that’s not how “people” are?
Other people are mostly like you. But maybe those here who spent their careers implementing Entertaining Ourselves To Death maws genuinely think that “people” (not them though) would genuinely, when left to their own devices and having beeng catered to every need, would just “consume” mindlessly onto death.
Sorry to Goomba anyone here.
> Of course we all know most humans are useless and unable to survive outside our culture,
All. Take away the “habitat” from an animal and they are helpless, useless, impotent.
sneak · · focus · HN ↗
<a href="https://www.goodreads.com/en/book/show/2819368-pump-six-and-other-stories" rel="nofollow">https://www.goodreads.com/en/book/show/2819368-pump-six-and-...
Both Pump 6 and The Machine Stops have examined precisely this in detail.