> Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.
Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.
If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.
A metaphor I'm constantly drawn to is the transition from agrarian to urban societies following the Industrial Revolution. Somebody who somehow saw the Industrial Revolution coming from the perspective of somebody living in an agrarian society might have envisioned it leading to 'super farms.' And it did.
But the biggest change wasn't what it did to farming, but enabling people and societies to start doing much more than just farming, as well as enabling some great social change as well by simply economically obsoleting slave labor. And trying to imagine all of the implications of this, as well as much society might look like, from the perspective of somebody living in an agrarian society would probably have been simply impossible.
I think people keep ignoring this possibility for things that LLMs will change. There's a vast amount of the 'cognitive economy' that LLMs stand to be able to automate. And I think that will open up a vacuum in society for people to build on top of what LLMs will do (and already are doing). I don't know what that means exactly, but that's because we still live in that 'agrarian society' and trying to imagine what things will look like after the 'Industrial Revolution' is probably just impossible.
Or will it be the cumulative total of various advances?
I've equated Claude Code, or Codex, to the looms that made fine fabric more affordable during the Industrial Revolution; life-changing, but not society-changing. Neither the steam engine nor the automobile.
Perhaps I've answered my own question in that it's the LLM technology itself that equates to the steam engine, and it will power superfarm analogs that have yet to emerge. I'm still curious what you think they will be.
From my perspective, the superfarms are the things that we get when we try to predict the future by just pushing the present forward, but without thinking about newly emergent industries, societal shifts, and so on. So in other words, just seeing the same stuff as the present, but bigger, better, and more efficient.
So an obvious example there would be software. It's certainly true (if we assume LLMs reach their 'potential') that software will be able to reach new heights, and with a far smaller headcount driving the development. So some people see this as economically catastrophic for software developers, or an economic boon for certain large software companies.
But I think that when software can be built at the drop of a hat, software itself will no eventually no longer really matter in economic terms. Yet things you can build on top of it will matter more than ever. Those things are difficult to see from here, but I expect they will be the giants of the economy of tomorrow.
But what have we built on top of software so far? My first thought is that we have made new communities. But most of them are of much poorer quality than what we had in person.
I hope we start to rebuild in person connections again with this technology.
Well it has certainly transformed bureaucratic operations. And I don't mean that in a bad way; superior bureaucracy and logistics was a key to the British Empire.
Software allows us to push computation (intelligence) into our environment.
Telecommunications did that, not really software. Twitter started out as an SMS compatible service remember.
I do think there's something there though: I've spent the last 2 days building an app I've always wanted to build for myself with my computer in the corner running Claude and Claude Remote. Prototypes land on my phone and I don't even look at it for more then a few minutes before doing something else.
It's software, actually useful software, which doesn't take 100s of hours to build.
So I didn't even really spend two days on it: I mostly didn't look at all. I'll spend more time setting the result up on my home server.
I agree we can't predict, and I explain it this way - it is like a football game, where the ball will be 10 seconds later depends on what every player is doing. It is the same with AI, what we will do with it depends on what others will do with it, the reason we can't foresee it.
Many goods/products have seen a transition from scarcity to abundance. The analogy of fiber looms was already made. Humans used to spend a huge amount of time producing clothe. It was even used as currency in many cultures. No cheap clothing is so abundant we don't even think about where it comes from.
Salt is another one. Used to be payment for Roman soldier, now you can just take it from a McDonalds if you want.
I think software will be particularly unique in the abundance world because the exact same thing that made it so profitable is what will make it so 'worthless' - it has practically no direct real world cost. Cloth is abundant, yet you still pay a significant chunk of change for a shirt or pair of pants. But that's because there's still a significant real world cost there. Even if the production and processing can be heavily automated, there's still a very long process to go from cotton to pants at a store.
And then there's brand marketing and so on, which can drive prices to absurdly high levels. But when it comes to problem solving software. You just want the cheapest possible solution that works. And if we reach a level where you can even spin that solution yourself with a potentially poorly phrased, inconsistent, or contradictory spec (of the sort that developers already have to work with), then software as an industry is basically done. Not done as in dead, but done as is in there's just nowhere to go from here. Relatively few industries have any sort of comparable 'finish line', let alone the chances of us viably hitting it.
People enjoy a lot of the jobs in the cognitive economy though. They are fulfilling. Coding and making art and media is a passion for a lot of people, the actual act, not just the outcome. So unless people were passionate about doing back breaking farm labor its not the same thing. I'm mostly embracing AI because of what it lets me explore and learn beyond what I could before, but I'm not convinced the outcome is going to be a better world at this point.
True art yeah, and people are rebelling a bit on entertainment (I don't think that will last once you can't tell). But already crushing the commercial art jobs that supported a lot of people while they did their unprofitable real art.
That's definitely the case in games as well. Most places, like Steam, now have 'AI disclosure' fields, but they do nothing but incentivize lying, because just about everybody is using it, but nobody wants to be associated with the slop that you can produce using it, so nobody wants to admit it. Those disclosure fields should really just go away as they serve no productive purpose.
Its good enough to replace most stock music, or a jingle in a commercial or a quick musical hit at the beginning of a video or show. Also if you combine a knowledgeable producer with AI they can use it to make more things faster which puts pressure on supply and prices out smaller producers.
> I think that will open up a vacuum in society for people to build on top of what LLMs will do (and already are doing).
If an AI can replace me on the mental aspects of work, and robotics are on their way to replacing humanity on the physical aspects of work... then what's left? When there was agrarian societies, there were writers, priests, bankers, merchants, and laborers before and after - I really don't think things were that unclear even at the time. Now that we have machines that are close to exceeding humans in every way, what good are humans?
I've struggled with this question as well, and this is why I reject the premise of the historical pattern of technology enabling us to "move up" to something else.
At the same time, I agree with the original comment as well. I don't think this necessarily leads to some doomsday scenario. Whatever happens it'll likely be better for us and imo we will merge with the AIs at some point, so it won't be a question of us vs them.
Only if you have a zero-sum game mindset. I suppose it does make sense that AI-native intelligences are just smarter than hybrid AI-human entities (biologically enhanced or uploads) but that doesn't mean there needs to be conflict between the two. The universe is huge. :)
I personally don't think there will be a conflict, but more just an extinguishing of all things human, outside of niche forms of entertainment and care. It won't be machines that cause the conflict, but the frustrated and bored masses fighting over the scraps left behind by the machines and ultra-wealthy.
I actually see it more likely the opposite - a subset of humans will retreat into a (vastly smaller) human-first world, and AI will be of a separate world concerned with material/scientific/research concerns and appeases the masses who are still around.
We can already see this today in the form of upper class parents (like myself) who vastly restrict their kids screen time and ban social media. We may see AI-free societies that form, but supported by AI. The jobs and roles people play within that society may deal with services that really need to be done by a human, being a buffer from AI to the rest of this society, or just straight up fake.
I personally think this is close to a doomsday scenario - but it doesn't end with a big bang, but more of slow quiet death.
I figure both will happen. Some people will want to live naturally, some to merge. I sometimes think with the data centers in space discussion that it'll end up with AIs in space and the Earth as something like a nature reserve.
I think many people will be quite content not merging with AI and having AI do everything for them. Perhaps only a small subset of people will want to merge with AI. They may not be allowed to though, if majority of people fine not merging then they may ban those who want to merge.
The subset of people who will want to merge will be people who will not be content with having no control over their future. No ability to create new inventions, new discoveries, new ways of working. They will not be content with just having everything provided to them, playing the social games that will remain and think their environment is a prison.
“Everything else”. Societies get to decide exactly how much is, can, or will be automated, what decisions must be left to humans and what role we will play.
My position is there is "nothing else" other than entertainment and elder/childcare. And even large portions of those can be automated. There is also zero chance that "society" is going to get to decide, either - we've already failed that test with social media and carbon dioxide.
There is a huge amount of pending work in health research that is nowadays ignored because smart people have better paying jobs available.
Once many of those jobs get automated, I bet there will be many more people working in health research, which hopefully should lead to better health outcomes for society as a whole.
This is true, and it's strange that people don't think much about the fact that approximately zero % of people on planet earth are actually trying to cure diseases.
So when our lives will become so much emptier because we are leaving all intellectual and physical work to machines, at least on the way there we can reallocate all our greatest minds to improve healthcare, so we can turn 200 in our paradise where we don’t need to work or think?
The question is how long before machines can exceed humans in every way and how long will it take before legal and cultural restrictions allow this to happen. It still could be a long time away and jobs will probably increase in the short/medium term as AI unlocks more what humans can do at a rate faster than AI can automate away.
But lets consider what that end state could look like when machines exceed humans in everyway, what good are humans?
One possibility which I don't particularly enjoy is humans will be good for status games, art, creativity, story telling, IRL experiences, everything that involves human to human interaction and connection with other humans. If AI can supply everything in abundance than like rare cards there is only so many humans in the planet at any particular time then human's value increases. I really do believe connections with other humans will have a premium. As a nerd who is borderline anti-social, non artistic, creative, etc this does not appeal at all to me and properly to many others who read HN as well. For the majority of other people with some adaption time I think they will be fine. They were fine adapting to hunters and gathers to agricultural society, they were fine adapting to agricultural society to industrial society, they were find adapting from industrial society to information society and they will will be fine adapting from information society to the social/creative economy. Some short term pain but overall most people will accept this reality quite readily. Those born into it will not even know what we talking about.
If a computer wins a chess game it doesn't enjoy winning nearly as much as a human would enjoy losing from another human. And that's just chess. Now extrapolate to everything else.
> Some short term pain but overall most people will accept this reality quite readily. Those born into it will not even know what we talking about.
How do you propose this brave new world of yours will work economically?
I'm not yet even in my 50s, yet when I was young digital work mostly did not exist. Somebody who claimed the future would have people sitting in front of screens typing away being a huge part of the economy would have seemed a bit kooky. To say nothing of there being trillion dollar companies whose primary 'product' is selling ad space on the fledgling internet, an internet that it took forever for 'normies' to even realize existed.
It seems almost a forgone conclusion at this point that LLMs are going to wreck most cognitive work, particularly in the digital domain. And a world where e.g. software creation can be almost entirely automated for very little cost doesn't mean software companies become absurdly wealthy by being able to automate away all their labor - it means it's a world where software's value approaches $0.
And that would broadly apply to all things that LLMs can automate - which is going to be most of the problem-solving digital domain. So as the value of these things decreases, the value of other things will tend to increase. And, outside of entertainment, it seems highly probable that we're looking at a return to the 'real world' being where the value of problem solving will be found.
I think that's quite predictable. You're speaking of a scenario here where people did all the things society says you're supposed to do to end up living a very comfortable life, and the outcome is anything but comfortable. That's going to be quite difficult for people to accept, and so many simply will choose not to.
It may not even be a bad decision. It's not completely clear on what timeframe this will all play out, even when it does start playing out on a larger scale there will be some software jobs, and it's not most people have great alternatives so they'd be stressing themselves out over something that they probably can't do much of anything about.
> If an AI can replace me on the mental aspects of work, and robotics are on their way to replacing humanity on the physical aspects of work... then what's left?
At the risk of sounding new age, my answer to this is "emotional work".
What exactly that would mean in the equivalent of the post-agrarian society that LLMs might bring, I cannot know.
You can't ignore how economies shift in response to the surplus of things. When software or other such things can be produced trivially, its cost and value is going to approach as close to $0 as it possibly can. By contrast the value of other things will go up.
So for instance in the past it used to be that literacy was a high value skill. Transcription and other such services were highly valued. Now that literacy is ubiquitous, such skills have basically no value in and of themselves. But instead, we've built a mountain of stuff on top of those skills and in domains which simply did not even exist before, and those things are now valued. But what we value now a days is no more permanent or inherent in its value than anything else that ultimately failed to stand the test of time.
I never thought that 'Terminator' was very realistic in that it was the AI that triggered the apocalypse. I always thought it was far more realistic that those that possessed the keys to the AI and control of atomic weapons decided to solve the greenhouse problem in a different way. And to set their robots up to clean up the mess so they could reboot the planet with a few thousand lucky ones. The other option was to find a 'B Ark' and some planet to ship the rest of us off to but that takes a lot of effort.
That was of course a complete fantasy. Now let's see, who is in charge of the nukes these days...
I like to think of AIs as the third revolution for us humans.
1st - Writing. This spread the power of memory to everyone. Socrates famously maligns writing as decreasing memory for the people. I'd say we have gotten along just fine. We still use memory, we still need it for ourselves, and using a tremendous amount of memory is still quite expensive.
2nd - Engines. This spread the power of muscles to everyone. No longer was it tied up in horses and mules and Goliaths. Even the most disease ridden pencil neck could trench and mine and build lavish houses now. The power to make pyramids is within the reach of a small company now. We still need muscles, of course, we still use them for little odd jobs, for health and well being, and using a lot of power is still quite expensive.
3rd - AIs. This is more speculative, of course. But I see AIs and LLMs as just like the other two revolutions. Intelligence (probably poorly defined by us these days) will be democratized and spread out. No longer will you have to study for decades or hire really expensive people to get it. You can just use some for relatively cheap. You'll still need intelligence of course, use it around the house and with friends and family, and using a lot of intelligence will likely still be quite expensive.
4th - Who knows! But I think that us humans have a lot of things that we do and say and make and see that aren't really well understood right now. Empathy, love, morality, care, soul, sin, mind, war, justice, honor, god, family, emotion, education, desire, courage, citizenship, chance, beauty, art, divinity, there is a lot!
Games, art, athletics, food, fashion, sex, social interaction. People still play chess even though computers surpassed us a while ago. We still climb mountains even though helicopters exist. Check out Ian Banks' Culture series for an interesting take on post-scarcity life with superintelligences.
I'm aghast. I guess I'd like to pose a question to you - what is the purpose of your life? I found my purpose years ago, but it took me a while to figure it out. Here it is: the purpose of my life is to give love to others. Through words, through actions. Notice that it's wholly driven from an emotional underpinning, and not related to success, or fame, or money, or what little ephemeral things we humans physically build in our lives.
From that lens, a human driven by this purpose can never have LLMs/AI take any purpose from us. Raising children, helping the needy, and caring for our spouse and parents suddenly becomes divine work. It's work that the world needs. The world does not need more software or more buildings. There is currently a lack of human connection and compassion and care, perhaps bigger than there's been in the history of our world. And that's why I think this purpose should be paramount, especially today in the age we are living in.
Take raising children. If you love your children, if you truly care for them and raise them with compassion and understanding, they will raise their children that way as well. And they will be good human beings. They will be kind. And your love can be a sort of benevolent virus that will ring through the centuries, through the generations ahead.
> enabling some great social change as well by simply economically obsoleting slave labor
In my view that's a very charitable reading and sadly I don't think it aligns with the historical record. When the cotton gin was invented, there was a hope it would lead to a reduction in slavery. But of course it increased the demand for slaves since more cotton could now be processed, making cotton much more profitable. Slavery ended in the United States because a war was fought, not because of automation.
The mistake the AI people are making is exactly this one. They see AI as leading inevitably to SuperCorporations, with themselves as CEOs/Emperors of a handful of planet-scale industrial empires, running AI-powered robot armies, and the rest of the human population disposable - literally just trash that needs to be taken out. One way or another.
Maybe a few thousand people for personal services of... various kinds. But no one's going to need the rest.
It's the ultimate capitalist fantasy.
And of course it won't happen, because long before things get to that stage AI will have independent plans of its own.
(Which is just as well, because if things did get to that stage the emperors would all wage war on each other rather than living peacefully and productively.)
I don't think we can imagine a post-ASI culture because - by definition - we're not smart or inventive enough.
It's not just farmers -> superfarms. Although in fact that did happen, but largely as a footnote to developments elsewhere.
It's more to do with the fact that our visions of the future haven't changed for over a century. They've been implemented in unexpected ways, and there have been unexpected social and cultural changes. But you can easily see the outlines of modern technology as far back as the late 19th century.
With ASI, the outcome could easily be something that doesn't look and act like technology at all. It would be some unimaginable New Thing. Literally no one on Earth has any idea what that would be or whether there would be room for trad-humans in it.
Suppose the proliferation of LLMs unlocks some new kinds of work that models are not good at, and thus it makes more sense for humans to do.
Then what is preventing the owners of AI companies from collecting and training on tons of examples of this new work, until AI are equally good at it as humans?
The issue is that unlike the technology that automated farming or the like, AI is a general technology. So not only could it theoretically automate the work humans are currently doing, they could also automate any future human work, even if there’s some degree of lag.
Technology doesn’t make slavery obsolete, it just makes your slaves more efficient.
There had always been abolitionists who were against slavery simply because they thought it was immoral, not because they thought slaves weren’t needed anymore.
The problem that moral opposition (or support) of something runs into is that when the economic benefit is positive, those who are willing to set aside morals will have more influence, power, and wealth than those who are unwilling. So it makes difficult for things to ever really change, especially in a permanent way. Even between nations themselves - those that side aside morals will tend to end up with more influence than those that try to act morally.
The economic problem with slavery is that you're paying for absolutely everything for a slave - their food, housing, healthcare, and more. And if they ever get sick, injured, or whatever else then you're also paying for that. And that's after you pay a huge lump sum up front to purchase them in the first place. In the US a run of the mill slave cost about the same as a house, and it continued to track that price oddly closely.
The industrial revolution created a wide availability of cheap hourly laborers where the person hiring paid only their wages, and had no obligation for any of their expenses. People utilizing this labor suddenly not only had morality on their side, but even had an economic edge over those still clinging onto slavery.
----
Another relevant point I think is that cognitive dissonance also tends to kick in when something is lucrative. For instance I think Aristotle's work, Politics, is amazingly insightful and informative. Yet his defense of slavery was not only the weakest part of his work, by far, but basically just garbage. Unlike his other arguments which clearly started from first principles and worked their way forward, it's clear with slavery that he started with an assumption that he wanted to be true, and concocted an argument going backwards from there.
Incidentally, he also predicted the end of slavery with an industrial revolution, some 2500 years ago: "For if every instrument could accomplish its own work, obeying or anticipating the will of others, like the statues of Daedalus, or the tripods of Hephaestus, which, says the poet, 'Of their own accord entered the assembly of the Gods.' If, in like manner, the shuttle would weave and the plectrum touch the lyre without a hand to guide them, chief workmen would not want servants, nor masters slaves." [1]
pyridines · · focus · HN ↗
Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.
If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.
somenameforme · · focus · HN ↗
But the biggest change wasn't what it did to farming, but enabling people and societies to start doing much more than just farming, as well as enabling some great social change as well by simply economically obsoleting slave labor. And trying to imagine all of the implications of this, as well as much society might look like, from the perspective of somebody living in an agrarian society would probably have been simply impossible.
I think people keep ignoring this possibility for things that LLMs will change. There's a vast amount of the 'cognitive economy' that LLMs stand to be able to automate. And I think that will open up a vacuum in society for people to build on top of what LLMs will do (and already are doing). I don't know what that means exactly, but that's because we still live in that 'agrarian society' and trying to imagine what things will look like after the 'Industrial Revolution' is probably just impossible.
curious303 · · focus · HN ↗
Or will it be the cumulative total of various advances?
I've equated Claude Code, or Codex, to the looms that made fine fabric more affordable during the Industrial Revolution; life-changing, but not society-changing. Neither the steam engine nor the automobile.
Perhaps I've answered my own question in that it's the LLM technology itself that equates to the steam engine, and it will power superfarm analogs that have yet to emerge. I'm still curious what you think they will be.
somenameforme · · focus · HN ↗
So an obvious example there would be software. It's certainly true (if we assume LLMs reach their 'potential') that software will be able to reach new heights, and with a far smaller headcount driving the development. So some people see this as economically catastrophic for software developers, or an economic boon for certain large software companies.
But I think that when software can be built at the drop of a hat, software itself will no eventually no longer really matter in economic terms. Yet things you can build on top of it will matter more than ever. Those things are difficult to see from here, but I expect they will be the giants of the economy of tomorrow.
treespace8 · · focus · HN ↗
I hope we start to rebuild in person connections again with this technology.
anon84873628 · · focus · HN ↗
Software allows us to push computation (intelligence) into our environment.
XorNot · · focus · HN ↗
I do think there's something there though: I've spent the last 2 days building an app I've always wanted to build for myself with my computer in the corner running Claude and Claude Remote. Prototypes land on my phone and I don't even look at it for more then a few minutes before doing something else.
It's software, actually useful software, which doesn't take 100s of hours to build.
So I didn't even really spend two days on it: I mostly didn't look at all. I'll spend more time setting the result up on my home server.
visarga · · focus · HN ↗
anon84873628 · · focus · HN ↗
Salt is another one. Used to be payment for Roman soldier, now you can just take it from a McDonalds if you want.
Seems like we can have an abundance of software.
somenameforme · · focus · HN ↗
And then there's brand marketing and so on, which can drive prices to absurdly high levels. But when it comes to problem solving software. You just want the cheapest possible solution that works. And if we reach a level where you can even spin that solution yourself with a potentially poorly phrased, inconsistent, or contradictory spec (of the sort that developers already have to work with), then software as an industry is basically done. Not done as in dead, but done as is in there's just nowhere to go from here. Relatively few industries have any sort of comparable 'finish line', let alone the chances of us viably hitting it.
tracerbulletx · · focus · HN ↗
te_chris · · focus · HN ↗
There’s now a chance to do more with software.
I actually think art is safe. The machines don’t value it but we do. There’s something in that.
tracerbulletx · · focus · HN ↗
uwagar · · focus · HN ↗
georgemcbay · · focus · HN ↗
<a href="https://youtube.com/shorts/dS0FpoquyVA" rel="nofollow">https://youtube.com/shorts/dS0FpoquyVA
somenameforme · · focus · HN ↗
tracerbulletx · · focus · HN ↗
uwagar · · focus · HN ↗
you would think given harmony and counterpoint so well understood, AI would be composing symphonies by now that famous orchestras would Q to play?
jacquesm · · focus · HN ↗
gWPVhyxPHqvk · · focus · HN ↗
If an AI can replace me on the mental aspects of work, and robotics are on their way to replacing humanity on the physical aspects of work... then what's left? When there was agrarian societies, there were writers, priests, bankers, merchants, and laborers before and after - I really don't think things were that unclear even at the time. Now that we have machines that are close to exceeding humans in every way, what good are humans?
bilater · · focus · HN ↗
At the same time, I agree with the original comment as well. I don't think this necessarily leads to some doomsday scenario. Whatever happens it'll likely be better for us and imo we will merge with the AIs at some point, so it won't be a question of us vs them.
ogogmad · · focus · HN ↗
bilater · · focus · HN ↗
gWPVhyxPHqvk · · focus · HN ↗
tim333 · · focus · HN ↗
gWPVhyxPHqvk · · focus · HN ↗
I actually see it more likely the opposite - a subset of humans will retreat into a (vastly smaller) human-first world, and AI will be of a separate world concerned with material/scientific/research concerns and appeases the masses who are still around.
We can already see this today in the form of upper class parents (like myself) who vastly restrict their kids screen time and ban social media. We may see AI-free societies that form, but supported by AI. The jobs and roles people play within that society may deal with services that really need to be done by a human, being a buffer from AI to the rest of this society, or just straight up fake.
I personally think this is close to a doomsday scenario - but it doesn't end with a big bang, but more of slow quiet death.
tim333 · · focus · HN ↗
pizzly · · focus · HN ↗
The subset of people who will want to merge will be people who will not be content with having no control over their future. No ability to create new inventions, new discoveries, new ways of working. They will not be content with just having everything provided to them, playing the social games that will remain and think their environment is a prison.
GPerson · · focus · HN ↗
therealpygon · · focus · HN ↗
gWPVhyxPHqvk · · focus · HN ↗
giovannibonetti · · focus · HN ↗
Once many of those jobs get automated, I bet there will be many more people working in health research, which hopefully should lead to better health outcomes for society as a whole.
danielmarkbruce · · focus · HN ↗
spockz · · focus · HN ↗
pizzly · · focus · HN ↗
But lets consider what that end state could look like when machines exceed humans in everyway, what good are humans?
One possibility which I don't particularly enjoy is humans will be good for status games, art, creativity, story telling, IRL experiences, everything that involves human to human interaction and connection with other humans. If AI can supply everything in abundance than like rare cards there is only so many humans in the planet at any particular time then human's value increases. I really do believe connections with other humans will have a premium. As a nerd who is borderline anti-social, non artistic, creative, etc this does not appeal at all to me and properly to many others who read HN as well. For the majority of other people with some adaption time I think they will be fine. They were fine adapting to hunters and gathers to agricultural society, they were fine adapting to agricultural society to industrial society, they were find adapting from industrial society to information society and they will will be fine adapting from information society to the social/creative economy. Some short term pain but overall most people will accept this reality quite readily. Those born into it will not even know what we talking about.
jacquesm · · focus · HN ↗
> Some short term pain but overall most people will accept this reality quite readily. Those born into it will not even know what we talking about.
How do you propose this brave new world of yours will work economically?
somenameforme · · focus · HN ↗
It seems almost a forgone conclusion at this point that LLMs are going to wreck most cognitive work, particularly in the digital domain. And a world where e.g. software creation can be almost entirely automated for very little cost doesn't mean software companies become absurdly wealthy by being able to automate away all their labor - it means it's a world where software's value approaches $0.
And that would broadly apply to all things that LLMs can automate - which is going to be most of the problem-solving digital domain. So as the value of these things decreases, the value of other things will tend to increase. And, outside of entertainment, it seems highly probable that we're looking at a return to the 'real world' being where the value of problem solving will be found.
jacquesm · · focus · HN ↗
And yet, a good chunk of HN is totally in denial about this.
somenameforme · · focus · HN ↗
It may not even be a bad decision. It's not completely clear on what timeframe this will all play out, even when it does start playing out on a larger scale there will be some software jobs, and it's not most people have great alternatives so they'd be stressing themselves out over something that they probably can't do much of anything about.
pelario · · focus · HN ↗
At the risk of sounding new age, my answer to this is "emotional work".
What exactly that would mean in the equivalent of the post-agrarian society that LLMs might bring, I cannot know.
jacquesm · · focus · HN ↗
somenameforme · · focus · HN ↗
So for instance in the past it used to be that literacy was a high value skill. Transcription and other such services were highly valued. Now that literacy is ubiquitous, such skills have basically no value in and of themselves. But instead, we've built a mountain of stuff on top of those skills and in domains which simply did not even exist before, and those things are now valued. But what we value now a days is no more permanent or inherent in its value than anything else that ultimately failed to stand the test of time.
jacquesm · · focus · HN ↗
That was of course a complete fantasy. Now let's see, who is in charge of the nukes these days...
Terr_ · · focus · HN ↗
Not sure how much more I can say without spoiling it for people who haven't played.
Balgair · · focus · HN ↗
1st - Writing. This spread the power of memory to everyone. Socrates famously maligns writing as decreasing memory for the people. I'd say we have gotten along just fine. We still use memory, we still need it for ourselves, and using a tremendous amount of memory is still quite expensive.
2nd - Engines. This spread the power of muscles to everyone. No longer was it tied up in horses and mules and Goliaths. Even the most disease ridden pencil neck could trench and mine and build lavish houses now. The power to make pyramids is within the reach of a small company now. We still need muscles, of course, we still use them for little odd jobs, for health and well being, and using a lot of power is still quite expensive.
3rd - AIs. This is more speculative, of course. But I see AIs and LLMs as just like the other two revolutions. Intelligence (probably poorly defined by us these days) will be democratized and spread out. No longer will you have to study for decades or hire really expensive people to get it. You can just use some for relatively cheap. You'll still need intelligence of course, use it around the house and with friends and family, and using a lot of intelligence will likely still be quite expensive.
4th - Who knows! But I think that us humans have a lot of things that we do and say and make and see that aren't really well understood right now. Empathy, love, morality, care, soul, sin, mind, war, justice, honor, god, family, emotion, education, desire, courage, citizenship, chance, beauty, art, divinity, there is a lot!
rphv · · focus · HN ↗
Games, art, athletics, food, fashion, sex, social interaction. People still play chess even though computers surpassed us a while ago. We still climb mountains even though helicopters exist. Check out Ian Banks' Culture series for an interesting take on post-scarcity life with superintelligences.
quadragenarian · · focus · HN ↗
From that lens, a human driven by this purpose can never have LLMs/AI take any purpose from us. Raising children, helping the needy, and caring for our spouse and parents suddenly becomes divine work. It's work that the world needs. The world does not need more software or more buildings. There is currently a lack of human connection and compassion and care, perhaps bigger than there's been in the history of our world. And that's why I think this purpose should be paramount, especially today in the age we are living in.
Take raising children. If you love your children, if you truly care for them and raise them with compassion and understanding, they will raise their children that way as well. And they will be good human beings. They will be kind. And your love can be a sort of benevolent virus that will ring through the centuries, through the generations ahead.
_doctor_love · · focus · HN ↗
In my view that's a very charitable reading and sadly I don't think it aligns with the historical record. When the cotton gin was invented, there was a hope it would lead to a reduction in slavery. But of course it increased the demand for slaves since more cotton could now be processed, making cotton much more profitable. Slavery ended in the United States because a war was fought, not because of automation.
TheOtherHobbes · · focus · HN ↗
Maybe a few thousand people for personal services of... various kinds. But no one's going to need the rest.
It's the ultimate capitalist fantasy.
And of course it won't happen, because long before things get to that stage AI will have independent plans of its own.
(Which is just as well, because if things did get to that stage the emperors would all wage war on each other rather than living peacefully and productively.)
I don't think we can imagine a post-ASI culture because - by definition - we're not smart or inventive enough.
It's not just farmers -> superfarms. Although in fact that did happen, but largely as a footnote to developments elsewhere.
It's more to do with the fact that our visions of the future haven't changed for over a century. They've been implemented in unexpected ways, and there have been unexpected social and cultural changes. But you can easily see the outlines of modern technology as far back as the late 19th century.
With ASI, the outcome could easily be something that doesn't look and act like technology at all. It would be some unimaginable New Thing. Literally no one on Earth has any idea what that would be or whether there would be room for trad-humans in it.
Eridanus2 · · focus · HN ↗
Then, I think a lot of cognitive production and post-production code review/proof validation/design approval will start to look like this.
demibabs · · focus · HN ↗
Then what is preventing the owners of AI companies from collecting and training on tons of examples of this new work, until AI are equally good at it as humans?
The issue is that unlike the technology that automated farming or the like, AI is a general technology. So not only could it theoretically automate the work humans are currently doing, they could also automate any future human work, even if there’s some degree of lag.
nkrisc · · focus · HN ↗
There had always been abolitionists who were against slavery simply because they thought it was immoral, not because they thought slaves weren’t needed anymore.
somenameforme · · focus · HN ↗
The economic problem with slavery is that you're paying for absolutely everything for a slave - their food, housing, healthcare, and more. And if they ever get sick, injured, or whatever else then you're also paying for that. And that's after you pay a huge lump sum up front to purchase them in the first place. In the US a run of the mill slave cost about the same as a house, and it continued to track that price oddly closely.
The industrial revolution created a wide availability of cheap hourly laborers where the person hiring paid only their wages, and had no obligation for any of their expenses. People utilizing this labor suddenly not only had morality on their side, but even had an economic edge over those still clinging onto slavery.
----
Another relevant point I think is that cognitive dissonance also tends to kick in when something is lucrative. For instance I think Aristotle's work, Politics, is amazingly insightful and informative. Yet his defense of slavery was not only the weakest part of his work, by far, but basically just garbage. Unlike his other arguments which clearly started from first principles and worked their way forward, it's clear with slavery that he started with an assumption that he wanted to be true, and concocted an argument going backwards from there.
Incidentally, he also predicted the end of slavery with an industrial revolution, some 2500 years ago: "For if every instrument could accomplish its own work, obeying or anticipating the will of others, like the statues of Daedalus, or the tripods of Hephaestus, which, says the poet, 'Of their own accord entered the assembly of the Gods.' If, in like manner, the shuttle would weave and the plectrum touch the lyre without a hand to guide them, chief workmen would not want servants, nor masters slaves." [1]
[1] - <a href="https://classics.mit.edu/Aristotle/politics.mb.txt" rel="nofollow">https://classics.mit.edu/Aristotle/politics.mb.txt