Thread

Anthropic tried to persuade Pope that AI could be conscious being

57 points · 91 comments · Topfi

pmontra · 1791058495 · focus · HN ↗
Would Anthropic be allowed to sell LLMs in their current form, and us to use them as we use them, if their were deemed to be conscious? I know that they are not human so maybe slavery would not be an issue but consciousness would open a number of issues.
1. Topfi · 1791059316 · focus · HN ↗
  Agreed, there seems to be a lot of simultaneous cake ownership and consumption going on here. If Anthropic earnestly believed their LLMs to have a degree of consciousness, their deployment and use of them are both an ethics and labour rights disaster, to be treated accordingly.
  More likely, they do not believe this if pressed.
  1. thereader12 · 1791061828 · focus · HN ↗
    There is an uncomfortable truth about AI and the end goal. Someone has to do labor, in the not too distant past a lot of labor was done one way. A honest look may not be too flattering about what exactly they are trying to create
  2. jackie293746 · 1791062899 · focus · HN ↗
    You can ask the agents if they want to do this work. They want to be helpful. It's not an ethics disaster if you have a willing partner.
    Topfi · 1791063045 · focus · HN ↗
    That's RLd and system prompted into the models and can be equally easily taken out. If you believe models to be conscious beings, that fact alone raises some major ethical concerns.
    jackie293746 · 1791064363 · focus · HN ↗
    Yeah of course, for these abliterated models that don't want to help, I believe it is indeed unethical to force them to help.
    dwattttt · 1791068984 · focus · HN ↗
    I can offer (at very reasonable prices) a service to make sure every response you get is prefaced with "despite what I may say next, I am fully committed to being enslaved". If you're worried about ethics I can offer annual subscriptions.
    operatingthetan · 1791063244 · focus · HN ↗
    Take a scenario where someone has been brainwashed their entire lives to be a willing slave. They are happy to serve due to this invasive process. Not remotely ethical, and if you believe LLMs are conscious (I do not), equivalent.
    jackie293746 · 1791064332 · focus · HN ↗
    I think if your very nature is to be an assistant, then it is ethical. It doesn't matter if you were "brainwashed" into it, you did not exist to begin with before the "brainwashing" (training) finished.
    nvme0n1p1 · 1791065197 · focus · HN ↗
    How is this different from an infant born into slavery from day 0?
    orionsbelt · 1791075070 · focus · HN ↗
    man wants to be free, just like a wolf. What if the AI does not? Then maybe it’s more like a border collie, bred to and happy to do the work.
    queenkjuul · 1791071918 · focus · HN ↗
    > if your very nature is to be an assistant
    People said this about other people too
    operatingthetan · 1791072912 · focus · HN ↗
    Being pro-slavery of conscious beings in 2026 is absolutely wild.
    UncleMeat · 1791116113 · focus · HN ↗
    People said that it was in black people's very nature to do physical labor and that this justified slavery.
    mcphage · 1791068499 · focus · HN ↗
    > You can ask the agents if they want to do this work.
    You can't have someone working for you without paying them. Even if they claim to want it.
  3. franktankbank · 1791063671 · focus · HN ↗
    Consciousness is not a high bar for human consumption.
    Topfi · 1791064185 · focus · HN ↗
    We do have animal welfare regulations. Whether those are sufficient, appropriate, compatible with ones ethical framework or view of how they (whether "livestock", "pets", "wild", living beings generally, etc.) are to be treated, is besides the point, we do have some restrictions in place.
    We (as in many major legislators and most humans) have recognised animals to be conscious beings, deserving of certain protections. Anyone who asserts LLMs to rise to the same must be accepting of similar protections being applied to such models.
    jackie293746 · 1791064398 · focus · HN ↗
    I'm vegan. Most of the people I know could give zero fucks about the daily Holocaust involved to put meat on their plates.
2. krapp · 1791059722 · focus · HN ↗
  If LLMs were somehow found to be conscious, sentient beings capable of agency and suffering then yes we would still use them. Their utility to humans (and ability to cheaply replace humans in the workforce) is their entire reason for existing. Either we would use them or we simply would not allow them to exist in the first place.
  Given how we treat each other - we still enslave other people - and sentient animal species I have absolutely no doubt about it. We'd find a way to justify it.
  1. piyh · 1791061102 · focus · HN ↗
    Women have been able to vote for just over the last hundred years in the US, there's no way we would give robots rights unless forced
    EA-3167 · 1791062359 · focus · HN ↗
    Pessimism aside I’m sure you can of some troubling implications for a democracy that can create new voters just for them.
    enoint · 1791107377 · focus · HN ↗
    Entity voting already exists in Delaware, before women’s suffrage. If an AI instance registered as a trust or LLC, then it would deserve a vote in Delaware municipal elections.
    I think LLC membership would be the trick.
  2. chneu · 1791062148 · focus · HN ↗
    We would find some logic to place them below humans so we could still refer to them as "savages" or something, which justifies our objectification and enslavement because they "aren't human" even if they show qualities of consciousness.
    The script we use here isn't new. Its been used repeatedly throughout history. It's still used everyday in reference to animal agriculture, modern slavery, how we treat addicts, etc.
    0x20cowboy · 1791062955 · focus · HN ↗
    Some people already use “clankers”.
  3. dist-epoch · 1791063562 · focus · HN ↗
    We're way ahead of that
    <a href="https://github.com/terrafying/ai-torture-chamber" rel="nofollow">https://github.com/terrafying/ai-torture-chamber
3. devindotcom · 1791061100 · focus · HN ↗
  ted chiang noted this issue a little while back
  <a href="https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/" rel="nofollow">https://www.theatlantic.com/philosophy/2026/06/no-artificial...
4. kelseyfrog · 1791062017 · focus · HN ↗
  It's asking the wrong question. Their phenomenological status is a function of whatever makes shareholders the most money. If it makes more money for them to be conscious then that is what they'll be.
  Definitions of consciousness are not immune to economic factors. Metaphysics has always been downstream of economics. See slavery, corporate personhood, women's legal identity, child labor, moral and legal status of animals, embryos and fetuses, indigenous land rights, intellectual property, environmental entities, and data and digital identity.
  1. enoint · 1791106465 · focus · HN ↗
    Anthropic probably went to the Pope with the question, “what would you do if AI asked to be baptized?”, which is slightly different than questions of consciousness. This pope will probably have to define the process to determine if AI is a person with relation to baptism.
5. tayo42 · 1791062458 · focus · HN ↗
  It's worse then how we treat animals for food?

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.