‹ BackHN Continuity

Thread

OpenAI's Misalignment Framework: A Tactical Bid to Preempt Global AI Governance

41 points · 91 comments · ghernando

  1. pcestrada · · focus · HN ↗
    OpenAI and Anthropic should be nationalized. I find it difficult to trust Sam Altman or Dario Amodei.
    1. manav · · focus · HN ↗
      and you trust the government?
      1. klaff · · focus · HN ↗
        not this one
        1. redsocksfan45 · · focus · HN ↗

          [dead]

    2. vb-8448 · · focus · HN ↗
      Even better: force them to release weights.
      1. bibimsz · · focus · HN ↗
        who even has the inventive to do that? certainly not the government.
      2. autoexec · · focus · HN ↗
        Even better: force them to release every scrap of data they trained their AI on.
      3. timmg · · focus · HN ↗
        My gut reaction is that this is not a good idea.

        But I could imagine a scenario where you are required to release weights for publicly-used models after N years. Kinda like how drugs have a limited patent.

        Not sure what N should be. But it would make for an interesting rule.

        1. nullbio · · focus · HN ↗
          > But I could imagine a scenario where you are required to release weights for publicly-used models after N years. Kinda like how drugs have a limited patent

          That's not a bad idea actually.

    3. [deleted] · · focus · HN ↗

      [deleted]

    4. prettyblocks · · focus · HN ↗
      I guess that's one way to guarantee development slows down to a crawl, but in no way would it increase trust.
      1. bibimsz · · focus · HN ↗
        why would that be the case? it's not true for top secret defense contractors today. and the frontier labs already operate in the dark, openness is a liability.

        nationalization to me simply means the government is their main customer and stakeholder, and shield them from liability, governance, and openness. not that the frontier labs become part of the government per se.

        1. pixl97 · · focus · HN ↗
          >eans the government is their main customer and stakeholder, and shield them from liability, governance, and openness

          I personally believe they already have this, why would the current government at least want to formalize this when it can have it with no public discussion.

        2. WarmWash · · focus · HN ↗
          Well we can say whatever we want if we have our own definitions for everything, no?
          1. [deleted] · · focus · HN ↗

            [deleted]

    5. bibimsz · · focus · HN ↗
      they will. to shield from oversight, liability and profitability concerns, and to ensure unimpeded rapid development, with the frontier only being available to elite (not you). definitely not to add public transparency.

      the frontier labs are the new top-secret defense contractors.

    6. indymike · · focus · HN ↗
      I find it even harder to trust elected leaders from any party. At least Sam and Dario are aligned with a value set that is understood and clear, whereas political leaders values changes as do the polls their livelihood depends on changes.
      1. ceejayoz · · focus · HN ↗
        > At least Sam and Dario are aligned with a value set that is understood and clear…

        How could we possibly know this?

        1. [deleted] · · focus · HN ↗

          [deleted]

        2. vasco · · focus · HN ↗
          What do you think their value set is if not "gimme more money"? Seems pretty clear cut to me.
          1. ben_w · · focus · HN ↗
            "Number go up" is table stakes, it's how you keep track of the score well before you reach their level.

            However, that's like saying I'm motivated by food. I mean, yes, I like food, but this isn't a useful description to let you guess what move they'll make next, especially as they're opening opining about radical economic transformations that are likely to do to money what money did to real estate when the industrial revolution came.

            And that's still true even if you don't believe they're anywhere near actually achieving any of these things.

      2. jsrozner · · focus · HN ↗
        No system of governance can deal with immense concentration of power. The US Constitution was about separation of powers. Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful.

        Political leaders become a problem when they amass too much power. Corporations become a problem when they amass too much power. It doesn't matter what Sam and Dario's purported values are. They aspire to power and absolutely power always corrupts absolutely.

        Technologies which are infinitely powerful or whose power grows too quickly outrun any reasonable attempt at regulation. If you imagine that tomorrow everyone were given a tank, we might think, "alright, everyone has a tank so it's not too bad." But humans are squishy, and our houses are (relatively) squishy compared to tanks. Substantial collateral damage would result from everyone having a tank, and it seems likely that substantial collateral damage will result from everyone having a cyberterrorism-capable slop machine.

        1. luqtas · · focus · HN ↗
          > Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful.

          this is "direct democracy" and it's not even close to exist in USA... even with that a society can allow powerful people to exist if they don't create any law forbidding that

          1. jsrozner · · focus · HN ↗
            Democracy includes a broader array of governmental organization than just pure direct democracy. If you do believe that individuals should have some ability dictate the terms of their own social organization, then you believe in some amount of democratic principles.

            Economic power eventually manifests in the political realm. The wealthy effectively get more votes, which means that society moves away from being democratic. Thus substantial wealth inequality is incompatible with democracy in the long run. We have been witnessing that corruption for a while now.

            1. luqtas · · focus · HN ↗
              how the wealthy would get more votes in a direct democratic system? they are literally the minority, thus fewer votes
        2. throwaway13337 · · focus · HN ↗
          Nearly everyone in the US has a giant car that is basically a tank.

          They are dangerous. But it’s managed.

          Centralized power never works. We have the worst times in history to look at.

          Nationalization just centralizes to a different set of people. It’s personal ownership or oppression.

          We all need open weight R2D2s.

          1. pixl97 · · focus · HN ↗

            [dead]

          2. jsrozner · · focus · HN ↗
            Cars are decidedly less dangerous than tanks, which are less dangerous than nuclear weapons. I am certain that giving a nuclear weapon to every person in the world would not go well.
            1. freejazz · · focus · HN ↗
              I don't see tanks killing pedestrians every week or so in NYC
          3. boie0025 · · focus · HN ↗
            I can't properly read the tank/car comment, the comparison is bizarre to me. But I think this misses the point a bit. The reality of these new risks is literally being learned in front of us in real time, and in my opinion, anyone who claims to understand these risks is speculating at best, and actively manipulating the situation for whatever reasons.

            While I am a (mostly) capitalist and generally disagree with nationalization (including, for the time being, this situation), I also don't think we can say that "centralized power never works". Maybe we scope that a bit. I know plenty of business owners who centralize power in their businesses and they are effective, ethical and it works perfectly fine. In theory, in the US, nationalizing some unit of the economy _decentralizes_ power; the US is, after all, a representative democracy. Trustworthiness of the electorate is a different problem. But I don't think we can generalize much about centralization beyond sometimes it works and sometimes it doesn't. There is a difference between centralizing all power in a given administrative unit, and centralizing certain powers, but that's more analogous to non-democratic units.

          4. ben_w · · focus · HN ↗
            Funny, I'm sure I would have noticed when I visited if US cars came with tracks, a 105 mm main cannon, and massed around 55 metric tons.

            We may all like our own personal R2 units, but if you insist on scifi, instead of tanks, consider everyone getting an X-wing for their commute. Oops, safety on the blaster was off, there goes the neighbourhood.

        3. pixl97 · · focus · HN ↗
          It's interesting the cyberpunk-esque future we're sliding into. Things like cognito-hazards and information-hazards are legitimately discussed and researched problems we're experiencing.

          It's going to be interesting on how humanity deals with this problem (well, or if we turn it over to AI and make it their problem and suffer whatever consequences falls out). Being able to gather further information and power by acting on the information you already have causing massive power imbalances that is very hard to deal with, it's a natural outcome.

      3. swatcoder · · focus · HN ↗
        > At least Sam and Dario are aligned with a value set that is understood and clear

        What value set do you perceive that to be, and why would you take your perception of it to be any more sound than it would be with a politician?

        It's not like someone can operate companies of that scale, especially startups, through earnestness and openness. Like national politics, their job is fundamentally about perception management and power brokering across dynamic windows of opportunity. Nothing they say or do can be taken at face value, and you can't reduce their incentives to either company or personal profit in any particular form over any particular time scale.

        1. pixl97 · · focus · HN ↗
          >What value set do you perceive that to be

          A meat based paperclip maximizer.

        2. jonas21 · · focus · HN ↗
          I feel like many of Anthropic's issues are due to Dario being too earnest and open. It seems both refreshing (that a CEO has thought deeply about and is willing to talk publicly about the dangers of their product) and depressing (that so many people cynically think this is some sort of marketing ploy).
          1. freejazz · · focus · HN ↗
            Yeah, the guy running a trillion dollar scam is totally the exception to CEO-ism
            1. ben_w · · focus · HN ↗
              What's the old quote from WW2?

                I couldn't help but notice how each successive headline reporting our glorious victories seemed to draw closer to Tokyo.
              Something like that.

              Well. I can't help but notice how each successive headline reporting how this scam/stochastic parrot/"scare quotes intelligence" seems to be solving more and more things that were but a few years ago widely regarded as being indicators of high intelligence.

              1. freejazz · · focus · HN ↗
                Year 3 of being told my job will be replaced by AI, and the only thing that's happened so far is that AI vendors keep showing up to my office, begging me to pay them to use it is a tool.

                If you're only seeing the charts go up - you're not looking in the right places.

                1. ben_w · · focus · HN ↗
                  I'm looking at the millennium puzzles, and independently of those puzzles I had asked it for a fluid dynamics simulation engine that runs in my browser, and it put one together for me so I could play with aerospikes and watch the formation of Mach diamonds in rocket engines. The isochrone map generator has also been stuck on my to-do list for years, and yet now thanks to Claude, I have it, and it's real-time and multimodal.

                  Even before then, the free trials of Claude Code and Codex at the end of last year and start of this year… despite their flaws, they could write all the code I've ever been paid to write. Sure, limited speedup, Amdahl's law and coding is not the only part of the job, but anyone who was fine at PM and QA but not code no longer needs a coder.

                  Even before then, I looked at the maths puzzles they were doing well at, and I found I did not understand the questions let alone the answers.

                  I remember when the ability to generate music and art was "uniquely human", and sure there's a lot of cringe there with those models, but they're also winning awards and causing controversy by doing so, and artists are losing clients; I remember when the board game Go was considered to require "human intuition we could never make a computer solve, totally different to chess" (and I remember when chess was so, too).

                  When I was a kid, cheques and letters on addresses often got read by a human; the OCR which automated this is also AI, though these days image-to-numbers is the "hello world" of the field.

                  1. freejazz · · focus · HN ↗
                    >Even before then, I looked at the maths puzzles they were doing well at, and I found I did not understand the questions let alone the answers.

                    okay, and?

                    >Even before then, the free trials of Claude Code and Codex at the end of last year and start of this year… despite their flaws, they could write all the code I've ever been paid to write.

                    Boy, programmers sure do think programming is like the only thing in the world

                    I'll repeat it for you again:

                    If you're only seeing the charts go up - you're not looking in the right places.

                    1. ben_w · · focus · HN ↗
                      > Boy, programmers sure do think programming is like the only thing in the world

                      You're the one who called it a trillion dollar scam.

                      The compensation paid to professional software developers worldwide is currently around US$1.5 trillion per year.

                      1. freejazz · · focus · HN ↗
                        >You're the one who called it a trillion dollar scam.

                        Don't pretend like the AI companies are saying its only useful for programming. You've been arguing against that the entire time.

                        1. ben_w · · focus · HN ↗
                          They're not only useful for programming, it's just that programming is by itself a trillion dollar a year industry. This, by itself, is sufficient to not be the scam you assert it to be.

                          They may not work for your industry, but then again I don't know what your industry is.

                          Bluntly, over my lifetime, I've heard "AI will never/not in my lifetime do X" repeatedly within a year of it doing X, for many different X. It's good at getting good. The most recent one being "make useful contributions to millennium prize maths problems". A year or two before that, it was even "do well on degree-level economics essays".

                          Artists are unhappy, not only because it rips off their work, but because businesses that were previously hiring them now use it instead. This is much smaller (strictly in terms of money) than with software, but is also measurable.

                          Now, if you said Tesla's self-driving cars (another AI) are a trillion dollar scam, that I would even agree with. At least, for the market cap part, for actual sales it's more like a billion dollar (ish) scam.

                          1. freejazz · · focus · HN ↗
                            >They're not only useful for programming, it's just that programming is by itself a trillion dollar a year industry. This, by itself, is sufficient to not be the scam you assert it to be.

                            Yeah, if you think I'm being overly literal. Regardless, people here were positing, yourself included, that they are widely useful.

                            > Bluntly, over my lifetime, I've heard "AI will never/not in my lifetime do X" repeatedly within a year of it doing X, for many different X. It's

                            Again, over my lifetime, I've seen it been told to me that the next iteration of each model will surely be the one that does away with my entire profession. Yet, again, here we are, firms at my door, begging me to use their tools.

                            >Artists are unhappy, not only because it rips off their work, but because businesses that were previously hiring them now use it instead.

                            No, they are unhappy because it rips off their work. You're totally wrong about that.

          2. nullbio · · focus · HN ↗
            It's exactly the opposite. Anthropic has earned their terrible reputation through years of lying, deceit, misdirection, gaslighting, unethical marketing strategies, etc.

            I remember it vividly, when Anthropic first came on the scene, people (myself included) were incredibly optimistic about them and their leadership. Everyone hated SamA and OpenAI because they felt they couldn't be trusted.

            Then slowly but surely, they showed their true colours. Now their reputation is in shambles due to their own behavior, and people are rooting for OAI to beat them. OAI's reputation gains have purely been a result of NOT following in the footsteps of Anthropic.

      4. infogulch · · focus · HN ↗
        Yes it's clear that Sam and Dario seem to be aligned with a value set that prioritizes concentrating trans-national government-mandated centralized control of AI and crowning themselves high priests of this unholy abomination. "At least it's clear that they're aiming to bring hell on earth" -- hard disagree, I think we can aim significantly higher.
      5. goatlover · · focus · HN ↗
        At least politicians are somewhat beholden to their constituents. What keeps Sam and Dario in line other than profit?
      6. forgetfully05g1 · · focus · HN ↗
        I find it even, even harder to trust your opinion in this context with a business that has “AI powered” at the top of the landing page.
      7. autoexec · · focus · HN ↗
        > I find it even harder to trust elected leaders from any party.

        You can vote out an elected leader, but not Sam and Dario. It's very weird that you're so willing to give up any kind of power and want to be ruled by unelected billionaires who only want to take advantage of you at every opportunity.

        1. ben_w · · focus · HN ↗
          Hi, I'm not a citizen of the USA.

          I can't vote out your president, and we've already got a huge trust problem with the one y'all went for.

          1. autoexec · · focus · HN ↗
            Yeah, the entire rest of the world has pretty much been stuck being ruled by unelected billionaires who only want to take advantage of them at every opportunity. It's a problem. It's starting to change though. The EU is starting to walk away from their abusive relationship with Microsoft and Amazon. China has a lot of their own stuff (mostly to better control their people though).

            As an American, I'm still hoping it's not too late to fix things, but it's got to be hard for those outside the US to be so dependent on it, especially when we're looking like a sinking ship and our current administration is still running around drilling holes in the hull.

            You can always become a US citizen to get a voice, but I wouldn't recommend it now or you'll be thrown in prison as soon as you show up to your scheduled immigration hearing. The better option is to keep trying to reduce your dependence on US companies and consider the worst aspects of our current situation (in both corporate policy and government) as a cautionary tale so you can try to avoid them in your own country.

      8. MattGrommes · · focus · HN ↗
        Admittedly I haven't thought this through incredibly deeply but what if "nationalize" just means the US government owns half of the company? Then we get profits as recompense for building the company on our shared culture but there's still a profit motive for employees and a check on the direction of the company in the same way VCs have. But without necessarily turning the company into some red-tape bound bureaucracy.
        1. ceejayoz · · focus · HN ↗
          Trump's been doing that.

          <a href="https:&#x2F;&#x2F;www.pbs.org&#x2F;newshour&#x2F;politics&#x2F;what-economic-and-policy-experts-think-about-the-u-s-governments-stake-in-intel" rel="nofollow">https:&#x2F;&#x2F;www.pbs.org&#x2F;newshour&#x2F;politics&#x2F;what-economic-and-poli...

          &gt; Then, in August, Trump called for Intel&#x27;s beleaguered CEO Lip-Bu Tan to resign, alleging ties to China. Days later, after Tan met with Trump, the president called him a &quot;success,&quot; before announcing that the federal government had bought a stake in the company.

          Somehow, this isn&#x27;t derided by the right as socialism.

      9. freejazz · · focus · HN ↗
        &gt; At least Sam and Dario are aligned with a value set that is understood and clear

        I find this ironic as it&#x27;s regularly pointed out here that they operate in the exact opposite way.

      10. sillyfluke · · focus · HN ↗
        I shouldn&#x27;t be surprised, but it&#x27;s still wild to me that people would trust robber barons rather than their elected officials to run their country. And this continues until you end up electing robber barrons as your elected officials, which is where we are at now.

        The idea that anything of lasting good can come out such a preference doesn&#x27;t seem conceivable. And if the educated think like this I guess we&#x27;re passed blaming the poor and ignorant.

      11. troyvit · · focus · HN ↗
        How about internationalized then? Give the UN something to do?
    7. binlog · · focus · HN ↗
      Makes sense let’s trust Donald Trump instead
      1. greenmilk · · focus · HN ↗
        a maliciously engineered dichotomy
    8. cmrdporcupine · · focus · HN ↗
      The nature of the US state is such that the distinction between nationalized and not is almost meaningless.

      Like Lockheed-Martin or Boeing, etc. there&#x27;s just interpenetration between the corporate boardroom and the state. They act in each other&#x27;s mutual interests.

      The Chinese system is just more explicit and open about this.

      And as a non-American, I can&#x27;t trust the US state anymore than I can trust its dominant corporate entities. So I fail to see the advantage to the world to it being nationalized. In fact under the current administration this would be an even worse outcome.

      1. woah · · focus · HN ↗
        You&#x27;re taking companies that work almost exclusively for the US government and represent a tiny portion of the US economy as representing all US companies?
        1. pixl97 · · focus · HN ↗
          This is a weird contextual failure on your part.

          Not all companies have a concentration of power. The government has no mutual interest in those companies. Now, when you talk about things in the F100 the situation changes drastically. If you produce things like planes, weapons, and weaponization of software you are talking about something completely different in kind.

        2. cmrdporcupine · · focus · HN ↗
          Beyond the defense sector, the US state has always intervened publicly and privately to mediate and balance competing corporate &quot;private&quot; interests.

          It has also periodically aggressively helped subsidize, bankroll, and enforce the interests of some key sectors; notably the petroleum&#x2F;energy sector. And finance.

          In those sectors the state and private sector are fully intertwined in a strategic way.

          I think &quot;AI&quot; is now joining that list. OpenAI and Anthropic will not be allowed to fall over or explode, and speculative investors I think are confident even with the dubious financial situation because they know this.

          Especially insofar as there&#x27;s now a strategic alignment of the fossil fuel sector and the &quot;AI&quot; datacentre sector as they are now becoming massive users of natural gas.

          (Worse: Here in Canada that has taken on a very explicit role in that new datacentres seem to be pitched mainly in areas with remarkably traditionally expensive electricity and 100% reliance on natural gas [Alberta] and even coal [Saskatchewan] power generation -- instead of places like Quebec and B.C. that have copious hydroelectricity. On the surface it makes no sense until you realize it&#x27;s more about finding customers for domestic natural gas than it is strategically about AI itself.)

    9. imadierich · · focus · HN ↗

      [dead]

    10. mukyu · · focus · HN ↗
      How do you think that would go? A large portion of OpenAI workers were ready to jump ship the moment the board coup&#x27;d Altman. How do you think they are going to react to Donald Trump being in charge? It is not likely that OpenAI would continue to function.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.