Thread

1. edot · 1791067681 · focus · HN ↗

   Absolutely based. Finally someone of stature in the industry calling this whole fear overblown. Bill Gates sounded like a nontechnical goofball in his Ezra Klein interview where he basically just screamed that the Terminator is real.

   There are lots of real worries (government use to suppress the people with minimal manpower or popular support, brainrot and fake news, unemployment due to the belief that LLMs can replace people, education collapse, etc.) we should instead be looking at. This whole rogue AI shtick is tiresome.

   1. Barrin92 · 1791070504 · focus · HN ↗

      >Bill Gates sounded like a nontechnical goofball in his Ezra Klein interview where he basically just screamed that the Terminator is real.

      I saw that episode too and he genuinely looked completely out of it, even in terms of his temperament and how he was coming at Klein for putting common questions in front of him, some people are genuinely starting to lose it.

      I also found the whole debate about cyber-security and 'rogue' software so bizarre because dangerous malware isn't a new thing, and it's often dangerous not because it's intelligent but just the opposite, because it's tiny, viral and fast. Which describes everything that kills humanity in far larger numbers than anything complex, big and intelligent

      1. jswelker · 1791071216 · focus · HN ↗

         Wasn't Bill Gates mostly just saying terrorists could use AI to build a bioweapon? That sounds legit to me.

         1. nradov · 1791072195 · focus · HN ↗

            It only sounds legit to people who don't understand biology and haven't done advanced laboratory work. Sort of like Michael Crichton novels: superficially plausible but not grounded in any real science.

            1. jswelker · 1791072713 · focus · HN ↗

               I see no actual argument here.

               1. nradov · 1791074479 · focus · HN ↗

                  You haven't made any argument either, so I suppose we're even.

                  1. jswelker · 1791075415 · focus · HN ↗

                     My argument was that it is reasonable to be concerned that human terrorists might use AI to assist with building bioweapons.

                     1. nradov · 1791075802 · focus · HN ↗

                        That's merely an assertion unsupported by any reliable evidence, not an actual argument. In short you're just making shit up.

                        1. jswelker · 1791078399 · focus · HN ↗

                           Well that's the nature of trying to prevent a thing that hasn't happened yet right? What would be reliable evidence except that terrorists already used AI to help build a bioweapon? I'm sure before 9/11 talking about terrorists using commercial airplanes as makeshift missiles seemed like scaremongering too.

                           I see zero technical reasons why it could not be done, so being concerned about prevention seems pretty reasonable. I'm not saying AI uses robotic arms to build a bioweapon unassisted or something, just that it dramatically empowers bad actors enough to make them capable of things they previously were not.

                           1. pixl97 · 1791134102 · focus · HN ↗

                              The last few years have shown me something about human behavior.

                              Before the thing happens: "This will never happen, it can't happen, you're making it up, stop being a scaremonger".

                              10 minute after the thing happens: "Of course, this always happened and it's always been this way and we just have to live with it".

                              We are quickly adaptable, but that may be risky if we snuggle up with death.

                        2. literalAardvark · 1791103845 · focus · HN ↗

                           It's quite hard to measure until a wet lab gets behind the filter access to benchmark it.

                           But if you extrapolate from the ability it has in fields that aren't too strictly filtered, it looks pretty scary.

                           There are arguments against doing that but at first glance it seems like we just don't really know, and we likely won't: if governments decide they're interested in AI gain of function capabilities they won't be broadcasting that or allowing public benchmarks.

                           1. zozbot234 · 1791106456 · focus · HN ↗

                              > But if you extrapolate from the ability it has in fields that aren't too strictly filtered, it looks pretty scary.

                              The closest unfiltered analogy to something as complex as chemistry or biology is most likely the softer fields like philosophy, the humanities and the softer end of the social sciences. Most practitioners and scholars in these fields would agree that AI is not nearly as compelling there as it might be in e.g. math, and that's putting it mildly and charitably.

                              Even coding shows the divide pretty well: AI writes code that manages to work (i.e. achieve its self-assessed functional goals) but the stuff is so unmaintainable that it ultimately poisons the AI's own context leading to mode collapse. This makes complete sense because maintainability is a soft objective that's especially hard to automatically optimize for in the short term, as part of a RL training run. The math folks themselves, too, now faced with a very real threat to their field from purportedly "hostile misaligned AIs", immediately zeroed in on education and exposition as something that LLMs are terrible at; with their abilities in systemizing and theory-building also being very much in question.

                        3. comeonbro · 1791127087 · focus · HN ↗

                           What an incredible illustration of human intelligence failing to generalize out of distribution.

Unofficial Hacker News client; not affiliated with Y Combinator.