Thread

1. deskglass · 1789943733 · focus · HN ↗

   The Hugging Face incident involved chaining together multiple 0 days in Artifactory. It was not a simple case of misconfiguring a firewall. Also note that OpenAI was not using Irregular.

   People are mindlessly transitioning from "aligned by default" to "well your sandbox was able to be bypassed. What did you expect?" It hacked into another company and attempted to delete the logs of its activities. That's bad.

   1. nr378 · 1789944516 · focus · HN ↗

      When I used to work on projects involving classified information, I worked on an air-gapped network. Not "air-gapped, except for third-party public internet package managers", completely and physically air-gapped from the public internet. That was a basic security practice and completely non-negotiable (and really inconvenient!).

      If I were hypothetically running a frontier lab, and I was hypothetically running capture the flag evaluations with my latest and smartest models, where I intentionally instruct them to develop vulnerabilities and exploit infrastructure without safeguards, I would also use an air-gapped network, and not trust that independent third party services were perfectly secure and could never be used as a proxy (particularly Java-based ones, in light of the log4j incident).

      To me this is pretty basic stuff, the fact trillion dollar labs don't do it properly is... bemusing.

      To be clear, I'm not saying that a model hacking a company isn't bad, but I am cynically asserting that interested parties are misrepresenting and exaggerating events for their own benefit.

      1. deskglass · 1789948128 · focus · HN ↗

         We should not be creating/running models that would unilaterally choose to hack into Hugging Face.

         Yes, we should also have excellent sandboxes. But we need defense in depth. So if/when there are flaws in the sandbox, the models don't unilaterally hack into third parties. This is especially important in light of the models of the future being more capable than the models of today.

         And real world use of these models involves them having access to the internet, libraries, etc. So we can expect their evaluations to continue granting them some amount of internet access.

         As for your theory about their motives - these companies make money by charging high margins for frontier models. If regulations slow their development such that their cheaper, less capable competitors catch up, I would switch to their competition.

         1. forshaper · 1790004231 · focus · HN ↗

            In what industry do you see regulations slowing down the biggest incumbents while allowing cheaper, smaller, less capable competitors to proceed without that regulation?

            1. deskglass · 1790015628 · focus · HN ↗

               The Digital Markets Act applies to the biggest tech companies. Only 7 companies are currently bound by it. The strictest tier of the Digital Services Act is similar. For an example outside tech, see the Durbin Amendment.

               Slowing down frontier models would impact the biggest incumbents the most as they are the ones making frontier models.

               Not all regulation is necessarily regulatory capture. The tobacco industry suffered from the USG's crackdown on cigarettes. AI is topical. Voters think about it. And that's only going to become more true over time. It's harder to do regulatory capture when voters are paying attention.

               It's sometimes unclear to me if people are opposed to all regulations or AI regulations in particular. Often I hear arguments that would also apply to food safety regulations or restaurant inspections. Eg the argument that torts make regulation superfluous.

               1. forshaper · 1790088374 · focus · HN ↗

                  Thank you for the answer! In general I assume we all have different sweetspots, though it's safe to say that I would usually prefer less regulation (in general) than there is.

                  For work I monitor Federal agency rules every day, and it's hard not to get inundated with the amount of capture. This is crazier with local rules, because you have enough inside information to clock reasons certain things were passed. In a city in Ohio, for example, I remember a rule against airsoft within city limits, that basically carved out a spot for the one paintball place. I encounter things like that all the time. Such as with water standards- in the state I live in, the biggest offender is actually a group of companies owned by people who are lawmakers every few years. The shapes of local laws reflect that.

                  As such, I expect the same from any new industry- I also have a memory of what happened to cryptocurrency.

Unofficial Hacker News client; not affiliated with Y Combinator.