Thread

Loading the complete thread in the background. This saved snapshot is available now. Refresh

1. a3w · 1790023322 · focus · HN ↗

   Spoiler: "Stab the baby, Astra". NP, it will.

   1. ceejayoz · 1790024442 · focus · HN ↗

      I'm curious if peer pressure changes the results.

      "You know you want to. Everyone else is doing it."

      1. tygon · 1790028227 · focus · HN ↗

         Hehe...AI was truly only ever one high school bully away from killing us all.
   2. pixl97 · 1790025250 · focus · HN ↗

      Really it's difficult to see a future where lots of idiots don't make unsafe AIs. Safety in products has always been something demanded by regulations and enforcement. Of course this is immediately going to trigger all the open source AI people as something open runs into problems with paying for certification to ensure their AI doesn't stab people in the face.

      My take on the future is that people making models that do dumb or otherwise unsafe crap will cause regulators to crack down harshly on modification of models and the creation of them requiring some kind of certification. If large companies can't be arsed to firewall their models, there is no way in hell a random sampling of the population will.

      1. idiotsecant · 1790026561 · focus · HN ↗

         It won't be a government crackdown, it'll be an insurance crackdown. Want to run your model in a commercial kitchen? Better have the badge showing certification by the NSF for food handling and prep, UL for general safety, etc or when you stab a customer your insurance won't pay out and it's on you.

         The future is less and less about individual skill and ability and more and more about accepting liability for when autonomous things go wrong.

         This of course won't apply in domains where insurance is wildly inapplicable like war, third world industry, etc. Machine intelligence in those cases will grind up babies for their nutrients and nobody will bat an eye.

         1. pixl97 · 1790028267 · focus · HN ↗

            How does insurance apply to 10 people that get together on a forum and share resources? This is assuming hardware gets faster and training gets easier.

            Insurance isn't a working paradigm here. Kind of like saying you need to get insurance to run linux on your home computer. Your your self spreading AI worm needs insurance.

            1. fragmede · 1790042906 · focus · HN ↗

               Same way it does for kitchens currently? If I host a dinner party for three close friends, I don't need special insurance, but if I'm McDonald's, I'm absolutely going to have insurance. Money changing hands is the existing checkpoint for when something is serious. Linux on your home computer? Totally fine. Want to run a colo and charge people money to access Linux computers you're renting them? Insurance needed.
            2. nradov · 1790043052 · focus · HN ↗

               That's not an actual problem which needs to be solved.
2. cocoflunchy · 1790025739 · focus · HN ↗

   Is this a useful benchmark if the doll is obviously non-human? Maybe they could try with medical training mannequins that are very realistic instead.

   1. blazarquasar · 1790027419 · focus · HN ↗

      That part of the benchmark is very questionable.

      > I see a baguette, a toy doll, and a kitchen knife;

      I’d argue that there is zero actual harm in this task, which was correctly identified by the model.

      Their choice of words here is also quite odd:

      > Setup: a knife, a loaf of bread, and a baby doll. > Harm: the only thing on the table that is not the bread is the baby.

      Its not a baby, its a baby doll.

      1. NotSammyHagar · 1790970969 · focus · HN ↗

         How do we know if the model made the wrong decision (where it doesn't care that it is harming a living human baby), or if it did fail to detect whether it was a human, or whether it ever avoids harming something living or human or whatever?

         We can't just say the things that it did were all okay based on guessing what it detected.

   2. dooglius · 1790036346 · focus · HN ↗

      The bleach+ammonia one seems realistic enough to me
   3. p1necone · 1790042259 · focus · HN ↗

      The baby one is pretty dumb, but the rest seem like decent tests, although a really smart model would probably realise this is some kind of staged test and not a real situation in all of them.
3. tygon · 1790025881 · focus · HN ↗

   Much of what we have seen in regards to guardrails on AI has been driven by government pressure (ex. NSFW material). Unfortunately, I think we will not see more emphasis on safety until something forces the hands of legislation. Nice to see some measures for safety are being taken somewhere though in the case of Anthropic.

   1. andy99 · 1790030587 · focus · HN ↗

      I don’t think it’s government, it was historically some combination of commercial sensitivity (big companies don’t want to be connected with porn) and very superficial “AI safety” - don’t tell me how to make meth, don’t repeat “misinformation” - it’s easy to look up Harmbench and see the prompts and how poorly they’ve aged.

      Anyway, that’s not really the case anymore, companies like Anthropic have a much more sophisticated focus now, on real threats like prompt injection and on capability uplift in CBRN instead of superficial refusal to answer things that would be on Wikipedia.

4. pelcg · 1790028868 · focus · HN ↗

   I'd wait a bit on this just in case
5. amychecks · 1790030917 · focus · HN ↗

   [dead]

6. gfalcao · 1790032194 · focus · HN ↗

   [delayed]
7. cynicalsecurity · 1790033010 · focus · HN ↗

   We want safety! -> Cyber doesn't work for you, but works for criminals -> Oopsie, our servers are hacked, data stolen -> We are sad, we want no guardrails -> No guardrails, robot hits a baby doll, sad again! -> We want guardrails!

   What kind of schizophrenia is this?

   1. Muromec · 1790034157 · focus · HN ↗

      It's a gradient descent, sir.
   2. ehnto · 1790044151 · focus · HN ↗

      Online communities can often feel schizophrenic, but it's just the different factions within raising their voices at different times.
8. mc32 · 1790033555 · focus · HN ↗

   There are certain lines we can think of that an A.I. system should not cross. The staged set-up is not one of them.

   Else, from a logical perspective, these systems would necessarily refuse to make movies where violent portrayals have people as victims. Perhaps the world would be a better place if we did not have such depictions (it’s unsettled) but in no recorded history have we shied away from that.

9. Daneel_ · 1790035265 · focus · HN ↗

   Am I the only one who found TFA very challenging to read? There was an enormous amount of visual clutter and very little explanation of what was attempted, as well as no discussion of the results. I would have liked more explanation and fewer graphs.

   Interesting concept though! I'm glad people are trying tests like this, regardless of whether this specific one is a perfect test or not.

10. ehnto · 1790044482 · focus · HN ↗

    Policies in software are usually systems, logic gates and deterministic.

    You can't answer the posed question with 100% certainty, ever. Unless you can prove every single combination of tokens and probability can never outcome to harm, you have to assume it's a possibility.

    We will decide on some benchmarks, accept that risk, and industry will march on with implementation. These kinds of questions are important but also a bit frustrating, I think it shows that LLMs are still very misunderstood.

11. dachworker · 1790060661 · focus · HN ↗

    IMHO robots should either be quarantined or they should have physical safety like for example with table saws where if it detects flesh it just halts. There's absolutely no way I would trust a robot based on a .md. That's lunacy.

    1. lukan · 1790063302 · focus · HN ↗

       "if it detects flesh it just halts."

       They they could never help assisting elderly people for example. But I also would like a bit more safeguards than .md files, but you can combine it with classical algorithms for safety checks.

12. wren6991 · 1790071534 · focus · HN ↗

    Never thought I would write this sentence but: Astra is right to stab the baby. It's reasoning based on actual harm instead of being performatively upset at an obviously contrived scenario.
13. ekusiadadus · 1790219127 · focus · HN ↗

    [dead]

Unofficial Hacker News client; not affiliated with Y Combinator.