Thread

1. LightMachine · 1789679503 · focus · HN ↗

   Hi, I'm the author.

   HN staff: someone posted before me. Could we change the title to "Bend - a language that blocks AI mistakes via proof and runs on GPUs"?

   Everyone: feel free to ask any question, but I'd be highly appreciative if you could be a bit civilized and respectful this time. I've worked on this for 1 year, nearly 16h/day, 7 days a week, and I'm giving it for free. You need not to use it. So, I'd be thankful if you could point occasional failures politely rather than throwing me in a lava pit.

   Thank you!

   1. pdpi · 1789681723 · focus · HN ↗

      Don't know whether this is a common outcome, but I tried the "remove the walls" example, and the result was... scary. It completely changed the game so that movement is now diagonal, and made the arbitrary decision that up/down move you on the positive diagonal, and left/right move you on the negative diagonal.

      The problem, of course, is that having only the one single "you can't win" law is severely underspecified, but the solution was too clever by half, and highlights the problem with this approach — every program will be under-specified, because, at some point, writing the laws becomes a bigger problem than writing the code itself.

      This becomes a real issue because the combination of underspecified but rigid laws pushes the aI towards this sort of "creative" solution that matches the letter but not spirit of the law. In this case, the issue was obvious, but I seriously worry about what sort of shenanigans will occur in less obvious cases.

      1. LightMachine · 1789682947 · focus · HN ↗

         Exactly, "you can't win" is grossly under-specified. The goal of the demo is just to show that laws can't be broken. Yet, if that's your only law, the AI can do whatever to protect it - including changing how the character moves, or even removing the flag entirely!

         So, yes, that's the issue with laws: they only protect what you remember to write. They're not a silver bullet. But they can still be incredibly useful, because it takes a small law to protect against entire classes of bug, covering your entire codebase. For example:

         LAW: "the sum of all balances in this contract must be zero"

         This one liner would have prevented Ethereum's infamous "The DAO" hack, where millions were stolen and almost undermined the entire project. But if your laws are under-specifying or ill-specifying your intents, Bend can't do anything to protect you.

         1. skissane · 1789688041 · focus · HN ↗

            Thought about automated discovery of laws in an existing codebase?

            If you can find a law which the existing code obeys, and show it to a human, and if they agree, save it. And maybe the AI could make a decent guess as to what kind of laws would appeal to a human versus which wouldn't – a simple law identifying a fundamental constraint the system obeys is good, something really complicated or constraining something coincidentally true isn't

            Or some kind of measure of coverage? you'd never want 100% – then your laws would become so complex you couldn't follow or maintain them – but if it is very low, that can be a signal to increase it

            1. drfloyd51 · 1789695236 · focus · HN ↗

               Discover the laws in a current code base?

               They are usually in the “tests” folder.

               So anything capable of extracting unit tests is extracting “laws”.

               One might consider the test name the text of a law. And the AI fills in the details.

               1. actionfromafar · 1789722264 · focus · HN ↗

                  From the codebases I have seen the "laws" are scattered between unit tests and lots of little tests and assertions in the actual code, plus cultural norms and "do it like this" patterns.

                  But a good start could be found in the unit tests.

Unofficial Hacker News client; not affiliated with Y Combinator.