- Correct by construction: the language makes invalid states or programs hard or impossible to express.
- Statically established: types, proofs, and static analysis establish properties before execution.
- Runtime-enforced: memory management, isolation, capability boundaries, and other runtime enforced properties.
- Empirically validated: program validation through tests, property-based testing, and fuzzing.
---
Along with being familiar, so it's easy to generate, is a huge part of why I'm building Zena: <a href="https://zena-lang.dev/" rel="nofollow">https://zena-lang.dev/
I don't have the AI-first rationale put into the public docs well just yet, but I mention some of it here: <a href="https://zena-lang.dev/guide/why-zena/#familiar-to-humans-and-to-agents" rel="nofollow">https://zena-lang.dev/guide/why-zena/#familiar-to-humans-and...
along with a doc in the repo on this topic: <a href="https://github.com/elematic/zena/blob/main/docs/design/ai-first-language.md" rel="nofollow">https://github.com/elematic/zena/blob/main/docs/design/ai-fi...
In short, the more deterministic, automated, checks the better. AI can deal with a pedantic language. I intend to add statically verified structured concurrency, units of measure, contracts, and eventually more and more formal methods into the language so it can be a familiar TYpeScript-like base with as many static guarantees as we can fit in.
I also think that fine-grained isolation, which Zena gets via Web Assembly, is critical for limiting the capabilities of generated code and the blast radius of bugs, vulnerabilities, and non-aligned behavior.
I do have an optimistic hope that a language also optimized for humans, readability and simple semantics especially, has value in the future, even when most code is generated. We'll see about that.
I think starting with a familar typescript-like base language is a good approach to this. This should be familiar enough for LLMs for the most part as long as additional features can be explained in a succinct system promopt/skill.
From experience with Zena, this is not true at all. Opus, Fable, Gemini Flash and Pro all barely make any syntax mistakes after a little is in context, and those are caught extremely early.
The one thing I do see sometimes is that agents sometimes don't take advantage of added features, but that's partially because the Zena code base doesn't use them as much yet. I'm working on skills and linter-based suggestions to use better patterns.
You should have working programs that demonstrate the features you want them to use, and then the skills. Working programs they can mutate in an RL gym.
> bc agents will naturally be bad at it due to a lack of examples.
Can we please as a community stop parroting these false premises as a basis of every argument against doing anything new? It's plainly obvious to anybody that uses LLMs on a regular basis that it's not true.
It might seem obvious but in truth is not - to be honest, a strictly typed language where models can play adversarial competitions against a strict compiler/linter does have advantages, as mentioned in the article itself but that's not related to training size, the advantage is exactly that it can generate synthetic useful data due to compiler/linter guarantees - regarding training size as long as some logic showcasing the constructs exists that is all that is needed. If you have 2 correct examples for each feature or language construct then the probability of the a LLM learning it and applying it is very much guaranteed, if you have thousands of diverging uses of patterns and constructs in the training data (with a lot of bad examples or wrong uses) a LLM might, I dare say will, actually perform worse, since the probabilities of what particular variation being the "correct" one to use are all over the place.
What can happen is sometimes patterns that are unique to certain languages aren't surfaced and so can't be "learned" but that is a different case, since you can add examples - or patterns that go beyond syntax (threaded code, process isolation, interaction between different modes of resolution, etc) where you need the agent to be able to understand and plan higher-level logic.
I wrote CSSex as a css pre-processor (similar but not the same as SASS/SCSS) and my "boss" at the time fed the existing cssex files to GPT (+2 years ago) and it was able to write CSSex just as fine as if it was writing something that was present in its training data set. It never introduced syntax bugs, the bugs it introduced were all relative to complex cascading styles rules in an existing large (for a definition of large in webapps) codebase, rules interactions (CSS), browser quirks, and complex logic to do what we "humans" wanted (or him in this case) and so on.
True, it’s also trained on millions of papers of programming language research, so you can write a language to operationalize that research and it does great with it. The millions of lines of typescript scaffold the syntax, the PL papers scaffold the semantics.
Agree, new language will not suitable for LLM because there is not enough documents about it on the internet. LLM should learn the usage of the langauge from documents. Yes, it can use new language but it's not natural as existing(which have a lot of documents) languages.
So my opinion is language is no matter anymore. Just prompting(describe the spec) matters. And there is babo language for this.
<a href="https://github.com/armbox/babo" rel="nofollow">https://github.com/armbox/babo
They should find the limitation of the natural language somehow, and use and distribute babo language will help find how it will works(or not). LLM model eveolves yet, so there is no such a approach, but the speed of evolution is decreased lately. So now is good time to dig into identify the limitation and boundary of the LLM.
This is absolutely not the case in my experience. I am building a very large embedded domain specific language for describing distributed systems. It looks like a small subset of Elixir, but with object-oriented syntax in a lot of places. (It’s called a choreography; there exist many other choreographic programming languages.)
Even though this programming language is absolutely nowhere in any large language model’s training set, they have so far done extremely well at extrapolating from the small set of examples I’ve given it when I need an agent to generate some tests or whatever for me.
Agents are fairly good even at languages designed to trick them. I built one[1] and it does make for a good benchmark[2] to see which LLMs are actually good. I think that a language which is largely similar to others will be a piece of cake for most and any advantage that an existing language will have will be minor enough to not matter.
Btw if folks have ideas of how to make Killswitch even harder for LLMs I’d appreciate them.
Alternatively, this might accurately identify Chinese models so you can avoid them if that's your goal
Personally I don't have much interest in using models that blatantly censor historical facts.
Tiananmen Square is an obvious one but I would be concerned that any model that censors that would also have other hidden censorship or more subtle biases I wouldn't catch
spankalee · · focus · HN ↗
---
- Correct by construction: the language makes invalid states or programs hard or impossible to express.
- Statically established: types, proofs, and static analysis establish properties before execution.
- Runtime-enforced: memory management, isolation, capability boundaries, and other runtime enforced properties.
- Empirically validated: program validation through tests, property-based testing, and fuzzing.
---
Along with being familiar, so it's easy to generate, is a huge part of why I'm building Zena: <a href="https://zena-lang.dev/" rel="nofollow">https://zena-lang.dev/
I don't have the AI-first rationale put into the public docs well just yet, but I mention some of it here: <a href="https://zena-lang.dev/guide/why-zena/#familiar-to-humans-and-to-agents" rel="nofollow">https://zena-lang.dev/guide/why-zena/#familiar-to-humans-and...
along with a doc in the repo on this topic: <a href="https://github.com/elematic/zena/blob/main/docs/design/ai-first-language.md" rel="nofollow">https://github.com/elematic/zena/blob/main/docs/design/ai-fi...
In short, the more deterministic, automated, checks the better. AI can deal with a pedantic language. I intend to add statically verified structured concurrency, units of measure, contracts, and eventually more and more formal methods into the language so it can be a familiar TYpeScript-like base with as many static guarantees as we can fit in.
I also think that fine-grained isolation, which Zena gets via Web Assembly, is critical for limiting the capabilities of generated code and the blast radius of bugs, vulnerabilities, and non-aligned behavior.
I do have an optimistic hope that a language also optimized for humans, readability and simple semantics especially, has value in the future, even when most code is generated. We'll see about that.
demibabs · · focus · HN ↗
Agents will naturally be bad at it due to a lack of examples.
ryuuseijin · · focus · HN ↗
spankalee · · focus · HN ↗
"Dart-style constructors, Swift-style pattern matching and Strings, Trio-style async cancellation, Scala-style sealed classes"
whattheheckheck · · focus · HN ↗
The future is cooked
spankalee · · focus · HN ↗
The one thing I do see sometimes is that agents sometimes don't take advantage of added features, but that's partially because the Zena code base doesn't use them as much yet. I'm working on skills and linter-based suggestions to use better patterns.
genxy · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
abletonlive · · focus · HN ↗
Can we please as a community stop parroting these false premises as a basis of every argument against doing anything new? It's plainly obvious to anybody that uses LLMs on a regular basis that it's not true.
demibabs · · focus · HN ↗
hnedeotes · · focus · HN ↗
What can happen is sometimes patterns that are unique to certain languages aren't surfaced and so can't be "learned" but that is a different case, since you can add examples - or patterns that go beyond syntax (threaded code, process isolation, interaction between different modes of resolution, etc) where you need the agent to be able to understand and plan higher-level logic.
I wrote CSSex as a css pre-processor (similar but not the same as SASS/SCSS) and my "boss" at the time fed the existing cssex files to GPT (+2 years ago) and it was able to write CSSex just as fine as if it was writing something that was present in its training data set. It never introduced syntax bugs, the bugs it introduced were all relative to complex cascading styles rules in an existing large (for a definition of large in webapps) codebase, rules interactions (CSS), browser quirks, and complex logic to do what we "humans" wanted (or him in this case) and so on.
ModernMech · · focus · HN ↗
theqoo · · focus · HN ↗
They should find the limitation of the natural language somehow, and use and distribute babo language will help find how it will works(or not). LLM model eveolves yet, so there is no such a approach, but the speed of evolution is decreased lately. So now is good time to dig into identify the limitation and boundary of the LLM.
[deleted] · · focus · HN ↗
[deleted]
ashton314 · · focus · HN ↗
Even though this programming language is absolutely nowhere in any large language model’s training set, they have so far done extremely well at extrapolating from the small set of examples I’ve given it when I need an agent to generate some tests or whatever for me.
dom96 · · focus · HN ↗
Btw if folks have ideas of how to make Killswitch even harder for LLMs I’d appreciate them.
1 - <a href="https://killswitch-lang.org" rel="nofollow">https://killswitch-lang.org
2 - <a href="https://bench.killswitch-lang.org" rel="nofollow">https://bench.killswitch-lang.org
the_duke · · focus · HN ↗
Good one. Though this might unfairly bias against Chinese models?
bluefirebrand · · focus · HN ↗
Personally I don't have much interest in using models that blatantly censor historical facts.
Tiananmen Square is an obvious one but I would be concerned that any model that censors that would also have other hidden censorship or more subtle biases I wouldn't catch
dom96 · · focus · HN ↗
1 - <a href="https://github.com/dom96/KillSwitch/blob/main/SPEC.md#abrupt-return-from-function" rel="nofollow">https://github.com/dom96/KillSwitch/blob/main/SPEC.md#abrupt...