Maybe I'm not that deep into programming, but I don't understand the hype about Zig?
I programmed in rust a bit and can't say I'm an expert, but in my view rust mostly-solved the memory management problem at compile time and without a GC, and it works very well. The biggest con and cost I've always seen repeated so far is that "it's slow to compile", and I get that, if you're past 250 crates the final --release link tends to become noticeable, but there were improvements to incremental compilation.
On the other hand - looking at the syntax from this post - Zig feels a blend of javascript, python and golang syntax that still requires memory management. So a nicer-written C that inherits all the issues from C? From the post: no functional programming, data mutation, memory leak, double-free, memory corruption.
Personally I'd rather trade a couple minutes of final link every time when this is the other option.
As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating, and I also think it addresses a long-standing problem in low-level programming. I'll get to the problem later, but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. It's never been done before (other languages have partial evaluation mechanisms that are almost as general, but they're offered in addition to, not as a replacement of, other features).
> rust mostly-solved the memory management problem at compile time and without a GC
"Mostly" does a lot of work here because 1., if you look at the implementation of very efficient, possibly specialised data structures - the very thing you reach for a low-level language for - they typically require unsafe, and 2., it still suffers from the problem C++ has had for decades, which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower (huge runtimes like TCMalloc help, but not enough, because they can't move pointers). This problem, of programs that start out fast, but after five or ten years of evolution need to spend a lot of effort to remain fast, is one of the things moving collectors were designed to solve, but they require moving pointers, which doesn't work in low-level languages that are not meant to have an FFI layer between them and the hardware.
To compete with the performance of moving GCs, which allocate through bumping a pointer, like on the stack, and free memory in bulk, low-level languages need to rely on arenas (which work based on a similar principle), and Zig is the first language that makes arenas almost user-friendly and hopefully sufficiently composable to withstand program evolution. Of course, time will tell how well this works in practice.
> the very thing you reach for a low-level language for - they typically require unsafe
There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks.
This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way.
There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need!
> which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower
What are you talking about? I've never encountered this and I've been using Rust for 10 years.
> There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks.
In general "unsafe" does not compose.
"if you keep up the safety invariants within an unsafe region"
Here's an article about the research on it which lays out the properties in simple terms: <a href="https://smallcultfollowing.com/babysteps/blog/2016/10/02/observational-equivalence-and-unsafe-code/" rel="nofollow">https://smallcultfollowing.com/babysteps/blog/2016/10/02/obs...
I'm curious why you think that statement is doing heavy lifting. It's much easier to write and verify that a few lines of code are correct than it is to write and verify that an entire program is correct. But that's the norm in C and Zig, and historically people haven't been very good at it. That's why we try to do it as little as possible.
You must reason about the invariants in unsafe code on a global level. In particular, you could have unsafe code in crate A, whose data are then used by crate B. It could be fine. But then crate B changes its implementation which now violates the invariant expectations of crate A.
> In particular, you could have unsafe code in crate A, whose data are then used by crate B.
Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A.
Also for what it's worth I get the impression you and treyd might be talking about slightly different things when talking about whether unsafe code composes. I believe treyd is referring to the RustBelt series of papers [0, 1], for which the statement "unsafe code composes" means (at a high level) that adding a module with a memory-safe API to a memory-safe system will result in a memory-safe system as long as the implementation upholds the safe semantics. Yes, the last bit can be a rather significant caveat, as you said.
What you're talking about seems more along the lines of needing to look beyond the boundaries of unsafe blocks to prove that the unsafe block upholds its invariants, which is also true. I think you only need to check within whatever safe encapsulation boundary is relevant, though, rather than globally.
> Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A.
This is a specifically crafted bad idea, but you could have module A use unsafe to craft a Vec<u8> that is safe to use to read or write, but not to grow or shrink. You declare an invariant that the receiver shalt not grow or shrink the Vec.
If B only reads and write, you're good. But if a future B breaks the invariant, bad things happen. As I said, specifically a bad idea; there's a much better type to use if the thing can't grow or shrink...
No real world example, because I don't think we've run into memory safety issues with unsafe in the Rust code base I work in... but we only use unsafe where it's required (syscalls and other FFI).
Fair. I suppose that even in such a scenario you shouldn't need truly global analysis to prove safety - in principle an analysis of A should reveal the soundness precondition on a safe API - though that's probably easier said than done.
plqbfbv · · focus · HN ↗
I programmed in rust a bit and can't say I'm an expert, but in my view rust mostly-solved the memory management problem at compile time and without a GC, and it works very well. The biggest con and cost I've always seen repeated so far is that "it's slow to compile", and I get that, if you're past 250 crates the final --release link tends to become noticeable, but there were improvements to incremental compilation.
On the other hand - looking at the syntax from this post - Zig feels a blend of javascript, python and golang syntax that still requires memory management. So a nicer-written C that inherits all the issues from C? From the post: no functional programming, data mutation, memory leak, double-free, memory corruption.
Personally I'd rather trade a couple minutes of final link every time when this is the other option.
pron · · focus · HN ↗
As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating, and I also think it addresses a long-standing problem in low-level programming. I'll get to the problem later, but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. It's never been done before (other languages have partial evaluation mechanisms that are almost as general, but they're offered in addition to, not as a replacement of, other features).
> rust mostly-solved the memory management problem at compile time and without a GC
"Mostly" does a lot of work here because 1., if you look at the implementation of very efficient, possibly specialised data structures - the very thing you reach for a low-level language for - they typically require unsafe, and 2., it still suffers from the problem C++ has had for decades, which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower (huge runtimes like TCMalloc help, but not enough, because they can't move pointers). This problem, of programs that start out fast, but after five or ten years of evolution need to spend a lot of effort to remain fast, is one of the things moving collectors were designed to solve, but they require moving pointers, which doesn't work in low-level languages that are not meant to have an FFI layer between them and the hardware.
To compete with the performance of moving GCs, which allocate through bumping a pointer, like on the stack, and free memory in bulk, low-level languages need to rely on arenas (which work based on a similar principle), and Zig is the first language that makes arenas almost user-friendly and hopefully sufficiently composable to withstand program evolution. Of course, time will tell how well this works in practice.
treyd · · focus · HN ↗
There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks.
This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way.
There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need!
> which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower
What are you talking about? I've never encountered this and I've been using Rust for 10 years.
dnautics · · focus · HN ↗
In general "unsafe" does not compose.
"if you keep up the safety invariants within an unsafe region"
This condition is doing a lot of heavy lifting.
treyd · · focus · HN ↗
I'm curious why you think that statement is doing heavy lifting. It's much easier to write and verify that a few lines of code are correct than it is to write and verify that an entire program is correct. But that's the norm in C and Zig, and historically people haven't been very good at it. That's why we try to do it as little as possible.
dnautics · · focus · HN ↗
You must reason about the invariants in unsafe code on a global level. In particular, you could have unsafe code in crate A, whose data are then used by crate B. It could be fine. But then crate B changes its implementation which now violates the invariant expectations of crate A.
aw1621107 · · focus · HN ↗
Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A.
Also for what it's worth I get the impression you and treyd might be talking about slightly different things when talking about whether unsafe code composes. I believe treyd is referring to the RustBelt series of papers [0, 1], for which the statement "unsafe code composes" means (at a high level) that adding a module with a memory-safe API to a memory-safe system will result in a memory-safe system as long as the implementation upholds the safe semantics. Yes, the last bit can be a rather significant caveat, as you said.
What you're talking about seems more along the lines of needing to look beyond the boundaries of unsafe blocks to prove that the unsafe block upholds its invariants, which is also true. I think you only need to check within whatever safe encapsulation boundary is relevant, though, rather than globally.
[0]: <a href="https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf" rel="nofollow">https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf
[1]: <a href="https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf" rel="nofollow">https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf
toast0 · · focus · HN ↗
This is a specifically crafted bad idea, but you could have module A use unsafe to craft a Vec<u8> that is safe to use to read or write, but not to grow or shrink. You declare an invariant that the receiver shalt not grow or shrink the Vec.
If B only reads and write, you're good. But if a future B breaks the invariant, bad things happen. As I said, specifically a bad idea; there's a much better type to use if the thing can't grow or shrink...
No real world example, because I don't think we've run into memory safety issues with unsafe in the Rust code base I work in... but we only use unsafe where it's required (syscalls and other FFI).
aw1621107 · · focus · HN ↗
toast0 · · focus · HN ↗
aw1621107 · · focus · HN ↗