Maybe I'm not that deep into programming, but I don't understand the hype about Zig?
I programmed in rust a bit and can't say I'm an expert, but in my view rust mostly-solved the memory management problem at compile time and without a GC, and it works very well. The biggest con and cost I've always seen repeated so far is that "it's slow to compile", and I get that, if you're past 250 crates the final --release link tends to become noticeable, but there were improvements to incremental compilation.
On the other hand - looking at the syntax from this post - Zig feels a blend of javascript, python and golang syntax that still requires memory management. So a nicer-written C that inherits all the issues from C? From the post: no functional programming, data mutation, memory leak, double-free, memory corruption.
Personally I'd rather trade a couple minutes of final link every time when this is the other option.
As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating, and I also think it addresses a long-standing problem in low-level programming. I'll get to the problem later, but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. It's never been done before (other languages have partial evaluation mechanisms that are almost as general, but they're offered in addition to, not as a replacement of, other features).
> rust mostly-solved the memory management problem at compile time and without a GC
"Mostly" does a lot of work here because 1., if you look at the implementation of very efficient, possibly specialised data structures - the very thing you reach for a low-level language for - they typically require unsafe, and 2., it still suffers from the problem C++ has had for decades, which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower (huge runtimes like TCMalloc help, but not enough, because they can't move pointers). This problem, of programs that start out fast, but after five or ten years of evolution need to spend a lot of effort to remain fast, is one of the things moving collectors were designed to solve, but they require moving pointers, which doesn't work in low-level languages that are not meant to have an FFI layer between them and the hardware.
To compete with the performance of moving GCs, which allocate through bumping a pointer, like on the stack, and free memory in bulk, low-level languages need to rely on arenas (which work based on a similar principle), and Zig is the first language that makes arenas almost user-friendly and hopefully sufficiently composable to withstand program evolution. Of course, time will tell how well this works in practice.
> the very thing you reach for a low-level language for - they typically require unsafe
There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks.
This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way.
There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need!
> which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower
What are you talking about? I've never encountered this and I've been using Rust for 10 years.
That's always been true in all the safe languages with unsafe escape hatches, except here these "primitives" are the main reason to reach for a low-level language in the first place - because they presumably require the control that low-level languages offer. Combining them in the same language might appeal to some and not to others who think that the high-level, safe parts are unnecessarily complicated because it needs to integrate with the low-level parts, and the low-level parts are unnecessarily complicated because they need to integrate with the safe parts. Anyway, some like this and some don't, but my point is that it's not "mostly solved".
> What are you talking about? I've never encountered this and I've been using Rust for 10 years.
Okay, but I've been doing low-level programming professionally for 25 years, and have encountered this over and over in large programs (over 500KLOC) as they evolve.
That's just not an accurate description of how you write Rust in practice. There's no separate "high level" and "low level" parts/forms of the language any more than the software development process already is all about building abstractions. You should be doing this in Zig, too.
It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.
They're not separate forms but they are separate modes, and it is precisely because the language tries to fit both these modes into the same language that both suffer. I fully understand the goal of trying to unify these modes into the same language (C++ does the same thing), but there have always been very experienced people who like this approach and those who dislike it, hence it's not "solved". Something is solved when there's a broad consensus it's solved, and there isn't one here.
I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.
I don't know exactly how specific you want to be, but sure, because we've come across this countless times in C++, which suffers from the exact same problem.
Suppose you're writing a program that's mostly high-level, say some kind of concurrent server, and it's large-ish, say around 1MLOC (most C++ programs I've worked on were significantly larger). Because the language is also a low-level language, it has low-level constraints, so:
1. It needs to use an AOT compiler, and consequently to get good performance you need to use less general mechanisms, such as direct (as opposed to dynamic) dispatch and even manual monorphisation (with generics/templates). These are viral, so they have to be carefully chosen (you can't monomorphise everything or you'll get machine code explosion). Five years later you need to make a big change that requires more generality, and then you either have to reconsider all of your manual optimisations, which is expensive, or go for more general constructs (dynamic dispatch) and the program gets slower.
2. It needs to use machine pointers (i.e. you can't enjoy a moving GC), and so you try to use the stack as much as possible (which you can't really do for anything dynamic), or suffer the high cost of malloc/free on individual objects. As the program evolves, you need to make things more general, and objects that could live on the stack now need to go on the heap, and objects that lived on the heap now may need to be shared among threads, in which case you often add the additional cost of refcounting GC. Of course, you want to use arenas in many cases, but they're very, very hard to use in C++ and Rust.
You'd be better off - performance-wise and maintenance-wise - with a good optimising JIT and a moving GC. This was exactly a problem with many C++ programs that didn't really need a lot of direct hardware interaction - everything worked great for a few years, and then the evolution and maintenance costs became really high (or the programs became slow).
Now suppose you're writing something low-level, i.e. you really need to interact with the hardware and/or OS directly a lot, and want to control everything - where everything is in memory, exactly when it's initialised, exactly when it's freed, exactly which operations are executed and when. But now you have a language that's also high-level, so it has a lot of implicitness that hides from you the things you want to see (and in Rust's case, you lose the safety). Best case scenario, you rely on disciplne and avoid implicit features, but then you also need to avoid much of the standard library.
Anyway, combining high and low level in the same language was C++'s dream: one language for everything. Of course, for a while we didn't know about the maintenance problems, as those appear only years down the line, but more importantly, there weren't really high-performance high-level languages back then. These days, with lessons learnt and with more options, I prefer a language that focuses on being high-level for high-level stuff, and a language that focuses on low-level for low-level stuff. If you really need both kinds, use two languages.
> I've been doing low-level programming professionally for 25 years
You haven't been doing any Rust though. You seem to think you can extrapolate your C++ experience to Rust. That's preposterous. The actual Rust programmers can't recognize this theoretical problem in their Rust programs.
Could that be because the language is fairly young? You don't see the "20-year-old legacy system" in Rust because it doesn't exist yet ;)
And if you look at other comments in this thread, many engineers have this mentality of "just use a crate, it's probably optimised already". They might not have performance problems immediately or obviously but it's more like ten thousand papercuts - a few allocations here and there, a few extra copies here and there and you've got a way slower program than it should have been.
Well, the problems don't start after 20 years but after 5 or so (depending on the size of the codebase and the rate of the application's evolution), and the reason there aren't many large and oldish Rust codebases isn't because the language is too young for that (work on it began twenty years ago, and it's been stable for over a decade); that's middle-aged for a programming language. When C++ was of a similar age, there were thousands of >1MLOC programs written in it. One reason is obviously because when C++ was of the same age, there weren't as many suitable high-level alternatives, and people just don't pick a low-level language for most large applications anymore. But most Rust fans at least on social media, have not actually had much experience with it or with low-level programming in general; I'm guessing most haven't worked on Rust projects with more than 10 full-time people on them (this isn't normal in the industry, BTW, as a lot of software lives in large programs). And again, there are people who can certainly live with these issues, but they are real, and many certainly find them troubling.
It's not theoretical, it's one of the main reasons many large applications abandoned C++, and there's absolutely no reason for it to not exist in Rust. All low-level languages suffer from expensive evolution for fundamental reasons - the reliance on an AOT compiler and the lack of movable pointers impose serious performance tradeoffs in large programs. Optimising JITs and moving GCs were invented, in large part, to address this very real problem, familiar to many low-level programmers who have maintained large codebases for a long time. It's also why large runtimes like TCMalloc were invented to assist as much as they can.
Most actual Rust programmers haven't maintained a large Rust program for a long time. Now, don't get me wrong - there are many C++ programmers who are fine with it, but many who aren't. What I find annoying is people without much experience in Rust assuming that everyone or almost everyone should like it, even though that's never been true for any language. I'm not saying Rust is bad by any means; in fact, I think it's better than C++ in a few ways. I'm explaining why I don't like it.
plqbfbv · · focus · HN ↗
I programmed in rust a bit and can't say I'm an expert, but in my view rust mostly-solved the memory management problem at compile time and without a GC, and it works very well. The biggest con and cost I've always seen repeated so far is that "it's slow to compile", and I get that, if you're past 250 crates the final --release link tends to become noticeable, but there were improvements to incremental compilation.
On the other hand - looking at the syntax from this post - Zig feels a blend of javascript, python and golang syntax that still requires memory management. So a nicer-written C that inherits all the issues from C? From the post: no functional programming, data mutation, memory leak, double-free, memory corruption.
Personally I'd rather trade a couple minutes of final link every time when this is the other option.
pron · · focus · HN ↗
As a long-time low-level programmer, and as someone working on a popular mainstream language, I find Zig fascinating, and I also think it addresses a long-standing problem in low-level programming. I'll get to the problem later, but the fascinating part is its use of partial evaluation (comptime) as a single coherent mechanism that replaces a myriad of other partial-evaluation mechanisms (macros, templates/generics, constexprs). That one mechanism is the core of the language, like macros are in lisps, and that design - whether you like it or not - is revolutionary. It's never been done before (other languages have partial evaluation mechanisms that are almost as general, but they're offered in addition to, not as a replacement of, other features).
> rust mostly-solved the memory management problem at compile time and without a GC
"Mostly" does a lot of work here because 1., if you look at the implementation of very efficient, possibly specialised data structures - the very thing you reach for a low-level language for - they typically require unsafe, and 2., it still suffers from the problem C++ has had for decades, which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower (huge runtimes like TCMalloc help, but not enough, because they can't move pointers). This problem, of programs that start out fast, but after five or ten years of evolution need to spend a lot of effort to remain fast, is one of the things moving collectors were designed to solve, but they require moving pointers, which doesn't work in low-level languages that are not meant to have an FFI layer between them and the hardware.
To compete with the performance of moving GCs, which allocate through bumping a pointer, like on the stack, and free memory in bulk, low-level languages need to rely on arenas (which work based on a similar principle), and Zig is the first language that makes arenas almost user-friendly and hopefully sufficiently composable to withstand program evolution. Of course, time will tell how well this works in practice.
treyd · · focus · HN ↗
There's a formal proof asserting that if you keep up the safety invariants within an unsafe region then that will not infect other code, even in the presence of arbitrary other correctly-written unsafe blocks.
This means you can build abstractions on top of these low-level primitives to keep it contained, so consumer code never has to even think about or know there's unsafe blocks in it. The type system lets you build very powerful abstractions so these go a long way.
There's a lot of woo-woo scare quoting around how much you actually have to use unsafe code in Rust. It's fairly uncommon to actually have to reach for them in practice. Most of my usage ends up being things like converting a &[u8] to a &str when I know it's already valid UTF-8 so I want to skip the linear-time validity check. Very rarely do I have to build data structures with complicated pointer juggling, because there's often a library that already does what I need!
> which is that over time, as program changes and evolves over years, things tend to drift toward the more general mechanisms that rely on malloc/free on an individual objects, and the program gets slower and slower
What are you talking about? I've never encountered this and I've been using Rust for 10 years.
pron · · focus · HN ↗
> What are you talking about? I've never encountered this and I've been using Rust for 10 years.
Okay, but I've been doing low-level programming professionally for 25 years, and have encountered this over and over in large programs (over 500KLOC) as they evolve.
treyd · · focus · HN ↗
It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.
pron · · focus · HN ↗
I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.
treyd · · focus · HN ↗
pron · · focus · HN ↗
Suppose you're writing a program that's mostly high-level, say some kind of concurrent server, and it's large-ish, say around 1MLOC (most C++ programs I've worked on were significantly larger). Because the language is also a low-level language, it has low-level constraints, so:
1. It needs to use an AOT compiler, and consequently to get good performance you need to use less general mechanisms, such as direct (as opposed to dynamic) dispatch and even manual monorphisation (with generics/templates). These are viral, so they have to be carefully chosen (you can't monomorphise everything or you'll get machine code explosion). Five years later you need to make a big change that requires more generality, and then you either have to reconsider all of your manual optimisations, which is expensive, or go for more general constructs (dynamic dispatch) and the program gets slower.
2. It needs to use machine pointers (i.e. you can't enjoy a moving GC), and so you try to use the stack as much as possible (which you can't really do for anything dynamic), or suffer the high cost of malloc/free on individual objects. As the program evolves, you need to make things more general, and objects that could live on the stack now need to go on the heap, and objects that lived on the heap now may need to be shared among threads, in which case you often add the additional cost of refcounting GC. Of course, you want to use arenas in many cases, but they're very, very hard to use in C++ and Rust.
You'd be better off - performance-wise and maintenance-wise - with a good optimising JIT and a moving GC. This was exactly a problem with many C++ programs that didn't really need a lot of direct hardware interaction - everything worked great for a few years, and then the evolution and maintenance costs became really high (or the programs became slow).
Now suppose you're writing something low-level, i.e. you really need to interact with the hardware and/or OS directly a lot, and want to control everything - where everything is in memory, exactly when it's initialised, exactly when it's freed, exactly which operations are executed and when. But now you have a language that's also high-level, so it has a lot of implicitness that hides from you the things you want to see (and in Rust's case, you lose the safety). Best case scenario, you rely on disciplne and avoid implicit features, but then you also need to avoid much of the standard library.
Anyway, combining high and low level in the same language was C++'s dream: one language for everything. Of course, for a while we didn't know about the maintenance problems, as those appear only years down the line, but more importantly, there weren't really high-performance high-level languages back then. These days, with lessons learnt and with more options, I prefer a language that focuses on being high-level for high-level stuff, and a language that focuses on low-level for low-level stuff. If you really need both kinds, use two languages.
brabel · · focus · HN ↗
You haven't been doing any Rust though. You seem to think you can extrapolate your C++ experience to Rust. That's preposterous. The actual Rust programmers can't recognize this theoretical problem in their Rust programs.
Pannoniae · · focus · HN ↗
And if you look at other comments in this thread, many engineers have this mentality of "just use a crate, it's probably optimised already". They might not have performance problems immediately or obviously but it's more like ten thousand papercuts - a few allocations here and there, a few extra copies here and there and you've got a way slower program than it should have been.
pron · · focus · HN ↗
pron · · focus · HN ↗
Most actual Rust programmers haven't maintained a large Rust program for a long time. Now, don't get me wrong - there are many C++ programmers who are fine with it, but many who aren't. What I find annoying is people without much experience in Rust assuming that everyone or almost everyone should like it, even though that's never been true for any language. I'm not saying Rust is bad by any means; in fact, I think it's better than C++ in a few ways. I'm explaining why I don't like it.