It's interesting how there's an increasing split of software engineers into those who're very happy with AI and leaning more into it, and those who're increasingly skeptical or dejected.
I don't know how this will play out, but I haven't seen anything like this before.
For my social circle, the distinction falls almost perfectly along the line of people who see software engineering as a craft / passion, versus those who see it as something that needs to be done to get paid.
The first group is generally disillusioned, for two reasons. First, it's an attack on their hobby. Second, they sort of can't let go: they want to own the product, they want to understand if the code is good or bad, and they spend a lot of time cleaning up after LLMs. Which, frankly, is not fun. You're no longer doing anything clever, you're just a janitor.
The second group is over the moon about it because they feel it gives them the elite powers previously only reserved for greybeards.
I don't think this is unique to coding. It's the same thing with writing. If you like writing, you probably hate LLMs. If you see writing as a boring chore that needs to be done, you are delighted to have a button you can press. Haha, take that all the influential bloggers, tech writers, etc. I can now do your job at 10x the speed.
The one thing that poisons the entire online discourse regarding LLMs to me is just simply not knowing who you're dealing with and what their problems are. This, like all things, is complex and nuanced.
For all my efforts to get LLMs to work for me, they've failed. I figure it is because the tech space I work in (Industrial Automation) simply doesn't have the training data that someone building webapps does. The few times I've put LLM to the test its failed horribly, and since my job has me existing in safety critical scenarios I don't humor what AI gives me at all.
So when I bitch about LLMs and their useless nature, its because of how I've experienced them in my craft. They get 95% of the information right, but that 5% it gets wrong can have very serious repercussions.
But when I'm talking with individuals online who see them as revolutionary? I don't know if their job is editing templates or formatting data or any other host of problems that AI can be revolutionary for. I don't know if they're full of themselves, having drank the AI koolaid. I don't know if they're just really deep into the LLM world and figured out things I have yet to learn (or am unwilling to pay for). But from a HN or Reddit discussion, they're all one in the same and without trying to figure out who they are, their opinions all carry the same weight.
I'm surprised to hear that. LLMs make it practical to prove safety-critical C code correct without the cost that has traditionally made formal verification prohibitive. If you haven't explored that area, it's worth a look.
My experience with C and C++ has also been excellent. I work on very large, high-performance C++ systems used to run airlines and airports, and LLMs have dramatically improved both the quality and the performance of our code.
That said, LLMs are tools, and like any tool they take skill to use well. It took me months of experimenting and learning from others to get where I am now.
I made mention on a reply to the other commentator but I feel the issue is due to lack of training data in the PLC/SCADA space. If an AI engine with the appropriate amount of training data was available to me I'm sure it could do the work just fine, but at the current state we have partial knowledge which is too similar to other languages. The result is agents which quickly hallucinate capabilities of the SCADA or PLC hardware and write out code that fundamentally won't work. The result of this is that I distrust AI to get it right enough that I won't kill an operator or blow up a production line. The automation world is too closed source and until the push for open standards that have begun to gain traction really takes off we're sort of stuck.
But then at the same time the code of an industrial system typically isn't the bottleneck in development. Selecting the hardware and waiting for the physical installation is a far bigger bottleneck on a project than what I do. As a result, in the end it isn't the "Correctness" of code in the sense of it is doing what it was instructed to do properly, but the correctness of matching software to hardware and that's where I've seen several failures in AI. If I tell it I have a Schneider Altavar VFD and I get code to interface with an Allen Bradley PowerFlex that's one issue. But if I get code to interface with a Altavar that expects specific parameter adjustments on the drive that haven't been set that's a separate issue and more typical for the faults I've seen within AI. If 99% of drives are configured to operate pumps and that's what AI learns for training, the moment I need it for a mill, aeration fan, or conveyor belt (which have distinct physical requirements) then we have the issues I've been seeing and it takes a trained eye to spot those deficiencies in my experience.
Thanks for the detailed reply. It’s always interesting to hear about real experiences from a different software domain.
So it sounds like the main problem is the under-specified hardware. I can definitely see how that would limit the usefulness of LLMs.
I used to work on consoles, including the N64, which are fairly low-level compared with a typical Windows, Linux, or Mac application. But you’re obviously working much closer to the hardware, which is a completely different world altogether.
bucket2015 · · focus · HN ↗
I don't know how this will play out, but I haven't seen anything like this before.
in_absentia · · focus · HN ↗
The first group is generally disillusioned, for two reasons. First, it's an attack on their hobby. Second, they sort of can't let go: they want to own the product, they want to understand if the code is good or bad, and they spend a lot of time cleaning up after LLMs. Which, frankly, is not fun. You're no longer doing anything clever, you're just a janitor.
The second group is over the moon about it because they feel it gives them the elite powers previously only reserved for greybeards.
I don't think this is unique to coding. It's the same thing with writing. If you like writing, you probably hate LLMs. If you see writing as a boring chore that needs to be done, you are delighted to have a button you can press. Haha, take that all the influential bloggers, tech writers, etc. I can now do your job at 10x the speed.
TheCapn · · focus · HN ↗
For all my efforts to get LLMs to work for me, they've failed. I figure it is because the tech space I work in (Industrial Automation) simply doesn't have the training data that someone building webapps does. The few times I've put LLM to the test its failed horribly, and since my job has me existing in safety critical scenarios I don't humor what AI gives me at all.
So when I bitch about LLMs and their useless nature, its because of how I've experienced them in my craft. They get 95% of the information right, but that 5% it gets wrong can have very serious repercussions.
But when I'm talking with individuals online who see them as revolutionary? I don't know if their job is editing templates or formatting data or any other host of problems that AI can be revolutionary for. I don't know if they're full of themselves, having drank the AI koolaid. I don't know if they're just really deep into the LLM world and figured out things I have yet to learn (or am unwilling to pay for). But from a HN or Reddit discussion, they're all one in the same and without trying to figure out who they are, their opinions all carry the same weight.
deterministic · · focus · HN ↗
My experience with C and C++ has also been excellent. I work on very large, high-performance C++ systems used to run airlines and airports, and LLMs have dramatically improved both the quality and the performance of our code.
That said, LLMs are tools, and like any tool they take skill to use well. It took me months of experimenting and learning from others to get where I am now.
TheCapn · · focus · HN ↗
But then at the same time the code of an industrial system typically isn't the bottleneck in development. Selecting the hardware and waiting for the physical installation is a far bigger bottleneck on a project than what I do. As a result, in the end it isn't the "Correctness" of code in the sense of it is doing what it was instructed to do properly, but the correctness of matching software to hardware and that's where I've seen several failures in AI. If I tell it I have a Schneider Altavar VFD and I get code to interface with an Allen Bradley PowerFlex that's one issue. But if I get code to interface with a Altavar that expects specific parameter adjustments on the drive that haven't been set that's a separate issue and more typical for the faults I've seen within AI. If 99% of drives are configured to operate pumps and that's what AI learns for training, the moment I need it for a mill, aeration fan, or conveyor belt (which have distinct physical requirements) then we have the issues I've been seeing and it takes a trained eye to spot those deficiencies in my experience.
deterministic · · focus · HN ↗
So it sounds like the main problem is the under-specified hardware. I can definitely see how that would limit the usefulness of LLMs.
I used to work on consoles, including the N64, which are fairly low-level compared with a typical Windows, Linux, or Mac application. But you’re obviously working much closer to the hardware, which is a completely different world altogether.