13,000 tests. That tells you everything you need to know. It doesn't matter how experienced you say you are, if AI is doing all the work including writing and running tests because I know no human is actually reviewing these. I've recently found tests written by Fable that don't actually do anything in a very large codebase. My point is not to advocate for doing any of this manually, but to highlight one of the problems brought about by pumping out so much code.
I hate to sound cynical, but I don't know if bragging about being AI-native is something any of us should be doing. We're really just pulling a lever on a slot machine and crossing our fingers.
You need to ensure the whole system is optimised for LLM. Getting human written code and trying to run like this will be a failure since you'll be missing essential guidelines particularly on the domain-specific design fundamentals that are acquired in the beginning in a human feedback loop before the automation really kicks in
> I've recently found tests written by Fable that don't actually do anything in a very large codebase.
tbf, I've seen these kind of tests as well at $dayjob, written by humans before gpt was a thing. That doesn't invalid your point. But it's still funny that we expect the machine to be perfect, before it can do something for us.
>But it's still funny that we expect the machine to be perfect, before it can do something for us.
We expect machines to be perfect at the things they should be perfect at, and when they aren't perfect we consider that a malfunction and try to make them closer to perfect. A calculator that can't do math correctly every time is a bad calculator. And until LLMs came along computers were just complicated calculators.
Now, however, we build the calculator with an LLM and just accept that it sometimes gets confuzzled and makes numbers up because "humans are no better at math."
It's not funny, it's the way things used to work and it's objectively worse that things can't work that way anymore.
dkobia · · focus · HN ↗
I hate to sound cynical, but I don't know if bragging about being AI-native is something any of us should be doing. We're really just pulling a lever on a slot machine and crossing our fingers.
fagnerbrack · · focus · HN ↗
_ink_ · · focus · HN ↗
tbf, I've seen these kind of tests as well at $dayjob, written by humans before gpt was a thing. That doesn't invalid your point. But it's still funny that we expect the machine to be perfect, before it can do something for us.
krapp · · focus · HN ↗
We expect machines to be perfect at the things they should be perfect at, and when they aren't perfect we consider that a malfunction and try to make them closer to perfect. A calculator that can't do math correctly every time is a bad calculator. And until LLMs came along computers were just complicated calculators.
Now, however, we build the calculator with an LLM and just accept that it sometimes gets confuzzled and makes numbers up because "humans are no better at math."
It's not funny, it's the way things used to work and it's objectively worse that things can't work that way anymore.
nijave · · focus · HN ↗
Me neither but we now have a likert scale on our performance review for it...