Thread

1. zero_shift · 1790612953 · focus · HN ↗

   I was reflecting on this on Saturday in an unformed way, trying to trace the lineage of a decision made at work.

   The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.

   The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.

   The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.

   This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.

   We were not building the feature because we wanted it. We were building it because we thought other people expected it.

   Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.

   Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.

   Where does the market's "action" come from? What is the driver?

   Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.

   This left me lost.

   Nobody in this ecosystem, I thought, is actually in control here.

   Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.

   So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.

   It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.

   Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

   1. danielmarkbruce · 1790614793 · focus · HN ↗

      In many (most perhaps, but hard to know personally) companies, most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions. They also aren't likely to be around (or held responsible) for decisions as they play out over years. So what has historically happened is that people use proxies for good decisions and understanding - which are clever sounding documents and presentations.

      This is a long winded way of saying "people made up clever/sensible sounding stuff". Now it's easier to do it with AI so the problem is worse. However, I'm not sure what you were looking for was ever really there - the "inscrutable machines" and "inscrutable incentives" were always quite inscrutable.

      1. ryaniscool · 1790617042 · focus · HN ↗

         Yes, people can be bad at their jobs. Yes, they can work in an industry and not know anything about the market but I refuse to believe that most people in most companies are just stumbling around faking it. I've never worked in a place like that. People like this exist but there are only a handful of people like this at every job.

         The hyperbole feels like it's been contrived to fit into the classic AI counterpoint: But humans do this too!

         1. TheOtherHobbes · 1790620509 · focus · HN ↗

            It's not counterpoint, it's a fact. Can you say with your hand on your heart that the typical user experience was acceptable before AI arrived?

            The reply was always "It's hard, and there are always going to be bugs."

            But it's not just about bugs. It's about systems that are hard to use, poorly designed, and opaque. And sometimes the opacity is deliberate. There are dark patterns, outright lies about what happens to data, and more or less obvious grifts.

            Was any of this truly great before AI arrived?

            Is it an accident that you can't kill an MS365 subscription if you have more than X amount of GB on OneDrive, but the actual usage includes all your spam and email attachments, and it doesn't show up unless you know where to find it, and the location is very non-obvious, and so is the cleanup and deletion process?

            Or that the eBay fee structure for international sales is utterly incomprehensible without automated help?

            Or that if you select Subscribe and Save on Amazon your cancellation date is something like two weeks before the next delivery?

            Or that if you sign up to Discord you need a mobile and an email for activation, but the site doesn't tell you this, and nor do tech support, who insist that a mobile is optional?

            And software quality - services down, records lost, records stolen, photos and documents deleted - is a whole other layer on top of that.

            AI is a moderately good solution to the first set of problems, because it can search and assemble information far more quickly than you can. So IME it's pretty damn good at tech support - not perfect, but better than DIY in many cases.

            Software quality? We'll see what happens. If tech stacks collapse over the next couple of years we'll know AI was terrible thing.

            I'm in the 'Too early to tell' camp. There's a fair chance they might. But if so it will be because of poor testing and design, not because of code review or lack of hand-rolled code. And it's not completely clear that quality levels wouldn't have dropped anyway.

            1. ryaniscool · 1790677013 · focus · HN ↗

               AI is here and things don't seem like they are getting any better any faster than they were before. Some people get it wrong and some people get it right...just like before.

               Also, you're completely missing my point. The author is talking about his company inadvertantly offloading decision-making to AI. The comment I was responding to was like "Well, it won't do that much worse than people". That type of thinking is a little dangerous to the future of our species.

Unofficial Hacker News client; not affiliated with Y Combinator.