Thread

Vote on which of Hacker News' challenges for AI have been met

202 points · 271 comments · stabbles

ben_w · 1790877906 · focus · HN ↗
Very pleased one of my predictions was totally wrong: <a href="https://news.ycombinator.com/item?id=23252711">https://news.ycombinator.com/item?id=23252711
Sure, sure, what LLMs make still isn't "efficient bug-free code": my prediction is falsified because while LLMs can write and train new models with machine learning, ML is fundamentally not advanced enough to throw arbitraty new tasks at like this.
1. FabCH · 1790878254 · focus · HN ↗
  Somewhat appropriate the site the OP links to is called „goalposts“ because as far as I can see, people keep shifting theirs.
  In your case, the comment you link to says „business tasks“ and you expanded it now to „arbitrary new tasks“. Those are not the same. An LLM today sure can do many many many business-speak conversion tasks.
  1. tripleee · 1790879112 · focus · HN ↗
    > An LLM today sure can do many many many business-speak conversion tasks
    Not reliably, and not without supervision. That's the main point. I'm trying really hard to figure out a workflow that doesn't require me to review the code and I just don't see how it's possible (yet)
    You either need a comprehensive test suite (which requires understanding the code in order to create) or you need to review the actual implementation code to make sure it does the right thing
    FabCH · 1790884224 · focus · HN ↗
    Code is a tiny part of "business".
    Most business is correspondence with people who want money from you and people you want money from.
    rstuart4133 · 1790914333 · focus · HN ↗
    The issue is that correspondence is legally enforceable [0]. LLM's are good and getting better, but if LLMs are giving enforceable undertakings, you want to very sure they are not going to promise something that will send the company broke.
    I'm not sure what risk a businessman is willing to accept, but I'd be asking for probabilities under once in a millennium. The latest round has improved considerably in their ability to follow instructions (thank $DEITY), but they aren't anywhere near that yet.
    [0] <a href="https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know" rel="nofollow">https://www.bbc.com/travel/article/20240222-air-canada-chatb...
    sokoloff · 1790935209 · focus · HN ↗
    People who demand risks to be lowered to once-a-millennium are not the type to go start or even run businesses.
    There’s nothing wrong with that, but starting a business means fading several once-a-year risks of failure and running even an established one means facing several once-a-century risks every year.
    user43928 · 1790929557 · focus · HN ↗
    No, you don't.
    I've stopped reviewing the code in my mobile app project months ago. I now only look at files changed and lines count in MRs. Functionality is best verified via manual QA testing.
    I know that people here are going to doubt the quality of my project and say that it is impossible, but they are clueless and have evidently not build a project in this way. Experiences from eg. corporate backend work are hardly relevant.
    It is clear to me that the fewer consequential mistakes people find during code review, their attention to code reviews is going to go down, to the point of also skipping them.
    I expect that for most development, not reviewing the code will be the standard by March of next year. Only critical code like authentication will be reviewed.
    suddenlybananas · 1790930497 · focus · HN ↗
    What's the app?
    user43928 · 1790935018 · focus · HN ↗
    Not public.
    It's a paid app, and after 6 months of work I expect to publish it this month.
    In other words, you will have to take my word for its quality.
    tripleee · 1790952522 · focus · HN ↗
    what do your prompts / your workflow look like?
    user43928 · 1790975221 · focus · HN ↗
    Nothing special at all, it just works.
    One thing that I do is have each change reviewed by another model. If I implement with Codex, I would have Claude do the review, or the other way around.
    I did not benchmark this against a review from a subagent with the same model, so I don't know if that in particular helps.
    sokoloff · 1790934958 · focus · HN ↗
    I have a task that I do once per year for a robotics team that I mentor: Roughly,
    Take this calendar of events and rank your preferences for the event lottery. Events are spread across 5 weeks, some are 20 minutes away, some are 4.5 hours away, some are Friday/Saturday, some are Saturday/Sunday, some are historically extremely competitive, some fill up in round 1, others don’t even fill after round 2. For the last two years, I’d written some scripts to scrape the event sites, find the addresses, ask Google Maps to give me driving distances and times, scrape prior year registration information to find which teams went and the strength of those teams, etc. It was several hours of effort.
    This year, ChatGPT was capable of doing almost all of that basic research and data conversion, filling out our internal spreadsheet. It probably still took 4 hours on the wall clock, but at 2% attention (5 minutes of human toil).
    I doubt I go a single workday without some kind of “I have an idea and I know there are disparate data sources out there; go find those and cross-correlate or extract the relevant data points.” question that is now 10-50x more efficient than 2 years ago.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.