Thread

Vote on which of Hacker News' challenges for AI have been met

202 points · 271 comments · stabbles

hatthew · 1790903353 · focus · HN ↗
My comment about humanity's last exam being a misnomer is included, and I proposed better ideas about what a last exam could look like. One of the things I said was "solve an open math problem" which has conclusively been done with Navier-Stokes (regardless of the controversy surrounding that). However, in the spirit of clarifying the goalposts, AI has only passed 1/6 of the tests I proposed. 17% is not a passing grade, so I'd say no, my challenge has not been met.
Another thing to note is that the (presumably AI-generated) summary of my challenge does not accurately represent what I wrote, listing only half the things I said and saying "or" rather than "and".
1. intelkishan · 1790929511 · focus · HN ↗
  Could you share your other 5 tests, if they are public?
  1. hatthew · 1790931333 · focus · HN ↗
    Here's my previous comment: <a href="https://news.ycombinator.com/item?id=42809902">https://news.ycombinator.com/item?id=42809902
    I didn't really design a comprehensive test, just listed a few examples of the sort of thing I imagine when I head the name "humanity's last exam". So take it with a grain of salt.
    dingdongditchme · 1790940161 · focus · HN ↗
    I like your list of "challenges" but have a personal issue with two of them:
    1. "improve uniteds' plane shedule" -> the word "improve" does a lot of heavy lifting there..
    2. Turing test for me is solved: "AI expert" is just moving the goal posts imho. The original turing test to my knowledge is about passing notes under a door. Obviously, llms can't communicate via hand-written notes and if that is the bar it will take a long time (or a specifically designed hand-writting machine) to really do this. As far as how much writing back and forth you can do it is clear that turing test is beat: I am wondering often enough on text sent to me from colleagues if it is generated, the same goes for comments here or any ol' website. In a standard llm session I don't really communicate differently than with a human and would not be able to tell the difference in an hour texting session or so. Of course if I ask it to count words or do something ridiculous I can find sus it out; but for all intents and purposes the chat-bot exists.
    Dylan16807 · 1791185504 · focus · HN ↗
    Making the judge an expert might be moving the goal posts. But what you're describing is not at all serving the purpose of a Turing test. Depending on the situation, having plausibly human-sounding work-related communication with a robot has been possible for decades. It's not a Turing test unless the main goal of the conversation is figuring out human versus AI. And it has to be a proper conversation going back and forth many many times.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.