Thread

Vote on which of Hacker News' challenges for AI have been met

202 points · 271 comments · stabbles

hatthew · 1790903353 · focus · HN ↗
My comment about humanity's last exam being a misnomer is included, and I proposed better ideas about what a last exam could look like. One of the things I said was "solve an open math problem" which has conclusively been done with Navier-Stokes (regardless of the controversy surrounding that). However, in the spirit of clarifying the goalposts, AI has only passed 1/6 of the tests I proposed. 17% is not a passing grade, so I'd say no, my challenge has not been met.
Another thing to note is that the (presumably AI-generated) summary of my challenge does not accurately represent what I wrote, listing only half the things I said and saying "or" rather than "and".
1. intelkishan · 1790929511 · focus · HN ↗
  Could you share your other 5 tests, if they are public?
  1. hatthew · 1790931333 · focus · HN ↗
    Here's my previous comment: <a href="https://news.ycombinator.com/item?id=42809902">https://news.ycombinator.com/item?id=42809902
    I didn't really design a comprehensive test, just listed a few examples of the sort of thing I imagine when I head the name "humanity's last exam". So take it with a grain of salt.
    dingdongditchme · 1790940161 · focus · HN ↗
    I like your list of "challenges" but have a personal issue with two of them:
    1. "improve uniteds' plane shedule" -> the word "improve" does a lot of heavy lifting there..
    2. Turing test for me is solved: "AI expert" is just moving the goal posts imho. The original turing test to my knowledge is about passing notes under a door. Obviously, llms can't communicate via hand-written notes and if that is the bar it will take a long time (or a specifically designed hand-writting machine) to really do this. As far as how much writing back and forth you can do it is clear that turing test is beat: I am wondering often enough on text sent to me from colleagues if it is generated, the same goes for comments here or any ol' website. In a standard llm session I don't really communicate differently than with a human and would not be able to tell the difference in an hour texting session or so. Of course if I ask it to count words or do something ridiculous I can find sus it out; but for all intents and purposes the chat-bot exists.
    lovich · 1790949299 · focus · HN ↗
    You can buy handwriting machines on Amazon.
    Can’t imagine it’d be too much work to hook up and LLMs output to one.
    <a href="https://www.amazon.com/s?k=handwriting+machine+for+letters" rel="nofollow">https://www.amazon.com/s?k=handwriting+machine+for+letters

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.