Thread

Vote on which of Hacker News' challenges for AI have been met

202 points · 271 comments · stabbles

Retr0id · 1790880368 · focus · HN ↗
Heh, there's one of mine: <a href="https://stoppels.ch/goalposts/?c=39727943" rel="nofollow">https://stoppels.ch/goalposts/?c=39727943
"GPT-4 looks at original ASCII art of a foot, not copied from the web, and says it is a foot."
The vote is currently 64% yes, 18% no.
Just now I asked Opus 5.5 to generate an ASCII art foot, and it did a passable job. It's not great, but it's a foot. Then I pasted it into ChatGPT (whatever they're serving to the free tier by default, which seems to be 5.6 Luna), and it said it was a "train/locomotive": <a href="https://chatgpt.com/share/6abeaa39-cc80-83ed-851f-29370db08964" rel="nofollow">https://chatgpt.com/share/6abeaa39-cc80-83ed-851f-29370db089...
Maybe it's Opus's fault for drawing a bad foot but I think it's fair to say LLMs are still pretty bad at ASCII art (without additional tool calling etc).
1. tedsanders · 1790881084 · focus · HN ↗
  For me, 6.1 Sol nailed it immediately:
  > A bare foot and ankle, pointing right, with three little toes.
  I wonder how much of the wide variation in perceptions of LLM capabilities is driven by the gulf between free models and frontier models. Luna getting something wrong is not always great evidence for LLMs be unable to do that thing.
  Edit: for curious skeptics without access to 6.1 Sol, I tried 3 times and it got it all 3 times. Convo share link: <a href="https://chatgpt.com/share/e/6abeb955-7614-832e-a5e1-b1bd134f1971" rel="nofollow">https://chatgpt.com/share/e/6abeb955-7614-832e-a5e1-b1bd134f...
  1. ben_w · 1790881375 · focus · HN ↗
    Case in point, this nonsense came out of 5.6 Luna: <a href="https://chatgpt.com/share/6abeae1e-42b4-83ed-8966-7e82ae0bef5c" rel="nofollow">https://chatgpt.com/share/6abeae1e-42b4-83ed-8966-7e82ae0bef...
    Like, is this an ice-cream? A tooth?
  2. tomalbrc · 1790882487 · focus · HN ↗
    Please share a link to the conversation, otherwise I am not buying it.
    Because "for me DeepSeek Flash 4.1 nailed it immediately", trust me bro.
  3. Kotlopou · 1790899150 · focus · HN ↗
    The share link doesn't work for me, it says I don't have access. I can see the one from the parent, so the problem is likely on your side.
    shagie · 1790902541 · focus · HN ↗
    [dead]

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.