Thread

Vote on which of Hacker News' challenges for AI have been met

202 points · 271 comments · stabbles

Retr0id · 1790880368 · focus · HN ↗
Heh, there's one of mine: <a href="https://stoppels.ch/goalposts/?c=39727943" rel="nofollow">https://stoppels.ch/goalposts/?c=39727943
"GPT-4 looks at original ASCII art of a foot, not copied from the web, and says it is a foot."
The vote is currently 64% yes, 18% no.
Just now I asked Opus 5.5 to generate an ASCII art foot, and it did a passable job. It's not great, but it's a foot. Then I pasted it into ChatGPT (whatever they're serving to the free tier by default, which seems to be 5.6 Luna), and it said it was a "train/locomotive": <a href="https://chatgpt.com/share/6abeaa39-cc80-83ed-851f-29370db08964" rel="nofollow">https://chatgpt.com/share/6abeaa39-cc80-83ed-851f-29370db089...
Maybe it's Opus's fault for drawing a bad foot but I think it's fair to say LLMs are still pretty bad at ASCII art (without additional tool calling etc).
1. tedsanders · 1790881084 · focus · HN ↗
  For me, 6.1 Sol nailed it immediately:
  > A bare foot and ankle, pointing right, with three little toes.
  I wonder how much of the wide variation in perceptions of LLM capabilities is driven by the gulf between free models and frontier models. Luna getting something wrong is not always great evidence for LLMs be unable to do that thing.
  Edit: for curious skeptics without access to 6.1 Sol, I tried 3 times and it got it all 3 times. Convo share link: <a href="https://chatgpt.com/share/e/6abeb955-7614-832e-a5e1-b1bd134f1971" rel="nofollow">https://chatgpt.com/share/e/6abeb955-7614-832e-a5e1-b1bd134f...
  1. ben_w · 1790881375 · focus · HN ↗
    Case in point, this nonsense came out of 5.6 Luna: <a href="https://chatgpt.com/share/6abeae1e-42b4-83ed-8966-7e82ae0bef5c" rel="nofollow">https://chatgpt.com/share/6abeae1e-42b4-83ed-8966-7e82ae0bef...
    Like, is this an ice-cream? A tooth?
  2. tomalbrc · 1790882487 · focus · HN ↗
    Please share a link to the conversation, otherwise I am not buying it.
    Because "for me DeepSeek Flash 4.1 nailed it immediately", trust me bro.
  3. Kotlopou · 1790899150 · focus · HN ↗
    The share link doesn't work for me, it says I don't have access. I can see the one from the parent, so the problem is likely on your side.
    shagie · 1790902541 · focus · HN ↗
    [dead]
2. ben_w · 1790881113 · focus · HN ↗
  Mm, I kina agree with the AI on this one:
  (_)(_)(_) represents the wheels
  They do look rather wheel-like; I have to assume you see them as toes though?
  It's like the duck-bunny picture to me. If I focus on the "wheels", I see a steam train locomotive (but perhaps I'm only seeing that because I read your comment?); if I look at the ankle I see a foot.
3. howunfortunate · 1790881681 · focus · HN ↗
  Readers: before you vote or comment, look at that foot.
  I think I would have failed this test!
4. hydrolox · 1790882009 · focus · HN ↗
  To be fair if a human was given a linear sequence representing ascii art you couldn't tell either
5. nonameiguess · 1790882051 · focus · HN ↗
  This is a Rorschach test, not a foot. If you'd shown this to me without telling me what it was meant to be first, I'd have guessed a crematorium.
  1. dec0dedab0de · 1790882187 · focus · HN ↗
    Has anyone done Roschach tests for AI? That would be an interesting study to see how different models responded.
    shagie · 1790903309 · focus · HN ↗
    Using <a href="https://rorschachgenerator.org" rel="nofollow">https://rorschachgenerator.org
    <a href="https://chatgpt.com/share/6abf02ae-9a40-83e9-a432-00bf064f60c8" rel="nofollow">https://chatgpt.com/share/6abf02ae-9a40-83e9-a432-00bf064f60...
    The images: <a href="https://imgur.com/a/ig6sn6I" rel="nofollow">https://imgur.com/a/ig6sn6I
    ... They're not what I would have described. For me, 99.something% flesh and blood with less than 1% metal, glass, and probably some microplastics...
    The first one I see a person with a big tall hat and a big nose.
    The second one... I do see the black statue with a figure in white in front of it.
    The third one is immediately two fish looking at each other.
    lbreakjai · 1790929881 · focus · HN ↗
    Where did you find those three pictures of my parents getting divorced??
6. adam_rb · 1790891616 · focus · HN ↗
  I think the problem is that you're using basing your conclusion from the cheap/dumb models available on the free tier of services. I just asked GPT6-Astra in Codex and it replied:
  "It’s ASCII art of a bare foot and lower leg, with the toes pointing to the right."
  No tool calling, just an immediate reply with the correct answer.
  1. brudgers · 1790892992 · focus · HN ↗
    Sure, but it’s already read the HN thread.
    akavi · 1790896565 · focus · HN ↗
    ...that's not how LLM training works.
7. asdfasgasdgasdg · 1790895647 · focus · HN ↗
  Opus 5.5 was able to parse and understand an ASCII art foot when I pasted one in.
8. hyperpape · 1790898525 · focus · HN ↗
  I'm a human, and that's not a foot, it's a smokestack.
  But as you pointed out, while that absolves ChatGPT, it makes Opus look worse.
9. janalsncm · 1790900996 · focus · HN ↗
  A better test would be using an image to ascii converter tool to rule out bad ascii drawing from Opus.
10. qnleigh · 1791052983 · focus · HN ↗
  I wouldn't have guessed foot either, and I'm a human.*
  *In case there's any doubt, there are three r's in strawberry.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.