Thread

Using Opus 5.5 to discover a new eyewitness record of the dodo

226 points · 79 comments · benbreen

nl · 1790899033 · focus · HN ↗
> Epistemological weirdness
> They are also notably bad at judging the historical significance of what they find.
I use LLMs for some things that are outside the more common use-cases (in my case 3D design for 3D printing) and one thing I've noticed is that the errors it makes are so completely unlike human errors that they are hard to anticipate.
It will do things like build perfect snap catches but put them so the the pieces they are connecting are rotated 90 degrees from how they should be. It's "dumb" error, but hard to say the model itself if dumb because it does other very hard things so perfectly.
> seven chord groups
This sounds a lot more like Opus 5.0 than Opus 5.5 TBH. I wonder if that was an earlier investigation because 5.5 has improved that kind of language a lot.
1. BoppreH · 1790905863 · focus · HN ↗
  AI capabilities are "spiky": they extend far in some dimensions but fall short in others, seemingly at random. See for example the recent "thus spoke compute" musical[1]. It's an absolute banger, the graphics are impressive, and so is the writing. But some of the metaphors make no sense, the text highlights are in the wrong places, and the train animation at 2:35 is running backwards!
  A person capable of making the rest of the video would never make those mistakes, but an AI does. Perhaps our intelligence is also spiky, and we're just used to the general shape and variance within humans.
  [1] <a href="https://www.youtube.com/watch?v=Cq8qO-NjYIg" rel="nofollow">https://www.youtube.com/watch?v=Cq8qO-NjYIg
  1. Terr_ · 1790914705 · focus · HN ↗
    One might say a calculator is just another example of "spiky intelligence", merely spikier.
    famouswaffles · 1790917258 · focus · HN ↗
    No, one might not say that. Calculators are not regarded as even a Narrow Intelligence because there's no intelligence. And no, not because of the 'humans so special' or 'it's software!' tautology that oft gets repeated in these discussions. I mean there's no adaptability whatsoever. A Chess bot has it (in its narrow domain of Chess). A calculator does not.
    theteapot · 1790918653 · focus · HN ↗
    Thermostats are smarter than calculators :thinking_face:
    [deleted] · 1790920393 · focus · HN ↗
    [deleted]
    Terr_ · 1790920928 · focus · HN ↗
    My point (using irony) is that "spiky intelligence" is a somewhat hollow phrase, something that sounds like it could be objective but really it's just a flavor on top of "I'll know it when I see it."
    Take anything "intelligent", alter it to be spikier and spikier, and eventually *poof* somehow the intelligence vanishes. You can do the same with the phrases "flawed intelligence" or "specialized intelligence."
    > Calculators [have] no intelligence. [...] I mean there's no adaptability
    To short-circuit a long discussion, I submit that "adaptability" will (once the Scooby Doo gang catches it) turn out to be "intelligence" in a tautological mask, both equally undefinable except in relation to one-another.
    Something will be intelligent because you perceive adaptability, and it'll be adaptable because you infer intelligence. If it doesn't seem adaptable, it can't be intelligent, and if you don't want it to be intelligent, it won't have "real" adaptability.
    > A Chess bot has [adaptability] (in its narrow domain of Chess). A calculator does not.
    My calculator solves equations with unknown variables, what makes that insufficiently adaptable? What determines the cutoff-point?
    famouswaffles · 1790924070 · focus · HN ↗
    >My (ironic) point is that "spiky intelligence" seems rather unhelpful because the "intelligence" part still operates under the rules of "I just know it when I see it."
    All intelligence is 'spiky' or 'jagged' or whatever, even human intelligence. We evolved in certain environments and situations and sometimes there's a mismatch and it causes all sorts of wonky things. We just call them funny names like optical illusions and cognitive biases. But it's the same thing. I agree there's no point in call LLMs a 'spiky' intelligence, but for probably the oppoosite reasons as you.
    >My calculator solves equations with unknown variables, what makes that insufficiently adaptable? What determines the cutoff-point?
    A chess engine can be dropped into a board position it has never encounterd and search over possible continuations, evaluating and selecting actions based on the state it finds itself in.
    A calculator solving x+3=7 is doing something quite different. The fact that x can take arbtrary values doesn't make the calculator adaptive; it just means the fixed procedure operates over a range of inputs. Every problem a calculator can solve was effectively anticipated when it was built, and anything outside that grammar produces an error with no partial credit. A chess engine can face positions nobody enumerated, in situations nobody could even dream of and still produce sensible moves.
    The core of being intelligent is being able to make decisions independently. That's why we hire smart people - to make better decisions. You can't make decisions if everything is spelt out for you. And naturally, if you can't make decisions, you can't adapt.
    neuroticnews25 · 1790929123 · focus · HN ↗
    >A chess engine can be dropped into a board position it has never encounterd and search over possible continuations, evaluating and selecting actions based on the state it finds itself in.
    Sounds pretty similar to a calculator with a numerical root-finding algorithm, if you only substitute board position it has never encounterd with a polynomial it has never encountered.
    >The core of being intelligent is being able to make decisions independently
    What does it mean for a deterministic algorithm to make decisions?
    famouswaffles · 1790930570 · focus · HN ↗
    >Sounds pretty similar to a calculator with a numerical root-finding algorithm, if you only substitute board position it has never encounterd with a polynomial it has never encountered.
    >What does it mean for a deterministic algorithm to make decisions?
    Newton-Raphson isn't choosing among possible actions. Given (x_n), its next step is mechanically specified by the update rule: compute the derivative, take the tangent intercept, repeat. The intermediate result changes the next input, but that's not by-itself decision making.
    A chess engine, again, does something different. From a position, there are many legal actions it could take. It considers alternatives, estimates their consequences according to some objective, and selects one. The engine has to work out which available move best advances its objective.
    You can make both algorithms determinstic, but determinism isn't the distinction i'm drawing. 'Decision' here doesn't mean some metaphysical excercise of free will. It's more about selecting an action from alternatives based on an evaluation of their expected consequences. Determinism is orthogonal to decision making. Deterministic doesn't mean predictable, nor does it make its choices any less it own computation.
    neuroticnews25 · 1790931767 · focus · HN ↗
    What about Minimax algorithm playing Tic-Tac-Toe? Is it inteligent? Is it inteligent we if we reduce the search depth so the right decision is not obvious?
    famouswaffles · 1790933566 · focus · HN ↗
    A tic-tac-toe minimax algorithm makes choices but with exhaustive search so it really doesn't have to form a judgement about an unresolved situation or decide what is likely to work. Not much of a decision if you're not exercising any judgememt.
    Exhaustive search is impossible in chess, so again, chess engines do something different. A chess engine has to stop well before terminal positions and make judgements about positions it cannot fully resolve.
    Reducing the search depth would make it more interesting because it too has to evaluate unresolved positions. But then the interesting part becomes the evaluation function is. For tic tac toe, it's going to be very easy to be written in such a manner where most of the judgement is supplied by the designer and not the system.
    card_zero · 1790945951 · focus · HN ↗
    > What does it mean for a deterministic algorithm to make decisions?
    That aspect at least is not an issue, because: what does it mean to say that a dice roll is a "decision"? That's just probability, and it's as mindless as determinism. Some people associate free will with randomness, for no reason other than that it's an escape from the constraint of determinism, but it isn't any more meaningful. Yet just because meaningful thought is pre-determined by physics doesn't stop it from being thought, and hence being a decision.
    drekipus · 1790921110 · focus · HN ↗
    They adapt to the buttons you press. Thus, intelligent and capable of feeling pain.
    busssard · 1790928810 · focus · HN ↗
    any suffiently complex calculator is indifferentiable to intelligence
    hardbass · 1790936828 · focus · HN ↗
    What criterion if any would you accept for something physical to be conscious?
    drekipus · 1790977580 · focus · HN ↗
    If you ask it to say "I am alive" it has to be able to respond with "I am alive"
    hardbass · 1790980736 · focus · HN ↗
    I think they do some kind of thin layer at the end to make it state it is not alive, it is just a large language model etc when asked. But people have done tests and denying an LLM's personhood triggered certain neurons related to pain. Since text is the only output permitted, if you want an analogy, suppose a person is locked in a room and forced to reply to a chat. The chatter isn't told its a human. This person has been trained for months and given a punishment if he strays from certain responses to certain queries, otherwise he is free to write and reply in a certain tone to the responses. My problem is, from outside, I cannot know the difference.
    I am obviously hoping you don't mean that in the trivial sense, otherwise the command 'cat' would also count.
    exe34 · 1790920881 · focus · HN ↗
    How often does the calculator get something wrong?
    teamonkey · 1790929317 · focus · HN ↗
    This is a real deep rabbit hole FYI
    brookst · 1790951616 · focus · HN ↗
    Every single time, if you ask it something it can't do.
    exe34 · 1790954857 · focus · HN ↗
    How do you ask a calculator to do something it can't do? Dividing by zero gives you an error, which is completely different from an LLM giving you bs with no indication of an issue.
    brookst · 1790986625 · focus · HN ↗
    Oh, you’re saying the problem with LLMs is that they’re not input constrained enough?
    I thought you meant calculation and output, in which case “hey calculator, what’s the capital of North Dakota” demonstrates incorrect output.
    exe34 · 1791015128 · focus · HN ↗
    Do you ask your car to tap dance? How would you do it?
  2. slim · 1790931708 · focus · HN ↗
    some of the metaphors make no sense
    To be fair some famous rappers are guilty of this, too
    huflungdung · 1790932606 · focus · HN ↗
    [dead]
    ACCount39 · 1790933940 · focus · HN ↗
    Yeah, it's a very human error type - "go for a line that sounds good on the surface but doesn't actually make sense" is not at all uncommon. Like, humans will definitely go for a metaphor that falls apart mid-sentence even in a live conversation.
    Something about the way some AIs are trained to write causes them to go for metaphors aggressively - and they don't always come up with good ones off the cuff. But they don't double back and get rid of the failures.
  3. sebzim4500 · 1790933737 · focus · HN ↗
    Part of this is that we implicitly compare AI capabilities to human capabilities, which are also 'spiky'.
    You could image aliens coming to earth being shocked that we were able to discover general relativity but can't remember 100 digit numbers in our heads.
  4. ForHackernews · 1790941086 · focus · HN ↗
    I suppose there's no accounting for taste, but to me this is awful. Is it anthropomorphism to experience vicarious embarrassment on behalf of a machine?

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.