Thread

1. charcircuit · 1790403333 · focus · HN ↗

   >The demo uses Claude Haiku rather than newer models like Opus or Fable. This is because I'm assuming that shopping agents of the future won't use newer, more expensive models,

   Why are we assuming that shopping agents are going to be using the model that is easiest to fall for prompt injections? Only testing a year old, small model is going to lead to a misleading conclusion.

   1. simianwords · 1790403499 · focus · HN ↗

      Author doesn’t understand cost dynamics of the industry - that prices keep falling.

      Author also needs to make a point, and can’t do so if they are using a powerful model.

      1. moomoo11 · 1790403990 · focus · HN ↗

         this is the issue most people don't understand, or maybe they're incapable of understanding.

         next year, astra tier model will be $2/1M or whatever (point is - cheaper) and whatever model is $10/1M will be insane like astra feels today.

         also, in any agentic workflow, you use models of various levels together... not just one model of one level...

         1. simianwords · 1790404641 · focus · HN ↗

            Also the author is deliberately chooses a model that is 1 year old and was supposed to be underpowered even then. If cost was deciding factor, they could’ve chosen Luna.

              Model                       AA Index   Cost/task
              Claude 4.5 Haiku (thinking)    17       $0.21
              GPT-6 Luna (max)               37       $0.07
              GPT-6 Sol (low)                34       $0.13
              GPT-6 Sol (medium)             40       $0.25
            
            
            
            

            Luna (low) scores higher and costs about 47x less per task and performs better.

            And Luna (max) scores 37 while still 30% of the cost of Haiku which scored 17.

            Why deliberately choose a model that is known to be poor and also costly?

Unofficial Hacker News client; not affiliated with Y Combinator.