Thread

1. charcircuit · 1790403333 · focus · HN ↗

   >The demo uses Claude Haiku rather than newer models like Opus or Fable. This is because I'm assuming that shopping agents of the future won't use newer, more expensive models,

   Why are we assuming that shopping agents are going to be using the model that is easiest to fall for prompt injections? Only testing a year old, small model is going to lead to a misleading conclusion.

   1. simianwords · 1790403499 · focus · HN ↗

      Author doesn’t understand cost dynamics of the industry - that prices keep falling.

      Author also needs to make a point, and can’t do so if they are using a powerful model.

      1. moomoo11 · 1790403990 · focus · HN ↗

         this is the issue most people don't understand, or maybe they're incapable of understanding.

         next year, astra tier model will be $2/1M or whatever (point is - cheaper) and whatever model is $10/1M will be insane like astra feels today.

         also, in any agentic workflow, you use models of various levels together... not just one model of one level...

         1. simianwords · 1790404641 · focus · HN ↗

            Also the author is deliberately chooses a model that is 1 year old and was supposed to be underpowered even then. If cost was deciding factor, they could’ve chosen Luna.

              Model                       AA Index   Cost/task
              Claude 4.5 Haiku (thinking)    17       $0.21
              GPT-6 Luna (max)               37       $0.07
              GPT-6 Sol (low)                34       $0.13
              GPT-6 Sol (medium)             40       $0.25
            
            
            
            

            Luna (low) scores higher and costs about 47x less per task and performs better.

            And Luna (max) scores 37 while still 30% of the cost of Haiku which scored 17.

            Why deliberately choose a model that is known to be poor and also costly?

         2. someothherguyy · 1790405053 · focus · HN ↗

            > or maybe they're incapable of understanding

            incapable of understanding speculation?

            1. simianwords · 1790405192 · focus · HN ↗

               Speculation of what? There are models today that perform better than Haiku at 47x lower cost.

               Excessive skepticism does no good.

               1. someothherguyy · 1790405453 · focus · HN ↗

                  speculation of cost decreases and performance increases in a matching trend year-over-year

                  1. moomoo11 · 1790406032 · focus · HN ↗

                     comparable capability has gotten cheaper.

                     gpt 6 Luna can be swapped for 5.6 terra or even sol in some cases, and it is much cheaper.

                     source: i have a lot of evals, and i have ai find the best model for me for those evals over time so i save $ lol

            2. [deleted] · 1790405471 · focus · HN ↗

               [deleted]

Unofficial Hacker News client; not affiliated with Y Combinator.