‹ BackHN Continuity

Thread

TabPFN and TabICL vs. tuned XGBoost: the model that doesn't train won 14/14

23 points · 12 comments · EfrainGaray

  1. 3eb7988a1663 · · focus · HN ↗

      XGBoost’s search optimized accuracy, and afterwards I also compare by area under the curve. Which means the “fourteen of fourteen on AUC” is against a boosting model that was not tuned for that metric. Tuning it for AUC would probably improve it there; I did not measure that.
    
    So, not a fair test?

    I also did not get why Xgboost had to count its training time for the inference. You only train once. I guess in some scenario, where someone says, "I need the best model now, you have five minutes on this singular dataset", but I have never been in that situation.

    I would feel better if scripts were released, because I am fairly dubious. I take it as a given that a tabular model has been pre-trained on all of the public benchmark datasets, but that is what it is.

    The slop was so meandering, I am not sure what is truth or not.

    1. aidiscoverywire · · focus · HN ↗

      [dead]

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.