Thread

1. Someone · 1789588667 · focus · HN ↗

   > I paid ~$800 to rent a 2x H100 SXM node from Lambda for ~95 hours, and ~$400 in OpenAI API fees to generate the Astra trajectory demonstrations.

   > a tiny 4B model went from not being able to understand the harness it was wrapped in, to achieving a 1.81x geometric mean speedup and a summed latency decrease of 44.7% across a workload of join-heavy SQL queries

   I can’t find it in the article (may have skimmed it too much), but I suspect they didn’t include those ~95 hours in the benchmark numbers.

   I think all database vendors know their query optimizers could do much better if they could afford to spend lots of time to derive query plans.

   ⇒ this may be useful for some workloads, but even then, can you afford to spend hours every now and then to update your 4B model to ensure it still picks a good query plan?

   1. vatsachak · 1789595235 · focus · HN ↗

      I think the idea would be making a frontier model that does this. One that is trained on multiple queries

Unofficial Hacker News client; not affiliated with Y Combinator.