Thread

Training a 4B model to produce 81% faster query plans than Postgres

702 points · 144 comments · polyphilz

hamilyon2 · 1789593005 · focus · HN ↗
Optimal plan construction is math-heavy, algorithm-heavy and vary even by workload. There are options like creating just-in-time indexes, so solution space grows even faster than article presents. Sometimes it is the query planner which is the slow part of total execution time.
LLM is kind of blunt weapon to use here. I am waiting rather for alphago style neural net heuristic.
1. yipinwong · 1789595402 · focus · HN ↗
  What if we use a hybrid model of using both query optimizer and LLM? Whichever produces better result, the database can use?
  - a question from someone with lack of DB depth, me.
  1. jaggederest · 1789597841 · focus · HN ↗
    This is about to bake your noodle:
    <a href="https://www.postgresql.org/docs/current/geqo-pg-intro.html" rel="nofollow">https://www.postgresql.org/docs/current/geqo-pg-intro.html
    Sesse__ · 1789626286 · focus · HN ↗
    GEQO is not to get a better plan than the traditional optimizer, it is to be able to get a plan at all when the query is large. And it's widely known for creating poor plans.
    jaggederest · 1789628617 · focus · HN ↗
    Yes, but it's exactly the kind of hybrid between a regular planner and something generative (writ broadly) that they were asking about. Practically speaking if you're hitting the GEQO you've already failed as a query writer unless it's a purely OLAP on a dedicated beefy machine.
    Sesse__ · 1789632158 · focus · HN ↗
    I think calling GEQO generative is a bit of a stretch; it's just a different way of searching through the same space with the same cost model. More or less devolving to “let's take a bunch of randomized join orders and see which one is best” :-)
    And yes, large joins is definitely for OLAP use. If you have 20-way joins for OLTP, you're either crazy or you're using an ORM.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.