‹ BackHN Continuity

Thread

Training a 4B model to produce 81% faster query plans than Postgres

702 points · 144 comments · polyphilz

  1. hamilyon2 · · focus · HN ↗
    Optimal plan construction is math-heavy, algorithm-heavy and vary even by workload. There are options like creating just-in-time indexes, so solution space grows even faster than article presents. Sometimes it is the query planner which is the slow part of total execution time.

    LLM is kind of blunt weapon to use here. I am waiting rather for alphago style neural net heuristic.

    1. yipinwong · · focus · HN ↗
      What if we use a hybrid model of using both query optimizer and LLM? Whichever produces better result, the database can use?

      - a question from someone with lack of DB depth, me.

      1. Sesse__ · · focus · HN ↗
        The immediate problem: How do you know which one is better without running them?
        1. scarmig · · focus · HN ↗

          [dead]

          1. Sesse__ · · focus · HN ↗
            This immediately halves your throughput.
            1. mattashii · · focus · HN ↗
              Only in the worst case when the plans are equivalent: If one plan is significantly faster, then it'll finish first, and the loser can get canceled before it finishes.
              1. 361994752 · · focus · HN ↗
                Good and bad plans can have orders of magnitude performance difference. The bad one can easily do enough damage cutting the performance in half before it is canceled.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.