Thread

WalShadow: Sub-second Postgres replication to ClickHouse from physical WAL

63 points · 10 comments · spathak

vmsp · 1789593272 · focus · HN ↗
They're not using `wal_level = logical`, which has been the "friendly" way of doing CDC on Postgres since ever, but are going straight to `wal_level = replica` which, afaik, has never really been used to build something atop of except Postgres' own replication.
This is very interesting. I'd never have guessed that it'd make such a difference. I also bet this is the sort of thing that would have never end up being implemented without access to coding agents. Having to figure out these protocol-level details is no longer the huge time sink it was
1. DenisM · 1789593533 · focus · HN ↗
  It’s probably brittle though? Replication implementation has to change in some ways from one version to another.
  1. saisrirampur · 1789612203 · focus · HN ↗
    Right, it takes some effort, but it’s worth it for the benefits of sub-second latency and reduced operational overhead. We expect the WAL format for existing commands to remain relatively stable across Postgres versions, while newer commands may need additional handling.

Unofficial Hacker News client; not affiliated with Y Combinator.