I think what we're seeing with every database company providing new full-text search capabilities is an example of AI coding productivity showing up in the real world.
It started with paradeDB and pg_search <a href="https://www.paradedb.com/blog/introducing-search">https://www.paradedb.com/blog/introducing-search
Timescale has pg_textsearch <a href="https://github.com/timescale/pg_textsearch" rel="nofollow">https://github.com/timescale/pg_textsearch
Neon and Databricks have Lakebase Search <a href="https://docs.databricks.com/aws/en/oltp/projects/lakebase-search" rel="nofollow">https://docs.databricks.com/aws/en/oltp/projects/lakebase-se...
Now PlanetScale.
AFAIK all of these are implementations of the BM25 algorithm. You can just tell an agent to read about BM25 and implement it in your system of choice. Cool to see. Seems like there's still a lot of juice to be squeezed out of how it's architected and integrated into each system, but you can't help but wonder if this will lead to aggressive commodification
I don't completely disagree with your hypotheses but it feels like the hard part of his TIN stuff isn't BM25 (which has been around for donkeys years) it's all the hardcore storage engine work around it. And is an LLM particularly good at e.g. segment merging under a thousand updates a second? I've had a few situations where I've been told "we've hit the perf floor" by Claude only to have persisted myself and shaved substantial amounts off still.
More damning for the theory might be that I think paradedb's pg_search predates the agentic coding by a few years?
andrenotgiant · · focus · HN ↗
It started with paradeDB and pg_search <a href="https://www.paradedb.com/blog/introducing-search">https://www.paradedb.com/blog/introducing-search
Timescale has pg_textsearch <a href="https://github.com/timescale/pg_textsearch" rel="nofollow">https://github.com/timescale/pg_textsearch
Neon and Databricks have Lakebase Search <a href="https://docs.databricks.com/aws/en/oltp/projects/lakebase-search" rel="nofollow">https://docs.databricks.com/aws/en/oltp/projects/lakebase-se...
Now PlanetScale.
AFAIK all of these are implementations of the BM25 algorithm. You can just tell an agent to read about BM25 and implement it in your system of choice. Cool to see. Seems like there's still a lot of juice to be squeezed out of how it's architected and integrated into each system, but you can't help but wonder if this will lead to aggressive commodification
ceuk · · focus · HN ↗
More damning for the theory might be that I think paradedb's pg_search predates the agentic coding by a few years?
philippemnoel · · focus · HN ↗
I do agree with the sentiment here that many of the other FTS-in-Postgres projects in recent years have been heavily enabled by vibe coding.