‹ BackHN Continuity

Thread

Tin: full-text search for Postgres

230 points · 98 comments · ksec

  1. groundzeros2015 · · focus · HN ↗
    Please read the Postgres manual. It has incredible built-in search capability.
    1. dragonwriter · · focus · HN ↗
      If you read deep into this, they claim much better performance than the built in search; they also imply that the built-in search is missing features they provide but don’t make clear which ones (I think it is just support in the same index for queries covering other conditions on other columns, because every other feature they claim seems to line up with the built in search features, which have been around for about 20 years.)
      1. zombodb · · focus · HN ↗
        Over Postgres' FTS, TIN provides at least:

          - superior performance
          - superior operational overhead
          - no second copy of data in tsvector form
          - BM25 scoring support with optimized top-k output
          - runtime configurable scoring knobs
          - expression-attached score boosting
          - sophisticated span query support -- this is proximity search on steroids (https://github.com/planetscale/lead/tree/main/tinql/docs)
          - lossless term positions
          - index-answerable negative expressions (find all docs that don't contain a word)
          - full document hit highlighting
          - optimized exact `count(\*)`
          - term expansion via any of fuzzy matching, wildcards, regular expressions, and dictionary ranges
          - intentionally smaller user-facing SQL API surface
        
        There's a lot we didn't cover in the announcement blog. I'm sure we'll do more as time goes on.

        As an aside, something I personally think is cool, and I suppose you can do this with Postgres' built-in `@@` too, is that you can use TIN's full query language (linked above) against any text datum. This is a valid query:

          SELECT pid, query 
          FROM pg_stat_activity 
          WHERE query ==> 'select OR copy'
        
        in other words, you don't need an index at all to use TIN's full query language against any text field in any query.
        1. groundzeros2015 · · focus · HN ↗
          For most applications an index is a feature not a bug because I can tailor it and inspect.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.