‹ BackHN Continuity

Thread

Hister: A private search engine for the pages you visit and the files you keep

741 points · 202 comments · bookofjoe

  1. taude · · focus · HN ↗
    Kind of related to this in that I built it to hoard knowledge from web pages I've visited along with implementing a Karpathy-style LLM Wiki, but the knowledge is collected automatically from sources I browse.

    I have it up on GitHub, but I don't think anyone should use my implementation.

    Loosely, what I built:

    * On each of my machines I have a cron job running that looks at all my web browser history (usualy it's inspecting the brower's SQLlite across firefox and chrome). If it matches my rule list: hacker news stories, certain reddits, etc. it'll grab the page, convert to markdown and drop in my Obsidian Vault incoming.

    * It has a whole de-duping architecture since I might open the same page on multiple machines. Uses the CloudFlare SQLITE D1 storage for tracking the processed links.

    * it'll then trigger the LLM to do some Karpathy wiki style taxonomy assignment to the articles, organize them, create an index etc.

    It's then available for my "bot" stuff to do writings for me.... I will probably write more about it at some point. I'm not certain it's totally useful and not just a yak-shave on hoarding knowledge.

    Ai-drafted article on this [1]

    Example AI-Drafted article based on some discussions the other day on Ollma vs LLama.cpp [2]

    [1] <a href="https:&#x2F;&#x2F;taude.xyz&#x2F;posts&#x2F;how-archivore-turns-browsing-into-a-wiki&#x2F;" rel="nofollow">https:&#x2F;&#x2F;taude.xyz&#x2F;posts&#x2F;how-archivore-turns-browsing-into-a-...

    [2] <a href="https:&#x2F;&#x2F;taude.xyz&#x2F;posts&#x2F;skip-ollama-run-llama-cpp-directly-on-a-mac&#x2F;" rel="nofollow">https:&#x2F;&#x2F;taude.xyz&#x2F;posts&#x2F;skip-ollama-run-llama-cpp-directly-o...

    1. verdverm · · focus · HN ↗
      &gt; have it up on GitHub, but I don&#x27;t think anyone should use my implementation.

      ditto, it&#x27;s an experiment in near-vibe coding, which also uses Typesense for queries using BM-25 &amp; RAG with fusion. I have the web search&#x2F;fetch&#x2F;crawl features persisting raw intermediate values (api responses, search result lists) because I might re-use them one day... at least good for auditability if I need to

      related, it is using Hister author&#x27;s prior project SearXNG as one of the search providers

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.