Thread

Hister: A private search engine for the pages you visit and the files you keep

741 points · 202 comments · bookofjoe

taude · 1789669128 · focus · HN ↗
Kind of related to this in that I built it to hoard knowledge from web pages I've visited along with implementing a Karpathy-style LLM Wiki, but the knowledge is collected automatically from sources I browse.
I have it up on GitHub, but I don't think anyone should use my implementation.
Loosely, what I built:
* On each of my machines I have a cron job running that looks at all my web browser history (usualy it's inspecting the brower's SQLlite across firefox and chrome). If it matches my rule list: hacker news stories, certain reddits, etc. it'll grab the page, convert to markdown and drop in my Obsidian Vault incoming.
* It has a whole de-duping architecture since I might open the same page on multiple machines. Uses the CloudFlare SQLITE D1 storage for tracking the processed links.
* it'll then trigger the LLM to do some Karpathy wiki style taxonomy assignment to the articles, organize them, create an index etc.
It's then available for my "bot" stuff to do writings for me.... I will probably write more about it at some point. I'm not certain it's totally useful and not just a yak-shave on hoarding knowledge.
Ai-drafted article on this [1]
Example AI-Drafted article based on some discussions the other day on Ollma vs LLama.cpp [2]
[1] <a href="https://taude.xyz/posts/how-archivore-turns-browsing-into-a-wiki/" rel="nofollow">https://taude.xyz/posts/how-archivore-turns-browsing-into-a-...
[2] <a href="https://taude.xyz/posts/skip-ollama-run-llama-cpp-directly-on-a-mac/" rel="nofollow">https://taude.xyz/posts/skip-ollama-run-llama-cpp-directly-o...
1. pwython · 1789670564 · focus · HN ↗
  I feel like this could be useful for my Karpathy-style LLM Wiki, but then again I think, in this day and age most of what I'm reading about is outdated in a few weeks. Not only are the tools/methods coming and going, but the ones that stick around drastically change. It would probably turn into a dump of antiquated markdown that could cause confusion.
  As far as the need for private search, well, I've already searched for or visited those pages, so...
  1. taude · 1789671686 · focus · HN ↗
    Yeup, as I mentioned, it kind of feels like a Yak-shave....but it was fun building it, and gave me some experience with agentic coding and new technologies I don't normally work with. It's nice to dig into for some research.
    The biggest win was the realization that both firefox and chrome maintain all the links you visit in a very queryable SQLite database. I've been poking at that for a lot of custom tools, like WHAT JIRA tickets am I paying attention to this week, etc....

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.