‹ BackHN Continuity

Thread

Hister: A private search engine for the pages you visit and the files you keep

741 points · 202 comments · bookofjoe

  1. asciimoo · · focus · HN ↗
    Ohi, author here! Thanks for posting Hister. Feel free to A.M.A. My first free software search project was Searx, a privacy respecting metasearch engine, but because of the limitations of the metasearch concept, I've decided to take a different approach.

    Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites. It stores extracted content with offline result previews, so information remains searchable even when the original page changes or disappears. It supports full text and semantic search, can run entirely on your own machine, and includes a web interface, command line tools, and an MCP endpoint for assistant integrations.

    Website: <a href="https:&#x2F;&#x2F;hister.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;hister.org&#x2F;

    Tiny read-only demo: <a href="https:&#x2F;&#x2F;demo.hister.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;demo.hister.org&#x2F;

    Ps.: It looks like our name conflicts with a registered trademark in the US. The owner of the other project has asked us to change it, so we’ll probably need to comply sooner or later.

    Name suggestions are welcome! Ideally, the new name should be relatively short, sound good, and have an available .org domain.

    Thanks!

    1. Capricorn2481 · · focus · HN ↗
      Thanks so much for this, I&#x27;m using it all the time. I self host a few things, but I&#x27;m using this the most.
    2. adfm · · focus · HN ↗
      histro.org is available.
      1. asciimoo · · focus · HN ↗
        Unfortunately, it is still considered too similar from a legal standpoint.
        1. adfm · · focus · HN ↗
          In that case, clipshot.org is also available.
        2. juliend2 · · focus · HN ↗
          historea.org is available.

          It sounds like history, storage and ethereal at the same time.

          1. _kidlike · · focus · HN ↗
            that exactly how the Greek word sounds :)

            ιστορία, if you wanna copy paste.

            and yes, the English word comes from the Greek word!

        3. golem14 · · focus · HN ↗
          I doubt it. Trademark “infringement “ only happens when the two parties compete in the same space. I think if the other party is the music party game, you’re pretty much in the clear. They may still sue you and lose unless you cave.
          1. TaLiTr · · focus · HN ↗
            Nothing stopping them from threatening an open-source dev. The &quot;&quot;recent&quot;&quot; incident of Hoarder rebranding to Karakeep comes to mind.
            1. golem14 · · focus · HN ↗
              Exactly. We should have a general kickstarter-like fund for cases like this. Maybe John Oliver is interested :)
    3. corndoge · · focus · HN ↗
      Are you aware of ArchiveBox?

      <a href="https:&#x2F;&#x2F;archivebox.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;archivebox.io&#x2F;

      What does Hister do differently? Search seems like a major differentiator, I&#x27;m wondering if leveraging the existing archivebox project for archival and implementing good search on top would be more efficient

      1. asciimoo · · focus · HN ↗
        The main difference I see is Hister focuses on creating an active knowledge base and finding information quickly, while ArchiveBox focuses on preserving web content for the long term.
        1. corndoge · · focus · HN ↗
          Thanks for answering. Do you think these dovetail? Both archive everything you browse, so that&#x27;s common functionality that could be factored out. I only want one archive, having two separate archives because one focuses on search and the other on long term archival is inefficient. What do you do for long term archival - or do you not have this use case?
    4. dbliss · · focus · HN ↗
      Does it work accross multiple computers? Ideally the service runs on a linux box on my tailnet, and my windows and mac systems share the same server.

      Edit: I RTFD - and it seems yes.

      1. asciimoo · · focus · HN ↗
        Sure, as long as you (and the browser extension) can reach the server, it can be used from as many machines as you want even in a multi-user setup.
        1. nottorp · · focus · HN ↗
          &quot;Optional global or personal access token used to authenticate extension requests.&quot;

          Looks like you can even set authentication up so you can run it at home but connect while you&#x27;re away too...

    5. dwedge · · focus · HN ↗
      It feels like I&#x27;m the only person using this but I&#x27;d like to throw another potential bookmark manager integration into the ring, cherry <a href="https:&#x2F;&#x2F;github.com&#x2F;haishanh&#x2F;cherry" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;haishanh&#x2F;cherry
      1. pava0 · · focus · HN ↗
        Not even a readme?
        1. dwedge · · focus · HN ↗
          Never even noticed that was missing and it&#x27;s probably why nobody else uses it. I took it from <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;selfhosted&#x2F;comments&#x2F;xyepiu&#x2F;cherry_an_open_source_selfhostable_bookmark&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;selfhosted&#x2F;comments&#x2F;xyepiu&#x2F;cherry_a... and <a href="https:&#x2F;&#x2F;cherry.haishan.me&#x2F;" rel="nofollow">https:&#x2F;&#x2F;cherry.haishan.me&#x2F; and just worked from the Dockerfile
    6. danielrmay · · focus · HN ↗
      Thanks for making Hister, I&#x27;ve been using it for a few days (~7k docs) and I&#x27;m impressed so far.
    7. Anonymous106 · · focus · HN ↗
      Histerekishi &#x2F; Histereki - れきし&#x2F;歴史 means history in Japanese. reki れき&#x2F;歴 is a suffix which means (history of)

      Histeri

      MyHister(i)

      Hyster(y)

      Also, I have been using your app for two months now. I have only had to rely on it a few times, but each time I did it worked beautifully. Thank you.

    8. Beijinger · · focus · HN ↗
      &quot;It looks like our name conflicts with a registered trademark in the US. &quot;

      So? Where are you based? For what class was the trademark filed? When was it filed?

      I doubt that he has any leverage, but I don&#x27;t know the background.

    9. asdfqwertzxcv · · focus · HN ↗
      Thanks so much for creating this. Installed last time it was posted and have been loving it. The MCP server and extensions and userscripts are great QOL additions, as well. Always wondered if something was out there like this and you answered my prayers! New name suggestion: MisterHistory
    10. Beijinger · · focus · HN ↗
      &quot;No mandatory cloud - A complete personal setup can run on one local machine.&quot;

      How does it sync via several computers?

      1. al_hag · · focus · HN ↗
        Tailscale is one option
        1. michaelje · · focus · HN ↗
          I built something similar at the start of the year, using tailscale for auth (multi user on my tailnet&#x2F;home network) and for access wherever I was, whether at home or on the road (all compute&#x2F;storage was on my Mac mini.) Worked a treat.
    11. jamienk · · focus · HN ↗
      Would like: * Local web page interface or even browser UI element (since extension needed anyway) * Ability to add notes to history * Flag if bookmarked, allow filtering &quot;bookmarks only&quot; * Keep old versions of pages * Human-readable text diff vs current live page
    12. yeahdef · · focus · HN ↗
      &quot;hister 2: histlectric histerloo&quot;
      1. mxuribe · · focus · HN ↗
        OMG, you win!!! :-D
    13. swyx · · focus · HN ↗
      is this like a pihole? is there a design difference you are going for here?
    14. Cider9986 · · focus · HN ↗
      What about &quot;searchy.me&quot;?
    15. [deleted] · · focus · HN ↗

      [deleted]

    16. mxuribe · · focus · HN ↗
      Hi @asciimoo , related to a name suggestion, how about something like...

      * chronilog.org ...as in, a log of one&#x27;s chronicles.

      * And if you will include this into KDE, then can use a &#x27;k&#x27; instead, such as kronilog.org :-)

      Both seem to be available. ;-)

      1. asciimoo · · focus · HN ↗
        This is a great suggestion, thanks! I&#x27;ll definitely add it to the list of candidates. My plan is to do a vote on our social platforms if we have a few decent candidates.
        1. consumer451 · · focus · HN ↗
          Genuinely curious, how could one fight pre-emptive domain squatters once any candidate is publicly suggested?

          When I have suggested names in other situations like this in the past, I spent the ~$10 to get the domain, and offered the transfer the free. Of course, not everyone would do this.

        2. pidgeon_lover · · focus · HN ↗
          What about Krionicle? References Chronological Chronicle, cryo- (freezing knowledge for later use), and a robotic toy from 2001
        3. skripp11 · · focus · HN ↗
          Unless you are really strapped for cash, register all the candidates before you start that voting.
    17. elektor · · focus · HN ↗
      Hi asciimoo, seems like this is the second time Hister is hitting the HN front page in a month, so congrats on the success!

      Question for you: For the less tech savvy of us on here, is there any chance Hister can be can hosted on something like Pikapods? <a href="https:&#x2F;&#x2F;www.pikapods.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.pikapods.com&#x2F;

      1. asciimoo · · focus · HN ↗
        Yes, that&#x27;s something I&#x27;d like to support. The main missing piece for a user-friendly hosting option such as PikaPods is a configuration UI. At the moment, customizing Hister requires editing a configuration file, which isn&#x27;t practical for this kind of hosted service.
        1. elektor · · focus · HN ↗
          Appreciate the response! I&#x27;ll be eagerly following Hister&#x27;s progress. For now, I&#x27;ve settled on a mix of Instapaper and using SingleFile uploads to Dropbox.
          1. cyanlimetea · · focus · HN ↗

            [dead]

    18. mircea · · focus · HN ↗
      On HN does it capture both the HN post page and the target page?

      E.g.: for this submission I would want both <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49743097">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49743097 and <a href="https:&#x2F;&#x2F;github.com&#x2F;asciimoo&#x2F;hister" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;asciimoo&#x2F;hister captured.

      1. asciimoo · · focus · HN ↗
        The extension captures the content of the opened tabs, it does not create new requests. If you open both, it captures both.
        1. iririririr · · focus · HN ↗
          You may be the first &quot;search engine&quot; capable of indexing instagram and other closed sites.
          1. asciimoo · · focus · HN ↗
            Exactly, this is the biggest advantage of the extension. It is fully invisible for the websites, so no captcha, anti-bot protection, no authentication issues, every common bottleneck of a classic crawler is solved by the browser&#x2F;user.
          2. asimovDev · · focus · HN ↗
            Oh, I haven&#x27;t considered the possibility of indexing things like Instagram or Discord with this. I am now much more interested than before
          3. brunousedtowrit · · focus · HN ↗

            [dead]

          4. hipjiveguy · · focus · HN ↗
            Wouldn&#x27;t Google be doing this already as they own Chrome of course, which knows what you&#x27;re looking at?
    19. Zizizizz · · focus · HN ↗
      hisect (history and bisect)

      Seekfold (seek and manifold)

      Seekdex (seek and index)

    20. 6510 · · focus · HN ↗
      I gather rss feeds from the websites I visit and it&#x27;s hard to express how interesting they are. The gut says it borderlines some random collection but that couldn&#x27;t be more wrong. I also enjoyed YaCy, that project should have a good amount of ideas for you. I kinda end up assigning more and more bandwidth until it gets in the way and I forget to enable it again. The turtle button on some torrent clients is a good invention.
    21. fooqux · · focus · HN ↗
      Histeria
    22. smellf · · focus · HN ↗
      Srchr
    23. culi · · focus · HN ↗
      The thing I most struggle with in this domain is recalling information from videos. I watch&#x2F;listen to a lot of hour+ lectures and I rely on this website deeply:

      <a href="https:&#x2F;&#x2F;filmot.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;filmot.com&#x2F;

      It lets you search YouTube transcripts. If you could somehow integrate video transcripts into this tool, I would be extremely interested in trying it out

      1. asciimoo · · focus · HN ↗
        We have a `yt-dlp` based extractor that can fetch the subtitles&#x2F;transcripts from the sites `yt-dlp` supports: <a href="https:&#x2F;&#x2F;hister.org&#x2F;docs&#x2F;extractors#ytdlp" rel="nofollow">https:&#x2F;&#x2F;hister.org&#x2F;docs&#x2F;extractors#ytdlp
        1. culi · · focus · HN ↗
          That&#x27;s wonderful but it doesn&#x27;t affect web-browsing right? If I watch a video with this extension, the transcripts are not gonna be indexed
          1. [deleted] · · focus · HN ↗

            [deleted]

          2. [deleted] · · focus · HN ↗

            [deleted]

    24. hefner1456 · · focus · HN ↗
      May I suggest Yahoox!
    25. aaron695 · · focus · HN ↗

      [dead]

    26. usernomdeguerre · · focus · HN ↗
      Been using hister for a number of weeks now; i&#x27;m coming across sites whose content would be better handled with a custom extractor; but it looks like extractors need to be bundled into the build in order to work? Is that correct?

      Put another way, i can&#x27;t write an extractor for Reuters and then point a config to it from my current hister binary?

    27. cyanlimetea · · focus · HN ↗

      [dead]

    28. g_host · · focus · HN ↗
      I love Hister so much.

      I ran it for most of this year but encountered some problems with it I couldn&#x27;t fix and thus have not had it hooked up to anything since June when I finally couldn&#x27;t take it anymore.

      In short, I serve a good number of apps from an Nginx reverse proxy. Maybe 25% of them are exposed to the WWW while everything else is limited to the LAN but I still get valid TLS for all of it.

      Hister, though, kept breaking my whole reverse proxy and I could never figure out EXACTLY why so I could fix it. After running fine for a few days, it would hog the whole server and everything else proxied by Nginx would become unreachable. I tried tuning the config for it to no avail.

      One day when I&#x27;m less lazy, I&#x27;ll probably hook it back up via it&#x27;s LAN IP to every machine I&#x27;ve got again. I REALLY liked that I could log my browsing history from any machine anywhere in the world without a VPN and I was really disappointed when I had to disable its config in Nginx.

      I still use it a lot to go find stuff I flagged as important quickly.

      I&#x27;m curious if this is something you&#x27;ve heard of before, or if I&#x27;ve got a one off problem here.

      I even ported the config to a brand new VM with NGINX and still had the same problem.

      1. dwedge · · focus · HN ↗
        Are you sure it was the proxy and not JavaScript in your browser? I&#x27;ve seen similar behaviour from one particular website where using it in a certain way causes it to process a lot of data slowly and block the main thread. This somehow persisted across tabs, even if I closed all the tabs and tried again that site was still hanging until that process finished. But if I used incognito or another browser it would he responsive.

        It might be something similar if all of your sites are subdomains. Try incognito at the same time next time

        1. g_host · · focus · HN ↗
          That&#x27;s a good call out but it is something I tested originally and ruled out.

          I&#x27;ve got some applications used daily by friends all over the world and the services would all become unavailable to them when this started happening. Only fix I found was restarting NGINX service and then it could happen again an hour later or 3 days later. Once I removed the proxy config for Hister from the service, the issue never happened again.

          I&#x27;ll repro the issue and get the details intoa. Github Issue this weekend.

      2. asciimoo · · focus · HN ↗
        I&#x27;ve never experienced&#x2F;heard similar behavior. Could you open a GitHub issue or join our Discord&#x2F;IRC to identify the root cause?
        1. g_host · · focus · HN ↗
          Yes, I should have some time this weekend to reproduce the problem and provide useful information for ya&#x27;ll to look at...even if it&#x27;s just to determine the problem is for me to fix and not a bug with Hister.
    29. rurban · · focus · HN ↗
      histsearch
    30. traktorn · · focus · HN ↗
      Regarding the name, how about Ascimo or Askimo? :)
    31. ggm · · focus · HN ↗
      Whatever you do, try not to chose a name which collides with nostrodamus&#x27; predictions about .. (ok, he actually used &quot;Hister&quot; to refer to the Danube it seems, but popular legend has another take which is .. unfortunate)
      1. brimwats · · focus · HN ↗
        ah yes, the Chancellor of Greater Germanie
    32. darkwater · · focus · HN ↗
      I will definitely try this in my home-lab!

      And for the name, what about &quot;Historex&quot; (although already taken as well) o &quot;Histearch&quot;, mesh-up of &quot;history&quot; and &quot;search&quot;?

    33. zuminator · · focus · HN ↗
      My suggestion is Lookdown. Kind of a play on words reminiscent of Markup&#x2F;Markdown --&gt; Lookup&#x2F;Lookdown. But also a lookdown is a distinctive looking fish with a cute concave profile so you&#x27;ve got an instant mascot.
    34. amai · · focus · HN ↗
      Call your project hipster. Just a one letter change ;-)
    35. scoot · · focus · HN ↗
      Hyster (not Hister as far as I can tell) is a registered trademark in the US. Regardless:

      &quot;The HYSTER trademark is filed in the category of Education and Entertainment Services&quot; [0], so you can safely ignore any demands to rename, as it doesn&#x27;t conflict. Embarrassing for them that their lawyers don&#x27;t understand even the basics of trademark law.

      Edit to clarify as some folks here are as confused as sHyster&#x27;s lawyers: Trademarks are not exclusive, they&#x27;re restricted to a category or categories. You can be Apple in the category of computers, but not in music if there&#x27;s already an Apple in that category (unless you have $500M to spare [2]).

        [0] https:&#x2F;&#x2F;www.trademarkia.com&#x2F;hyster-77843354
  [1] https:&#x2F;&#x2F;tmsearch.uspto.gov&#x2F; for a more authoritative source than [0], but it doesn&#x27;t allow deep linking
  [2] https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Apple_Corps_v_Apple_Computer
      1. scoot · · focus · HN ↗
        &gt; Embarrassing for them that their lawyers don&#x27;t understand even the basics of trademark law

        Or more likely do, but hope that you don&#x27;t...

    36. ptaffs · · focus · HN ↗
      Not a Lawyer, but Hister is a common, old word. Unless you&#x27;re in the same business as the trademark holder, I would think you have grounds to continue? I note you have the domain name registered, and any deep-pocketed hostile trademark owner would already have reclaimed that. Unfortunately, legal advice is expensive, and the system is open to abuse.
    37. jsmo · · focus · HN ↗
      Hey thanks! Looks cool but while using the demo I searched for &quot;interest&quot; or &quot;*interest&quot; which I would expect to find <a href="https:&#x2F;&#x2F;nlnet.nl&#x2F;themes&#x2F;" rel="nofollow">https:&#x2F;&#x2F;nlnet.nl&#x2F;themes&#x2F; which was one of 28 pages with &quot;Areas of special interest&quot; as the H1 title innerText and yet the side panel populated the page for &quot;Internet&quot; (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Internet" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Internet). I assume there&#x27;s some fuzzy search stuff going on here any thoughts?
    38. SimplGy · · focus · HN ↗
      What about “terry”? (His-terry”
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.