‹ BackHN Continuity

Thread

From the creator of Redis; run LLM locally with ds4

359 points · 103 comments · fibo

  1. HoldOnAMinute · · focus · HN ↗
    How is this different from other LLM runners?
    1. pydry · · focus · HN ↗
      My instinctive reaction from the readme is that it isnt. It's apparently a vibe coded knock off of llama.CPP.
      1. jminnl · · focus · HN ↗
        The llama.cpp guys don't get nearly enough credit for their work. Though the quality of the codebase is dropping over time, it is still quite high compared to most of the alternatives, and it is still one of the most stable ways to run a large variety of models.

        Definitely worth looking at if you have only a single 5090 is ninfer, and various hardware specific forks (3090, 4090).

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.