Thread

Qwen Image 2.1

740 points · 199 comments · jmillikin

mdp2021 · 1789912749 · focus · HN ↗
How do you use this model locally, similarly to using `llama-server -m <model>`?
(I mean: outside direct or substantial use of Python, and running the Neural Network in the most efficient way.)
1. utopiah · 1789913176 · focus · HN ↗
  why not just as you suggested i.e. <a href="https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.html#llama-server" rel="nofollow">https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.... then get the result either via a UI or wget/curl it back?
  1. mdp2021 · 1789913434 · focus · HN ↗
    I am not sure that llama.cpp also supports image generation models.
    utopiah · 1789913646 · focus · HN ↗
    it's multimodal, see <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md" rel="nofollow">https://github.com/ggml-org/llama.cpp/blob/master/docs/multi...
    exe34 · 1789913964 · focus · HN ↗
    Multimodal doesn't guarantee input and output.
    > Currently, we support image, audio and video input.
    utopiah · 1789915865 · focus · HN ↗
    Seems I'm missing something. Does this model support other inputs?
    Image outputs are supported, videos I'm not sure but I don't think that's an output, just a preview of the equirectangular example, so, same question here, what does this model outputs that isn't supported?
    [deleted] · 1789916261 · focus · HN ↗
    [deleted]
    exe34 · 1789917509 · focus · HN ↗
    It's a diffusion model, completely different from autoregressive attention models.
    utopiah · 1789922089 · focus · HN ↗
    like <a href="https://github.com/ggml-org/llama.cpp/tree/master/examples/diffusion" rel="nofollow">https://github.com/ggml-org/llama.cpp/tree/master/examples/d... ?
    At the risk of stating the obvious llama.cpp isn't just about LLaMa as <a href="https://github.com/ggml-org/llama.cpp/blob/master/src/llama-arch.h#L13" rel="nofollow">https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... someone else pointed out.
    exe34 · 1789926056 · focus · HN ↗
    Aha I was wrong. Thanks for sharing that!
    utopiah · 1789926295 · focus · HN ↗
    no worries, I was wrong too, it is multimodal but only for inputs apparently, so for now there seem to only be text as output but no image as output
    mdp2021 · 1789917711 · focus · HN ↗
    Not all architectures are supported by llama.cpp . The GGUF format encodes the NN in a standardized way, but then you need code that can use that NN structure.
    I understand that llama.cpp could only output text, last time I checked (I do not know how to find a good source for that though).
    See <a href="https://github.com/ggml-org/llama.cpp/blob/master/src/llama-arch.h" rel="nofollow">https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... , the
    enum llm_arch {
    ...
    utopiah · 1789922613 · focus · HN ↗
    I haven't used llama.cpp for image generation either but I recall an issue about it. Unfortunately I can't pinpoint it now and there is the older closed issue <a href="https://github.com/ggml-org/llama.cpp/issues/4408" rel="nofollow">https://github.com/ggml-org/llama.cpp/issues/4408 so unless mtmd supports also multimodal outputs out of the box safe to assume output is still limited to text.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.