Thread

1. hbarka · 1789982542 · focus · HN ↗

   If Jev is fundamentally trained using RLCD while you’re building on a Qwen model that was trained using RLHF, how can the resulting model be considered Jev-like?

   1. mohsen1 · 1789986029 · focus · HN ↗

      I can't find it but saw that if you give Jev English alphabet as choices and ask it in a loop what model it is, it would say Qwen

      also tried myself: <a href="https:&#x2F;&#x2F;console.typesafe.ai&#x2F;playground?share=shr_1690a3160f19c2e4851b6a2883bcf17a790" rel="nofollow">https:&#x2F;&#x2F;console.typesafe.ai&#x2F;playground?share=shr_1690a3160f1...

      1. bityard · 1789993110 · focus · HN ↗

         That is not how models work.

         Unless specifically told in a system prompt, the pile of weights has absolutely no knowledge of itself. You could hypothetically train it to answer such questions, but nobody bothers to do this, and ALL &quot;knowledge&quot; embedded in the weights is probabalistic anyway.

         (I feel like this should be common knowledge in LLM discussions on HN by now.)

         1. spiderfarmer · 1789993477 · focus · HN ↗

            Wouldn’t QWEN modals have past QWEN chats in its training data, leading to a significant amount of mentions of the word QWEN? Just the question “what model are you” would have been answered deterministically multiple times and they’re now part of the weights.

Unofficial Hacker News client; not affiliated with Y Combinator.