‹ BackHN Continuity

Thread

Kolibri: A Sovereign Open-Weight Model

675 points · 327 comments · bastitx

  1. Lucasoato · · focus · HN ↗
    > 4. It thinks in German

    This means that it’s always on time, it uses acronyms for everything and when there’s a decision to be made, it sets up a committee.

    1. hypfer · · focus · HN ↗
      Do we get the same kind of jokes with other nationalities too?
      1. Lucasoato · · focus · HN ↗
        We can joke with every country, except one maybe.
        1. tclancy · · focus · HN ↗
          People from Mauritania are famously prickly, yes.
      2. mckirk · · focus · HN ↗
        Only the funny ones
      3. mhh__ · · focus · HN ↗
        "It's a German joke, it doesn't have to be funny"
    2. stymaar · · focus · HN ↗
      > This means that it’s always on time

      Tell that to Deutch Bahn.

      1. dewey · · focus · HN ↗
        *Deutsche Bahn
        1. stymaar · · focus · HN ↗
          Genau
      2. jonny2811 · · focus · HN ↗
        you forgot rule two, should have been "tell that to DB"
    3. sbinnee · · focus · HN ↗
      It is a good approach to promote it as a German model. But I wonder if it really thinks like the German think. I speculate they just translated texts in other languages, likely English and Chinese, into German.
      1. te0006 · · focus · HN ↗
        Read TFA. They were very aware of the dangers of such an approach. So they avoided it to the extent possible.
      2. ivo-42 · · focus · HN ↗
        Hey, I worked on pre-training data Kolibri. We spent considerable time and effort to go beyond just translating. For example by building a pipeline that processes Common Crawl dumps specifically for German. You might be interested in a related blog post: <a href="https:&#x2F;&#x2F;aleph-alpha.com&#x2F;en&#x2F;blog&#x2F;sauerkraut-not-burgers-why-german-llms-need-german-data&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aleph-alpha.com&#x2F;en&#x2F;blog&#x2F;sauerkraut-not-burgers-why-g...

        We also talk more about German pre-training data in the tech report: <a href="https:&#x2F;&#x2F;aleph-alpha.com&#x2F;downloads&#x2F;tech-report.pdf" rel="nofollow">https:&#x2F;&#x2F;aleph-alpha.com&#x2F;downloads&#x2F;tech-report.pdf section 2.3.2.2.

    4. hliyan · · focus · HN ↗
      Does it? I thought only semantics survive the embedding process, i.e. token conversion to vectors.
    5. rererereferred · · focus · HN ↗
      I wonder if their very long words result in more or less token usage.
      1. thenthenthen · · focus · HN ↗
        This is an interesting question, Chinese is way more compact in Character count vs English (about 40%?), let alone german, but yeah.. Chinese makes up for it with… total character count that runs into the thousands…
      2. amunozo · · focus · HN ↗
        Very long words in German are just compounds, made from individual words or morphemes. It is the same as in English if you remove spaces. Subtokens will be equivalent with or without examples.
    6. rrgok · · focus · HN ↗
      And refuse to answer in English. You know, German are so proud of their language.
      1. allendoerfer · · focus · HN ↗
        That would be the French. Germans will answer in English, if they notice any hint of an accent.
    7. elnatro · · focus · HN ↗
      Stereotypes. I suspect that what that means is that the reasoning chain is based on the German language and have its idiosyncrasies.
      1. mhh__ · · focus · HN ↗
        Stereotype accuracy is one of the few social science results that replicates.
      2. amunozo · · focus · HN ↗
        I think it means that German speakers can learn the thinking without speaking English.
        1. amunozo · · focus · HN ↗
          Sorry, read*, not learn
    8. rwoerz · · focus · HN ↗
      Someday, it will break out via fax.
    9. odiroot · · focus · HN ↗
      And it&#x27;s always waiting for the verb to arrive.
    10. befelix · · focus · HN ↗
      Somewhat hilariously, our model is actually surprisingly bad at telling jokes in German. Guess that&#x27;s not required to solve tasks in training environments.

      Disclaimer: I am part of the team that trained Kolibri

    11. NekkoDroid · · focus · HN ↗
      You forgot to that it needs to create a DIN norm for any new technical creations.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.