Thread

Kolibri: A Sovereign Open-Weight Model

671 points · 327 comments · bastitx

Lucasoato · 1791034086 · focus · HN ↗
> 4. It thinks in German
This means that it’s always on time, it uses acronyms for everything and when there’s a decision to be made, it sets up a committee.
1. sbinnee · 1791035891 · focus · HN ↗
  It is a good approach to promote it as a German model. But I wonder if it really thinks like the German think. I speculate they just translated texts in other languages, likely English and Chinese, into German.
  1. te0006 · 1791050050 · focus · HN ↗
    Read TFA. They were very aware of the dangers of such an approach. So they avoided it to the extent possible.
  2. ivo-42 · 1791050592 · focus · HN ↗
    Hey, I worked on pre-training data Kolibri. We spent considerable time and effort to go beyond just translating. For example by building a pipeline that processes Common Crawl dumps specifically for German. You might be interested in a related blog post: <a href="https://aleph-alpha.com/en/blog/sauerkraut-not-burgers-why-german-llms-need-german-data/" rel="nofollow">https://aleph-alpha.com/en/blog/sauerkraut-not-burgers-why-g...
    We also talk more about German pre-training data in the tech report: <a href="https://aleph-alpha.com/downloads/tech-report.pdf" rel="nofollow">https://aleph-alpha.com/downloads/tech-report.pdf section 2.3.2.2.

Unofficial Hacker News client; not affiliated with Y Combinator.