Thread

1. killerstorm · 1789733470 · focus · HN ↗

   This is, basically, text-to-LoRA with some extra stuff.

   I.e. it basically takes text, computes and embedding and makes a LoRA adapter out of this embedding.

   Note that it is equivalent to a recurrent module attached to a transformer. Dynamically generated weights (proposed in the article) are computationally equivalent to multiplicative-gating network with fixed weights. Basically just a beefier variant of GLU operating on a slightly larger state.

Unofficial Hacker News client; not affiliated with Y Combinator.