Thread

1. rayiner · 1790121833 · focus · HN ↗

   > Visual FoxPro is a 32-bit program, and that decides more than it appears to.

   Has anyone studied what in the models causes this sentence structure? Is it some consequence of the architecture that leads it to blurt out a fact and then only later the significance of that fact?

   1. lloydatkinson · 1790128748 · focus · HN ↗

      Not sure but I'm getting really sad reading anything and seeing this AI speak everywhere.
   2. captn3m0 · 1790159860 · focus · HN ↗

      RLHF. The reviewers that rated the models preferred it.

      1. rayiner · 1790163942 · focus · HN ↗

         But were there other outputs that constructed sentences more naturally?
      2. Scramblejams · 1790263786 · focus · HN ↗

         A whole culture suffers due to the awful taste of a few. The more things change…
   3. antonvs · 1790215154 · focus · HN ↗

      My unsubstantiated theory is that it’s due to large amounts of human slop in the training data: marketing blurbs, slide decks, motivational writing, copy designed to sell ad impressions, posts on LinkedIn, and so on. This style is what you get when you pile all that slop together and stir vigorously, i.e. when you train a model on it.

      Sentences like the one you quoted seem to be deliberately attempting to create a hook that’s supposed to lure the reader into reading further to find out what, in this case, “that decides”. But when that deliberately manipulative style is jammed into every other sentence, the result is terrible.

Unofficial Hacker News client; not affiliated with Y Combinator.