Thread

1. richwater · 1785957598 · focus · HN ↗

   One thing that plagues [insert current FAANG] is the large amount of corpus knowledge that is outdated/misleading or just plain wrong. I'm curious how this addresses that if it's deriving the reward function from the corpus itself.

   1. seahyinghang8 · 1785980875 · focus · HN ↗

      (founder of castform here!) - having worked at FAANG / big tech, i totally get this. our example was on gitlab's open source company handbook but i think a real company's corpus is way more messy and has many sources of truth.

      a few ideas i have yet to validate are: - prioritize recently updated docs when generating the training questions (assumption those docs are more correct than others) - actually including contradicting documents that talks about the exact same topic might be a good training example - ideally the model should surface all the relevant info it can find, and explain what it has found. (usually contradiction comes from the fact that the later document is the updated stance) - you could also mine high quality Q&A from public slack / communication channels where questions were asked and someone else in the team linked some docs / answer. those are strongly validated "ground truth" answers

      1. richwater · 1786026566 · focus · HN ↗

         Thanks! I think those could all be good heuristics.

Unofficial Hacker News client; not affiliated with Y Combinator.