Thread

1. linux_devil · 1785992531 · focus · HN ↗

   Why do we need to train the model to solve for retrieval within the org, so we have to keep training it whenever new dataset is introduced , or am I missing something here ?

   1. kumama · 1785994302 · focus · HN ↗

      (founder of castform here) the model you post-train should ideally learn general patterns & search strategies over your dataset that should transfer to new docs you add to the search corpus (unless its super out of distribution)

      1. srvraw · 1786006827 · focus · HN ↗

         could you share more about what you mean by "general patterns & search strategies"? I can think of it being along the lines of searching over specific tables or databases for queries in certain context. It's an exciting line of work and I'm interested because I need something like this for the problem I'm solving atm. So, I'd like to understand how the training generalizes

         1. kumama · 1786027497 · focus · HN ↗

            yup! it’s mostly about getting better at using the right search keywords.

            for more complex multi-hop question, it's also about knowing which sections of a document to look up and in what order.

Unofficial Hacker News client; not affiliated with Y Combinator.