Thread

Japanese used bookstores see 5x sales surge as books are being bought by the ton

96 points · 130 comments · speckx

patall · 1790265425 · focus · HN ↗
Can someone explain why the old books could really be relevant. I get the pre-nuclear steel analogy, but why is this relevant given how much more modern texts exist. A few years ago, millions of yahoo groups were erased but now a few thousand books are what is needed to run a successful AI company? I mean, it can barely be about the information in those books (that would be very often outdated), but just for a little more text (with ever less marginal gain), what is the benefit?
1. Aurornis · 1790267046 · focus · HN ↗
  > but now a few thousand books are what is needed to run a successful AI company?
  They're scanning millions of books.
  It's the diversity of text that helps. One of the lessons we've learned is that more training data leads to better models. Even old books have different mixes of word sequences that will improve the model. The returns are diminishing, but when you have the pipeline set up to ingest it you might as well keep adding to the dataset.

Unofficial Hacker News client; not affiliated with Y Combinator.