Well, since llms need matterial for self-education, someone still had to be inteligent enough. Or llms will freeze their “intelect” at some point.
Why would it freeze if they can create more training material. Its already been shown synthetic training data is just as good if not better.
> Its already been shown synthetic training data is just as good if not better.Source, please! I'm a machine learning layperson but how can this not be overfitting?
For example, an LLM can prove a new theorem in math, verify it in Lean (so it is definitely true), and then prove more things based on that.AI-generated data might not always be that useful, but at least in this case it obviously is.
hyperionultra · · focus · HN ↗
BenzeneDream · · focus · HN ↗
timcobb · · focus · HN ↗
Source, please! I'm a machine learning layperson but how can this not be overfitting?
azakai · · focus · HN ↗
AI-generated data might not always be that useful, but at least in this case it obviously is.