No, whether or not it's intentional, maybe can be debated. But there's definitely an experience of a model losing horsepower quickly after launch.
Has there ever been any measurement of this, of any sort? Honest question. I frequently see a plural of anecdotes to that effect, but I've not seen a concrete statement of fact or measurement that could be scrutinized or tested in any way.
If so, please share. This should be measurable, and I'm glad this project is measuring it.
Answers in the form of additional anecdotes, stated with even greater passion but still lacking a statement that could be tested and falsified, would validate my exact concern.
Official tweet from Tibo from OpenAI confirming they ran experiments tweaking “juice” (reasoning effort mapping values) and have reverted them: <a href="https://x.com/thsottiaux/status/2076495156757577895" rel="nofollow">https://x.com/thsottiaux/status/2076495156757577895
As he confirmed, for at least a period of time, and for some users, “Sol xhigh” was actually “Sol high”, etc.
solenoid0937 · · focus · HN ↗
solfox · · focus · HN ↗
jyoung8607 · · focus · HN ↗
If so, please share. This should be measurable, and I'm glad this project is measuring it.
Answers in the form of additional anecdotes, stated with even greater passion but still lacking a statement that could be tested and falsified, would validate my exact concern.
dannyw · · focus · HN ↗
As he confirmed, for at least a period of time, and for some users, “Sol xhigh” was actually “Sol high”, etc.