People keep asking for this w/ Codex, too, and I really regret that both labs seem inclined to listen.
If you ask a thinking/research type question in 'Chat' versus 'Work' mode in these products -- say, something complex about politics, or do a multiturn business strategy, or want to work thru a new concept, you get very different answers.
The harness, steering, etc. in the chat/reasoning products is so much better for this type of question (that doesn't require code as a primary substrate).
Even as someone who mainlines like, 7 coding agents at all times, I regret that productivity fever will mean the regression of think-first-act-later AI UX.
LLMs are inherently deterministic. The way everyone deploys LLMs leads to non-deterministic results, but there's nothing† stopping providers from offering deterministic evaluation if they choose.
All the sources of randomness are under the control of the provider, even if today's deployment structures mean providers introduce extra randomness due to the concurrent nature of the evaluation. Serialise the computation, feed it from a pRNG, and you have a fully deterministic result. But providers don't want to offer a deterministic result, and especially not one as fragile, expensive, and inefficient as a full serialisation would be.
†: For variants of "nothing" that include cost and deployment challenges.
This is technically true, but when people talk about randomness, its not only about same input-> different output, like temperature>0 and the things you said.
Its also about very similar inputs -> different outputs. Even with everything you said, yes, same input would result consistently into same output, but sliightly different input and you might get completely different/semantic answer.
Sherveen · · focus · HN ↗
If you ask a thinking/research type question in 'Chat' versus 'Work' mode in these products -- say, something complex about politics, or do a multiturn business strategy, or want to work thru a new concept, you get very different answers.
The harness, steering, etc. in the chat/reasoning products is so much better for this type of question (that doesn't require code as a primary substrate).
Even as someone who mainlines like, 7 coding agents at all times, I regret that productivity fever will mean the regression of think-first-act-later AI UX.
steve1977 · · focus · HN ↗
firmretention · · focus · HN ↗
andrewaylett · · focus · HN ↗
All the sources of randomness are under the control of the provider, even if today's deployment structures mean providers introduce extra randomness due to the concurrent nature of the evaluation. Serialise the computation, feed it from a pRNG, and you have a fully deterministic result. But providers don't want to offer a deterministic result, and especially not one as fragile, expensive, and inefficient as a full serialisation would be.
†: For variants of "nothing" that include cost and deployment challenges.
MikhailTal · · focus · HN ↗
Its also about very similar inputs -> different outputs. Even with everything you said, yes, same input would result consistently into same output, but sliightly different input and you might get completely different/semantic answer.
andrewaylett · · focus · HN ↗