IMO by asking Jev underspecified questions like this, you're essentially using it as a random number generator (similar to the dice example). On actual NLP problems (including ones with uncertainty under human review) it does appear to be well calibrated: <a href="https://leonardgrazian.com/blog/jev-calibration/" rel="nofollow">https://leonardgrazian.com/blog/jev-calibration/
Jev should ideally respond in a non-random way, it should just list out the probabilities.
I suppose trying to interface with the model like this is like asking an LLM how many times the letter E appears in a word - just not the correct way to ask that given its model
cannedbread · · focus · HN ↗
jezzamon · · focus · HN ↗
I suppose trying to interface with the model like this is like asking an LLM how many times the letter E appears in a word - just not the correct way to ask that given its model