I wonder if as a hack, some of the shortcomings mentioned could be addressed through prompting.
E.g. "Approach this problem iteratively. As you form a hypothesis, track the confidence you have in various explanations you're considering, what evidence you're weighing to support each, and the unresolved questions you're holding onto. Log all that for later inspection.
Be methodical when evaluating evidence and only accept facts you have verified. At every stage, gauge how much each possible next step resolves uncertainty, and discard options unlikely to advance progress. Divide the functions I described into subagents responsible for each, and coordinate with them as you work."
The worry I would have is that an LLMs stated "confidence" is probably not calibrated well - maybe asking it what evidence supports its conclusion and what evidence would change it would be a better approach?
rkagerer · · focus · HN ↗
E.g. "Approach this problem iteratively. As you form a hypothesis, track the confidence you have in various explanations you're considering, what evidence you're weighing to support each, and the unresolved questions you're holding onto. Log all that for later inspection.
Be methodical when evaluating evidence and only accept facts you have verified. At every stage, gauge how much each possible next step resolves uncertainty, and discard options unlikely to advance progress. Divide the functions I described into subagents responsible for each, and coordinate with them as you work."
apercu · · focus · HN ↗
rkagerer · · focus · HN ↗
(I'm definitely one of the bigger "AI" skeptics out there but am nonetheless fascinated by these questions).