Thread

AI chatbots give wrong answers to financial queries 'most of the time'

157 points · 89 comments · 1vuio0pswjnm7

ehe78qhe · 1789967399 · focus · HN ↗
Article is just a vague summary of <a href="https://www.saturnos.com/report/artificial-authority">https://www.saturnos.com/report/artificial-authority
Anecdotally, current models seem to be decent at general personal finance principles - certainly better than the majority of personal finance education that people get exposed to unless they seek it out and read a variety of books and sources. But I wouldn't trust them with direct decision making with actual money due to the training lag time on current tax policy, etc.
1. NitpickLawyer · 1789970815 · focus · HN ↗
  > certainly better than the majority of personal finance education that people get exposed to unless they seek it out and read a variety of books and sources.
  Also, models are now good enough that you can give them chapters from "authoritative" books, and they'll integrate that and come up with better answers even if their "vanilla" answers were average. And they'll tailor stuff to your particular situation. It's funny that the "agentic" stuff is only used in coding mostly, while it can and does work in other fields as well.
  As always, you kinda need to check it (at least spot check) but all in all I'd agree it's better than the average stuff you used to find with a quick google search.
  1. legostormtroopr · 1789974843 · focus · HN ↗
    So if you know a book that has the information you need, you just need to upload it into the model to get the right answer.
    Isn't that a bit circular - if you already know the authoritative source, why ask a model?
    theshrike79 · 1789977426 · focus · HN ↗
    It's basically a fancy context aware ctrl-f to the book.
    I use this regularly with RPG manuals. I _know_ the stuff, but don't remember every detail by heart. And just ctrl-f:ing through a Mörk/Pirate Borg -style PDF isn't really productive (they're "artistically" laid out). But I can just ask an AI bot that has the pdf indexed like "how does the medical kit work?" and it'll give me a summary along with the relevant rolls within seconds.
    LorenPechtel · 1790018411 · focus · HN ↗
    Is there any local version of an AI that can do this sort of thing?
    theshrike79 · 1790061077 · focus · HN ↗
    Most likely about three billion - all vibe coded and most of them with super pretty domain names and launch pages :D
    The only complicated bit is the PDF indexing. I would personally suggest using a proper cloud model in a massive datacenter to do it if the source is even a bit fuzzy. (lots of tables, fancy layouts etc). A "normal" PDF can just be converted to markdown in seconds.
    After that the local stuff can be managed by any local model that has a tool to use bash or read files.
    I have a Discord bot that can ingest a RPG PDF and stores it in a sqlite vector database for search, took me maybe an evening to build it with Claude.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.