Thread

Using Opus 5.5 to discover a new eyewitness record of the dodo

226 points · 79 comments · benbreen

jamienk · 1790908837 · focus · HN ↗
My dad died and had many many notebooks of his journals with very hard-to-read handwriting. Is it worth the effort to scan all of these so I can feed them in and go to work. Seems like so much minutia is out there, ready to be meta-understood.
1. komali2 · 1790910256 · focus · HN ↗
  I set up an OCR flow using local models on all my many tens of journals stretching back the last 30 years.
  I would say it's about 80% accurate, which means it's missing enough key words to make a lot of it uselessly unintelligible. I can easily compare the images against text I turn up in a grep which is nice if I'm looking for something.
  Allegedly Claude set up a system for retraining for my handwriting, but it would require me to manually revise several hundred pages by hand so I don't think I'll ever do it.
  <a href="https://github.com/508-dev/journal-ocr" rel="nofollow">https://github.com/508-dev/journal-ocr
  1. jiggawatts · 1790947594 · focus · HN ↗
    Accurate text OCR from bad handwriting is still very much a "bleeding edge" frontier capability that isn't practical with local models.
    GPT 6.1 and Gemini Flash 3.8 both do pretty well, their OCR of your sample image is only "wrong" in the sense that the original has typos and they corrected some inadvertently and/or filled in gaps where you had "unintelligible" in the canonical text.
    If you have the budget and want the best possible results, you need to run each image through multiple models and then combine the outputs into a final "merge these" prompt. Better scanning helps too, your sample image is rotated and you used a phone in low light. Try a DSLR or a flatbed scanner and process only one page at a time instead of two at once.
    jamienk · 1790948019 · focus · HN ↗
    'run each image through multiple models and then combine the outputs into a final "merge these" prompt' << How to do this? This would be an amazing workflow to get documented. This could be the start of a full-service company "send us a bunch of notebooks, get back HIGH QUALITY text version"
    jiggawatts · 1790948888 · focus · HN ↗
    Literally "just" what I outlined! That's the brilliant thing with LLM-based automation, you don't need a massive piece of complex software, just "ask" in English.
    Roughly:
    Get API keys for multiple vendors or just use OpenRouter (but availability of frontier models tends to be limited). Alternatively, Azure Foundry has everything except Google models, so just two subscriptions is enough.
    Run the same prompt and same input image through each of your chosen models.
    Then feed the smartest model the original image together with the collected output texts. Use a prompt along the lines of "Merge these attempts to OCR together into an corrected and improved combined version, taking special care to exactly preserve the original's typos, etc, etc..."
    You can do this manually, it's just fiddly. It's not hard to automate, most of the "code" is English instructions!
    The downside of this approach is the cost: even the "light" frontier models are a few cents per page, which is not so bad until you're doing this 5x or 10x times per page and suddenly scanning a notebook can set you back tens of dollars, more than buying a good novel at a book store.
    I picked up on this technique back when GPT 4 was released. People noticed that it could translate ancient Akkadian, but only if you ran the prompt through 4x times and merged. I tried this with a few random samples I found online and the merged translations were generally better than the "official" ones, even thought the individual attempts were unreadable gibberish.
    There are already scripts/tools floating around for this!
    Look into OpenRouter Fusion, Consensus AI, Multi-Model Debate, etc... or just whip up something yourself.
    komali2 · 1791002053 · focus · HN ↗
    Sounds kinda like exactly what I set up in the repo I linked in the original comment heh... Minus the openrouter calls.
    komali2 · 1791001878 · focus · HN ↗
    That's the workflow in the GitHub repo I linked, I guess the person you're replying to didn't look. You could just change the calls to be to frontier models instead of local ones if you want.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.