Any lawyers here who have used AI agents heavily for their work? From what I've heard, they're currently very good at searching, analyzing and drafting documents like contracts and patents, but some say they suck at interpreting the law.
I’m sure there are a wide variety of experiences out there, but here’s my perspective as a former biglaw associate and current solo litigator:
I have had some success using frontier models from the last 6ish months, but only when I can break up my work into discrete and verifiable tasks. For example, I had ~15k pages of discovery I needed to dig through for a summary judgment motion. Instead of just asking Claude to find the best evidence, I asked it first to run a clean, high quality OCR pass (it was almost entirely PDFs). Then I had it generate embeddings and write some reusable python scripts to make keyword and semantic searching easy for agents. While I was writing the brief, I would routinely ask my agent (Claude Code) to use both keyword and semantic searching to find the best evidence supporting whatever assertion I was trying to make. I trusted it because there were traces I could follow.
In other cases/situations, I’ve tried just giving a model access to all the docs and saying “write a brief arguing X,” but it’s always terrible at this. It writes briefs with lots of evocative jargon and rhetorical flourish, but a low signal-to-noise ratio.
Again, I’m sure others’ experiences differ based on workflow, legal area, etc.
keeda · · focus · HN ↗
droidjj · · focus · HN ↗
I have had some success using frontier models from the last 6ish months, but only when I can break up my work into discrete and verifiable tasks. For example, I had ~15k pages of discovery I needed to dig through for a summary judgment motion. Instead of just asking Claude to find the best evidence, I asked it first to run a clean, high quality OCR pass (it was almost entirely PDFs). Then I had it generate embeddings and write some reusable python scripts to make keyword and semantic searching easy for agents. While I was writing the brief, I would routinely ask my agent (Claude Code) to use both keyword and semantic searching to find the best evidence supporting whatever assertion I was trying to make. I trusted it because there were traces I could follow.
In other cases/situations, I’ve tried just giving a model access to all the docs and saying “write a brief arguing X,” but it’s always terrible at this. It writes briefs with lots of evocative jargon and rhetorical flourish, but a low signal-to-noise ratio.
Again, I’m sure others’ experiences differ based on workflow, legal area, etc.
Otterly99 · · focus · HN ↗
Are they separate searches, or some sort of weighted search?