‹ BackHN Continuity

Thread

Astra for Law

589 points · 689 comments · vertigoruntime

  1. cbg0 · · focus · HN ↗
    No word on model hallucinations in the blog post.

    <a href="https:&#x2F;&#x2F;artificialanalysis.ai&#x2F;models&#x2F;gpt-6-astra?omniscience=omniscience-hallucination-rate" rel="nofollow">https:&#x2F;&#x2F;artificialanalysis.ai&#x2F;models&#x2F;gpt-6-astra?omniscience...

    1. heaney-555 · · focus · HN ↗
      The benchmark you linked to shows GPT-6 Astra having the lowest hallucination rate of all tested models.
      1. jdiff · · focus · HN ↗
        The blog post makes no mention. If the rate isn&#x27;t 0% it should be mentioned for a field like this.
        1. TomGarden · · focus · HN ↗
          Agreed. Hallucinations and reliability are the main hurdles to anything being &#x27;agi&#x27; in my book
          1. Tanjreeve · · focus · HN ↗
            The definition of AGI is whatever a frontier lab wrote a blog about doing in the last week.
        2. simianwords · · focus · HN ↗
          why this maximalism? There&#x27;s nothing that has 0% hallucination including humans. lets use reasonable baselines.
          1. drakythe · · focus · HN ↗
            Humans can be sanctioned and fined, and eventually disbarred if they continue to lie in court filings.
            1. john_strinlai · · focus · HN ↗
              presumably that will remain true for the lawyers using the product
              1. jdiff · · focus · HN ↗
                So those lawyers will likely want to know about the likeliness, strategies to adapt, and overall mitigations for hallucinations considering that. It might be helpful if the company launching this product including such information in their blog post rather than ignoring it in a field with such a high cost of getting it wrong.
                1. john_strinlai · · focus · HN ↗
                  agreed, but i dont think there are many (any?) product launch blogs that cover all the fine details and legal minutia of the product.

                  that all typically comes in pages of terms of service, purchase contracts, SLAs, and that sort of thing. not in the initial marketing post.

                  i would like to think any lawyer wanting to use the product will either get that information or choose not use the product.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.