Yesterday I tried to google "can the Halifax Wanderers still make the CPL playoffs?"
So obviously what appears right at the top is the AI summary, which told me "they've already secured their #4 position and made the playoffs". I knew this wasn't true, and I guess I could have just scrolled down a bit further and found my answer but now I was curious.
So I said "that's not true, they're still #5, what I want to know is _could they still make the playoffs_"
It says they've got an upcoming game against Ottawa, and if they win their chances are good. That game has already taken place, so I correct it again and finally I get a reasonable answer.
My question is: what's the point of the AI in the search engine if it itself isn't going to use the search engine first before answering? Like, I can't wrap my head around that. The answer is on the same page as its hallucination. It could have done a cursory look around before first hallucinating something completely false, and when corrected the first time giving me outdated information. It's meant to be A SEARCH ENGINE!
The AI answer dominates the page, and is usually wrong, so it's just a waste of the most valuable space on the page.
The ads have proliferated, and rarely have what I need.
Video search results also take up a lot of the page, and are almost never what I want if I haven't clicked over to the Video tab. Again, just a big waste of space.
Speaking of the Video tab its accuracy/relevancy has gotten worse too.
The organic results have deteriorated, and are still competitive, but are no longer clearly and consistently better than Duckduckgo.
So it's duckduckgo as the default for me on all devices now. If I don't find what I want, which is maybe 30% of the time, I switch over to G just by adding !g to the query. G might have what I need in about half of those cases.
You can see why a lot of people just ask an LLM to do the web searching for them.
Across the board, this is all quite bad compared to web search from 10+ years ago.
If there's a single thing which has really pushed me away from Google, it's simply google.com devoting so much of the page to things that aren't organic web search.
The AI answer dominates the page, and is usually wrong, so it's just a waste of the most valuable space on the page.
I started noticing that for a lot of niche questions, AI overview often uses things like Reddit comments as the source, which are often partially or completely wrong. Then it gets reformulated with the typical confidence of an LLM.
Very scary because at the same time I notice people taking the AI overview answers as truth.
The web search gives the impression that it's grounding its answers, but if you ask it for something new or non-existent it'll still make shit up.
I'm in several facebook groups for cars and bikes, and holy smokes, someone asks a question and usually the top response is someone literally pasting in a screenshot of a google AI generated response....which is wrong like, solid 50% of the time. And then you point it out and people get aggro on you. I hate it.
I have literally seen multiple screenshots now of people pasting Google Gemini answers in an attempt to tell other people whether something they're doing is legal or not in a particular state, as it relates to NFA item firearms in the USA... And they seem to take it as the gospel truth. People are literally risking federal prison over information that Gemini pulled out of the ass-end of some internet forum of multiple cletus-intellect level individuals talking about their homemade 3D printed silencers. And then Gemini further hallucinates and fabricates new ways to parse that information different than how the original person wrote it. It's astonishing.
This phenomenon is so widespread and I can't see it as anything other than serious AI psychosis.
I think people want to feel useful and helpful in general, and now a lot of people are getting this feeling by using AI to answer questions. And it's like you said they get so mad when you point out that their AI screenshot is wrong or, even more obviously, that the person who asked could also just read the google gemini blurb. They get mad because you're invalidating their feeling of being helpful.
Last year someone went missing on the water in my area. There was a lot of talk on facebook during the search, and a coordinated facebook group. And every post would have people posting their chats with ChatGPT, or saying they used AI to analyze the weather and currents on the day he went missing to figure out he probably landed on this or that shore (he didn't). These posts are equivalent to the mediums and psychics who also come in with these unhelpful advice posts.
I think fundamentally it's the same as how AI is giving some people an extremely strong feeling of being an expert, and how their psyche seems to lash out to protect that feeling when it's threatened.
Another facebook anecdote, I recently saw someone post in a local genealogy and history group some pure AI slop infographics. When people called him out he said that it's AI but he carefully vets all the information and that they need to stop being luddites or they'll be stuck in the past blah blah devolving into calling them liberals and the like. But his infographics were all incredibly obviously wrong, to anyone who had ever seen a map it's obvious that the coastlines were incredibly wrong, and the city weren't in the correct locations at all. So what can you even do with someone like that? They're just going to retreat into the safety of the chatbot to continue their "research".
Whatever the tool, the problem is that it’s wrong, not that it’s done by AI. Like, why are we conflating the two - is it because it’s now easier for someone to produce a wrong artifact that visually looks ok and therefore tricks our brains? And people get upset because a prior quality heuristic is now obviously broken?
Yeah I think so. I mean it's a new social norm - previously if you replied with "hey I read this article here and it says X" it would be really easy to check the source and either say yeah this looks good or disprove it.
Now people will say "I asked AI to give me a map of ocean currents and it definitively says the person is here!" and the argument that "look it's AI, it's probably wrong" just does nothing for a lot of people. Doubly so if the information they want is at the forefront of Google, after all, Google is a big reputable company, so of course what it says is correct.
Yes I agree that the AI itself isn't the problem, but the way it's being pushed and promoted(just ask anything!) is leading to this.
If you say it's obviously AI nonsense you get a reply that "AI is just a tool, it's part of our daily life now, criticize the content and not what tool was used", you see that so much on here. But then if you try to criticize the content the person won't believe you because the AI says it's correct.
Because this new tool will also tell you, in (mostly) human language, why the wrong artifacts it createds are correct. And it's arguments are very convincing to a lot of people. AI is clearly creating new problems that didn't really exist before, you can't say it's just a tool like any other when this is the only tool where its users form parasocial relationships with it.
It's not only the only tool that tries to manipulate users, it's also one that is certain to lie often and at random. Lies are inherent to the technology. It's a bad tool for the things it's being used for.
The actual scary part is that people already manipulate and poison the google overviews on purpose.
It is very easy to bring your bs to the TOP spot on google at the moment with carefully crafted reddit comments and blogs, and it's not hard to get to the traffic and clicks this way.
This is what the new SEO is about, I know a bunch of people who are doing this and they are praying it is not getting fixed anytime soon.
Yeah, but even religion is not up to the task for how backwards people's thinking gets at times.
You have people here in this thread - and in every other AI thread - seeing themselves as exploited for information for LLM companies to profit. This is such a backwards, scale-inverting perspective that it shouldn't be possible to hold in a rational mind, and yet.
What a sleek way of positioning your comment as something that can't reasonably be disagreed with whereas it's nothing more than a pure opinion of yours. I am in awe.
> I'm currently reading a book on a local mafia boss, and the best the church could do, when he was finally popped, was refuse to host his funeral.
Not to defend the church, but what else should they have done?
If I type exact document identifiers, the LLM either tells me that I have either mistyped something or that I am searching for a specific document. Hell yeah, I am doing exactly that! Give me the link instead of babbling about it!
Even scarier: my dad, who has been actually been getting good advice about his blood pressure issues (hair-trigger too low with his meds). It's at least coming up with citations from Cleveland Clinic and such, but still, it scares me, even if he has a good sense of how to ask it questions.
I started noticing that for a lot of niche questions, AI overview often uses things like Reddit comments as the source, which are often partially or completely wrong.
If only! I've seen it sourcing reddit comments, but when you follow the link, the comment is either completely unrelated or does not say what the LLM claims it is saying.
In most instances where I've seen it, it's on niche topics where a Reddit post may be the easiest place to find any relevant content.
I expect somewhere in the AI harness there's a branch that effectively mandates "You must always answer."
Which results in it grasping at Reddit comment straws.
And which, in typically modern Google fashion, nobody there gives a shit about fixing, because the PR made their top-level AI KPI go brr.
Google is increasingly proof that you can't run an effective scaled tech company without a Gates or Jobs who occasionally uses the fucking product themselves and descends with fiery vengeance on whatever team drifted too far from user alignment.
>it's on niche topics where a Reddit post may be the easiest place to find any relevant content.
But isn't the whole point (I mean, aside from making infinite money or world domination or whatever) of the AI that it'll do the hard work of finding the information? Like, that's the whole value proposition of the AI summary in the search results, right?
How would somebody go about mass-poisoning forum posts like this? If Google has a business relationship with Reddit, I assume it wouldn't be server side - but at the same time I don't know how a regular redditor could poison his text posts without it being noticeable in a human reading.
Normally. That's literally what SEO has been about, even pre-LLMs. It's not Google doing it, they don't benefit from their own AI suggestions being bad. Reddit doesn't have any obvious interest here too. But the millions of businesses and millions of people angry at AI, both have plenty of reasons to spam the life out of any site that allows users to post comments, and in both cases, LLMs themselves make it harder to filter and much easier to scale.
This is another area where DDG shines - it doesn't have a commercial relationship with Reddit, so doesn't push it in its results.
DDG's 'Search Assist' AI overviews are pretty basic but they tend to be sourced from Wikipedia and traditional media outlets rather than forums and blogs - they can be out of date, but are generally a lot less "spicy" than the sources that Google use.
I guess Google is much better-optimised for engagement but that's not what I'm looking for, meaning that DDG is a better choice for me.
I've been fairly impressed by DDG for a few years now. I find everything I'm looking for just fine, and the AI summaries are actually helpful on occasion.
Poisoning the well with fake/made up reddit posts is incredibly common these days.
Certain people have realised that if you start feeding the machine this poisoned data eventually it reaches millions of eyeballs. It's the new SEO.
For the first time in history, the Reddit community is actually useful. And they don't even have to change anything. Just be "the Reddit community". Always with a self confidence three levels higher than the underlying substance allows. Threads that go deeper and deeeeeper and deeeeeeeeeper without ever carrying anything of actual value.
Future scientist will find out: Most of the weaknesses AI has come from Reddit, Instagram and Youtube being part of the training material. Sure it makes the training material broader by a large degree. But not deeper, to say it nicely.
Hugsbox · · focus · HN ↗
So obviously what appears right at the top is the AI summary, which told me "they've already secured their #4 position and made the playoffs". I knew this wasn't true, and I guess I could have just scrolled down a bit further and found my answer but now I was curious.
So I said "that's not true, they're still #5, what I want to know is _could they still make the playoffs_"
It says they've got an upcoming game against Ottawa, and if they win their chances are good. That game has already taken place, so I correct it again and finally I get a reasonable answer.
My question is: what's the point of the AI in the search engine if it itself isn't going to use the search engine first before answering? Like, I can't wrap my head around that. The answer is on the same page as its hallucination. It could have done a cursory look around before first hallucinating something completely false, and when corrected the first time giving me outdated information. It's meant to be A SEARCH ENGINE!
safety1st · · focus · HN ↗
The AI answer dominates the page, and is usually wrong, so it's just a waste of the most valuable space on the page.
The ads have proliferated, and rarely have what I need.
Video search results also take up a lot of the page, and are almost never what I want if I haven't clicked over to the Video tab. Again, just a big waste of space.
Speaking of the Video tab its accuracy/relevancy has gotten worse too.
The organic results have deteriorated, and are still competitive, but are no longer clearly and consistently better than Duckduckgo.
So it's duckduckgo as the default for me on all devices now. If I don't find what I want, which is maybe 30% of the time, I switch over to G just by adding !g to the query. G might have what I need in about half of those cases.
You can see why a lot of people just ask an LLM to do the web searching for them.
Across the board, this is all quite bad compared to web search from 10+ years ago.
If there's a single thing which has really pushed me away from Google, it's simply google.com devoting so much of the page to things that aren't organic web search.
microtonal · · focus · HN ↗
I started noticing that for a lot of niche questions, AI overview often uses things like Reddit comments as the source, which are often partially or completely wrong. Then it gets reformulated with the typical confidence of an LLM.
Very scary because at the same time I notice people taking the AI overview answers as truth.
kibibu · · focus · HN ↗
swozey · · focus · HN ↗
gambiting · · focus · HN ↗
walrus01 · · focus · HN ↗
lukan · · focus · HN ↗
Obscurity4340 · · focus · HN ↗
ethbr1 · · focus · HN ↗
Makes you wonder if the 20s will be looked back on like the 80s
Obscurity4340 · · focus · HN ↗
wartywhoa23 · · focus · HN ↗
Obscurity4340 · · focus · HN ↗
zem · · focus · HN ↗
Obscurity4340 · · focus · HN ↗
JDups · · focus · HN ↗
I think people want to feel useful and helpful in general, and now a lot of people are getting this feeling by using AI to answer questions. And it's like you said they get so mad when you point out that their AI screenshot is wrong or, even more obviously, that the person who asked could also just read the google gemini blurb. They get mad because you're invalidating their feeling of being helpful.
Last year someone went missing on the water in my area. There was a lot of talk on facebook during the search, and a coordinated facebook group. And every post would have people posting their chats with ChatGPT, or saying they used AI to analyze the weather and currents on the day he went missing to figure out he probably landed on this or that shore (he didn't). These posts are equivalent to the mediums and psychics who also come in with these unhelpful advice posts.
I think fundamentally it's the same as how AI is giving some people an extremely strong feeling of being an expert, and how their psyche seems to lash out to protect that feeling when it's threatened.
Another facebook anecdote, I recently saw someone post in a local genealogy and history group some pure AI slop infographics. When people called him out he said that it's AI but he carefully vets all the information and that they need to stop being luddites or they'll be stuck in the past blah blah devolving into calling them liberals and the like. But his infographics were all incredibly obviously wrong, to anyone who had ever seen a map it's obvious that the coastlines were incredibly wrong, and the city weren't in the correct locations at all. So what can you even do with someone like that? They're just going to retreat into the safety of the chatbot to continue their "research".
edmundsauto · · focus · HN ↗
gambiting · · focus · HN ↗
Now people will say "I asked AI to give me a map of ocean currents and it definitively says the person is here!" and the argument that "look it's AI, it's probably wrong" just does nothing for a lot of people. Doubly so if the information they want is at the forefront of Google, after all, Google is a big reputable company, so of course what it says is correct.
Yes I agree that the AI itself isn't the problem, but the way it's being pushed and promoted(just ask anything!) is leading to this.
JDups · · focus · HN ↗
JDups · · focus · HN ↗
edmundsauto · · focus · HN ↗
kergonath · · focus · HN ↗
autoexec · · focus · HN ↗
deaton · · focus · HN ↗
gloosx · · focus · HN ↗
It is very easy to bring your bs to the TOP spot on google at the moment with carefully crafted reddit comments and blogs, and it's not hard to get to the traffic and clicks this way.
This is what the new SEO is about, I know a bunch of people who are doing this and they are praying it is not getting fixed anytime soon.
latentsea · · focus · HN ↗
phoronixrly · · focus · HN ↗
tripzilch · · focus · HN ↗
dinkblam · · focus · HN ↗
TeMPOraL · · focus · HN ↗
You have people here in this thread - and in every other AI thread - seeing themselves as exploited for information for LLM companies to profit. This is such a backwards, scale-inverting perspective that it shouldn't be possible to hold in a rational mind, and yet.
Matl · · focus · HN ↗
TeMPOraL · · focus · HN ↗
wartywhoa23 · · focus · HN ↗
And what's really going on then, according to your very rational mind?
smallmancontrov · · focus · HN ↗
creaturemachine · · focus · HN ↗
JadeNB · · focus · HN ↗
Not to defend the church, but what else should they have done?
latentsea · · focus · HN ↗
latentsea · · focus · HN ↗
com2kid · · focus · HN ↗
My random blog was highly ranked for a fair number of technical topics.
anilakar · · focus · HN ↗
breakingcups · · focus · HN ↗
motbus3 · · focus · HN ↗
flyinghamster · · focus · HN ↗
krige · · focus · HN ↗
If only! I've seen it sourcing reddit comments, but when you follow the link, the comment is either completely unrelated or does not say what the LLM claims it is saying.
kombookcha · · focus · HN ↗
ethbr1 · · focus · HN ↗
I expect somewhere in the AI harness there's a branch that effectively mandates "You must always answer."
Which results in it grasping at Reddit comment straws.
And which, in typically modern Google fashion, nobody there gives a shit about fixing, because the PR made their top-level AI KPI go brr.
Google is increasingly proof that you can't run an effective scaled tech company without a Gates or Jobs who occasionally uses the fucking product themselves and descends with fiery vengeance on whatever team drifted too far from user alignment.
thfuran · · focus · HN ↗
But isn't the whole point (I mean, aside from making infinite money or world domination or whatever) of the AI that it'll do the hard work of finding the information? Like, that's the whole value proposition of the AI summary in the search results, right?
ethbr1 · · focus · HN ↗
And is still likely wrecking their query unit economics in order to avoid hemorrhaging search to LLMs.
Propping search usage up and putting the expense in the massive AI capex probably looks better to investors though.
TeMPOraL · · focus · HN ↗
kombookcha · · focus · HN ↗
TeMPOraL · · focus · HN ↗
roryirvine · · focus · HN ↗
DDG's 'Search Assist' AI overviews are pretty basic but they tend to be sourced from Wikipedia and traditional media outlets rather than forums and blogs - they can be out of date, but are generally a lot less "spicy" than the sources that Google use.
I guess Google is much better-optimised for engagement but that's not what I'm looking for, meaning that DDG is a better choice for me.
seanw444 · · focus · HN ↗
Groxx · · focus · HN ↗
techdmn · · focus · HN ↗
One of the most disappointing things to me is people I previously respected appealing to the authority of LLMs.
Grimburger · · focus · HN ↗
Certain people have realised that if you start feeding the machine this poisoned data eventually it reaches millions of eyeballs. It's the new SEO.
bitmasher9 · · focus · HN ↗
pino83 · · focus · HN ↗
Future scientist will find out: Most of the weaknesses AI has come from Reddit, Instagram and Youtube being part of the training material. Sure it makes the training material broader by a large degree. But not deeper, to say it nicely.
someonebaggy · · focus · HN ↗
pino83 · · focus · HN ↗