A few years ago anything you wanted to search for would generate hundreds of pages of results. From sources all over. If you structured your query well you could dramatically cut down the irrelevant results and then spend a bit of time skimming through the offerings until you found your target.
Searching felt like a hunt, it could take you to far flung corners of the web, and hitting the jackpot on some obscure forum was part of the fun.
Now searching anything feels incredibly frustrating. You know what you are looking for is out there somewhere but it feels impossible getting to it. Basically we have only two main routes now, Google and Bing (and I swear Bing is just copying Google’s homework). I’ve tried a handful of others and they all fall very short.
Something has changed dramatically under the hood.
It seems like the results are very shallow and no amount of prodding will get either search engine to produce anything besides the laziest output. Any subject has a limited pool of sites they lean on and then pretend that that’s all of the Internet.
It generalizes very quickly and if you try to counter by putting things in quotations it gives up.
Even in the rare moments I’ve buckled and used their stupid AI, it takes even more time browbeating the program to just deliver the damned specifics.
What is going on??
How do we get our old search back?
I swear every day of trying to navigate the web feels worse than the day before.
How did we get here?
There’s got to be something better, right? Or something we can do to pave a new structure that benefits the denizens of the web better than the lazy slop our tech overlords scrape off the table for us. What can we do?
SEO.
People with bad info put more effort into SEO than people with good info. Google changing focus from getting people good results to getting people to stay on their site. Powers that be filling the web with misinformation. There’s always been conspiracies and misinformation on the internet, but now there’s major money in pushing it.
SEO. Everybody is blaming corporations but nobody today can develop a search engine that can’t be spammed. If they could, they would.
If only we (the collective users) spent more time creating a framework and vision of how the Internet should develop and aggressively pushed back on attempts to deviate from it…
When the people refuse to convene a governing body with teeth, the vacuum is filled with opportunists.
In a sense we are suffering from our own success. A place where anybody can build anything they want digitally. Left a ripe field for unrestrained commercial interests to infest the whole thing. Now it is bloated with human and AI slop to such a degree it smothers everything else and every major tech push is tuned towards data scraping every morsel.
True, but that reeks of victim blaming. The spammers are to blame here, not the mathematical inevitability of antispammers being behind on the arms race. Also, fucking AI.
There are leaked emails from Google (from many years ago) where it was proposed to purposely degrade search quality to increase ad clicks. While some at Google responded with dismay as this, it is not a great leap to assume that this faction eventually won and they did in fact deliberately do that. Advertising is Google’s main revenue source, they are clearly motivated to do so.
Ever since the deployment of machine generated results for search engines, there has always been a competition between the search engines and various others using search engine optimization to get to the top of the results. It isn’t worth it for Alphabet or Microsoft to fix their search engines if they can push you to AI, so their non-AI results naturally degrade.
Search quality has been degrading for years, even before AI was involved.
But it feels it got much worse much faster over the last 2 years.
The one thing “AI” truly excels at.
I think they intentionally made search engines worse to push people into using LLM’s
Which are amazing now, but not to worry, LLMs will be complete dog shit too once they pull the rug.
They already are dogshit.
If your LLM prompt isn’t about something common or mainstream is has an incorrect response most of the time. Unfortunately due to how they work, they will present the incorrect answers with the same confidence as correct answers.
LLMs seem like their only use is writing reports or resumes that no one needs to read. Everything else is better done by hand.
Episode of the Better Offline podcast: The Man That Destroyed Google Search
Sheesh, the description sounds like a Behind the Bastards episode.
Can’t wait to give it a listen. Thanks
Money.
Anytime you think it’s not about the money, it’s about the money.
I’ve been using metasearch engines (engines that pass your query to multiple search engines so you can aggregate them easily) and the two I’ve been using were pretty good in my experience. 4get.ca and degoog.org.
I’ll check them out, thanks
Because data is never free. Google, duckduckgo, kagi, they all cost money one way or another whether that be a subscription service or by forcing websites to be shit so your only option is using their LLMs. Yes the reason why the internet has gone to shit is because of LLMs
I’ve went through hundreds of search results on duckduckgo, even when I make it intentionally broad and genetic that wikipedia would cover it, it still doesn’t come as the first search result. Search engines have gone to shit because LLMs make the most money in terms of data surveillance. Meaning (take for example google) they have a profile of you, they have queries to train, they have data that can be sold to advertisers. The only reason LLMs are being pushed so much isn’t because its intelligent, it’s because it makes the most money.
There is a “search engine” that hasn’t gone to shit, and that’s called marginalia search, It’s not exactly a search engine where you can type a url and it will bring you to a page, Instead it compliments your general search engine as a research aid and you can find some golden sites.
I’ve personally used it for data sheets on IC electronics and their implementations. Unfortunately I’ve found it insufferably difficult on duckduckgo (like multiple pages deep for a datasheet, something that should pop up on the first page), and I don’t trust the output of LLMs (large lying models).
So yea, hopefully this helps.
Others have addressed what I’m about to say, but there is an answer older than LLMs: advertising.
Google made an amazing search product, one used by so so many people from all over. This made them a perfect platform for advertising, and as time passed and it became their primary focus, they realized they could make even more money people spent more time on their own site.
How do you do that, when the product people are there to use is aimed to redirect you elsewhere? Make it take longer to find the thing they are looking for. Search was incentived to work less well so that users saw more ads and for longer.
Integrating AI into search results introduced new ways to keep people on Google itself, including by presenting the answers themselves directly. But it started earlier.
I’ll have to check that out. Thanks
the reason why google search results went to shit is because google did some basic maths.
99% of search queries actually ask for information that you can find on one of the top websites. only about 1% actually care about anything besides that.
so if you deliver results only due to a rather superficial algorithm, you will still retain about 99% of your userbase, but the computation cost for finding these results would go down dramatically. so you save a lot of money on electricity, while serving almost as many ads. it pays off for google.
Because “Fuck you, pay me”
SEO did this. It started before LLM and chatGPT. Blogger make money be generating trafic. So they wrote shits in a format that please search engine.
That’s why if you search “search into file Linux”, instead of just a grep command line, you end up on 2000 lines article with introduction, paragraph, explanation, blablabla. Because search engine boost this kind of content. Title, introduction paragraph that repeat the title, subsections, etc…
yeah, most slop content already existed before chatgpt and was written by humans. kind of like a student is asked to write “a minimum of 500 words for their homework assignment”, too many bloggers copied that attitude to their blogs.
To my old ears, ‘bloggers’ sounds way too much like the self-employed people with a web-site about their hobby, that they are trying to make a living through.
But there were already content farms, that paid an army of underpaid people to write something about a subject they had no clue about and plaster it with SEO keywords in order to attract ad revenue.
SEO and decent search results were having a fist fight for years, then AI showed up and shot it in the face.
If I remember correctly, around 5 or 6 years ago, Google Search leadership changed such that the ads org started running it. The focus since then has been on advertisers rather than regular users.
This is literally the definition of enshittification: the company first focuses on gaining users, then switches the focus to business customers, then squeezes both of them.
In Cory Dowctorows latest book he describes that it was revealed that google not only neglected the search function, but deliberately made it worse. If you have to search several times to find a good result, they can show you several ads.
https://pluralistic.net/2024/04/24/naming-names/
It was a triumph for enshittification, a deliberate decision to make the product worse in order to make it more profitable, under the (correct) belief that the company’s exclusivity deals to provide search everywhere from Iphones and Samsungs to Mozilla would mean that the business would face no consequences for doing so.
Wow, what a great site. I like the cut of this guy’s jib.
Doctorow is awesome. He’s a great author too. I’d recommend https://en.wikipedia.org/wiki/Walkaway_(Doctorow_novel) and https://en.wikipedia.org/wiki/Little_Brother_(Doctorow_novel)
Thanks! This is what I was thinking of, but ironically I couldn’t find the links/articles lol
