Difficulty searching AskMeFi on Google

February 26, 2021 9:13 AM

This morning I searched Google for a recent AskMeFi post I previously read and Google wasn't able to find it and I'm wondering what gives.

The post I was looking for is this one on replacing Lastpass. I eventually found it via manually going through the recent posts in the "Computers & Internet" category.

My search in Google was entered as follows, and filtered for "Past month":
lastpass site:ask.metafilter.com

Google returned my search with: "Your search - lastpass site:ask.metafilter.com - did not match any documents."

I have done this type of Google search on AskMeFi successfully hundreds of times in the past. Since the post does exist on AskMeFi and it is from the past month, I can't fathom why Google was unable to locate it in a search.

Anyone else experience something similar? Any guesses as to why this happened or what I might do differently?

posted by Goblin Barbarian to MetaFilter-Related at 9:13 AM (12 comments total)

I've encountered this frequently--Google's index of Metafilter in general seems pretty spotty. I'd wonder what Metafilter is doing in terms of sitemaps and whether that might help.
posted by Wobbuffet at 9:27 AM on February 26, 2021

Google's index of MeFi is okay, but a lot of stuff slips through the cracks. Try searching the thread title in quotes and you'll see it in their crawl of the Computers & Internet category page, as well as the questions before and after it, but not the thread itself.
posted by Rhaomi at 9:31 AM on February 26, 2021

the google date restriction thing seems wonky. Even if the result you were looking for does not turn up, using your search without any such restriction clearly shows results within the last year but adding the within the last year restriction returns nothing for me.
posted by juv3nal at 9:35 AM on February 26, 2021 [1 favorite]

btw the one post you were looking for does show up with the same search terms/time restriction using duckduckgo.
posted by juv3nal at 9:44 AM on February 26, 2021 [4 favorites]

Just searching the last 20 FPPs I've put up, there's definitely something I don't understand about Google's results for this one. When I search for its title in quotes, I get 8 results--none from Metafilter, but a couple that look like other sites using Metafilter content. When I add site:metafilter.com, I get no results. When I search for the first line of the post text, it's the top result.
posted by Wobbuffet at 9:45 AM on February 26, 2021

Incidentally, searching for the next 10, I can't find this one or this one at all. So my rough guess right now would be a 10% miss rate.
posted by Wobbuffet at 10:00 AM on February 26, 2021

So I see something that might definitely be a problem for Metafilter Google SEO, quickly looking at the markup of a random AskMe question, cortex et al:

Even if I view the page (anonymously) via HTTPS, and see the page source, or get the page via curl, I see a *lot* of http:// links, including the

<link rel="canonical" ...>

being *http* instead of *https*. Google prefers https over http but only sometimes listens to canonical tags.

To test that a bit, I tried a few searches in site:ask.metafilter.com, clicked on result with the network inspector turned on and seeing a reference first to http://ask.metafilter.com/444/foo which returns a 301 redirect to https://ask.metafilter.com/444/foo. Which in my experience google doesn't like and could result in some weirdness in getting things into the index or being ranked highly.

So, there might be a real chance that fixing the canonical tag and other meta tags to be https might be a good idea since my impression is that un-logged in viewing always redirects to https now.
posted by skynxnex at 12:01 PM on February 26, 2021 [10 favorites]

I prefer the search here. It lets me first target the post, then more broadly look for comments if I want. Though I'd probably check tags first.
posted by Pronoiac at 12:03 PM on February 26, 2021 [1 favorite]

When I search for its title in quotes, I get 8 results--none from Metafilter,

Whereas when I do that search, my first result is the thread on Metafilter (incognito or not). With site:metafilter.com, I _only_ get the original post as a result. (Same results with multiple browsers, both over my home ISP and mobile network). So in general it seems to work as expected, but not for you or maybe some other people.
posted by thefoxgod at 2:15 PM on February 26, 2021 [1 favorite]

So in general it seems to work as expected, but not for you or maybe some other people

Sort of--turns out those results changed between this morning and now, and I see what you see too. The missing ones are still missing though.
posted by Wobbuffet at 2:25 PM on February 26, 2021

Google has been doing ... creative ... things with their search results in general to be "helpful." Which results in a bunch of stuff not getting included in results when previously they would have. I've noticed this recently when searching for stuff on Reddit and Stack Overflow as well. For example, searching a site for foo and bar might not return a result that appears in a search for foo, bar, and biff, even though the latter should be a subset of the former.
posted by Candleman at 7:47 PM on February 26, 2021 [2 favorites]

"ask metafilter lastpass replacement" turned up that question as the first result.
posted by GuyZero at 12:12 PM on March 1, 2021