Userpages closed to indexing? January 23, 2005 2:01 PM Subscribe
Revisiting this.
Metafilter is the biggest site that I can edit a page (my user page) on. I'd like to have "I am the [my name] that went to so and so school, and so and so high-school and did so-and-so" so that if someone googled my name looking for me, they could find that. However, userpages are marked NOINDEX for robots, for to stop villians from using google to harvest stuff, I assume. Given that it would be trivial to write a script to go through all the user pages and snarf anything, and given that these are The Internets, is there a reasonable expectation that an email address or other information you put on a publicly viewable page will *not* be harvested? Doesn't everyone have a spam-catcher address for this purpose?
Metafilter is the biggest site that I can edit a page (my user page) on. I'd like to have "I am the [my name] that went to so and so school, and so and so high-school and did so-and-so" so that if someone googled my name looking for me, they could find that. However, userpages are marked NOINDEX for robots, for to stop villians from using google to harvest stuff, I assume. Given that it would be trivial to write a script to go through all the user pages and snarf anything, and given that these are The Internets, is there a reasonable expectation that an email address or other information you put on a publicly viewable page will *not* be harvested? Doesn't everyone have a spam-catcher address for this purpose?
As long as it's on the web somewhere, and hyperlinked from somewhere else, Google will find it. Unless you have a very common name, having the info on your Mefi user page won't make much of a difference.
posted by Saucy Intruder at 2:32 PM on January 23, 2005
posted by Saucy Intruder at 2:32 PM on January 23, 2005
It's right that the userpages aren't cached as Metafilter has quite a high pagerank. It'd be trivial to fill your userpage up with links to viagra sites, and if the pages were indexed, only google would notice.
The robots.txt setting should stay. If you want people to find you on the internet, then (to use a common phrase), "get your own fucking blog"
More importantly, I think that the "comments by users" pages should also not be indexed. Every time I try and find anything on metafilter using google it always leads me to one of these pages. This must (a) impact the server performance, and (b) isn't a satisfactory result for a search.
posted by seanyboy at 2:40 PM on January 23, 2005
The robots.txt setting should stay. If you want people to find you on the internet, then (to use a common phrase), "get your own fucking blog"
More importantly, I think that the "comments by users" pages should also not be indexed. Every time I try and find anything on metafilter using google it always leads me to one of these pages. This must (a) impact the server performance, and (b) isn't a satisfactory result for a search.
posted by seanyboy at 2:40 PM on January 23, 2005
Slightly off-topic, but inspired by the link to the wiki - how often is the [links.mefi] page compiled? I'm on there, but with an old URL that I removed from my profile around a year ago.
As long as it's on the web somewhere, and hyperlinked from somewhere else, Google will find it
After 11 months of waiting, I'm not convinced this is 100% true.
posted by nylon at 2:45 PM on January 23, 2005
As long as it's on the web somewhere, and hyperlinked from somewhere else, Google will find it
After 11 months of waiting, I'm not convinced this is 100% true.
posted by nylon at 2:45 PM on January 23, 2005
nylon, you're not trying hard enough.
I bet if I linked to your site, you'd be there in a month.
posted by seanyboy at 2:50 PM on January 23, 2005
I bet if I linked to your site, you'd be there in a month.
posted by seanyboy at 2:50 PM on January 23, 2005
I'm actually pretty glad that Googling my real name doesn't find my Metafilter user page. Please leave it that way.
posted by nicwolff at 8:00 PM on January 23, 2005
posted by nicwolff at 8:00 PM on January 23, 2005
Agreeing with seanyboy, I find the "comments by users" pages every time I google+metafilter something and it's not very good, I mean I do find the thread I'm looking for eventually by scrolling that page, it would just be neater to find the thread at once instead. ;)
posted by dabitch at 3:21 AM on January 24, 2005
posted by dabitch at 3:21 AM on January 24, 2005
You are not logged in, either login or create an account to post comments
posted by Capn at 2:14 PM on January 23, 2005