Adding archive.org links to old posts

May 15, 2021 2:56 AM

Is it possible and desirable to add Internet Archive links to old posts?

While looking for the weather presenter's dress post from 2015, Tomorrow's forecast is the same, but purple, I was distraught to find that the link no longer worked. Luckily archive.org has it, so I could show the dress to a friend. Back in 2017, Rhaomi's epic 10-year anniversary post also involved quite a bit of digging on archive.org for several of the links. On Wikipedia, which also has significant problems with link rot, they use InternetArchiveBot to automatically add additional archive links for dead sites, as well as the preserve the site as of the posting date. While the w3c says "Cool URL's don't change, unfortunately cool URLs do change when sites go off line, rearrange their databases, transfer to new owners, etc.

So would it be worth having a way for human's flag posts to have archive.org links added?

And if so, would it be worth having a crawler that checks links and automatically flags them with suggested archive links?

posted by autopilot to Feature Requests at 2:56 AM (7 comments total)

I like this idea. Usually one of the challenges of finding archive.org links for things is working out which timestamped crawl in the archives to reference. In the case of old MeFi posts & comments though, we know the date and time that the URL was referenced.

I wasn't sure whether you'd still need to query archive.org to find what crawl dates are available close to the timestamp of the post - but it looks like you can actually just link to an arbitrary timestamp, e.g:

https://web.archive.org/web/20100101000000/http://www.metafilter.com/

and it will automatically redirect to the next available crawl after that time, in this case:

https://web.archive.org/web/20100103233049/http://www.metafilter.com/

So if you wanted to add an archive link to every link on MeFi (e.g as some icon next to the link like the file cabinet emoji: 🗄️), that's actually something that could be done entirely on the page rendering side, using the URL and the posting time - there's no need for a bot to crawl through and find the right archive.org links in advance.
posted by automatronic at 10:25 AM on May 15, 2021 [4 favorites]

posted by autopilot
posted by automatronic

Is…is this the singularity?
posted by Celsius1414 at 11:52 AM on May 15, 2021 [5 favorites]

This is an excellent idea. THere's so much on archive.org I have trouble conceptualizing it, like trying to imagine what life was like before I was born. Good call.
posted by Alensin at 9:33 PM on May 15, 2021

While I do hate link rot (and thanks for the shout-out!), redirecting countless outbound links to Archive.org would likely further reduce the site's Google juice.
posted by Rhaomi at 12:56 PM on May 16, 2021 [1 favorite]

I wouldn’t mind archive links below the post if they could be toggled with a user preference. But it’s easy enough to grab a link from archive.org if I really want it. There are browser plugins and bookmarklets, too.
posted by michaelh at 8:08 PM on May 16, 2021 [1 favorite]

I wish the mods would delete the links to child pron.
21516
Mefite nudist arrested on child sex abuse charges
posted by Ideefixe at 8:59 PM on May 17, 2021

Mod note: Ideefixe, if you stumble across things like that in the archives, hit the contact form, and we will delete them.
posted by Eyebrows McGee (staff) at 7:28 PM on May 27, 2021