We Need a Hero March 14, 2013 8:38 AM Subscribe

@mathowie: "I think RSS is so important that I'd take a job (leaving MeFi) at any startup aiming to make an improved Google Reader (w/ social features). [...] I'm serious, and feel free to email me. MetaFilter can continue with the employees running things."

I think that this maybe merits some serious discussion here, pronto.

So I slept on it, had a scrumptious muffin, did some fancy meditation exercises, and yep, STILL REALLY REALLY ANGRY ABOUT GOOGLE READER.

And I'm not the only one. pb is "extremely upset." Warren Ellis is losing "the most effective tool [he] has." Ezra Klein uses it "more than any single web site or app save Gmail." Felicia Day is having none of it. Small-scale blogs are in danger. A Change.org petition has surpassed 50,000 signatures. Reader's death trended above a new pope on Twitter. It seems Google has kicked the entire blogosphere in the collective chin.

But a light shineth in the darkness. Clearly, a competent, no-frills feed reader is vitally important to not only the sort of people who read Metafilter -- bloggers, webmasters, curators, journalists -- but to the founder himself, to the point he'd be willing to leave the project he's built for 13 years. But there's no need to split the gang up, right? So why don't we game it out here.

Questions to consider:

Is this something that MeFi would be capable of hosting?

Should it be in-house, or could we partner with an existing project?

Would it even be possible to persuade Google to open-source Reader's code?

Exactly how much of Reader would we be able to replicate? Interface? The fetch mechanism? APIs?

Would it be available to everyone, or just registered Mefites?

Would it charge a fee? I'd be willing to pay a subscription for a trustworthy Reader clone hosted by a site as longstanding as MeFi.

Most importantly, names... read.metafilter.com? feeds.metafilter.com? rss.metafilter.com?

It's a big endeavor, but it would be fit right into Metafilter's mission to filter the best of the web, while providing a big beacon for all the literate, web-savvy people left out in the cold by Google's caprice. There are plenty of alternatives springing up, but MeFi could provide the imprimatur of reputation and experience to make for the very best solution. It could be the smart, effective alternative to Reader, just like AskMe is to Yahoo Answers or Answers.com.

We have the userbase, the technical knowhow, the desire, the connections. Let's make it happen. For the sake of future FPP sources everywhere, let's help save the small-scale web.

posted by Rhaomi to MetaFilter-Related at 8:38 AM (198 comments total) 33 users marked this as a favorite

What kind of muffin was it?
posted by Brandon Blatcher at 8:39 AM on March 14, 2013 [63 favorites]

Banana nut. Because Google is being both nutty and bananas, amirite?
posted by Rhaomi at 8:41 AM on March 14, 2013 [14 favorites]

I would pay cash money for a Mefi RSS reader, one off fee or monthly. If it synced with desktop clients I'd be absolutely delighted.
posted by Happy Dave at 8:43 AM on March 14, 2013 [4 favorites]

I've been quite impressed by Feedly after giving it a whirl tonight and this morning.
posted by Rock Steady at 8:43 AM on March 14, 2013 [1 favorite]

Yes, please. Let's make this happen, or "adopt" a service.
posted by boo_radley at 8:44 AM on March 14, 2013

Nah, let him take a job working on this elsewhere, we can be free of this terrible tyranny!
posted by Drinky Die at 8:44 AM on March 14, 2013 [2 favorites]

Also, minor(?) semantic point:

It's a big endeavor, but it would be fit right into Metafilter's mission to filter the best of the web...

Is that Metafilter's mission? Haven't the mods explicitly said that posts or links are not about being the "best of the web" per se?

Otherwise, creating an open source version of the reader would be really good and sort of a no brainer. *fingers crossed*
posted by Brandon Blatcher at 8:44 AM on March 14, 2013

Would it charge a fee? I'd be willing to pay a subscription for a trustworthy Reader clone hosted by a site as longstanding as MeFi.

Yes. Because the alternative is to be beholden to ads or otherwise incentivised to act against your user's interests. Which is OK if you're aiming for casual users, but a serious tool like the feed reader should have the guarantee of stability that a sustainable funding model would provide.
posted by acb at 8:44 AM on March 14, 2013 [5 favorites]

This is the best idea-- then I only have to go to one website ever. Before, I went to two. Please take my money!
posted by goHermGO at 8:45 AM on March 14, 2013 [19 favorites]

Man I'd pay small amounts of money on a regular basis for a Google Reader clone.

Meanwhile, Feedly rubs me the wrong way with its unclear separation of different blogs, NetVibes is hideous, and NewsBlur and The Old Reader refuse to import for the moment.
posted by griphus at 8:46 AM on March 14, 2013 [1 favorite]

I care about RSS, but TBH I care about MeFi more. Which raises concerns on both sides of this for me. The idea of Mathowie leaving gives me the willies. On the other hand, I don't want MeFi's community to be overwhelmed and dissipated if it introduces an RSS service that becomes the new Reader. Perhaps I'm overreacting.
posted by snuffleupagus at 8:46 AM on March 14, 2013 [2 favorites]

We Need a Hero

I'M HOLDING OUT FOR HERO....

Great, now I have Bonnie Tyler and visions of tractors playing chicken stuck in my head. THANKS A LOT.
posted by entropicamericana at 8:47 AM on March 14, 2013 [17 favorites]

Is this something that MeFi would be capable of hosting?

My guess is we'd need to buy like seventy-three new giant-sized servers.
posted by shakespeherian at 8:47 AM on March 14, 2013 [2 favorites]

I'd subscribe. I have no skills to offer, but I'd be happy to be a steady income source.
posted by bonehead at 8:47 AM on March 14, 2013 [3 favorites]

griphus: Feedly rubs me the wrong way with its unclear separation of different blogs

If you use the "All" view of Feedly with the "Titles" display option, it is visually almost identical to Google Reader, as far as I am concerned.
posted by Rock Steady at 8:47 AM on March 14, 2013 [6 favorites]

Rock Steady: "I've been quite impressed by Feedly after giving it a whirl tonight and this morning."

How does feedly make its money? I don't see a subscription option or price list anywhere.

Jesus, Google, thanks for making the future some bullshit bingo nonsense where I have to insist on knowing that companies are actually interesting in my money before I can consider using them.
posted by boo_radley at 8:48 AM on March 14, 2013 [7 favorites]

I'd subscribe. I have no skills to offer, but I'd be happy to be a steady income source.

Yep.
posted by gaspode at 8:49 AM on March 14, 2013

Mathowie should totally kickstarter this.
It worked for Andy Baio and XOXO.
posted by DigDoug at 8:49 AM on March 14, 2013 [6 favorites]

Great, now I have Bonnie Tyler and visions of tractors playing chicken stuck in my head. THANKS A LOT.

Here, let me help you with that.

NO OSCAR! NO!
posted by snuffleupagus at 8:49 AM on March 14, 2013

THE WORLD NEEDS A HERO

Also, Feedly's desktop client can be made to replicate Google Reader pretty well, but if you want a mobile client theirs is absolutely hopeless.
posted by Holy Zarquon's Singing Fish at 8:49 AM on March 14, 2013

All I want is Tumblr (a modestly skinnable content platform), with the ability to add external feeds, but more importantly with comments done right. Since Tumblr seem preternaturally inept with regard to comments, the marketplace is wide freaking open.
posted by seanmpuckett at 8:50 AM on March 14, 2013 [5 favorites]

KICKSTARTER! That was the other thing I meant to mention. A successful one would prove there's real sustainable money behind all the outrage.
posted by Rhaomi at 8:51 AM on March 14, 2013 [1 favorite]

Really really what's needed is a site that does Tumblr's dashboard PROPERLY. Because what the hell?
posted by seanmpuckett at 8:51 AM on March 14, 2013 [4 favorites]

Yeah but you couldn't announce the kickstarter on MeFi, could you.
posted by seanmpuckett at 8:51 AM on March 14, 2013 [9 favorites]

To put some numbers on this: I've been paying $20 every 30 days to access my regular newspaper on the web for the past few months. That's worth less to me than an RSS reader with a good interface.

GReader both largely replaces the newspaper for me (most especially for the comics pages) and also is a critical work tool---it's how I keep up on academic journals. So, for me, $5/month is a bargain. I'd pay more.
posted by bonehead at 8:51 AM on March 14, 2013

I don't even use an RSS reader (I tried different ones a lot until it became clear to me that it's just not how I read the internets), but I'd throw some money at a mefi kickstarter project for one.
posted by rtha at 8:52 AM on March 14, 2013 [3 favorites]

gosh people just go buy a radio
posted by The White Hat at 8:52 AM on March 14, 2013 [6 favorites]

I think it would be a good test run to allow only mefites (up to the start date of this project or some arbitrary date) join and use it at first. I think a couple dollar a month investment from users isn't that particularly absurd. I am more and more of the opinion that, as many people are saying, if you aren't paying, you are the product. The 2 best services that I still love and adore online (aside from google reader) are Livejournal and Metafilter, and both are things I pay/have paid for. I think people don't want to pay too much, which is why I say 2 or 3 bucks a month. Five is right on the edge of acceptable, and maybe could be a premium tier or something. Donations and such wouldn't be a bad way to help beyond the monthly contribs, but wouldn't be the primary income source.

Would this be something the Mozilla project might look into, perhaps? Or archive? Archive seems so big, but their servers always seem so... slow. I can see a kickstarter, but I worry that might lead it to be too big too fast. I'm more of slow and steady growth. While we want big, we don't want to get so big we turn people off.

And yes - please - custom skins.
posted by symbioid at 8:53 AM on March 14, 2013 [1 favorite]

But I really don't want Matt to leave, so can we maybe skip that part?
posted by rtha at 8:53 AM on March 14, 2013 [35 favorites]

Why is this our business? mathowie talked about doing something outside of Metafilter, and suddenly it becomes a Metafilter project?

That tweet was mentioned in the Google Reader thread on the blue. We don't need a second discussion of it here.
posted by mokin at 8:54 AM on March 14, 2013 [2 favorites]

I really really like MetaFilter, I'm sad to see Reader go, I think this idea would fit the MeFi brand/mission pretty well.

I am a developer, I am between contracts, if someone is doing something non-profity and wants extra hands they can contact me and I might possibly be able to help (depending on timing and paid jobs).
posted by dickasso at 8:55 AM on March 14, 2013

How does feedly make its money? I don't see a subscription option or price list anywhere.

My Feedly is serving up some small banner ads in the right sidebar (Books, "Gifts"), which seem to be related to my feed choices, despite Adblock running as a Chrome extension.

So there's that.
posted by notyou at 8:56 AM on March 14, 2013

But I really don't want Matt to leave, so can we maybe skip that part?

Yeah, that did seem like a particularly 2013 version of an O. Henry twist to me too.
posted by MCMikeNamara at 8:56 AM on March 14, 2013 [2 favorites]

Holy Zarquon's Singing Fish: Also, Feedly's desktop client can be made to replicate Google Reader pretty well, but if you want a mobile client theirs is absolutely hopeless.

Well, shit. You appear to be right about that. Why no "All" view in mobile, Feedly? Damn it. Sign me up for a donation/subscription to MeFeed.
posted by Rock Steady at 8:57 AM on March 14, 2013

Why is this our business? mathowie talked about doing something outside of Metafilter, and suddenly it becomes a Metafilter project?

Because a social layer involving mefites would be rediculously awesome? It's sort of what I wanted the G+ group to be, but reader originally did social sharing better than anything else I've used and we already have the smart people.
posted by jaduncan at 8:57 AM on March 14, 2013 [3 favorites]

gosh people just go buy a radio

hush grandperson (not ma or pa-ist and go back to sleep).
posted by Brandon Blatcher at 8:58 AM on March 14, 2013

I installed Fever last night. Works pretty well. The guy who sells it founded Mint (or something like that). Maybe a MeFiFever mashup?
posted by tayknight at 8:59 AM on March 14, 2013 [1 favorite]

I just paid Newsblur $12 and I don't even know if I like it yet. PANICKING HERE. This is a golden opportunity for someone.
posted by insectosaurus at 9:00 AM on March 14, 2013

It pisses me off no end that they've done this. For no damned reason, either. The service is practically self-sustaining, extremely popular and probably costs them next to nothing.

Google also recently killed support for Active Sync for private (non-paying) users, because they're trying to move to an open source format/protocol. Gmail and Outlook users are both affected. I tried to set up a synced-to-Mac Outlook Google group calendar this morning and now have to research other alternatives.
posted by zarq at 9:00 AM on March 14, 2013 [1 favorite]

mokin: "Why is this our business? mathowie talked about doing something outside of Metafilter, and suddenly it becomes a Metafilter project?"

We're emotionally invested in Google Reader? We'd support his idea, but want him to stay here?
posted by boo_radley at 9:01 AM on March 14, 2013 [2 favorites]

If you use the "All" view of Feedly with the "Titles" display option, it is visually almost identical to Google Reader, as far as I am concerned.

Is there a way to enable to sidebar blogroll to be always on?
posted by griphus at 9:02 AM on March 14, 2013

I've thought about this a bit since I posted that tweet. Obviously, I was super pissed yesterday (still am, but understand why things happen the way they do with big companies and stagnating services) and I must admit I posted that in the heat of the moment a bit. I don't think I could physically leave MeFi completely ever really, too much of the plumbing goes through my inbox and bank accounts, so no worries that I'm running for the hills.

I've considered "Build It Here at MeFi!" a little bit since someone mentioned it yesterday and there are three really hard problems in it. The easiest thing is the obvious, cloning some sort of reading UI that lets you go through feeds and follow a zillion blogs, and though it's non-trivial to just basically start with a clone of Reader, that's the easiest of problems.

The first hard problem is the update/crawler. Google got theirs wired to their Googlebot that was already scouring the web looking for updates and this became the competitive advantage that no one could match. I remember bloglines back in the day would update feeds hours and sometimes a day after a blog updated but Google Reader was always within minutes of a post going up. I don't have the first clue how to get a reliable update bot scouring the entire web, it's not something easy a random person could build, maintain, and support, or base an entire system off it.

The next hard problem is the API. Much of Reader's current fans don't even touch the website (that's where I spend nearly 100% of my interaction with Reader, in a browser) as they instead use phone clients, desktop clients, and use the API as merely a centralized updates system. This would be a huge endeavor for anyone to build, maintain, and support.

The last hard problem and it was even hard for Google from what I've heard internally, was providing search for all your feeds. I barely used this but some people found this to be a killer app. I heard this was a massive storage issue, to basically cache every blog post from every blog people followed, essentially, forever.

Overall, the problems are pretty great and Google Reader is closing really soon, so I don't see how it's possible to tackle even the easiest problems all on my own, which is why I mentioned it'd be great to work with someone else that had maybe solved some of these problems already. So far I got a few casual emails from people saying they were interested in my thoughts, stuff you'd share over a phone call, etc, but nothing major so no worries that I'm dropping MeFi anytime soon.
posted by mathowie (staff) at 9:07 AM on March 14, 2013 [47 favorites]

OK - big cheer to hear that mathowie's not going anywhere.

That said, so far Feedly's been treating me alright. Seamless move was indeed my experience.

So far, so good.
posted by Sophie1 at 9:11 AM on March 14, 2013

Oh, I totally forgot to even mention the financial aspects. I know a lot of important people in journalism rely on Google Reader, but honestly, I think it's a tough thing to make it work economically. It's pretty geeky, you'd attract a few thousand customers at first, but the tough thing would be running something on an infrastructure like Google's doing things Google has been doing for the last 8 years with any sort of reliability, all using a small team employees.

I honestly don't think there's enough of a market to support much of anything, which is why you see all these replacement reader apps doing more user-friendly casual magazine looking apps, to bring in more than just veteran reporters and programming geeks.
posted by mathowie (staff) at 9:12 AM on March 14, 2013 [3 favorites]

NewsBlur's source code might yield lessons on the site crawler and API side of things.
posted by Municipal Hare at 9:13 AM on March 14, 2013

I'd be happy to go yay yay yay and pump my fist in supprt
posted by angrycat at 9:15 AM on March 14, 2013 [2 favorites]

I'm just chiming in to say that I'd kick in for this, if only from seeing my girlfriend's despair over the news. I've never been a big RSS dude — especially since a bunch of the music blogs I follow have been shutting down (RIP Mutant Sounds) — but I'd pay to make her happy.
posted by klangklangston at 9:15 AM on March 14, 2013 [2 favorites]

RIP Mutant Sounds

CAN WE PLEASE HAVE SOME GODDAMN GOOD NEWS TODAY
posted by griphus at 9:16 AM on March 14, 2013 [3 favorites]

mathowie: "Overall, the problems are pretty great and Google Reader is closing really soon"

Hey, let's not think about creating a 1:1 copy of reader, immediately. There's different ways to roll out a service incrementally and still have it be useful. I know what you're saying, but there's ways to approach this that might not be evident, because the obvious solution has carved out such an enormous rut in peoples' minds.
posted by boo_radley at 9:17 AM on March 14, 2013 [1 favorite]

I hadn't thought about the search aspect. That's a really hard problem to solve. I don't envy anyone trying to tackle that. I didn't use it much, but if we learned anything from Massless' comments, it's that everyone used the tool a bit differently.
posted by DigDoug at 9:18 AM on March 14, 2013

Hey, let's not think about creating a 1:1 copy of reader, immediately.

Well, my initial thought was what could we make with the things we have? All the apps that work off the Reader API now just need to make their own API/update bots. The other problems are solved, and that seems like a more likely path for us to follow than me starting something from scratch tomorrow.
posted by mathowie (staff) at 9:18 AM on March 14, 2013 [2 favorites]

mathowie: "Well, my initial thought was what could we make with the things we have?"

There we go. Mind if I hit you up on gchat later?
posted by boo_radley at 9:23 AM on March 14, 2013

We Need a Hero

Once again, I ask you to vote #1 quidnunc as your hero, all-around good guy and sorta-mystic. His qualities as hero include:

1. He regularly ventures forth from the common world to a world of supernatural wonder;
2. He encounters numerous challenges in such worlds;
3. Returns to our common world with increased knowledge and powers;
4. Accommodates both French and Italian breads;
5. Often provides you with reading material from yesterday's Post, depending how properly supplicant you are.

So please vote #1 quidnunc kid in the upcoming vacancy created by mathowie's departure. We thank mathowie for his years of service, and wish him well, but now it is time for us to embrace #1 quidnunc and the future of tyranny and abject abuse only he can give us. #1 QUIDNUNC!
posted by Capt. Renault at 9:24 AM on March 14, 2013 [13 favorites]

Capt. Renault: "Once again, I ask you to vote #1 quidnunc as your hero, all-around good guy and sorta-mystic. His qualities as hero include:"

Wrong account, quidnunc.
posted by boo_radley at 9:25 AM on March 14, 2013 [7 favorites]

SOMEONE MAKE A GOOD REPLACEMENT AND TAKE MY MONEY.
posted by rmd1023 at 9:25 AM on March 14, 2013 [2 favorites]

CAN WE PLEASE HAVE SOME GODDAMN GOOD NEWS TODAY

I'm no longer feeling deathly ill.
posted by shakespeherian at 9:26 AM on March 14, 2013 [11 favorites]

Wrong account, quidnunc.

You flatter me, but alas, I am not quidnunc. Despite the facial reconstruction surgeries.
posted by Capt. Renault at 9:27 AM on March 14, 2013

We thank mathowie for his years of service, and wish him well, but now it is time for us to embrace #1 quidnunc and the future of tyranny and abject abuse only he can give us.

You misunderstand, the tyranny and abuse does not flow down to the users from User #001. It flows up from the users to #001. Besides,

WHAT
THE
HELL
QUIDNUNC

doesn't have the same je ne sais quoi.
posted by entropicamericana at 9:27 AM on March 14, 2013

CAN WE PLEASE HAVE SOME GODDAMN GOOD NEWS TODAY

I probably just passed my first organic chemistry exam.
posted by ocherdraco at 9:28 AM on March 14, 2013 [32 favorites]

A long time ago, pre GReader, I used a php script called Gregarius to parse and show RSS feeds. It hasn't been updated in years, but the source still exists (I'm sure there are dozens of RSS parsers that work in a similar fashion).
posted by tayknight at 9:28 AM on March 14, 2013

I'm no longer feeling deathly ill.

I probably just passed my first organic chemistry exam.

Why do I have a feeling these two good newses are related?
posted by griphus at 9:29 AM on March 14, 2013 [2 favorites]

In just a couple of days, I could make a really crappy web-based RSS reader that wouldn't scale.

2 ????
3 PROFIT!
posted by Zed at 9:32 AM on March 14, 2013 [4 favorites]

RSS made what Twitter does possible more than ten years ago, in a way that didn't belong to one private company, and which didn't require debasement with a stupid neologism. Can we rewind the internet please and try again?
posted by General Tonic at 9:35 AM on March 14, 2013 [3 favorites]

ocherdraco has, under instruction from her cruel and domineering mother, been poisoning me slowly in her mansion in Rio.
posted by shakespeherian at 9:37 AM on March 14, 2013 [2 favorites]

We don't need another hero
We don't need a New Dear Leader
All we want is life beyond
The Google Reader
posted by It's Raining Florence Henderson at 9:39 AM on March 14, 2013 [4 favorites]

Google Reader is complicated due to the scale & some of the features mathowie listed. It'd be a mistake to replicate it and end up with the same technical & financial problems.
Self-installs such as Fever will fill the gap for some, others will get by with services that merely sync what's been read and force the client software to download all of the feeds.

(In my view, the web is crying out for a modern interpretation of Tim Berners-Lee's vision of a distributed web with creating treated as the equal of consuming. Basically we need a top-notch web app for sharing text/images/links and subscribing to feeds. Something built to run as self-installs on low-end hosting but also able to support multiple users & be offered as a hosted service. Something so user-friendly it stands a chance of drawing some content away from the centralised services that are either locking up or shutting down. Ideally, hosting firms should fund something like this, as they have a vested interest in keeping the web decentralised.)
posted by malevolent at 9:42 AM on March 14, 2013 [4 favorites]

Gregarious was brilliant, until it became abandoned freeware.
posted by Faintdreams at 9:44 AM on March 14, 2013

Regarding the Hard Problem of crawling feeds, would it not be possible to have a request-based system rather than one that automatically brings everything to you? Load up ReaderClone, click a refresh button, and have the site/app query all your feeds right then and there, instead of constantly in the background.

Search isn't a dealbreaker, either. I would have no problem with Google dispensing with the enormous search backlog if it meant being able to keep Reader running.
posted by Rhaomi at 9:45 AM on March 14, 2013 [3 favorites]

The release date for the next Gentleman Bastards book was just announced this morning!

Okay, I feel a bit better now. Still have to read the second one, but more is better!
posted by Holy Zarquon's Singing Fish at 9:49 AM on March 14, 2013

Search isn't a dealbreaker, either. I would have no problem with Google dispensing with the enormous search backlog if it meant being able to keep Reader running.

Search over the last 30 days and starred items would be a good compromise.
posted by jaduncan at 9:51 AM on March 14, 2013 [1 favorite]

I got Tiny Tiny RSS running on my shared hosting account this morning. So far, it's working as a replacement for me. I don't have instantaneous updates because I don't have root access on my shared account, so I'm making do with an hourly cron job to update the feeds.

However, being forced to do something else because no new feeds are due for 45 minutes isn't necessarily a bad thing for my productivity.

I also bought an email account with a paid email host. I've been meaning to dump Gmail for a while and this was just the motivation I needed.
posted by COD at 10:00 AM on March 14, 2013 [1 favorite]

What makes sense to me is just trying to contribute to NewsBlur, Vienna, etc. and see what happens. For example, I haven't checked whether NewsBlur has an architecture that supports plugins for user creation/authentication, but if it did, that could make it interesting as an addon to a community like MeFi or to enterprises like, say, news organizations or higher ed institutions that would enjoy automatic account provisioning and private installations with automatic subscriptions and private sharing.

What I'm saying is that while I get that end users are understandably worried about continuity of service, something developers concerned about the future of RSS might prefer to think about is how to make existing projects a lot more flexible and interesting.

Incidentally I'd pay double NewsBlur's fee to have it be an appendage of MeFi with my handle and global sharing to other MeFi users.
posted by Monsieur Caution at 10:07 AM on March 14, 2013 [1 favorite]

Yeah I would gladly have limited search functionality and somewhat laggy updates in exchange for keeping the rest. I need my shamed dogs and captioned cats! (and my webcomics!)
posted by brilliantine at 10:16 AM on March 14, 2013

Look, truthfully? All I want is a feed sync facility as I switch between Linux, Windows and ricochet between two machines regularly,
posted by Samizdata at 10:19 AM on March 14, 2013 [5 favorites]

Regarding the Hard Problem of crawling feeds, would it not be possible to have a request-based system rather than one that automatically brings everything to you? Load up ReaderClone, click a refresh button, and have the site/app query all your feeds right then and there, instead of constantly in the background.

My intuition, though I do backend web programming only rarely, is that you'd want to be very careful giving users that level of control over your bandwidth. A sensible caching policy and an (probably invisible to users) enforced cool-down period after a feed refresh could probably go a long way towards ameliorating that concern, but it still merits some caution.
posted by invitapriore at 10:22 AM on March 14, 2013

CAN WE PLEASE HAVE SOME GODDAMN GOOD NEWS TODAY

I had a really nice chicken sandwich for lunch today, and have a social event this evening that I'm looking forward to.
posted by jbickers at 10:34 AM on March 14, 2013 [2 favorites]

CAN WE PLEASE HAVE SOME GODDAMN GOOD NEWS TODAY

There's a sale at Penney's!
posted by entropicamericana at 10:39 AM on March 14, 2013 [4 favorites]

Popping in to say thank $DEITY our fearless leader isn't leaving, and that I would pay for a Reader service, even without search.
posted by immlass at 10:45 AM on March 14, 2013

I would totally fund this if only because MeFeed is such a perfect name!
posted by Wretch729 at 10:49 AM on March 14, 2013 [2 favorites]

yes, but is that mehfeed meefeed myfeed?
posted by nadawi at 10:51 AM on March 14, 2013 [6 favorites]

Surely it should be FeedMe?
posted by ocherdraco at 10:51 AM on March 14, 2013 [16 favorites]

FeedMeh for those who pronounce it ~~wrong~~ the other way.
posted by rtha at 10:52 AM on March 14, 2013 [7 favorites]

FeedMeh SeeMore
posted by MCMikeNamara at 10:53 AM on March 14, 2013 [18 favorites]

So, what are thoughts on the feasibility of a Reader clone that:

- does on-demand updating (with a cooldown/limit), rather than automatic

- searches only starred items and/or last 30 days, rather than everything ever

- has a basic API based on open standards that make it easily compatible with any front-end people might want to use with it

These two alone would make Reader salvageable for Google (apart from the cynical strategic maneuvering), but could it make it achievable for a smaller project?
posted by Rhaomi at 10:58 AM on March 14, 2013

CAN WE PLEASE HAVE SOME GODDAMN GOOD NEWS TODAY

It's Friday tomorrow.
posted by urbanwhaleshark at 11:02 AM on March 14, 2013 [2 favorites]

I have no problem with those caveats, Rhaomi. Honestly, if Reader wasn't made by Google, it probably never would have occurred to me to search my feed, but because it is Google, not being able to search seemed like a huge negative (before they activated it), if that makes any sense.
posted by Rock Steady at 11:05 AM on March 14, 2013

Great, now I have Bonnie Tyler and visions of tractors playing chicken stuck in my head. THANKS A LOT.

Let me help you with that.
posted by octobersurprise at 11:12 AM on March 14, 2013

It's interesting to think about how an ideal RSS feeder would manage something like Lifehacker. It posts a ton of content every day most of which I don't care about but once in a while there's something highly relevant that I want to save. In Google Reader I just scan and star, then mark all as read trusting the starred items to be easier to find later because of the crutch of Google's search capability, but its not a perfect system.

In the old days I would visit the website and bookmark things I cared about using a folder system. Better organized but impossible for more than a few sites at once.

So how would the (ideal fantasy) MeFeed/FeedMe handle this?
posted by Wretch729 at 11:15 AM on March 14, 2013

It's Friday tomorrow.

That's the worst news for those of us with multiple Friday EOD deadlines.

*closes tab, bears down*
posted by notyou at 11:15 AM on March 14, 2013 [3 favorites]

As an ignorant luddite, I've only just (!) started using these RSS doobies, and I wonder why I ain't never done it before. Given his obvious skills, I hope mathowie does get an offer to become involved in a Reader replacement (or that one is built by the copious talent within MeFi) while still maintaining his chairpersonship of Metafilter Industries LLC - mark me down with those who would gladly pay for such a service.
posted by the quidnunc kid at 11:20 AM on March 14, 2013

As an ignorant luddite, I've only just (!) started using these RSS doobies

I apparently am even more ignorant and luddish -- I never heard of google reader and still don't quite understand what it is.
posted by JanetLand at 11:33 AM on March 14, 2013 [2 favorites]

Also, it's fascinating to me that all of these web/blog/tech bigwigs used Google Reader just like little old me. I always assumed that there was a much better paid service that you could use if your job was actually web-related, but free Google Reader was good enough for my modest needs.
posted by Rock Steady at 11:34 AM on March 14, 2013 [1 favorite]

Considering quidnunc managed to get doobies out of it, I'm not sure what it is either anymore.
posted by griphus at 11:35 AM on March 14, 2013 [2 favorites]

So how would the (ideal fantasy) MeFeed/FeedMe handle this?

In my view, there is no need to have bookmarks (stars) and search within the RSS reader considering the user base of services like Instapaper and Evernote. An ideal feed reader would have awesome sharing capabilities and integration with IFTTT so that you could save and organize stuff however made the most sense for you. Robust mobile apps would be a must as well.

Your Lifehacker is my Serious Eats. I get a ton of recipes in my feed, most of which I'm not interested in, but some I want to save. The feed has the full text of recipe posts. If I am interested in one, it takes just seconds to send it to my Evernote email, add a notebook and tags and move on, all within Reader.
posted by payoto at 11:42 AM on March 14, 2013 [2 favorites]

I seriously don't understand in what way Facebook fills the RSS niche for people. Were you only using Google Reader to see the most linkbaity articles about $current_political_topic or $celebrity_mishap?
posted by shakespeherian at 11:43 AM on March 14, 2013 [14 favorites]

still don't quite understand what it is

Well it's a doo-ba-dee-doo. Yes it's a doo-ba-dee-do.

*takes another puff*

I mean ...

*exhales*

... a doo-be doo-be doo-be doo-be doo-ba-dee-doo.
posted by the quidnunc kid at 11:44 AM on March 14, 2013

So it's looking like all of the enthusiasm for writing an open-source replacement has waned a little, but if anyone is still thinking of doing it and needs developers, please memail me. I'm available.
posted by suetanvil at 11:51 AM on March 14, 2013

Insightful comment payoto. So fine, let Evernote or Instapaper take care of the organization. Using IFTTT makes sense too, I need to learn more about that service.
posted by Wretch729 at 11:58 AM on March 14, 2013

Wretch729: "In the old days I would visit the website and bookmark things I cared about using a folder system. Better organized but impossible for more than a few sites at once. "

If you use Firefox, ScrapBook might be able to help you for the things you used to star.
posted by Chrysostom at 12:07 PM on March 14, 2013

I wish I could help out with a project like this, however I have no applicable skills. To be honest, I never even used Google Reader, but seeing the seething nerdrage and utter despair coming from my partner last night--I wish I could do something to help you all!

the technical knowhow, the desire, the connections. Let's make it happen.
...I don't have any of those things to contribute, sadly. I can bake a damn good pie or cheesecake though. Do you need those?
posted by inertia at 12:11 PM on March 14, 2013

Digg to the rescue!

Probably the first time those words have ever been used in that order.
posted by Rock Steady at 12:13 PM on March 14, 2013 [5 favorites]

So I haven't read every comment here, but...

Would it even be possible to persuade Google to open-source Reader's code?

Googlers have floated this idea internally but it's essentially impossible to disentangle Reader from the rest of the guts of Google. Even if you had all the code it would be next to useless without all the backend systems it relies on.

FWIW, man, you have never heard such a shitstorm inside a company as I did today.
posted by GuyZero at 12:14 PM on March 14, 2013 [41 favorites]

Fuck Digg. They were already at the top of my personal internet shit list for the way they wrecked the site and erased all user contributions without warning during their big redesign. Why should I ever trust them with anything ever again?

FWIW, man, you have never heard such a shitstorm inside a company as I did today.

Do tell...
posted by Rhaomi at 12:17 PM on March 14, 2013 [2 favorites]

By which I mean, what's the internal view? I was under the impression that even Page uses Reader regularly. Who's behind the decision, and how much dissent is there? There's no way they don't know how much hatred they're getting for this fiasco. Heck, I wouldn't be surprised if most Googlers use it daily.
posted by Rhaomi at 12:21 PM on March 14, 2013

I'll be very surprised if this decision doesn't affect confidence in their other products, even by people who didn't use Reader. I had been seriously considering porting my cell phone number to Google Voice. Now, there's no way I'll do that and take the risk that they kill Voice or start charging an insane amount for it.
posted by payoto at 12:34 PM on March 14, 2013 [2 favorites]

I don't need no stinkin' RSS reader.

*slams door, crosses arms, glares at door*
posted by deborah at 12:40 PM on March 14, 2013

OK, here's the scalable plan: Dropbox.

What we need is an open-source library that can be embedded in desktop/mobile/web RSS apps, capable of caching RSS content to the user's local Dropbox and then exposing that content to the app via something as close as possible to the Google Reader API.

This avoids the storage and bandwidth problems of scaling (making them Dropbox's problem) and the syncing problem of having your current state accessible across platforms and readers, and eases the adoption problem by mimicking an existing API. It should also make search feasible. The one problem is that you'll need enough space on each of your devices for as much of an archive as you want to be able to search on that device, and enough space on Dropbox for your entire archive -- but if having a large archive is valuable to you, Dropbox doesn't actually cost much.

Basically you end up avoiding all but the easiest of Matt's hard problems (mimicking the API), by replacing the centralized Google service with a little alien library that crawls into all the competing RSS readers and teaches them to use the same networked filesystem for synced storage. Should work, and should be cheap enough to Kickstart.

It could offer other synced-storage solutions as backends too, of course. I hear Google started one of those recently.
posted by jhc at 12:41 PM on March 14, 2013

I think a reader that was run by Mefites, whether officially or no, would be awesome. I'd throw money at that thing. I've never participated in Kickstarter (not even for Ryan North's awesome Choose-Your-Own-Adventure version of Hamlet!), but I would so participate in a Kickstarter for this. I recognize the implausibility, really, of the whole idea... But, man, it's a beautiful dream.

My whole internet is built around Reader and Metafilter. There are (rarely) moments when I have nothing new in Reader and no new posts on Metafilter.... and it's as if the internet has ended. The two combined would be amazing, for me.
posted by meese at 12:42 PM on March 14, 2013 [3 favorites]

payoto: I'll be very surprised if this decision doesn't affect confidence in their other products, even by people who didn't use Reader.

Today a thing I googled was "paid email provider". Never thought I'd do that.
posted by Rock Steady at 12:52 PM on March 14, 2013 [7 favorites]

For a first iteration I would cut storage or search as much as possible. Also, think of the fact that it ain't up to the minute as a selling point. I would go with a once a day email. You can get super extra points if you make the email "beautiful" and focus on readability and typography and whatever net hipsters are into.

That leaves you with a dashboard where people can manage subscriptions. And some "social".

I'm thinking you could also sub to anything other people have, so I can follow Ezra Klein and get all the stuff he gets.

Maybe this isn't google reader but it could be a legit product.
posted by Ad hominem at 1:26 PM on March 14, 2013

In my view, there is no need to have bookmarks (stars) and search within the RSS reader considering the user base of services like Instapaper and Evernote

and

OK, here's the scalable plan: Dropbox.

Maybe the solution is to have something that delivers Reader-like functionality by knowing how to communicate with the services the user already has.
posted by a snickering nuthatch at 1:29 PM on March 14, 2013

Make it pay per number of feeds, so people with thousands can subsidize people with 5.
posted by Ad hominem at 1:29 PM on March 14, 2013

"I'll be very surprised if this decision doesn't affect confidence in their other products, even by people who didn't use Reader."

Yep, I'm determined to be as unreliant on Google as possible, starting yesterday. Switched to Feedly, Firefox, going to unlink all my accounts and funnel everything into my paid email provider. Over the next four months I'll ween myself off Google Refine (it is truly brilliant and I need it for academic reasons for now). I was only just transitioning into Google Docs/Drive, so I'll leave that party early. Plus pissed me off a while back when they threatened to shut down my blogs because I wouldn't give up my years-old username (iamkimiam, obvs) for their stupid non-anonymity policy.

All the other things that will come up, I'll deal with accordingly.

When you start thinking about things from this perspective, you realize how pervasive Google is, and how hard it is (and possibly futile) to truly get away from it. It's worth my doing though, even if only for dignity and self-respect. As odd as that sounds.
posted by iamkimiam at 1:38 PM on March 14, 2013 [4 favorites]

OK, here's the scalable plan: Dropbox.

Not that people aren't having valid reactions here, but just so you know: Dropbox could go out of business anytime too.

Keep using gmail, leave gmail, whatever. Just be clear that wherever you're going is no more or less likely to survive 8 years than Google Reader was.

And before you say "put it on a hard drive you physically own" - most of the hard drives I've physically owned have lasted less than 8 years.
posted by GuyZero at 1:49 PM on March 14, 2013 [3 favorites]

Just a thought (maybe obvious to people that have thought about this before): rather than having a uniform update rate, have an adaptive update rate per feed. The crawler can notice that lifehacker has a ton of updates, so it can fetch lifehacker more often. The crawler will notice that Steve Yegge updates his blog like once a year (to be generous- get on it, Steve!) so it fetches that more rarely.

I imagine adaptive updates would work like this:

Every feed F is associated with an update delay D
When a feed F is added, D_F := D_default.

When you fetch a feed,
call the number of new items N
D_F := 2*(D_F) / (N + 1)
T_F = Now

When your crawler has a spare thread and wants to fetch a feed, select the feed that has highest score:
(Now - T_F) / D_F

This way, every time it checks a feed that doesn't have any items, it will check that feed about half as frequently as it did before; if it checks a feed an it's got 1 new item, it will check that feed about as often as it was, and if it has multiple new items, it will check more frequently. Of course, you'd want to clamp the D_F values within some range (maybe no less than 5 minutes, no more than 1 week).
posted by a snickering nuthatch at 1:53 PM on March 14, 2013 [4 favorites]

Switched to Feedly, Firefox

To beat a dead horse here: Feedly could die anytime. Success could be worse than failure as they then become an attractive acquisition target.

And Firefox is complete funded by a search ad revenue sharing deal with Google. Feel free to look up the financial docs for the Mozilla Foundation.

From Mozilla's 2010 & 2011 audited financial statement:

"Mozilla entered into a contract with a search engine provider for royalties which expires November 2014. The previous contract term expired in November 2011. Approximately 85% and 84% of royalty revenue for 2011 and 2010, respectively, was derived from this contract. The receivable from this search engine provider represented 77% and 64% of the December 31, 2011 and 2010 outstanding receivables, respectively."

They don't name Google in this doc but this is not exactly a state secret.

Like I said, de-google yourself all you want. But that alone doesn't solve any problems.
posted by GuyZero at 1:54 PM on March 14, 2013

Part of me really wants to de-Google after this, but it's kind of infuriating to know (a) how hard that is, viz. Mozilla's entanglement; (b) that my main alternatives for a lot of things are Microsoft and Apple, both of whom I also want to punch in the face; and (c) that I was using Reader in the first place not because it was free and/or ubiquitous but because it was the best, and it sucks to deliberately not use a product that is legitimately well-designed and really handy, and turn instead to one with significantly less utility, because of who its corporate overlord is.
posted by Holy Zarquon's Singing Fish at 2:08 PM on March 14, 2013 [10 favorites]

> Yeah but you couldn't announce the kickstarter on MeFi, could you.

He can post it to Projects.
posted by ardgedee at 2:10 PM on March 14, 2013 [1 favorite]

I'll be very surprised if this decision doesn't affect confidence in their other products, even by people who didn't use Reader.

Man, I use Gmail and Google Docs (Drive. Whatever.) every damn day, but I'm seriously considering jumping ship. I don't know if I'm being temporarily irrational and pissy, but it's awfully tempting right now.
posted by brundlefly at 2:11 PM on March 14, 2013

> Would it even be possible to persuade Google to open-source Reader's code?

Googlers have floated this idea internally but it's essentially impossible to disentangle Reader from the rest of the guts of Google. Even if you had all the code it would be next to useless without all the backend systems it relies on.

It's the downside of Google's services: Free services are great, up until the provider decides they're not getting enough return on the effort. It would be interesting if Google were to provide it as a paid subscription service -- even at $20/year, it'd probably be cheaper than what most competitors could provide, and have more depth and breadth than competitors could dream of (for reasons discussed above). There'd be a stark drop in subscriber base, but that's not necessarily Google's problem either...
posted by ardgedee at 2:15 PM on March 14, 2013

I can't begin to grasp why Google can't (or won't) monetize Reader. They're the middleman for the most signal-drenched technorati on the web. They have intricately detailed statistics on what millions of netizens choose to read, save, subscribe to, share. It's ludicrous they don't find such a treasure trove of valuable insight worthy enough to preserve.
posted by Rhaomi at 2:42 PM on March 14, 2013 [10 favorites]

Looks like a lot of people are having the same idea.
posted by brundlefly at 2:53 PM on March 14, 2013 [3 favorites]

FWIW, man, you have never heard such a shitstorm inside a company as I did today.

Google has been continuously changing since it started, but someday someone will write a history of this current period of transformation (kicked off by Google+?). I imagine the Reader shutdown has caused at least some Googlers to realize that Google of 2013 is considerably different from the Google of 5 years ago. And perhaps to realize that the change is irreversible.

(Google SWE, 2010-2012)
posted by jjwiseman at 3:22 PM on March 14, 2013 [2 favorites]

According to Forbes,

Basically, Google Reader was once a sort of hyper-insulated social network. It was the smartest social network on the internet, ...

and I'm betting this is what really signed its death warrant.

Google is determined to sacrifice all other Goods on the altar of Google+ if they have even the slightest overtone of social network to them, all unaware that they have set up a kind of Golden Calf to worship and lost their way. Too bad there's no Moses striding in from the wings to get them back on track in this particular remake.

But there is a kind of Devil: Facebook, aka Zuckerberg.

I've been guessing that the competition there is almost sexual, and since Brin spouted off with that near-delusional rant about smartphones being "emasculating", I'm sure of it.
posted by jamjam at 4:03 PM on March 14, 2013 [2 favorites]

I would read this slashfic.
posted by kagredon at 4:06 PM on March 14, 2013 [2 favorites]

iamkimiam: Google Refine has already been dropped by the Google and open-sourced (it was always open source actually). It is now called OpenRefine. There is already a healthy development community around it.
posted by rockindata at 4:07 PM on March 14, 2013 [3 favorites]

So I've been thinking about this a lot since I raged out about it last night. Personally, I'm going to be taking steps to disentangle Google from its central place in my life, and to buffer myself against this sort of thing in the future.

But I'm not really worried that I'm not going to sort out a feed reading solution - if nothing else, the little app I use on my phone and tablet is perfectly capable of doing all the crawling on its own, for example, if I give it some wifi bandwidth. It was just awfully nice to have it in synch across all of my devices, including the laptop and the desktop and the other laptop and the other desktop and so on.

What I'm really worried about is what this signals for the entire ecosystem of RSS/Atom, and more generally for the open-access, machine readable web.

A lot of us are serious professional nerds, and we are probably going to put time into client level solutions. What I'm more worried about, and what I'm going to put time into at the organization where I work, is that the server/provider/author/infrastructure level of things could also really use our help. Those of us who have platforms to work on need to be working to make feeds and other forms of democratic machine-readability prominently visible, better understood, and available for as many resources as possible.
posted by brennen at 4:15 PM on March 14, 2013 [3 favorites]

I got no inside info, I'm just wondering what the true reasons are.
posted by Ad hominem at 4:25 PM on March 14, 2013

I think they could monetize reader, they could also integrate it into G+, the could even let it operate at a loss. RSS could conflict with their strategy, which may be "everything lives in g+, it is publishing platform and distribution" not "g+ is an aggregator for RSS".

I think this is the strategy, in general, of an entire industry. I called it a conspiracy while ranting about it last night, but of course that's not really the right word. I don't think it's a conspiracy in the sense that a bunch of people sat down in a room and said "let's collude to kill feeds" so much as it's a bunch of collective action on the part of a set of people who all want to wholly own the act and the machinery of publishing things on the web.

Google wants to own everything and Facebook wants to own everything and so on. They're competing with each other on that level, but regardless of who wins or whether it's even possible for anybody to win, that's a goal that's going to lead them to do things that marginalize channels outside of frameworks like Google+ and Facebook.
posted by brennen at 5:04 PM on March 14, 2013 [1 favorite]

This is one reason it's so dangerous to become dependent on things owned by corporations. Privacy, ownership of data -- both worth considering. Lack of control over the existence of something that's integral to your life - this is a bright, shining example of this prospective problem.

I've wanted something like Google Reader, but I haven't found it in myself to commit to it because I don't want my settings, data, etc. stored on Google's server somewhere -- at least not the only copy.

I'd love a really great RSS reader that runs on my local machine. I know multiple OS's are a problem, but not an insurmountable one, no?
posted by amtho at 5:22 PM on March 14, 2013

"Google Refine has already been dropped by the Google and open-sourced (it was always open source actually). It is now called OpenRefine. There is already a healthy development community around it."

I was peripherally aware of this, but had my old Google Refine app installed and was none the wiser about the realness of OpenRefine. Am downloading it now. You've made my day, rockindata!
posted by iamkimiam at 5:24 PM on March 14, 2013

I think that is very true, they want all the clicks. How they arrived at the current strategy, instead of some other strategy, will make a great book for Steven Levy or someone.

They are also making some moves with CalDAV, making the API whitelist only, but they are whitelisting MS for WP. Are they going to whitelist Apple? IPhone and ICal may be out in the cold.
posted by Ad hominem at 5:31 PM on March 14, 2013

Looks like the "OpenRefine" app is still Google branded...so I guess I'm already using OpenRefine?
posted by iamkimiam at 5:37 PM on March 14, 2013

I'm writing my own version. It will be awesome and will live as long as I do.
posted by 922257033c4a0f3cecdbd819a46d626999d1af4a at 5:50 PM on March 14, 2013 [1 favorite]

I'm a software engineer who works on web apps and also builds big-ass databases. I volunteer as tribute.
posted by Alison at 8:19 PM on March 14, 2013 [5 favorites]

I would pay a subscription for a decent reader that synched across different OS and device types.
posted by arcticseal at 8:40 PM on March 14, 2013 [1 favorite]

I'm a software engineer who works on web apps and also builds big-ass databases. I volunteer as tribute.

Don't stick around for the melee. Just grab a pack and run - RUN!
posted by His thoughts were red thoughts at 8:43 PM on March 14, 2013 [2 favorites]

I will pay for this and I don't even use Reader.
posted by scrump at 9:35 PM on March 14, 2013

"I don't think it's a conspiracy in the sense that a bunch of people sat down in a room and said "let's collude to kill feeds" so much as it's a bunch of collective action on the part of a set of people who all want to wholly own the act and the machinery of publishing things on the web."

Tragedy of the commons. Or "Why I'm not a libertarian."
posted by klangklangston at 10:35 PM on March 14, 2013 [3 favorites]

jhc: "OK, here's the scalable plan: Dropbox.

What we need is an open-source library that can be embedded in desktop/mobile/web RSS apps, capable of caching RSS content to the user's local Dropbox and then exposing that content to the app via something as close as possible to the Google Reader API.

This avoids the storage and bandwidth problems of scaling (making them Dropbox's problem) and the syncing problem of having your current state accessible across platforms and readers, and eases the adoption problem by mimicking an existing API. It should also make search feasible. The one problem is that you'll need enough space on each of your devices for as much of an archive as you want to be able to search on that device, and enough space on Dropbox for your entire archive -- but if having a large archive is valuable to you, Dropbox doesn't actually cost much.

Basically you end up avoiding all but the easiest of Matt's hard problems (mimicking the API), by replacing the centralized Google service with a little alien library that crawls into all the competing RSS readers and teaches them to use the same networked filesystem for synced storage. Should work, and should be cheap enough to Kickstart.

It could offer other synced-storage solutions as backends too, of course. I hear Google started one of those recently."

Sounds lovely. I have about 16Gb of free Dropbox storage currently (which I did mention at some point donating a chunk to a MetaFilter share but no body got back to me)...
posted by Samizdata at 10:38 PM on March 14, 2013

arcticseal: "I would pay a subscription for a decent reader that synched across different OS and device types."

Can only favorite once. Le sigh.
posted by Samizdata at 10:54 PM on March 14, 2013 [1 favorite]

The first hard problem is the update/crawler. Google got theirs wired to their Googlebot that was already scouring the web looking for updates and this became the competitive advantage that no one could match. I remember bloglines back in the day would update feeds hours and sometimes a day after a blog updated but Google Reader was always within minutes of a post going up. I don't have the first clue how to get a reliable update bot scouring the entire web, it's not something easy a random person could build, maintain, and support, or base an entire system off it.

I've actually done the crawling part, it's not too difficult. I had a system that both had a set list of feeds and one that would scour the web looking for new ones. I actually had a cool system where the crawling would be done by EC2 instances and my desktop would only download zip files of the feeds.

The updating-in-near-real-time thing would be difficult, but not impossible if you base a system on hadoop/hbase it should be pretty trivial to scale. You just have to avoid any kind of centralization.

And for less popular feeds you only need to update 3-4 times a day - there will probably be a 'long tail' of tons of feeds with just one or two subscribers, and you can update those when those people log on.

I've never done a public API before, and obviously you have to maintain consistency and compatibility if you don't want to piss people off, but making an API itself isn't very difficult. The trick is, you have make sure you use your API internally. On powershop I did everything with a JSON API and static HTML files. If you're using an your own API internally then obviously you're going to do a good job of maintaining it.

One of the reasons why it was easy for Amazon to get into the API businesses is because they had all their departments create APIs for other departments to use, rather then having people talk to each other and ask for things. Then all they had to do to start selling them was open those APIs up to people outside the company.

Now Search however, would be difficult. Especially search as good as google's. Just doing a plain text search would be possible, but matching Google's ability to find relevance would be a hard problem. I never really used the search feature myself, though.

NewsBlur's source code might yield lessons on the site crawler and API side of things.

Actually, if the code is well designed and easy to setup, it might be possible to create a "Metafilter Reader" in a weekend or two.

I'm actually going to try setting that up on my own server for my own personal use - I don't want to get stuck using another 3rd party service

My intuition, though I do backend web programming only rarely, is that you'd want to be very careful giving users that level of control over your bandwidth. A sensible caching policy and an (probably invisible to users) enforced cool-down period after a feed refresh could probably go a long way towards ameliorating that concern, but it still merits some caution.

What you could do is, when a reader hits the site, if a feed is stale download a fresh one (or enqueue a download with high priority), if it's not over X hours old then you don't. (Really crazy idea: Have users install a browser addon that downloads feeds itself if they want faster refreshes for their feeds)

There's also atompub which should update you when feeds update, rather then needing to use a polling mechanism.

Digg to the rescue!

Because Digg has soooo much credibility when it comes to not screwing over their users...

Googlers have floated this idea internally but it's essentially impossible to disentangle Reader from the rest of the guts of Google. Even if you had all the code it would be next to useless without all the backend systems it relies on.

If I was on the reader team, I'd be trying to spend my 20% time on a version of reader that uses Hadoop/HBase instead of google stuff, so it could be open-sourced.

Not that people aren't having valid reactions here, but just so you know: Dropbox could go out of business anytime too.

What I'd really like to see is a system you can run "on your own server" with some kind of method to set things up for users who don't want to do that (like how you can run wordpress on your server, or host a blog on wordpress.com). Maybe newsblur will do that, but we'll have to see how easy it is to actually install and use on your own system.

Just because you pay for something doesn't mean the company won't go out of business.
posted by delmoi at 12:09 AM on March 15, 2013 [7 favorites]

Further to Jpfed's thoughts on adaptive updating [1]: ideally, the crawler would get each new post as soon as possible after it's published but without wasting resources by checking too frequently. To solve that, run a learning algorithm that initially looks at the entire history of the site and uses something like Bayesian learning to model how frequently and when new posts appear. Then poll each site based on, roughly, when new posts are expected to appear.

So, to first order, a site that's updated hourly will be polled approximately hourly, and a site that has one post a week will be polled no more than every few days. But in more detail, the algorithm should (in principle at least) be able to spot patterns in posting times and use that information. If site A posts weekly and always on a Thursday, then only poll on Thursdays. If site B is only ever updated between the hours of 10:00-16:00 EST, then only poll during those hours.

By taking the entire history (or a large sample of it) as a baseline, the algorithm is less likely to be thrown off by temporary anomalies (for example, someone taking a few days' holiday). It might even be able to spot patterns like "when site C comes back from a few days of quiet it publishes a flurry of new posts in the space of a few hours, so go into frequent-polling mode for the next 9 hours". Someone who knows more about Bayesian learning methods and related techniques could say more about this.

Another factor to take into consideration: false positives (i.e. polling when there's no new post) have a constant cost of wasted resources, whereas false negatives (i.e. failing to poll while there's a new post waiting to be collected) has a cost that depends upon the number of people who follow that site. So for sites with more followers the algorithm should err on the side of polling more frequently, to avoid keeping lots of people waiting.

[1] And yes, I know this is hardly the most urgent aspect of the problem to solve, but it's an interesting theoretical problem to think about over breakfast.
posted by logopetria at 12:11 AM on March 15, 2013 [1 favorite]

Further to Jpfed's thoughts on adaptive updating [1]: ideally, the crawler would get each new post as soon as possible after it's published but without wasting resources by checking too frequently. To solve that, run a learning algorithm that initially looks at the entire history of the site and uses something like Bayesian learning to model how frequently and when new posts appear

I think that's kind of a bad idea. Some blogs might be updated rarely, but they are probably updated sporadically as well, meaning you might get a burst of activity, a bunch of posts in a row, then nothing for a long time.

What you want to do, I think, is check popular feeds regularly, check medium popularity feeds 2/3 times a day, and check feeds with a handful of users when those users log on, if they haven't been accessed in a while (and again, this would be done by putting them at the top of a queue, so that you don't end up with bursts of unexpected bandwidth).

And remember you can use atompub to get notifications when some sites are updated, which mostly solves the problem for you, if sites implement it.

Also, downloading feeds doesn't really even take that much bandwidth If you had, erm, Google fiber you could easily download millions of feeds per hour.

Assuming 100k per feed, that's 10,000 per gigabyte. Which costs 10 cents to download on amazon EC2. So sweeping a million feeds would cost about $1, just in terms of bandwidth costs. Or take 8 seconds on a Google fiber connection. Or 80 seconds over FiOS.
posted by delmoi at 12:27 AM on March 15, 2013 [1 favorite]

Tragedy of the commons. Or "Why I'm not a libertarian."

I think this is more of an "enclosure of the commons" situation.
posted by brennen at 12:52 AM on March 15, 2013 [1 favorite]

What do people think of this proposal from Marco Arment (of Instapaper):

The last thing we need is a format war — with Reader’s shutdown in July, we don’t have time for one. An obvious idea that many have proposed (or already implemented) is to make a new service mirror the (never-officially-documented) Google Reader API. Even if it also offers its own standalone API for more functionality, any candidates to replace Google Reader should mirror the fundamentals of its API.
[...]
Like it or not, the Google Reader API is the feed-sync “standard” today. Until this business shakes out, which could take years (and might never happen), this is the best way forward.

(whitecedar mentioned it on the thread on the blue, but it didn't appear to get any follow-up there.)
posted by logopetria at 1:53 AM on March 15, 2013 [2 favorites]

So, to first order, a site that's updated hourly will be polled approximately hourly, and a site that has one post a week will be polled no more than every few days. But in more detail, the algorithm should (in principle at least) be able to spot patterns in posting times and use that information. If site A posts weekly and always on a Thursday, then only poll on Thursdays. If site B is only ever updated between the hours of 10:00-16:00 EST, then only poll during those hours.

I considered this, but thought it would use too much space for the marginal benefit.

Another factor to take into consideration: false positives (i.e. polling when there's no new post) have a constant cost of wasted resources, whereas false negatives (i.e. failing to poll while there's a new post waiting to be collected) has a cost that depends upon the number of people who follow that site. So for sites with more followers the algorithm should err on the side of polling more frequently, to avoid keeping lots of people waiting.

This is an excellent point. Some sort of weighting would be appropriate- not only by number of followers, but also by how much someone cares about the feed. I really don't care if I miss a post on /r/programming.rss. But if Rich Hickey says anything, I want to hear about it. Short of installing a wireless microphone on him, it would be nice to be able to tell my client that.
posted by a snickering nuthatch at 5:08 AM on March 15, 2013

delmoi, it occurs to me that Reader already holds the answer for adaptive updating for anyone who wants it. If you click on the detail view for a feed, it shows you detailed statistics on its update history, including average number of posts per week and charts showing the days of the week and times of the day that have received the most updates recently (and when items have been read). There are similar stats in the Trends section, which lists the most popular feeds you read, the most inactive, the most clicked-on, etc.

I don't know how easy it would be to pull this data out of Reader automatically, but doing so would enable replacements to be vastly more efficient at fetching updates.
posted by Rhaomi at 5:20 AM on March 15, 2013 [2 favorites]

I think the backend is fairly straightforward, you could build a simple one in a day or so. Like Jpfed says, you've got Boing Boing and Mefi and other high-traffic, regularly updated feeds. Cron is more than sufficient for those since you've got a very high chance of having new content whenever you check.

For the sporadic updates you have to be clever. In a lot of cases you can probably just grab the headers of the feed and use the cache control or etag information to tell if it's changed. That should be a very lightweight operation so you should be able to do it frequently.

Another idea is to use the clients to do some work for you. You'd have to do some fiddling with cross origin but I suspect you could let your browsers fetch some feeds and pass on the information that there's an update to the servers to parse.

As long as your fetching is quite well separated from your content display tiers you ought to be able to scale them separately. Also you could parse the rss into self-contained html fragments and stick them directly into memcache, fetch with nginx's built in memcache client. You might also be able to use varnish's esi support in some way, I don't know what the overhead there is per-include though.
posted by Skorgu at 5:28 AM on March 15, 2013 [1 favorite]

(That said I've been a paid newsblur user since they took away sharing from Google reader and I'm happy with that so there's no way I care enough to write any of that.)

This is actually the final spur I needed to dump Google as much as possible. I already host enough of my own infrastructure that I'm just going to bite the bullet and add email to that list. I was holding out for a Good Webmail Client but I think I'll just have to make do with k9mail and thunderbird.
posted by Skorgu at 5:32 AM on March 15, 2013

I considered this, but thought it would use too much space for the marginal benefit.

Actually, I retract this. The marginal space costs would be fairly low- about a half of a kilobyte per feed.
posted by a snickering nuthatch at 5:57 AM on March 15, 2013

Not really FPP material, but anyone looking to host more of their own online life might be interested in an open source ITTT/Yahoo Pipes.
posted by a snickering nuthatch at 6:14 AM on March 15, 2013 [1 favorite]

Has anyone mentioned Skimr yet? It's a Mefi Project from petrkral and it seems to do RSS, er, really simply.
posted by chavenet at 6:23 AM on March 15, 2013

I can't begin to grasp why Google can't (or won't) monetize Reader.
Same reason they killed off, or froze, Google Answers.
When your main business is raking in $50 billion a year, nearly all of it from advertising, and most of that from little text boxes, you just can't be bothered dealing with a tiny business like Answers where people pay $5 or $50 or whatever to get a specific answer to a specific question. At most Answers probably did a few million in revenue. Making Reader a subscription product, at $20 a year (which somebody mentioned) would similarly bring in a few million, or a few tens of millions, in revenue, and some fraction of that in profit. Meanwhile, the calculus is probably that if they can drive you to the original web pages of the Reader feeds in other ways (search, News, Google+, or something new), they'll make a lot more money serving ads on those sites or on referring Google sites. IOW, killing off Reader and writing off its potential monetization (millions) could eventually net them billions elsewhere.
posted by beagle at 7:21 AM on March 15, 2013

I wonder if that calculus accounts for the massive credibility blow and loss of trust in Google as a reliable platform for data, though. Not to mention the intangible but serious harm this does to the ecosystem of bloggers, writers, reporters, etc. that depend on RSS to digest, curate, and distribute content. The apparent internal surprise within Google suggest not, which is troubling. Do they really understand the web that little?
posted by Rhaomi at 7:33 AM on March 15, 2013 [2 favorites]

delmoi, it occurs to me that Reader already holds the answer for adaptive updating for anyone who wants it. If you click on the detail view for a feed, it shows you detailed statistics on its update history, including average number of posts per week and charts showing the days of the week and times of the day that have received the most updates recently (and when items have been read).

The feed itself contains that data: the time stamps of the last N posts (where N is usually about 20, but can be more)

But like I said, I don't think it should be based on how often the feed is updated. If it's updated once a week, and you check it once a week, you could have a feed that's updated on Monday and doesn't get refreshed until Thursday

Just because a feed doesn't update frequently doesn't mean you can assume the content isn't time-sensitive for people.

In a lot of cases you can probably just grab the headers of the feed and use the cache control or etag information to tell if it's changed.

Yup. But remember the files are pretty small usually 60-100kb, depending on how many items are included. The hard part is just updating the database.

What I would suggest is using a hash of all the item IDs (each item in an RSS/Atom feed is supposed to have has a unique identifier)

Another idea is to use the clients to do some work for you. You'd have to do some fiddling with cross origin but I suspect you could let your browsers fetch some feeds and pass on the information that there's an update to the servers to parse.

Not without a browser addon, I don't think

Honestly I'm kind of pissed about losing google reader, but I'm not sure if this is really a bad thing for the RSS ecosystem. The only risk is that a lot of people who use google reader might just give up on RSS entirely. Maybe just get on twitter and subscribe to their favorite blogs on twitter.

But if most people stay with RSS, it could actually start to see some growth again. The return of sharing features is an obvious one and it could be done in an open, standards compliant way. Remember "trackbacks" that were a thing for a while? Why not have every reader have it's own feed of shared items that people could subscribe too?

What about including a standard commenting mechanism for blogs with comments?

I really wish Google gave people more time though. I don't understand how they can justify keeping google wave open for years while only giving people three months to find a replacement for google reader.

Why not give people until June 1st 2014? That would give developers time to get something really scalable and feature-rich out there.

___

Meanwhile, the calculus is probably that if they can drive you to the original web pages of the Reader feeds in other ways (search, News, Google+, or something new), they'll make a lot more money serving ads on those sites or on referring Google sites. IOW, killing off Reader and writing off its potential monetization (millions) could eventually net them billions elsewhere.

Here's a way Google could have made a ton of money off Reader: Correlate reader preferences with youtube preferences, make google reader a great interface for viewing youtube subscriptions.

You could have a 'recommended videos' section (based on what it thinks you'll like, as well as the specific feeds you're currently looking at)

Not everyone wants video, but some people like it. And getting youtube views from Reader would mean getting video ads in front of people.
posted by delmoi at 7:44 AM on March 15, 2013

What I would suggest is using a hash of all the item IDs (each item in an RSS/Atom feed is supposed to have has a unique identifier)

I helped implement a feed reader for the WI state legislature's intranet. Those item IDs are absent a lot of the time; when the ID was absent I just hashed the text contents of the item. A side benefit of this is that if the feed producer edits the content, the new hash isn't present in your "marked as read" list of hashes, and so it displays the item as new.
posted by a snickering nuthatch at 7:57 AM on March 15, 2013 [1 favorite]

the new hash isn't present in your "marked as read" list of hashes, and so it displays the item as new.

This is awesome, why not do that even when the item ID is there?
posted by jaduncan at 10:23 AM on March 15, 2013

I'm just here to root root root for the good guys, and hoping something awesome springs forth in the next month or two.
posted by Theta States at 11:04 AM on March 15, 2013

As an avid Google Reader user, I also hope something comes forth but as a product manager these discussions are extra fascinating (though some of it is incomprehensible to me technically as I'm more of the marketing/client relations kind of PM). My Trello board fingers are itching.

Re: search. It's not a function I used much but if someone wanted search is there a way to use Google's own search as part of this? Just because they aren't pulling RSS for these sites, they are presumably trawling them anyway, so could the search be performed on Google and then just filtered against the URLs in someone's personal RSS list to display only some of the results? Kind of a Google Site search except the sites are individualized?
posted by marylynn at 12:52 PM on March 15, 2013

Is it completely misguided to ask why RSS services don't combine their crawling? It seems like almost everyone effectively wants exactly the same information. This seems like a field that would naturally reward consortiums.

Multiple RSS services simultaneously crawling the same data would be incredibly redundant.
posted by tychotesla at 1:11 PM on March 15, 2013

I think this stuff is hard enough to monetize as-is without having to spend resources coordinating development (and splitting revenue) across several companies. If Company X can make the True Google Reader Successor that makes Company Y and Company Z's attempt at it irrelevant, they get a lot of more out of it than if they team up with Company Y and Company Z.
posted by griphus at 1:17 PM on March 15, 2013

There are efforts to build a shared repository of web crawl data: CommonCrawl, founded by Gil Elbaz (ex-Googler, actually the guy who created AdSense).

I've never been able to figure out the status of that project, and I'm guessing they've focused more on doing a comprehensive crawl than on an up-to-date crawl.
posted by jjwiseman at 1:55 PM on March 15, 2013 [1 favorite]

In addition to the ID's being missing there is the tremendous pain to deal with all the other spectacularly malformed RSS out there in the world.

What I did for this was to use several rss parsing libraries. If one failed, move to another as a fallback. In the .NET world, the ServiceModel dll parses well-formed rss very fast, and Argotic parses pretty much everything else (but slowly).
posted by a snickering nuthatch at 2:28 PM on March 15, 2013

My RSS reader will hit Metafilter Projects in a week or two, I'm probably a month in now of coding. I've never used Google's version (or even aware of it), so it is very different than what you are used to. A version optimized for phones won't be ready for a while after that.

It's funny, one of my fears would be that Matt wouldn't like it, because it has changed how I use Metafilter, a lot, so I was relieved to see this post saying that RSS readers were important to him. It would have been a drag to pull out all the MF specific code.

I am hoping for lots of comments, feel free to me-mail me if you want a me-mail back when it goes up on Projects.
posted by rakish_yet_centered at 2:36 PM on March 15, 2013 [2 favorites]

In addition to the ID's being missing there is the tremendous pain to deal with all the other spectacularly malformed RSS out there in the world. While there are Feed Validators out there you can't just support well formed feeds because that would eliminate a lot of people's feeds from showing up at all. Then of course there probably is some amount of work trying to prevent malicious or obnoxious content from showing up.

Just handling all the well-formed feeds is kind of a lot of work, given how screwed up the aggregator world became.

feedparser is an "ultra-liberal" parser that can handle "RSS 0.90, Netscape RSS 0.91, Userland RSS 0.91, RSS 0.92, RSS 0.93, RSS 0.94, RSS 1.0, RSS 2.0, Atom 0.3, Atom 1.0, and CDF feeds. It also parses several popular extension modules, including Dublin Core and Apple’s iTunes extensions."

It tries to handle invalid feeds, including ones with malformed XML. It also handles 28 date formats, attempts to sanitize feed content to keep it safe, and makes use of HTTP caching features. It's been around for 10 years, but given the somewhat moribund state of aggregation, it might still be the best place to start.
posted by jjwiseman at 3:00 PM on March 15, 2013 [1 favorite]

I am a fairly-inexperienced software developer (mostly silly Arduino and python projects done as hobbies in my spare time) but I would devote whatever additional time I have to doing shit work on a project like this. Google Reader is literally my information lifeline, it is of incredible importance to just about every facet of my life (school, work, personal, everything).

Please feel free to me-mail me.
posted by downing street memo at 3:06 PM on March 15, 2013

delmoi: If I was on the reader team, I'd be trying to spend my 20% time on a version of reader that uses Hadoop/HBase instead of google stuff, so it could be open-sourced.

The Reader "team" is apparently one person, who works on it part-time, according to a few Tweets.
posted by zsazsa at 5:48 PM on March 15, 2013

Right now I have my starred items saved to a private delicious bookmark by using IFTTT - yes, I still use delicious. To solve the storage problem, you'd need multiple tools that allowed easy sharing and/or saving to [Evernote, delicious, local text file, email] whatever.
posted by soelo at 6:09 PM on March 15, 2013

I like RSS, but it's too much work.

It just seems more pleasant having a bookmark menu on your toolbar with your favorite 40 news feeds.
posted by four panels at 6:15 PM on March 15, 2013

I like griphus, but I don't believe he cares about Mutant Sounds.
posted by OmieWise at 6:18 PM on March 15, 2013

Your favourite 40 feeds? Check the thread on the blue for people posting their stats: I'm a light user with about 100 feeds; some people are up to a thousand. Some of those feeds will update 20 times a day, others only once every six months, but any one item could be important to my job or the non-profit work I do. A bookmark menu just doesn't cut it for anything more than casual browsing. RSS is for people who need to do their own filtering and curating from raw sources.

And that's not to put down people with different needs - just that people who don't use RSS don't understand that it's not just for news or what's popular; it's for sifting through piles of info so it can be extracted and used.
posted by harriet vane at 7:51 AM on March 16, 2013

> It just seems more pleasant having a bookmark menu on your toolbar with your favorite 40 news feeds

I had more than 400 subscriptions on Google Reader when I started cleaning it up in preparation for the exodus.
posted by The corpse in the library at 9:17 AM on March 16, 2013

RSS is way less work than checking a selection of bookmarks.

Not only for the reasons listed above, but also because it requires fewer clicks to get to updates. I like seeing pictures of cats with funny captions, I like seeing pictures of dogs with shaming confessions, and I like seeing new recipes. So, here I am, sitting at The Internet, and I want to find new things to read. Without RSS, I would have to go to icanhascheezburger to see if there are new posts, dogshaming to see if there are new posts, and serious eats to see if there are new posts. And then when I'm done, I'd have to go back to each one again. Over and over again, clicking and re-clicking to websites just to check if there are updates. But with Google Reader? I just keep that one webpage open, and I get to see, rather quickly, whether new stuff is available for me to read.

Of course, I don't just like three blogs. I like MANY blogs. Without RSS, keeping up with all the Internet I like to read would be like a full-time job. Ugh.

Literally, I have Reader open on my computer all the time. I only ever close that page when I turn my computer off (which is rare). A five second scroll through my list of subscriptions, and I immediately know how much internet is waiting for me, and which parts of the internet exactly.
posted by meese at 9:44 AM on March 16, 2013 [2 favorites]

Out of interest, rather than because I've looked in detail, is this something that any of the distributed social networks (e.g. Diaspora & Indenti.ca) would do well, possibly if forked? It certainly looks like a possibility.

That said, I use Opera's built in RSS, but it's definitely nothing special, so maybe I'm not the best person to be talking about this.
posted by ambrosen at 9:59 AM on March 16, 2013

Dietr Bohn of The Verge I think said it best when he likened RSS to the time shifting nature of the DVR. Stumbling into bookmarks or wading into Twitter gets me a snapshot of what is happening right now but doesn't expose me to everything from the people I care to follow. Twitter is great when I need to find where the food truck currently is but I find it hard to follow someone's thoughts or conversations unless I am super obsessive. I find with RSS I can follow more things more deeply than I could without.
posted by mmascolino at 10:05 AM on March 16, 2013

I'm like the only person who thought rss was an awkward, expensive bandage on the overwrought muddle of "web technology", and sees moments like this as a call to return to netnews, yes?
posted by ead at 10:21 AM on March 16, 2013

Considering the level of Yahoo hate around these parts, if Yahoo released something identical to Google Reader, would everyone spontaneously combust?
posted by davejay at 1:18 PM on March 16, 2013 [1 favorite]

That would be My Yahoo, wouldn't it?
posted by The corpse in the library at 1:21 PM on March 16, 2013 [1 favorite]

I'm like the only person who thought rss was an awkward, expensive bandage on the overwrought muddle of "web technology"

Honestly, RSS is a pretty shitty format, from the perspective of "I have to write code to produce / consume this". Atom is somewhat nicer: while it has its problems, it's one of the few XML formats that really strikes me as sensible and appropriately specified. That said, it still suffers from the inherent problems of the whole "our feed is a separate resource from the canonical version of the document, and you must frequently poll at least one endpoint per feed" model. We would all be vastly better off if the baseline tech we had gotten instead was some semantics, ignorable by browsers, for describing a given HTML document as a feed. At the right historical moment, you could pretty well have done it with an <article> tag and maybe a little bit of stuff in the header. Maybe if there hadn't been all that XML¹ in the air back in the late 90s, or if Dave Winer had been as smart as he thought he was, something like this would have stuck instead.

What happened is that we got RSS and friends. The important thing about this isn't that it's a great technology in the particulars. The thing is that it's a well-understood, widely-supported standard with a toolchain. We could do better, but not by abandoning the things that RSS provides to wholly-owned platforms like G+ and Facebook. Every time we lose a viable protocol that operates outside the control of multi-billion-dollar actors with unfathomably massive resources, the network as a whole becomes vastly more impoverished. Another chunk of the commons is suddenly surrounded by a fence and patrolled by thugs with no allegiances beyond the whims of the new landlord.

and sees moments like this as a call to return to netnews, yes?

There is something to be said for this idea, in metaphorical/aesthetic if not extremely literal terms.

¹ I pause here to think for a moment about SOAP, spit once in disgust like Clint Eastwood is always doing in The Outlaw Josey Wales, and return to my thesis.
posted by brennen at 3:25 PM on March 16, 2013 [2 favorites]

Yeah, I get the common non-corporate aspect. Just wish it was built on something with slightly more reasonable semantics for quoting, syncing, identifying and referencing, and a little more text payload per megabyte. It's saying something that even netnews wins on that count, considering how wretched the formats and protocols in the associated rfcs are.
posted by ead at 5:09 PM on March 16, 2013 [1 favorite]

Considering the level of Yahoo hate around these parts, if Yahoo released something identical to Google Reader, would everyone spontaneously combust?

We could all chalk it up to Marissa Mayer being the actual brains behind Google all those years.

That said, it still suffers from the inherent problems of the whole "our feed is a separate resource from the canonical version of the document, and you must frequently poll at least one endpoint per feed" model. We would all be vastly better off if the baseline tech we had gotten instead was some semantics, ignorable by browsers, for describing a given HTML document as a feed.

Nothing prevents you from slapping some CSS/XSL on an Atom feed and make it look like a web page if you want. Modern browsers actually do support this technology just fine.

Maybe if there hadn't been all that XML in the air back in the late 90s, or if Dave Winer had been as smart as he thought he was, something like this would have stuck instead.

That's kind of what XML + XSL/XSLT were intended to do. However, it's way more work then just slapping together two different ways of formatting whatever's in your database.

Think about it: if you're a developer and you have a database of articles, would you rather just write two functions in your favorite programming language, or one function and then some crazy XSLT transform?

In reality, developers just consider the stuff in their database to be the 'canonical' version, and the page/feed versions are just versions.

At the right historical moment, you could pretty well have done it with an <article> tag and maybe a little bit of stuff in the header.

What exactly is it that you don't think can be done with Atom's <entry> tag that could be done with an <article> tag in an HTML document?

RSS does totally suck compared to Atom, though.
posted by delmoi at 6:35 PM on March 17, 2013

Trying out other RSS readers. [sigh] This is a PITA.
posted by desuetude at 5:47 PM on March 18, 2013 [2 favorites]

Nothing prevents you from slapping some CSS/XSL on an Atom feed and make it look like a web page if you want. Modern browsers actually do support this technology just fine.

Yeah, I'm aware. In practice, however, it's not what (all but a few) people do, and it'd usually be more painful to construct than an HTML document. I've been expressing admiration for Atom; I think it's pretty decent for its niche. But it's still more bureaucracy than most people want to shoehorn stuff other than obvious feeds into. (Not that much, really, but I have noticed that it really doesn't take much to disinterest most programmers.)

What exactly is it that you don't think can be done with Atom's <entry> tag that could be done with an <article> tag in an HTML document?

Suddenly there is that special vibration in the air that only really emerges when the conditions are right for a fight about XML. I know I sort of stirred it up. I take it back. Maybe I should burn some sage or something.

Anyway, It was an off-the-cuff thought, but after some reflection I still think we'd be better off if the dominant "feed" technique we'd gotten had been a minimal, fault-tolerant vocabulary within the markup language everyone was already (sloppily, enthusiastically) using for everything else. (Like, say the kind of vocabulary that teenagers using bad regexps in terribly written Perl could have worked with pretty handily. The sort of thing that people who have not so much as heard of a parser can still produce and consume more-or-less correctly.)
posted by brennen at 10:07 PM on March 18, 2013

Anyway, It was an off-the-cuff thought, but after some reflection I still think we'd be better off if the dominant "feed" technique we'd gotten had been a minimal, fault-tolerant vocabulary within the markup language everyone was already (sloppily, enthusiastically) using for everything else. (Like, say the kind of vocabulary that teenagers using bad regexps in terribly written Perl could have worked with pretty handily. The sort of thing that people who have not so much as heard of a parser can still produce and consume more-or-less correctly.)

So you want to fight about XML, huh?

Rather than argue about this enthusiastic embrace of Worse Is Better, I should just say that there is a fair amount of pain caused by the dominance of tools that don't know much about the structure of data. Regex and line oriented tools are not a good fit for hierarchical data. As long as we use hierarchical formats, it would be nice if good tools for dealing with those were available and popular.
posted by a snickering nuthatch at 10:47 AM on March 19, 2013

I'm not sure who said it first, but this is not the time for a standards fight, it's the time to implement to the standards as they are practiced.
posted by jepler at 11:45 AM on March 19, 2013 [1 favorite]

I'm not sure who said it first

Vitruvius, probably.
posted by Chrysostom at 12:19 PM on March 19, 2013 [3 favorites]

/sub MeRead

(Isn't crawling easier with a Reader since you only have to crawl the RSS feeds of the subscriptions people actually have?)
posted by anotherpanacea at 6:31 AM on March 27, 2013

Has any semblance of internet-wide consensus been reached on the best successor to Reader? I'd like to get on board whichever one has the most momentum in case social features are ("re")introduced.

I was surprised to see, when Reader's axing was announced, that Google basically considered it a superuser product. I specifically liked it because I'm not a techie, and just wanted a barebones, unfinicky way to have my blogs come to me.
posted by threeants at 5:25 PM on March 27, 2013 [1 favorite]

Rock Steady: If you use the "All" view of Feedly with the "Titles" display option, it is visually almost identical to Google Reader, as far as I am concerned.

Holy Zarquon's Singing Fish: Also, Feedly's desktop client can be made to replicate Google Reader pretty well, but if you want a mobile client theirs is absolutely hopeless.

Rock Steady: Why no "All" view in mobile, Feedly? Damn it. Sign me up for a donation/subscription to MeFeed.

For anyone following this in Recent Activity or otherwise, you may want to know that Feedly has updated their iOS app to include a "All/Titles" view option. It's still not quite as good as the mobile Google Reader interface, but it's not bad.
posted by Rock Steady at 12:35 PM on April 1, 2013

Annoyingly, the Android app got an update too, but for some reason they don't think it was worth adding a "titles only" option in that version.
posted by Holy Zarquon's Singing Fish at 3:41 AM on April 2, 2013

iPad also needs to wait for "Titles Only" it appears. Guess it was just iPhone for now. Sorry for the false hope.
posted by Rock Steady at 10:36 AM on April 2, 2013

I'm just popping into this thread to see if a Metafilter Feed Reader has gained any traction. Matt, please do this. Please kickstarter it or whatever else you need to do to make it happen. There are obviously many alternatives out there, and many others are thinking the same thing about building their own. But Metafilter is the only place I trust. I may have gripped about the mobile UI on metafilter a time or two, but I think that could be overcome. :)

It could even follow the mefi model of free with ads and pay for no ads. Of course in this case, monthly makes more sense. Heck, with Matt's weight behind it, it could be the type of service that expands the rss audience.

Overall, I think metafilter would be the best curator of such an entity. Please please make this happen. You have the social capital to get some really talent parties interested.

Please hope me.
posted by [insert clever name here] at 12:56 PM on April 7, 2013

Feedly update: The Android app now has a "title only" option! It does not, in fact, display only the title of each article.

I can only conclude that the developers have some kind of cruft-Tourette's at this point.
posted by Holy Zarquon's Singing Fish at 11:58 AM on April 8, 2013 [1 favorite]

How hard would it be to simply clone the existing Reader interface, both desktop and mobile? That's what everybody wants anyway.

If Google can't or won't open-source the backend, they can at least make the interface more open to copying.
posted by Rhaomi at 5:35 PM on April 8, 2013 [1 favorite]

I'd always thought the Google Reader interface was horrible until I started looking for a replacement. While it was ugly, at least it's simple and straightforward.
posted by The corpse in the library at 11:06 AM on April 9, 2013 [1 favorite]

« Older Read The Fantastic Article | Six Degrees of Kevin errrr... Kickstarter Newer »

You are not logged in, either login or create an account to post comments

MetaTalk

We Need a Hero March 14, 2013 8:38 AM Subscribe

Tags

Share