More RSS Tools: Managing Content with Pipes

This post first appeared on the Idealware Blog in March of 2009.

I’m continuing with follow-up topics from my RSS article, Using RSS Tools to Feed your Information Needs. Last week, I discussed integrating content with websites, and this week I’m going to dive into one of the more advanced ways to work with RSS content. This gets a little geeky, but it really shows off some of the sophistication of this technology.

The article provides numerous examples of RSS sources, but all in the form of web sites, blogs and web services that offer you one or more streams of information. If you want to narrow your view beyond the feeds available on a site, say, because you are only interested in Idealware posts about CRM tools or the ones written by Steve Backman, then you need a tool that will refine your search. Alternatively, you might want to put a section containing news stories relevant to a particular issue on your site, but want some control over the sources, as well as the subject matter. For this amount of control over the content you retrieve, you want to use something like Yahoo! Pipes.

Pipes is an RSS mashup editor. It’s a tool that looks a bit like Microsoft’s Visio, where you drag boxes onto a grid and draw relationships between them. But it’s not a layout or flowcharting tool; instead, it’s a visual mapping and filtering tool that lets you identify sources and then apply rules to those sources before merging them into an aggregated feed. To break that down, let’s say that your goal is to either monitor talk about a bill, or, maybe, to publish a section on your web site titled “What they’re saying about bill 221b” (I made that bill up). You have identified eight blogs that have good posts on the subject, and these are blogs that you trust to properly represent the issues and not, in any way, malign or confuse your efforts.

In Pipes, you can select all eight as sources, and then set up a filter to block any posts that don’t reference “221b”. The resulting RSS feed — which you can then subscribe to our republish — will isolate the posts that are relevant to the bill from your selected sources.

For example, here’s that pipe that will allow you to skip Michelle, Heather, Paul, Laura, Eric and my posts and just see Steve’s:

Picture 2.png

Another, more advanced example: You have an organizational Twitter feed that you want to republish to your site But you only want to publish your posts, not your individual replies. In Twitter, a reply is always identifiable by the very first character, which will be an “@” sign. Twitter RSS items arrive in the format “yourtwitterid: tweet”, so any reply will start with “yourtwitterid: @”. Setting up a Yahoo Pipe filter to block any result with “: @” in the text will isolate your posts from the replies. You can add a “Regex” (e.g. Search/Replace) command to replace “yourtwittername:” with nothing in order to publish just the tweet. The pipe will look like this:

Picture 1.png

If you play with Pipes (Yahoo! ID required, otherwise free), I highly recommend starting with an example like mine or this one by Gina Trapani to get the feel of it. Save your pipe, and you can subscribe to it — it updates automatically, and you don’t have to make it public for it to work.

Google has it’s competing Google Mashups tool in private beta, and similar tools are popping up all over the web. I talk a lot about how RSS is the technology that allows us to manage the information on the web. Pipes let us refine it. It’s great stuff.

Look for more RSS talk on OPML files and Google Reader in my upcoming posts.

Share Button