Laying Pipes

wood.tang — Fri, 30 Nov 2007 16:55:57 -0500

Since I’ve been on an RSS kick this week, I wanted to pass along a tip about Yahoo Pipes, a pretty slick, albeit nerdy, “composition tool to aggregate, manipulate, and mashup content from around the web.” In a nutshell, it lets you mix, match, slice, and dice web searches and RSS feeds to your heart’s content.

I found it after one of my editors gave me assignment to track down what people outside of Chicago are writing about our local sports teams. Unless I wanted to comb through thousands of search results or new feeds, I had to ﬁgure out a way to set up an efﬁcient, automated search on each of the teams that could ﬁlter out stuff by the local rags.

My ﬁrst instinct was to set up a big search in Google News, then subscribe to the RSS feed of the results. I consulted with a friend who’s a renowned Google ninja, and he suggested I try Pipes instead, because it would allow me to exclude any news source I wanted, suppress duplicate items, and mix it all up into one nice feed.

I’ve been playing with Pipes for the last couple days, and it’s not for the faint of heart. Anyone who’s had experience with a workﬂow or integration development environment (webMethods, represent!) will instantly recognize the design palette. It’s ultra-programmy, but I’m guessing anyone who thinks they need to consolidate, ﬁlter, and sort RSS feeds won’t be daunted by that. What’s nice though is that there are a number of example pipes that other smart people have developed already, so if you’re not up for building your own, you can still jump right in.

I’m interested to see if anyone else is doing anything cool with Pipes, so leave your tips in the comments.

”Laying Pipes” was written by Matt Wood for 43Folders.com and was originally posted on November 30, 2007. Except as noted, it's ©2009 Merlin Mann and licensed for reuse under CC BY-NC-ND 3.0. "Why a footer?"