More on Yahoo Pipes and building custom RSS

craigslist yahoo pipes

As mentioned, one of my arcane job specialties is programming in Clipper, a DOS database programming language that was big in the late 80’s – early 90’s. And while it may seem hard to believe, there are still about a zillion Clipper apps out there. I’m currently working with a multi-national client who has a Clipper app originally written in 1988 that runs their business in one country.

I found them on Craigslist by using Yahoo Pipes, which is a powerful way to pull in RSS data from multiple sources. Craigslist only allows you to search one site at a time. With Pipes, I can search multiple sites at once, then filter out what I don’t want. This is important when searching for “Clipper”, unless you want a bunch of hair stylist and dog groomer jobs in your search results (because such ads often have the word “clipper” in them.

Yahoo Pipes lets you build RSS feeds with a drag and drop interface.

First, go to any Craigslist and search for what you want. Right-click the RSS icon on the results page, choose Copy Link Location, and save that in a text file. It’ll look something like this.

“http://losangeles.craigslist.org/search/jjj?query=Clipper+-barber+-stylist&catAbbreviation=jjj&format=rss”

This is the Craigslist search for “Clipper” excluding “barber” and “stylist.” It appears to only take a few terms for exclusion, which is ok, because Pipes can exclude the rest.

– I cut and pasted that search into Fetch Feed then continued the process, changing the area name until I got several areas in California to search.

– Unique tells it to combine items with duplicate titles, as Craigslist ads are sometimes posted multiple places.

– Filter is quite powerful. I told it to ignore any items with “hair”, “groomer”, “dog”, or “pet” in the description.

– Sort sorts by date, descending.

– Pipe Ouput is the result.

You don’t need to be a programmer to do this. You can search for anything on Craigslist this way, as well as on other sites.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.