sorted by: recent | see : popular

Content Tagged with pipes + Web2.0

Call them pipes, teqlos, dapps, modules, mashups or whatever else but fact is that recently we have seen a good number of new services that allow developers and users to build mini-apps and mashups that mix and re-mix data. Here we run through 5 applications that allow you to mix, rip and mash your data, looking at the data input, output, REST support, suggested use, and required skill level:

Yahoo Pipes

Yahoo! Pipes is a GUI web app that lets you create new data feeds by remixing syndication feeds (RSS, Atom, RDF). Pipes takes in feeds from around the web, letting you sort, join, and analyze the feeds items before outputting them in RSS or JSON. It also has a good query builder module that lets you grab feeds based on URL parameters. Yahoo! has also created a community around the service, letting users publish and remix other people’s pipes. The resulting data from the pipes can even be used for other mashups, as Teqlo has done.

Ideal for:Pipes is best suited for mashups between well formed feed data with Yahoo! services such as Search, Local, Flickr, or even Google Base, since the modules are already included. Programming experience is limited to an understanding of procedural programming control structures (loops, logical tests) and aided by the visual interface.

Examples: Apartments near something (Craigslist and Yahoo! Local). eBay Price watch (eBay RSS API).

Teqlo

Teqlo is a new widget-based mashup application. You build mashups by dropping specialized widgets onto the canvas and specifying interactions between them. For instance, you can map the results of an eBay search by dropping an eBay search widget and Google map widget on the canvas. Then you connect the two widgets by specifying an interaction such as when an item is selected in the eBay widget, add a marker on the Google map. The application is then accessed by a webpage with the active AJAX widgets. Other widgets include Google Calendar, Gadgets, Spreadsheets, LinkedIn search, DabbleDB search, YouTube viewer, contact lists, and to do lists.

The service is currently in beta, so they have a limited number of modules and have not turned on publishing to the web yet.

Ideal for: Teqlo is a high level masher best suited for non-programmers. Users create interactions between widgets by specifying an action in one widget causing a reaction in another. However, Teqlo’s high level approach means most of its power lies with its developers ability to craft useful widgets and interactions.

Examples: Examples are not public yet, but an example Teqlo is covered in their blog.

Proto

Proto is a Windows based mashup application meant to join your desktop apps with the web. You need the Windows application to both create and use the mashups. It’s component based, joining your desktop and web apps by pulling data from your desktop applications, such as Outlook, and feeding it into online web components, such as Yahoo! maps. Proto has the Visual Basic for Applications development environment (VBA IDE) and Adobe Flash baked in, so you can create your own modules to pull and display data from your applications. Proto also has a light database it uses to broken and manipulate data between the application and online component.

Ideal for: Proto takes some familiarity with database concepts and hopefully VBA experience so that you can program your own modules. Their 5 minute intro is indicative of the experience level you need to really use the program. Since Proto allows you to share your mashups, non-programmers can also use Proto for their library of pre-existing mashups.

Examples: The intro video provides a good example of the program, but downloading the viewer is needed to view modules like the restaurant viewer or more enterprise minded Salesforce reporter.

Dapper

Dapper is a web based application for generating XML for website content. You create “Dapps” (web services) by using Dapper’s virtual browser to grab content from web pages. Dapper is trained by feeding it several example urls that hold examples of content you’re interested in. Dapper looks at the similarities between the pages to take a guess at the important content to pull from the page. After Dapper has analyzed the page, you can narrow down the fields on the page you want to track. For instance, the titles of stories on Digg. Dapper can then output the content you select from the page in various formats (XML, JSON, HTML, and YAML) and incorporate that data to trigger alerts or even map locations found in the feed. Each Dapper application, “Dapp”, is published to the community for anyone to use.

Ideal for: Dapper takes minimal programming experience and is useful for making well structured feeds for pages that don’t have them already. Their demo movie is a good place to start.

Examples: Fidget is a tool that lets you find videos of your favorite bands based on searches carried out by Dapper.

OpenKapow

OpenKapow is the industrial strength version of Dapper. It’s a desktop app that programs RSS feeds, REST apps, and web clips through a browser interface. You can use OpenKapow to make a web robot to pull from a web page like Dapper, but can direct that bot to navigate web pages (including form submission), carry out loops, branches, recover from errors, and accept user input at any point in the process. OpenKapow has a community where developers can share their robots to be used and remixed by other users.

Ideal for: OpenKapow is ideal for serious web scrapping. It takes basic knowledge of procedural programming and web markup to use.

Examples: Here is a robot that logs into Gmail and outputs your email in XML. Here’s another one that searches deep into TechCrunch’s posts for a keyword.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

It takes effort to explain the significance of a new product when the immediate benefit to consumers may not be so obvious, and the awkwardly named “pipes” from Yahoo! is no exception. The product name is taken from the world of UNIX where a pipe is a conduit for the transfer of data between applications, while with the Yahoo product it is a conduit for data between web services. In a basic form Yahoo! Pipes allows you to take data from one or more sources and to bring it together, for example - to aggregate a group of feeds.

But Yahoo! Pipes goes beyond what just pipes are and what pipes do though as the application provides functions (or as they are called in the app - modules) that will perform a variety of different actions. There are modules available to prompt the user for input (a variety of input types), different operators to count, loop, cut, count, sort and merge data along with a variety of string and date functions. Because of this already broad base of available functions, Yahoo! Pipes is more akin to a shell scripting environment for the web rather than just a simple conduit between applications. It works like a visual procedural programming language with the output of the process dropping out at the bottom, in the form of text output, RSS, SMS alerts of even JSON. You can use feeds, user input or other pipes as input.

The beauty of the application is with its simplicity - a user can take any sources, user input requests or the above mentioned module and drag+drop them into place and then connect the pipes. Within minutes I had built an application (also known as a pipe, they should probably change the name as not everything can be a pipe) that would search for ‘Techcrunch’ in a variety of feeds, bring that data together, sort it and filter it for unique results. I saved the application and published it, from where I can now execute it at any time and receive the output in a variety of formats. I can take a copy of an existing pipe (application, argggh) and use it as a base template for my own pipe and I can browse an existing library of pipes.

Pipes can take any feed as input, and combined with the already available list of functions proves to be very powerful - my mind is still buzzing thinking about all that can be done with Pipes. I think some of the terminology needs to be cleared up, there needs to be a better introduction on the main page - but besides that this product is fantastic. It was inevitable that such a product would be released, and it is very good for Yahoo! that they managed to be the first of the big web companies to release such a product. The fact that they include Google Base as a default source in Pipes shows that the web is much more about interoperability than the desktop ever was or ever will be.

See Anil Dash, Tim O’Reilly and Jeremey Zawodny for more.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.