sorted by: recent | see : popular

Content Tagged with DATA + RSS

Making data portability suck less.

The Google data APIs ("GData" for short) provide a simple standard protocol for reading and writing data on the web. GData uses either of two standard XML-based syndication formats: Atom or RSS. It also has a feed-publishing system that consists of the Atom publishing protocol plus some extensions (using Atom's standard extension model) for handling queries.

»The Gnip API provides notification of activities (events) occurring in a variety of services and, whenever possible, a guid that identifies the activity itself vis a vis the service it was created on. Activity examples include a user "tweet" (twitter), a user "dugg" (digg), a user creating a blog post, etc.

pil ce qu'il me faut !!! :)

Dapp Factory sirve para hacer RSS de cualquier sitio.

"David Robinson, Harlan Yu, William Zeller, and Ed Felten, all of Princeton's Information Technology Policy Center, suggest that government officials abandon the dream of developing usable web sites, and instead focus on providing raw public data such as

outil pour extraire des données de pages web et les tranformant en flux xml

I’ve been coding a couple of scripts that run on 5 minute intervals to grab RSS/Atom feed data from http://mysql-dba.com and import that into a MySQL database. It idea is to create a search function for the site that will look at all past data from the aggregated feeds. Since there are multiple pollers running at different intervals I decided to use Innodb for the read/write nature of the poller/processing scripts.

This is very simple so far - and as such I felt it should be documented from the start unlike many of my other projects. Here’s the feed table that is storing the information from the RSS feeds.


mysql> show create table feed_items\G
*************************** 1. row ***************************
Table: feed_items
Create Table: CREATE TABLE `feed_items` (
`id` bigint(20) NOT NULL auto_increment,
`rss_site_id` int(11) NOT NULL,
`item_title` varchar(255) NOT NULL,
`item_link` varchar(255) NOT NULL,
`item_description` varchar(255) NOT NULL,
`item_guid` varchar(255) NOT NULL,
`item_pubDate` varchar(255) NOT NULL,
`Creation_time` datetime NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `rss_site_id` (`rss_site_id`,`item_link`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)