I’ve been coding a couple of scripts that run on 5 minute intervals to grab RSS/Atom feed data from http://mysql-dba.com and import that into a MySQL database. It idea is to create a search function for the site that will look at all past data from the aggregated feeds. Since there are multiple pollers running at different intervals I decided to use Innodb for the read/write nature of the poller/processing scripts.
This is very simple so far - and as such I felt it should be documented from the start unlike many of my other projects. Here’s the feed table that is storing the information from the RSS feeds.
mysql> show create table feed_items\G
*************************** 1. row ***************************
Table: feed_items
Create Table: CREATE TABLE `feed_items` (
`id` bigint(20) NOT NULL auto_increment,
`rss_site_id` int(11) NOT NULL,
`item_title` varchar(255) NOT NULL,
`item_link` varchar(255) NOT NULL,
`item_description` varchar(255) NOT NULL,
`item_guid` varchar(255) NOT NULL,
`item_pubDate` varchar(255) NOT NULL,
`Creation_time` datetime NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `rss_site_id` (`rss_site_id`,`item_link`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)