≡ Menu

Debian / Ubuntu Python: Set Planet Venus To Combine Two Or More RSS Feeds

How do I setup “Planet Venus” under Debian or Ubuntu Linux to downloads news feeds (RSS) published by web sites and aggregates their content together into a single combined feed?

From the project home page:

Planet is a flexible feed aggregator. It downloads news feeds published by web sites and aggregates their content together into a single combined feed, latest news first. It uses Mark Pilgrim’s Universal Feed Parser to read from CDF, RDF, RSS and Atom feeds; Leonard Richardson’s Beautiful Soup to correct markup issues; and either Tomas Styblo’s templating engine or Daniel Viellard’s implementation of XSLT to output static files in any format you can dream up.

You can use this tool to combine two or more feeds easily i.e. this software act as a “Feed Aggregator”. The following instructions were tested on:

  1. Ubuntu Linux 12.04 LTS.
  2. Ubuntu Linux 16.0.4 LTS.
  3. Debian Linux lastest stale.

Installation

Type the following apt-get command:
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install planet-venus

OR
$ sudo apt update && sudo apt install planet-venus
Sample outputs:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libltdl7 libmhash2 libpython-stdlib libpython2.7 libpython2.7-minimal libpython2.7-stdlib libraptor2-0 librasqal3 librdf0 libtidy-0.99-0 libyajl2 python
  python-beautifulsoup python-chardet python-feedparser python-html5lib python-htmltmpl python-httplib2 python-librdf python-libxml2 python-libxslt1 python-minimal
  python-pkg-resources python-portalocker python-six python-utidylib python2.7 python2.7-minimal
Suggested packages:
  raptor2-utils rasqal-utils librdf-storage-postgresql librdf-storage-mysql librdf-storage-sqlite librdf-storage-virtuoso redland-utils python-django python-genshi
  python-lxml python-doc python-tk python-setuptools python2.7-doc binutils binfmt-support
The following NEW packages will be installed:
  libltdl7 libmhash2 libpython-stdlib libpython2.7 libpython2.7-minimal libpython2.7-stdlib libraptor2-0 librasqal3 librdf0 libtidy-0.99-0 libyajl2 planet-venus python
  python-beautifulsoup python-chardet python-feedparser python-html5lib python-htmltmpl python-httplib2 python-librdf python-libxml2 python-libxslt1 python-minimal
  python-pkg-resources python-portalocker python-six python-utidylib python2.7 python2.7-minimal
0 upgraded, 29 newly installed, 0 to remove and 0 not upgraded.
Need to get 6565 kB of archives.
After this operation, 27.2 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmhash2 amd64 0.9.9.9-7 [87.0 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial/main amd64 libyajl2 amd64 2.1.0-2 [19.6 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 libraptor2-0 amd64 2.0.14-1 [154 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial/main amd64 librasqal3 amd64 0.9.32-1 [163 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-minimal amd64 2.7.12-1~16.04 [339 kB]
Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7-minimal amd64 2.7.12-1~16.04 [1294 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-minimal amd64 2.7.11-1 [28.2 kB]
.....
..
...
Unpacking python-libxslt1 (1.1.28-2.1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Setting up libmhash2:amd64 (0.9.9.9-7) ...
Setting up libyajl2:amd64 (2.1.0-2) ...
Setting up libraptor2-0:amd64 (2.0.14-1) ...
Setting up librasqal3:amd64 (0.9.32-1) ...
Setting up libpython2.7-stdlib:amd64 (2.7.12-1~16.04) ...
Setting up python2.7 (2.7.12-1~16.04) ...
Setting up libpython-stdlib:amd64 (2.7.11-1) ...
Setting up python (2.7.11-1) ...
Setting up libltdl7:amd64 (2.4.6-0.1) ...
Setting up libpython2.7:amd64 (2.7.12-1~16.04) ...
Setting up librdf0:amd64 (1.0.17-1build1) ...
Setting up libtidy-0.99-0 (20091223cvs-1.5) ...
Setting up python-pkg-resources (20.7.0-1) ...
Setting up python-chardet (2.3.0-2) ...
Setting up python-six (1.10.0-3) ...
Setting up python-html5lib (0.999-4) ...
Setting up python-htmltmpl (1.22-10.1) ...
Setting up python-httplib2 (0.9.1+dfsg-1) ...
Setting up python-librdf (1.0.17.1+dfsg-1.2ubuntu1) ...
Setting up python-libxml2 (2.9.3+dfsg1-1ubuntu0.1) ...
Setting up python-utidylib (0.2-9build1) ...
Setting up python-portalocker (0.5.5-1) ...
Setting up python-feedparser (5.1.3-3build1) ...
Setting up planet-venus (0~git9de2109-4) ...
Setting up python-beautifulsoup (3.2.1-1) ...
Setting up python-libxslt1 (1.1.28-2.1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...

How do I use planet venus?

Type the following command to create a default planet for planet.cyberciti.biz in /home/apache/domains/planet.cyberciti.biz/http:
$ mkdir -p /home/apache/domains/planet.cyberciti.biz/http
$ cd /home/apache/domains/planet.cyberciti.biz/http
$ planet --create linuxnews

To edit this default planet called linuxnews, enter:
$ cd linuxnews
$ vi planet.ini

Sample outputs:

# Global configuration
[Planet]
 
name            = nixCraft Linux News Planet 
link            = http://planet.cyberciti.biz/
owner_name      = nixCraft
owner_email     = webmaster@server1.cyberciti.biz
output_theme    = theme
cache_directory = cache
output_dir      = output
feed_timeout    = 20
items_per_page  = 60
log_level       = DEBUG
 
# RSS/Feed subscription configuration
# [Feed-RSS-URL]
# name = Web Site Name Here
 
[http://feeds.cyberciti.biz/Nixcraft-LinuxFreebsdSolarisTipsTricks]
name = nixCraft: Linux Tips, Hacks, Tutorials, And Ideas In Blog Format
 
[http://rss.slashdot.org/Slashdot/slashdotLinux]
name = Linux Stories From Slashdot
 
[http://feeds.feedburner.com/linuxtoday/linux]
name = Linux Today
 
[http://www.reddit.com/r/linux/.rss]
name = Reddit /r/Linux

Save and close the file.

Build default planet right away

Type the following command:
$ planet --verbose planet.ini
Sample outputs:

INFO:planet.runner:Socket timeout set to 20 seconds
INFO:planet.runner:Building work queue
INFO:planet.runner:Updating feed http://rss.slashdot.org/Slashdot/slashdotLinux
INFO:planet.runner:Updating feed http://feeds.cyberciti.biz/Nixcraft-LinuxFreebsdSolarisTipsTricks
INFO:planet.runner:Updating feed http://www.reddit.com/r/linux/.rss
INFO:planet.runner:Updating feed http://feeds.feedburner.com/linuxtoday/linux
INFO:planet.runner:Loading cached data

Test the results in your favorite web-browser:
http://planet.cyberciti.biz/linuxnews/output/atom.html
OR
http://localhost/linuxnews/output/index.html
Test the RSS result in your preferred rss reader:
http://planet.cyberciti.biz/linuxnews/output/atom.html
OR
http://localhost/linuxnews/output/atom.html
Sample outputs:

References:

Planet venus project home page.

Share this tutorial on:
{ 3 comments… add one }
  • db15 December 15, 2012, 3:14 pm

    Hi, is there a way for pagionation in the Venus Planet? You know, if i want e.g. 50 latest feeds on first page, a oldest on the second page and go on.

    • nixCraft December 18, 2012, 7:20 pm

      Use items_per_page, see this page for more info.

      • db15 December 19, 2012, 9:22 am

        Ok, but how to show older feeds in template? items_per_page eg. limits to show 50 newest feeds, and I want to have 150 feeds, 50 feeds per page. Is there a way to add something to template that will have <> eg. to browse for newset to oldest feeds?

Security: Are you a robot or human?

Leave a Comment

You can use these HTML tags and attributes: <strong> <em> <pre> <code> <a href="" title="">


   Tagged with: , , , , , , ,